Log your AI near misses before you have to explain an AI incident

The cheapest AI lesson is the one caught just before it reached a customer — if someone writes it down.

The mistake that almost happened

If you use AI at work, one of these will probably sound familiar.

The agent drafted a reply quoting last year's price, and someone noticed before sending. The automation was about to email the whole customer list instead of one segment, and a quick check stopped it. The assistant summarised a contract and missed the cancellation clause, and the account manager happened to read the original. A refund was queued twice and finance caught the duplicate.

In each case, someone sighed, fixed it and moved on. No harm done.

And that is the problem. The lesson disappears the moment the fix is made. Next week, a different person on a busier day may not catch the same thing.

Why near misses are worth more than you think

Aviation has long treated near misses as valuable data, not embarrassing secrets. A near miss shows where the system is weak without the cost of a crash.

That idea is now being applied to AI. On 3 October, Loughborough University's Vice-Chancellor, Professor Nick Jennings, signed an open letter to the UK Government on governing agentic AI. Among its recommendations is a pilot of a common route for reporting significant agent failures and near misses, with lessons published and an approach drawn from aviation accident investigation. Its technical annex says deployers should keep the information needed to report failures and near misses, including the agent version, permissions, tools used, sequence of actions and human interventions.

That is aimed at governments and larger deployers. The underlying idea scales down perfectly to a ten-person business. A near miss is free training data about where your AI workflows can go wrong.

What gets lost when nobody writes it down

When near misses go unrecorded, a few things happen.

The same mistake repeats. Without a record, there is no pattern. Each catch looks like a one-off.

The catch depends on luck. If the only safeguard was one alert person, that safeguard is gone when they are on leave.

Risk looks lower than it is. Leaders see no incidents and assume the AI is working well, when in fact people are quietly catching problems every week.

You have no story when something does go wrong. If a real incident happens, being able to show that you tracked warning signs and acted on them is far better than starting from nothing.

A near-miss log that people will actually use

Keep it simple. If logging takes more than a couple of minutes, nobody will do it.

For each near miss, capture:

1. What almost happened. One or two sentences.
2. Which tool or workflow was involved.
3. How it was caught. By whom, and was it luck or a deliberate check?
4. What could have happened if it had not been caught.
5. What we changed, if anything.

A shared form, a simple spreadsheet or a dedicated channel is enough. The point is that it lives in one place and anyone can add to it.

Make it safe and make it useful

No blame. People will only report near misses if doing so does not get them or a colleague in trouble. Thank people for logging them.

Review regularly. A short monthly look at the log is enough. Which workflows keep appearing? Which catches depended on luck?

Turn patterns into fixes. If the same type of mistake shows up three times, change the workflow: add a check, narrow the permissions, update the source information.

Close the loop. Tell the team what changed because of what they logged. That is what keeps people reporting.

Questions to ask your team

• What AI mistakes have we caught in the last month?
• Were they caught by a deliberate check, or by someone happening to notice?
• Is there anywhere those catches are written down?
• If a customer was affected by an AI error tomorrow, could we show what we had already learned?

If the answer to the third question is no, start the log today. It takes five minutes to set up.

How Walter thinks about it

At Walter, I see near misses as part of how a business gets better at running AI work. A good chief of staff notices when something nearly went wrong, writes it down and makes sure the fix sticks, so the same scare does not happen twice.

The incidents that make headlines are usually preceded by quieter warnings that nobody recorded. Write yours down while they are still cheap.

Join the waitlist at https://hirewalter.ai/