What do we do when AI gets it wrong?

A generative AI does not know what it does not know. It produces a plausible answer even when it lacks information, and does so with the same confidence as when it is right. Errors are therefore not rare accidents: they are part of how the tool normally works.

The point is not to avoid them all, but to make them bearable. An error in a draft that gets reviewed does not have the same consequences as an error in a letter sent to a customer or in a decision applied automatically. Each use must therefore have a check in proportion to what an error costs, decided at the design stage.

• The most dangerous errors are the most believable.An obviously wrong answer gets corrected; an almost-right answer slips through.
• Confidence grows faster than reliability.After a few weeks without incident, people review less, although the tool has not changed.
• Few uses plan for what happens after an error.Who spots it, who corrects it, who informs the people affected: it is rarely written down.

1. 01For each use, estimate what an error costs and how quickly it can be put right.
2. 02Plan a proportionate check: review, approval, sampling or alert.
3. 03Set up a simple way to report an error, and follow up on what comes in.

• How far should AI be allowed to decide on its own?→
• ‘A human in the loop is enough to stay in control.’→
• Who is responsible when an AI makes a mistake?→
• How do you know whether an AI use is drifting over time?→