Why most AI pilots never reach production

Most organisations we meet have already run an AI pilot. Rather fewer are running one in production. The gap between the two is not usually the model — it is everything around it.

The demo problem

A pilot is built on data somebody prepared, run by the person who built it, judged on examples they chose. Production is none of those things. The data arrives messy, the operator has other work to do, and the examples arrive in whatever order reality sends them.

That is not a criticism of pilots. It is what a pilot is for. The mistake is treating a successful one as evidence that deployment is a formality.

Three things that actually stop it

Nobody owns the output. A model produces a suggestion. Someone has to accept it, and be accountable when it is wrong. If that person has not been named before launch, the system stalls the first time a suggestion is questioned.

It lives outside the workflow. If the tool is a separate tab, using it is extra work. The pilots that survive are the ones where the output appears inside the system people already have open — the ERP, the helpdesk, the CRM.

No one can explain a decision. In a regulated sector, "the model said so" is not an answer. You need to know what was asked, what came back, and who approved it. Retrofitting that after launch is expensive; adding it during the build is nearly free.

How to tell early

Before the pilot starts, write down the number you expect to move — handling time, error rate, cost per document — and measure it as it is today. If nobody can name that number, the pilot cannot succeed, because there is no definition of success.

Then ask who will own the output, and where it will appear. If those two questions have answers, the pilot has a route to production. If they do not, you are funding a demonstration.

The honest version

AI is very good at the repetitive, judgement-light parts of work that consume a surprising share of everyone's week. It is not good at deciding what your business should improve. Pick the process first, then the tool.