Key takeaways
- Demos optimise for applause; production systems must survive Monday morning.
- Start with a measurable job, not a choice of model or technology.
- Deliberately constrained agents beat ambitious, unpredictable ones.
- Most AI value comes from data plumbing, guardrails and integration — not raw capability.
- Ship the smallest valuable version, measure it like a new hire, then compound.
The demo trap
Every week we see the same pattern: a slick AI agent demo that answers three curated questions flawlessly, everyone applauds, budget gets approved, and six months later the thing is quietly switched off. The problem was never the model. The problem was that a demo optimises for applause, while a business system has to survive edge cases, angry customers, messy data and a CFO asking what it actually returned.
At Evocative we build AI agents that do work, not agents that do theatre. That distinction shapes every decision we make, from scoping to deployment.
Start with a job, not a technology
The first question we ask is never "which model?" It is "what specific, measurable job is this agent replacing or accelerating?" An agent that "helps with customer support" is a demo. An agent that "drafts first-response replies for tier-1 tickets, cites the correct policy, and escalates anything involving refunds over €200" is a system you can measure, trust and improve.
We insist on a definition of done that a non-technical stakeholder can read. If we cannot express the value in a sentence a business owner cares about — hours saved, response time cut, conversion lifted — the project is not ready to build.
Constrain the agent on purpose
Counterintuitively, the most valuable agents are the ones we deliberately limit. Open-ended autonomy looks impressive in a demo and behaves unpredictably in production. We give agents:
- A narrow, well-defined scope so behaviour stays predictable and testable.
- Tools instead of guesswork — real API calls, database lookups, verified sources — rather than letting the model improvise facts.
- Clear escalation paths so the agent knows what it must hand back to a human.
- Guardrails and refusal logic tuned to the actual risks of the business, not generic safety theatre.
A constrained agent that handles 70% of cases reliably beats an ambitious one that handles 95% of cases usually. Reliability compounds trust; unpredictability destroys it.
Ground everything in the client's reality
Generic models know the internet. They do not know your pricing, your inventory logic, your tone of voice or the exception your operations team has been living with for years. The real engineering work is rarely prompt-writing — it is retrieval, data plumbing, and integration.
We spend a disproportionate amount of time on the unglamorous layer: cleaning knowledge bases, structuring context, connecting the agent to the systems where truth actually lives. This is where demos cut corners and where production systems earn their keep.
Measure like it is a colleague
Once an agent is live, we treat it like a new hire on probation. We log every interaction, sample outputs, track where it fails and feed those failures back into the design. Key metrics we hold ourselves to:
- Task success rate — did it actually complete the job, not just respond?
- Human intervention rate — how often does a person need to correct it?
- Time-to-value — measured against the manual process it replaces.
- Cost per resolved task — because token bills are real and margins matter.
An agent without observability is a liability. If you cannot see what it is doing, you cannot trust it, and you certainly cannot improve it.
Ship small, then compound
We deploy the smallest version that delivers genuine value, put it in front of real users, and expand from there. This keeps risk low, builds organisational trust in AI, and surfaces the real requirements that no workshop ever predicts. A team that has seen one agent quietly save them hours is far more ready for the next.
The uncomfortable truth is that most AI value comes from boring reliability, not from breakthrough capability. The magic is in the plumbing, the constraints and the measurement — the parts nobody demos.
The bottom line
Impressive demos are cheap. Systems that colleagues rely on, that pass audits, and that show up in the P&L are hard — and that difficulty is exactly the point. We build for Monday morning, not for the boardroom applause.
If you are weighing up an AI agent and want it to earn its place rather than gather dust, we would be glad to talk it through with you and pressure-test the idea before a line of code is written.
