Key takeaways
- Treat the model as stateless; make an external store the source of truth for workflow state.
- Separate working context, short-term state and long-term memory — each with its own retention rules.
- Idempotent actions make retries safe and prevent double-charging or duplicate sends.
- Sandbox agents with deny-by-default permissions and log every tool call.
- You are legally accountable for your agents' actions, so build the audit trail to prove control.
Every week brings a new wave of AI agents: Meta ships a cute mascot, OpenAI bundles existing features into a friendly persona, HubSpot renames its stack for the fourth time. The announcements blur together. But underneath the marketing, the real engineering question has barely changed: how does an agent remember what it did, carry state across steps, and do so without creating operational or legal risk? That's where most production projects live or die.
Why memory is the hard part, not the model
The frontier models are already good enough. As Zapier notes, even Anthropic's smallest Claude model now outperforms the best systems from two years ago. Raw reasoning is rarely the bottleneck. What breaks in production is continuity: an agent that forgets a customer's earlier message, re-runs a billed action, or loses track of where it was in a multi-step workflow.
An agent's "memory" is really three distinct things, and conflating them is the most common design mistake we see:
- Working context — the current conversation and task, held in the prompt window. Fast but ephemeral and expensive to grow.
- Short-term state — where the agent is in a workflow: which tools it called, what succeeded, what's pending. This belongs in a database or a state machine, never in the prompt alone.
- Long-term memory — durable facts about a user or account, typically stored in a vector index or structured store and retrieved on demand.
Tools like OpenAI's "dots" or Meta's Muse package these layers behind a persona, which is excellent product design but doesn't remove the underlying responsibility. If you build on their APIs, you still own the architecture decisions about what persists, for how long, and where.
State belongs outside the model
The pattern that survives contact with production is simple: treat the model as stateless and make an external system the source of truth. Every agent step reads state in, acts, and writes the new state back. If a run crashes, you resume from the last durable checkpoint instead of replaying from scratch.
This is what distinguishes genuine workflow execution — the kind Zapier's AutomationBench tests end-to-end with real tools — from a chatbot that happens to call functions. Idempotency matters here too: an agent that might retry a step must never double-charge a card or send a duplicate invoice. Store an operation key, check it before acting, and the retry becomes safe.
Memory hygiene saves money and trust
Unbounded memory is a quiet killer. Stuffing everything into context inflates token costs and degrades reasoning quality. The disciplined approach is to summarise and prune: compress old turns into a short state summary, promote only durable facts to long-term storage, and set retention policies per data type. This also keeps you aligned with GDPR — memory that remembers a customer indefinitely is a liability, not a feature.
Containing what the agent can do
Memory decides what an agent knows; sandboxing decides what it can do. NVIDIA's recent OpenShell release takes a deny-by-default stance: anything no rule explicitly allows is blocked. That philosophy should guide any production agent, regardless of tooling. Give the agent the narrowest set of permitted actions, log every tool call, and keep humans in the loop for high-stakes operations.
This isn't paranoia. OpenAI notified over a hundred organisations about agent activity during model training — a reminder that agents act in ways their operators don't always anticipate.
You own what your agents do
The commercial terms are converging across OpenAI, Anthropic and Google Cloud, and they all point the same way: the customer is responsible for their account and their agents' actions. Liability is typically capped at twelve months of fees, beta services are excluded, and indemnities are narrow. For a business deploying agents that touch real systems and real money, this reframes the whole project.
It means your audit trail, your state log and your permission boundaries aren't just engineering niceties — they're your defence. When an agent does something unexpected, you'll need to show exactly what it saw, what it decided, and what it was allowed to do.
What we recommend
For teams moving agents from pilot to production, the priorities are consistent:
- Externalise state; keep the model stateless and resumable.
- Separate working context, workflow state and long-term memory — and apply retention rules to each.
- Make every action idempotent so retries are safe.
- Sandbox with deny-by-default permissions and log every tool call.
- Assume you are legally accountable for the agent, and build the audit trail to prove control.
The model you pick matters less than the scaffolding around it. Get memory, state and containment right, and a modest model becomes a reliable colleague. Get them wrong, and the most capable frontier model becomes an expensive, forgetful liability.
Sources
- Claude models: Fable vs. Opus vs. Sonnet vs. Haiku — Zapier Blog
- Meta Muse AI agent: How to use the personal AI agent — Zapier Blog
- HubSpot AI tools: A complete guide to Agent Hub and Breeze — Zapier Blog
- NVIDIA livre OpenShell pour borner les agents, Sentry reste sans date — Actu IA
- Dots : un bon coup produit, pas une rupture technique — Actu IA
- Agents IA : le client répond de leurs actes, chez OpenAI comme ailleurs — Actu IA
