Agents that can explain what they did
An agent that takes actions on your behalf and cannot account for them afterwards is not a product, it is a liability. The engineering that separates the two is unglamorous and mostly about constraint.
Design
Scope the tools before the reasoning
The instinct is to hand an agent broad capability and let the model work it out. In practice the reliable systems have a deliberately narrow tool surface, where each tool does one thing and validates its own inputs.
An agent that can issue arbitrary queries against your database will eventually issue one you did not want. An agent with four specific, parameterised operations will not.
Approval gate: reversible actions pass, irreversible ones stop
Where a human belongs
Full autonomy is rarely the right target
The useful question is not how autonomous the agent can be. It is which actions are cheap to reverse.
- Run unattendedReading, retrieving, drafting, summarising, classifyingCheap to reverse. Wrong output costs a re-run.
- Ask a person firstSending, paying, deleting, anything a customer seesExpensive to reverse. Keep the gate until evidence says otherwise.
- Do not automate yetOpen-ended access to production systemsThe failure mode is unbounded and the tooling is not there.
Non-negotiable
Traces are not optional
When an agent does something surprising, and it will, you need the inputs, the tool calls, the intermediate reasoning and the final action stored and searchable.
Without that you cannot debug it, you cannot improve it, and you certainly cannot explain it to a customer or a regulator.
Questions
Frequently asked
- How reliable are AI agents in production today?
- Reliable enough for narrow, well-scoped tasks with a human on anything irreversible. Not reliable enough for open-ended autonomy over systems that matter. Anyone telling you otherwise is selling a demo.
- Which agent framework do you use?
- LangGraph and the Model Context Protocol where they fit, and plain application code where a framework would add indirection without value. Most production agents are simpler than the frameworks assume.
- How do you test something non-deterministic?
- A task set with a pass criterion per task, run repeatedly, tracking a rate rather than a single result. You measure how often it completes correctly, not whether it produced one exact string.
Also from us
- AI consulting servicesDeciding whether to build, what it will cost and what your data can actually support.
- AI integration servicesConnecting a model to your existing stack without breaking your permission model.
- Hire AI developersSenior engineers inside your team, committing to your repo and teaching as they go.
Thinking about putting an agent in front of real users?
Tell us which actions it would take and who it would take them for. That usually determines the whole design.