How to tell a real agent from a wrapped prompt

Almost every firm in our Agentic AI area now uses the word agent. Some are shipping software that plans, acts and recovers. Others have a prompt template with a loop around it. The gap matters because the second kind cannot be operated safely in production, and you usually cannot tell from the demo.

The four-question test

You can run this on a vendor call without seeing any code. The value is less in the answers than in how quickly they arrive. A team running real agents has hit all four problems already and will have opinions. A team that has not will improvise.

Why the failure question comes first

A wrapped prompt has one failure mode: it returns something wrong and stops. A real agent has many, because it took several steps and some of them succeeded. That distinction produces completely different engineering. Teams who have built the second thing will talk about partial completion, compensating actions and idempotency without being prompted. Teams who have not will describe better prompts.

If the answer is that failures are rare, that is not reassurance. It means either the system is not doing much, or nobody is measuring.

Permissions tell you the maturity

Ask what the agent can actually do in your systems and how that is scoped. The healthy answer involves a narrow service account, a per-action allowlist, and a kill switch somebody non-technical can reach. The answer to worry about is a shared credential with broad access, because it means nobody has yet had the conversation with a security team.

The four layers, and which one they build

A working agent needs a planner that breaks a goal into steps, a tool layer with real permissions, a memory of what has been tried, and an evaluation harness that catches a bad run before a person does. Only the planner touches the model directly. Ask which of the four the vendor builds and which they expect you to already own, because the unspoken answer is usually the evaluation harness, and that is the one that takes longest to build yourself.

What a good pilot looks like

Pick a process with volume, a clear success metric and a tolerable error cost. Ticket triage, invoice matching and claims intake all qualify. Run six to twelve weeks. Insist on seeing the failure log weekly rather than a summary at the end, because the shape of the failures tells you whether the thing will survive contact with a real queue.

What not to pilot: anything where a wrong action is expensive and hard to reverse. That is a phase-two problem, and starting there is the most common way these projects get cancelled.

The honest caveat

A wrapped prompt is not useless. For summarising, drafting and classification it is often the right tool and cheaper to run. The problem is only the mismatch: buying a wrapped prompt while planning for an agent, and then discovering the gap during a production incident. Name which one you are buying in the SOW.