Until recently, AI inside companies answered questions. It summarized documents, drafted text, and classified requests. If it got something wrong, a person read the answer and decided what to do.

Now AI is starting to act. Agents connected to ERPs, CRMs, and internal APIs can create orders, change customer records, issue refunds, open tickets, and message customers.

The promise is real: less repetitive work, faster service, processes that keep running outside business hours. So is the risk. An agent that can execute an operation can execute it wrongly, twice, for the wrong customer, or at a volume nobody anticipated.

My position: natural-language instructions are not controls. An agent connected to business systems must operate within verifiable technical limits — the same ones we would require of any other system or person with access to those operations.

The problem is not that AI makes mistakes. It is that nobody defined what happens when it does

Writing “never grant discounts above 10%” in a prompt is not a security policy. It is a suggestion to a probabilistic model, which can be bypassed by a malicious input, an ambiguous context, or a simple misinterpretation.

If the discount API accepts 40%, the agent can grant 40%.

The right question is not “how do we convince the AI to behave?” but “what does the system allow, regardless of what the AI decides?”

That distinction changes the entire design. Control moves out of the prompt and into the architecture: API authorization, value limits, business validations, approvals, and audit records.

Five controls every enterprise agent needs

1. Minimal, explicit permissions

A customer service agent that looks up orders does not need permission to change prices. An agent that opens tickets does not need access to financial data.

For each agent, define which operations it can perform, on which data, and within which limits. This applies both to the tools exposed to the model and to the credentials it uses behind them.

A practical rule: if you would not give that permission to an intern on their first day, do not give it to the agent without an additional control.

2. Identity: on whose behalf is the agent acting?

When an agent changes a record, the system needs to know who requested the action. Agents that use a single, broadly privileged service credential erase that information — and allow any user, through the agent, to do what they could not do themselves.

The safer design is for the agent to operate with delegated identity: its effective permissions are the intersection of what the agent may do and what the user who invoked it may do.

3. Human approval proportional to risk

Not every action needs approval. Requiring confirmation for every lookup defeats the purpose of automation. Requiring it for nothing turns the agent into an operational risk.

A simple classification helps:

Operation typeExampleSuggested control
ReadCheck an order’s statusExecute directly, with logging
Reversible, low-impact changeUpdate a contact phone numberExecute directly, with logging and the ability to revert
Change with limited financial impactRefund up to a defined amountExecute with hard limits per operation and per period
Critical or irreversible changeCancel a contract, change bank details, large refundMandatory human approval before execution

Illustrative classification. Each company should set its own limits based on its risks and obligations.

The approval request should show the approver what will be done, with which data, and why — not just a “confirm” button.

4. Idempotency and transactional control

Agents repeat actions. A slow call gets retried. An interrupted workflow restarts from the beginning. Without protection, that means duplicate refunds, double orders, and the same message sent several times to one customer.

Operations exposed to agents must be idempotent: executing the same request twice must produce the same result as executing it once. In practice, this requires a unique key per operation, verified by the target system.

Multi-step operations — reserve stock, charge the customer, send a notification — need a clear strategy for failures halfway through: undo the earlier steps, or resume from where the process stopped.

5. Complete audit trail and recovery

For every action, record who requested it, which agent executed it, which tool was called, with which parameters, what the result was, and — when applicable — who approved it.

This record is not only for investigating incidents. It lets the company answer customers, auditors, and regulators, measure the quality of the automation, and decide with evidence which operations can be given more autonomy.

An example: the refund agent

Consider a fictional e-commerce company that wants to automate refunds for orders with delivery problems.

Naive design: the agent receives an administrative credential for the payments API and a prompt instruction: “only grant refunds when the delay is proven.”

What can go wrong: a customer writes a carefully crafted message and convinces the agent there was a delay. A network glitch makes the agent retry the call, and the refund is processed twice. Afterwards, nobody can tell which conversation triggered the operation.

Controlled design:

  • the agent checks delivery status directly in the logistics system instead of trusting the customer’s account;
  • the refund tool only accepts orders belonging to the customer authenticated in the conversation;
  • refunds up to a defined amount run automatically; above it, they go to an approval queue;
  • every request carries an idempotency key — retrying the call does not create a new payment;
  • there is a daily total limit per agent, and the capability is suspended automatically when it is reached;
  • every refund records the conversation, parameters, result, and approver.

Illustrative example, unrelated to any real client.

The agent remains useful — most cases are resolved in seconds. But the worst possible scenario becomes bounded and traceable.

Why this has become urgent

Platforms are making it ever easier to connect models to real systems. The .NET 11 launch in November 2026 highlighted tool integration through MCP (Model Context Protocol) and workflows with Microsoft Agent Framework. The Python and JavaScript ecosystems are moving in the same direction.

The easier it is to connect, the more likely an agent reaches production before anyone has designed its limits.

The security community has responded too. The OWASP Top 10 for Agentic Applications catalogs the main risks of agent-based applications — including tool misuse, excessive permissions, poorly handled delegated identity, and autonomous actions without adequate oversight. It is a solid reference for anyone who needs to justify controls to leadership.

Accountability versus control

In an earlier article, Who signs off on AI decisions?, I discussed who answers for the decisions of an automated system. That is a question of accountability.

This article addresses the practical consequence: once you decide who is accountable, you must make sure the system only allows what that person is willing to sign. Accountability without technical control is just a promise.

Where to start

  1. Inventory the operations that are candidates for automation and classify them by impact and reversibility.
  2. Start with read-only and reversible operations, measuring the agent’s quality before expanding its autonomy.
  3. Review the APIs that will be exposed: do they enforce business limits on their own, or did they assume only a well-behaved UI would ever call them?
  4. Define approval, limits, and auditing before production, not after the first incident.
  5. Have a kill switch: the ability to suspend an agent immediately without affecting the rest of the system.

Frequently asked questions

Isn’t a good prompt enough to control the agent?

No. A prompt guides behavior but does not enforce limits. Effective controls must live in the APIs, permissions, and business rules of the target system.

Does every agent action need human approval?

No. Approval should be proportional to risk. Lookups and reversible changes can run automatically, with logging. Critical or irreversible operations should require prior approval.

Can I connect an agent directly to my ERP?

Technically, often yes. The question is with which permissions, on which operations, and with which safeguards. It is usually safer to expose a dedicated service layer for the agent, with its own validations and limits, than to grant broad ERP access.

How do I know if the agent is performing well in production?

Measure completion rate, escalation rate to humans, errors, reversals, and complaints linked to agent actions. A complete audit trail is what makes that measurement possible.

Conclusion

AI agents can deliver meaningful operational gains. But autonomy without technical limits is not innovation — it is exposure.

Before allowing an agent to change business data, assess which operations can be automated and which controls they require.

Through my AI and data architecture services, I help companies design integrations between agents and existing systems with permissions, approvals, auditing, and failure recovery defined from day one. Commercial engagements are delivered by IVATEC.