The first generation of generative AI waited for a person to ask a question. The person supplied the objective, relevant context, and timing. The model produced an answer, then stopped. This was useful, sometimes remarkably so, but the operating responsibility remained entirely with the user.
An agentic system is expected to continue. It receives an objective, decides what information it needs, uses tools, checks progress, handles ordinary setbacks, and returns when the work is complete or a decision exceeds its authority. That change from response to continuation is small in language and large in engineering.
Many products now call themselves agents because they can make several model calls or trigger an API. Those capabilities are ingredients. The harder question is whether the system can carry responsibility through the untidy conditions of a real company.
A prompt is an instruction, not an operating model
A good prompt can produce an excellent answer. It cannot define who may approve a refund, which customer record is authoritative, how long an unresolved case may wait, or what to do when the finance system and the signed contract disagree.
Those are operating questions. They require policy, permissions, source hierarchy, time limits, evidence, and named human authority. A production agent needs all of them.
Take a simple request: follow up with every prospect who attended yesterday's webinar. A model can draft messages in seconds. The real workflow asks more. Which attendees are existing customers? Who has already opted out? Which accounts have an open opportunity? What language should each message use? May the system send directly, or must a salesperson approve? What happens when the CRM owner is missing? When should silence lead to another message, and when should outreach stop?
The prompt describes an intention. The operating model turns that intention into safe, repeatable work. Confusing the two produces polished output attached to a fragile process.
Agency begins when the system can continue
Continuation requires a working loop. The system observes the current state, chooses a permitted action, executes it through a tool, inspects the result, and updates its plan. It repeats until it reaches a completion condition or an escalation condition.
Each part of that loop can fail. The source may be unavailable. A tool may return a partial result. The record may contain conflicting dates. The plan may be technically possible and commercially wrong. The completion condition may be ambiguous.
A credible agent makes these states visible. It does not treat every tool response as success. It verifies material changes, preserves the connection between evidence and action, and records why it chose the next step. It can resume after interruption without inventing progress that never occurred.
This is where persistent context matters. The system needs the state of the case, the history of actions, the applicable policy, and the facts it has confirmed. It should not force the model to rediscover the whole world on every turn. At the same time, context needs boundaries so that an old assumption does not silently control a new case.
The demo gap
Demonstrations select a clean path. Production discovers every path the demonstration did not include.
An invoice agent looks convincing when the supplier name, purchase order, amount, and currency match. The useful test begins when the supplier changed its bank account, the purchase order was split across departments, the PDF contains two invoices, or the amount differs because a credit note arrived in another email.
A support agent performs well when the answer exists in one current article. Production sends a customer whose account spans three products, whose contract includes a negotiated exception, and whose earlier case was closed with the wrong diagnosis.
A scheduling agent succeeds when calendars are current and participants are known. Real work includes an executive assistant protecting travel time, a customer using a different time zone, a room that appears free but lacks the required equipment, and a meeting that cannot move because it precedes a regulatory filing.
The demo gap is not mainly a model benchmark problem. It is a coverage problem. The system needs a defined response for incomplete data, contradictory evidence, tool failure, policy ambiguity, and actions that cannot be reversed. Production quality comes from finding these states early and designing their treatment.
Boundaries make autonomy useful
Autonomy without a boundary is difficult to trust and almost impossible to manage. The system should know its role, permitted resources, action limits, completion standards, and escalation owners.
Permissions are the first boundary. A customer operations agent may read account history and draft a service credit. Issuing that credit may require approval above a threshold. Changing contractual terms belongs elsewhere. The tool layer should enforce these limits even if the model proposes a broader action.
Policy is another boundary. Written rules need to be translated into conditions the system can apply and evidence it can cite. If a refund depends on delivery date, product condition, and customer tier, the agent should collect those facts and show how they support the decision. A confident sentence is not evidence.
Time creates boundaries too. An agent may wait for a supplier response for two business days, send one reminder, then escalate. Without time rules, unattended work can remain technically active while operationally abandoned.
Good boundaries do not make an agent timid. They let it move quickly inside a field where the organisation has decided speed is safe. The system can complete routine work without seeking approval for every step because its authority was designed in advance.
Memory must earn the right to persist
Long-term memory is attractive because it promises a colleague who learns. It is also a direct route for errors to compound.
Suppose a supplier accepts one late purchase order during an emergency. If the agent stores that event as a general supplier rule, the next order may bypass the normal process. Suppose a salesperson describes a prospect as price-sensitive after one call. If that observation becomes permanent account truth, future communication may inherit a weak assumption.
Memory needs classes. Stable facts, such as a legal entity number or approved payment term, can persist with source and effective date. Preferences can persist with an owner and a way to revise them. Interpretations should carry confidence, provenance, and expiry. Temporary case details should close with the case.
Writeback also needs a threshold. Some observations can be suggested for human confirmation. Others can be committed automatically when supported by an authoritative source. Conflicts should create a review item rather than allowing the newest statement to overwrite the old one.
The useful measure of memory is not how much the agent retains. It is whether retained context improves future work without weakening control over what the company believes to be true.
Exceptions reveal the real system
Teams often design the normal path in detail and treat escalation as a final box labelled human review. That box hides the hardest work.
A useful escalation identifies the blocked objective, the facts already established, the conflicting rule or missing authority, the actions attempted, and the exact decision required. It reaches the person who can decide while there is still time to act.
The human response should return to the workflow cleanly. If a controller approves paying an invoice despite a missing goods receipt, the agent should record the approval, finish the case, and avoid turning one authorised exception into a new standing rule.
Exception data is also the best product roadmap. Repeated missing purchase orders may point to an upstream procurement problem. Frequent language escalations may reveal where local knowledge is thin. A cluster of customer cases may expose a product defect. The agent should help the company see these patterns, not merely survive them.
The frontier is operational
Model improvements will continue to expand what a system can reason about and execute. That progress matters. It will not remove the need for authority design, source discipline, evaluations, observability, and accountable management.
The most credible agentic systems will look less magical over time. Their work will appear in ordinary queues. Managers will know what they own. Auditors will be able to inspect material decisions. People will intervene at defined points and receive enough context to act quickly.
That is the frontier worth building toward: systems that continue with purpose, stop for good reasons, and leave the organisation able to explain what happened. Agency earns its place when the work survives contact with reality.
This is an operating-model question. Our AI Company Operating Model thesis sets the company boundary, Governed AI sets the product boundary, and The EU AI Act Is Product Architecture explains why the distinction matters in Europe.