Source : INDIA TODAY NEWS
A chatbot that gets something wrong produces a bad answer. You read it, you reject it, you regenerate it. The damage is contained in the output. An agent that gets something wrong can leave your systems in a bad state. It can issue a payment twice, update a record it should not have touched, or abandon a workflow halfway through with no record of which steps completed. The output is not the artifact anymore. The state of the world is.
advertisement
The pipeline itself has changed shape. Generative AI was short: prompt, model, output. Agentic AI is long: intent, reasoning, authorization, tool invocation, external action, state transition, verification, recovery. The first architecture produces information. The second produces consequences. And nearly every reliability question that matters in production lives in the parts of that second pipeline that have nothing to do with the model.
That is the shift I do not think the industry has fully absorbed. We are still debating model accuracy as if it were the whole question, while quietly wiring probabilistic systems into tools, databases, payment rails, and other agents. Model reliability still matters, but once AI participates in stateful, multi-step workflows it becomes necessary and insufficient. The reliability of an agent is no longer a property of the model. It is a property of the runtime architecture surrounding the model.
What the benchmarks already tell us
The evidence for this is not speculative. Sierra’s -bench, one of the few benchmarks that evaluates agents against stateful environments, introduced a metric called pass^k: the probability that an agent completes the same task successfully across all k attempts, not just once. The results should unsettle anyone deploying agents in production. Even the best function-calling agents tested succeeded on fewer than half the tasks, and consistency across eight attempts fell below 25 percent in the retail domain.
Read that carefully. Same system, same task, same tools. A demo proves capability. Production requires consistency, and a runtime that knows when consistency has failed.
The multi-agent picture is similar. A UC Berkeley team analyzed more than 1,600 execution traces across seven popular frameworks and built the first systematic taxonomy of why these systems fail: 14 failure modes across specification and system design, inter-agent misalignment, and task verification. Many of these failures were not attributable to model reasoning alone. They arose from system design, coordination, and verification problems: ambiguous roles, lost context, agents that never checked whether an action succeeded before moving on. The authors demonstrate performance gains from refining system design rather than relying solely on better models, which is this essay’s argument in empirical form.
The problems are older than the models
Here is the uncomfortable part for the AI industry: many of these “new” reliability problems are not new. They are distributed-systems problems appearing in a new setting. The moment a model begins coordinating calls across external tools and stateful services, its runtime inherits the failure modes of distributed systems, whether its designers intended one or not: partial failure, retries, timeouts, concurrency, dependency collapse, ambiguous delivery.
One scenario makes this concrete. An agent initiates a payment through an external API. The API executes the transfer, but the response times out before the agent receives confirmation. The agent now faces genuine ambiguity. Did the action fail? Did it succeed and only the acknowledgment get lost? Should it retry? If the operation is not idempotent, retrying may charge the customer twice. Nothing in that failure involves a hallucination, and, crucially, more reasoning cannot fix it. If the transfer executed and the acknowledgment vanished, no amount of intelligence recovers information that never arrived. Resolution requires architecture: idempotency keys, durable state, reconciliation against the system of record.
Hyperscale engineers wrote the playbook for this long before agents existed. Amazon’s Builders’ Library documents how retries without idempotency can duplicate side effects, and why client request tokens make a repeated call resolve to the same outcome as the first. The companion guidance on timeouts, retries, and backoff with jitter explains why naive retry logic turns a small failure into a synchronized storm. None of this was written for AI. All of it now applies to AI, because an agent is a distributed-systems client that happens to reason.
The probabilistic-deterministic boundary
What is genuinely new is the character of the component doing the deciding. A language model is probabilistic by construction. Many of the invariants it now touches cannot be. A payment cannot be “probably executed once.” An authorization boundary cannot be “usually respected.” A ledger cannot be “mostly consistent.” An irreversible customer action cannot be “approximately committed.”
So the core architectural job of an agentic runtime is translation: converting probabilistic reasoning into controlled deterministic execution. The model can propose what should happen. The runtime must establish whether the action is authorized, whether the state the model believes in is still current, whether the action has already occurred, whether it is safe to retry, whether it actually succeeded, and whether the result satisfies the invariants the business cannot bend. The model proposes. The runtime commits.
Take the word commit seriously, because it names the largest gap in current agent architecture. Production systems need to distinguish among stages that today’s frameworks blur together: proposing an action, authorizing it, attempting it, committing it, verifying the resulting state, and recovering or compensating when something in that chain breaks. A model emitting “transfer $500” is not $500 leaving an account. A tool returning an error is not proof the transfer failed. A successful API response is not proof the business transaction completed. Database engineers solved a version of this with atomicity and rollback, and the analogy is useful but imperfect: many external actions cannot be rolled back at all, which is exactly why explicit commit boundaries and compensating actions matter more for agents, not less.
I think of this as the agentic commit boundary: the point where a proposed action crosses from reasoning into consequence. What the runtime demands before allowing that crossing should scale with the consequence. Reading a public document needs almost nothing. Deleting production data needs staging, validation, and a checkpoint to recover from. Moving money needs explicit authorization, idempotency, and verification against the ledger. Which suggests a better definition of autonomy: not how much an agent can decide, but where it is permitted to commit state without additional assurance.
Where reliability actually lives
Once execution is understood this way, as a sequence of controlled state transitions rather than a model producing an answer, the location of reliability becomes much easier to see. It distributes across a stack with six layers. Model reliability: can the model reason adequately for the task? Context reliability: did the system retrieve the right information and keep stale or conflicting context out of the decision? Tool reliability: did the right API execute with the right parameters and semantics? State reliability: does the system know what has happened, what is pending, and whether an action took effect? Workflow reliability: can a multi-step process survive timeouts, partial failures, and substitutions without corrupting the outcome? And governance reliability: was the system authorized, can its behavior be reconstructed afterward, and can a human intervene when the stakes demand it?
The value of the stack is not the categories themselves. Agent reliability is compositional and end-to-end, an emergent property of the entire execution path, and it cannot be inferred from model accuracy. Every additional consequential step is another place where context, state, authorization, or execution can diverge from expectation, which is precisely the behavior pass^k makes visible. A frontier model embedded in a runtime that cannot answer “did that payment actually go through” is an unreliable agent, and no amount of model improvement fixes it.
Read in order, the stack becomes a single progression that an architect can actually build against. Intelligence: can the model determine an appropriate action? Authority: is this particular execution permitted to take it? State: does the system know what has already happened? Commit: can the proposal safely become external state? Verification: did reality change as expected? Recovery: what happens when it did not, or when the answer is unknowable? Provenance: can the entire sequence be reconstructed afterward? Miss a stage, and production will eventually find it for you.
This is why orchestration is quietly becoming the real reliability layer, and why the word deserves a more rigorous meaning than connecting agents. What production systems need is an agentic control plane: the layer that owns identity, authorization, context assembly, workflow state, retries, idempotency, fallback, verification, escalation, audit, and recovery. Its job is enforcing invariants around probabilistic execution, not coordination for its own sake. Even so, Anthropic’s own engineering guidance distinguishes workflows, where code paths are predefined, from agents that direct their own tool use, and recommends starting with the simplest solution possible, which may mean not building an agentic system at all. Every layer of control plane adds complexity, latency, cost, and failure modes of its own. The right goal is not the most elaborate agent architecture but the minimum architecture capable of safely carrying the consequences being delegated to the system.
Fallback deserves particular skepticism. The intuition that if Model A fails you route to Model B treats models as interchangeable parts, and they are not. Two models can expose compatible APIs while differing materially in instruction interpretation, tool selection, structured-output behavior, ambiguity resolution, and refusal behavior. API compatibility is not behavioral compatibility; syntactic substitutability does not guarantee semantic substitutability. Fallback is therefore not redundancy. It is a semantic compatibility problem: the runtime must know whether the substitute preserves the behavioral assumptions under which the workflow was validated. Traditional infrastructure failover generally assumes a stronger contract of behavioral equivalence than model substitution can safely assume.
Identity, meanwhile, is really a delegation problem. Traditional access management asks who a principal is and what it may access. Agentic systems require a harder question: who delegated this authority, for what objective, over which resources, under what constraints, for how long, and with what right to delegate further. An agent should not inherit the full authority of whoever invoked it, which is what happens when it borrows a human user’s session. Its authority should be bounded: task-specific, resource-specific, time-limited, revocable, and auditable. OWASP’s AI agent security guidance points the same direction, calling for least-privilege tool permissions, step-up authentication for critical actions, idempotent high-impact operations, and systems that fail closed when authorization or audit logging fails. The principle underneath is simple to state and hard to build: a model may be technically capable of invoking a hundred tools, and that says nothing about whether this execution should hold the authority to use any of them. An agent’s capability should never imply its authority.
Observability changes shape too. Traditional monitoring asks whether the service is up and requests are succeeding. Agentic systems demand decision provenance: not reconstructing everything the model thought, but reconstructing what the system knew, what it decided, what authority it held, what it attempted, and what actually changed. The OpenTelemetry generative AI semantic conventions are beginning to standardize this, defining trace structures for agent invocations, model calls, and tool executions so an execution path can be rebuilt after failure. That is a deliberately narrower ambition than model explainability, and a far more achievable one.
Evaluating systems, not models
Which is where benchmarking must go. A benchmark can tell you something real about a model. It cannot tell you whether a workflow recovers correctly after partial failure, avoids duplicate actions, respects authorization boundaries, or escalates when uncertainty becomes unacceptable. Those are system properties, testable only against the system, with pass^k-style consistency measured against real state rather than static prompts on a leaderboard. Governance frameworks already point this way: NIST’s Generative AI Profile organizes its suggested actions around governance, content provenance, pre-deployment testing, and incident disclosure, which are process and system controls, not model metrics. Model evaluation measures capability. System evaluation measures dependable execution. The second increasingly determines whether agents belong in production at all.
Generative AI made reasoning probabilistic. Agentic AI puts probabilistic reasoning in charge of consequential action, and that is a fundamentally different engineering problem. The industry spent the first era improving the intelligence inside the model, and that work succeeded. The next frontier is the boundary around it: the systems that determine when intelligence may act, what it may change, how the change is verified, and what happens when reality diverges from expectation. The defining question of agentic AI will not be whether a model can reason its way to the right action. It will be whether the runtime can ensure that the intended action occurs under the right authority, against the right state, without unsafe duplication, and can recover when any of those assumptions fails. The model supplies the intelligence. The runtime decides what that intelligence is allowed to make real.
Dr. Aditya Vikram Kashyap is an AI governance researcher and practitioner whose work focuses on institutional accountability, agentic AI, model risk, and the governance of AI in regulated institutions. His broader work examines emerging technology, financial services, and systemic technology risk. Views expressed are his own.
– Ends
SOURCE :- TIMES OF INDIA
