The AI Accountability Gap: Why Agent Governance Can’t Wait

The Announcement

Kore.ai has released new survey data revealing a striking contradiction at the heart of enterprise AI adoption: more than half of organizations (53.2%) have deployed autonomous AI agents without fully understanding how those agents will behave. The findings paint a picture of governance lagging badly behind deployment velocity. Nearly four in five organizations (79.4%) have required manual reversals after autonomous agent actions, 62% have delayed deployments specifically due to concerns around control and observability, and 41.7% report that AI agent failures have directly caused revenue loss. Alongside the survey, Kore.ai announced Artemis, a new AI-programmable platform designed to help enterprises deploy governed multi-agent systems in days rather than months.

The Bigger Picture

This data lands at a moment when the enterprise AI deployment conversation has shifted from “should we?” to “how fast?” The problem is that speed without governance is how you get revenue-impacting failures, compliance exposure, and the kind of headline-making AI incidents that set entire categories back by years.

The AI Accountability Gap Is Real and Growing

The term Kore.ai uses, “AI accountability gap,” is an accurate description of a structural problem. Organizations are deploying agents into workflows that touch customers, finances, and operations, while their governance frameworks are still at the prototype stage. The 79.4% manual reversal rate is especially telling. That’s not a statistic about rare edge cases. That’s a majority of enterprises discovering, after the fact, that their autonomous systems made decisions they weren’t prepared to stand behind.

ECI Research’s 2025 AI Builder Summit survey found that 44% of enterprise AI leaders have only moderate confidence that AI agents can act autonomously without human intervention. The Kore.ai data suggests that confidence gap is well-founded. Enterprises aren’t just worried in the abstract; they’re correcting live production errors at scale.

The 41.7% revenue loss figure deserves particular attention from ITDMs. This isn’t a theoretical risk. A meaningful share of organizations deploying agents have already absorbed a financial hit from doing so without adequate governance. For any ITDM building a business case around AI agent deployment, this number should be part of the risk calculus from day one.

What This Means for ITDMs

The economic argument for moving fast on AI agents is real. Automation of repetitive tasks, faster decision cycles, reduced headcount pressure on routine work. But the Kore.ai data reveals a hidden cost structure that most business cases don’t account for: the cost of failure, correction, and remediation when ungoverned agents act unexpectedly.

ECI Research’s 2025 AI Builder Summit survey also found that two-thirds of enterprise AI leaders have already implemented multi-agent collaboration in live or pilot workflows. That number confirms that multi-agent architectures are no longer experimental. But wide deployment without a matching governance layer creates compounding risk, because agent-to-agent coordination failures are harder to detect and attribute than single-agent errors.

For ITDMs, the practical implication is this: governance infrastructure is not a phase-two consideration. It belongs in the architecture from the start. Platforms that can instrument agent behavior, enforce decision boundaries, and trigger human review when thresholds are crossed are no longer nice-to-have features. They’re table stakes for any enterprise operating in regulated industries or with customer-facing agent workflows.

What This Means for Developers

From a technical standpoint, the 62% deployment delay rate signals something important: developers and platform teams are already aware that current tooling doesn’t give them the observability they need to ship confidently. The manual reversal problem is largely an observability and rollback architecture problem. If you can’t observe what an agent decided and why, you can’t build reliable corrective mechanisms.

Artemis, Kore.ai’s new platform, is positioned to address exactly this. The claim that enterprises can deploy governed multi-agent systems “in days instead of months” is ambitious, and the technical credibility of that claim will rest on how Artemis handles the hard parts: decision logging, audit trails, policy enforcement across agents, and integration with existing enterprise identity and access management systems. Developers evaluating multi-agent platforms should be pushing hard on those specifics rather than taking deployment speed at face value.

The broader architectural lesson from the Kore.ai data is that multi-agent systems require a fundamentally different reliability engineering approach than single-model inference pipelines. Orchestration, state management, and failure containment aren’t just infrastructure concerns. They’re core design decisions that need to be made before the first agent touches production data.

What’s Next

Governance Will Become the Primary Buying Criterion

The current market dynamic, where 53.2% of enterprises are deploying agents they don’t fully understand, is not sustainable. Regulatory pressure is building across the EU AI Act, SEC disclosure requirements, and sector-specific frameworks in financial services and healthcare. As those regulations mature, enterprises that have deployed ungoverned agents will face retroactive remediation costs that dwarf what upfront governance investment would have required.

We expect governance capability to become the primary enterprise buying criterion for multi-agent platforms within 12–18 months. Right now, organizations are prioritizing deployment speed. By 2026, the organizations that moved fastest without governance in place will be spending significant engineering cycles on remediation, while those that built governance in from the start will be scaling. Vendors that can credibly demonstrate compliance-ready agent governance, with audit logs, policy enforcement, and rollback mechanisms, will command premium pricing and longer contract terms.

The Human-in-the-Loop Question Will Be Redefined

The 79.4% manual reversal rate is not evidence that AI agents are fundamentally broken. It’s evidence that most organizations haven’t yet defined the right boundaries for autonomous action in their specific business contexts. That’s a solvable problem, but solving it requires deliberate work: mapping which decisions carry acceptable automation risk, which require human review, and which should never be autonomous. Platforms that help organizations draw and enforce those lines programmatically will be far more valuable than those that simply offer faster deployment.

The market framing will shift from “autonomous vs. supervised” to something more granular: graduated autonomy with context-aware escalation. Expect the leading platforms in this space to build increasingly sophisticated policy engines that allow enterprises to tune agent autonomy by workflow type, data sensitivity, and risk threshold, rather than applying a single governance posture across all use cases.

Authors

  • Paul Nashawaty

    Paul Nashawaty, Practice Leader and Lead Principal Analyst, specializes in application modernization across build, release and operations. With a wealth of expertise in digital transformation initiatives spanning front-end and back-end systems, he also possesses comprehensive knowledge of the underlying infrastructure ecosystem crucial for supporting modernization endeavors. With over 25 years of experience, Paul has a proven track record in implementing effective go-to-market strategies, including the identification of new market channels, the growth and cultivation of partner ecosystems, and the successful execution of strategic plans resulting in positive business outcomes for his clients.

    View all posts
  • With over 15 years of hands-on experience in operations roles across legal, financial, and technology sectors, Sam Weston brings deep expertise in the systems that power modern enterprises such as ERP, CRM, HCM, CX, and beyond. Her career has spanned the full spectrum of enterprise applications, from optimizing business processes and managing platforms to leading digital transformation initiatives.

    Sam has transitioned her expertise into the analyst arena, focusing on enterprise applications and the evolving role they play in business productivity and transformation. She provides independent insights that bridge technology capabilities with business outcomes, helping organizations and vendors alike navigate a changing enterprise software landscape.

    View all posts