The News
The EU AI Act’s transparency requirements are now in force, marking a regulatory inflection point for enterprise AI deployments across Europe and any organization operating within its reach. Cathal McCarthy, Chief Strategy Officer at Kore.ai, has been advising enterprise leaders on navigating the regulation and argues that the AI Act is ushering in what he calls an “accountable era,” one in which organizations must demonstrate active oversight, monitoring, and accountability for AI systems acting on their behalf. His central warning is pointed to companies that treat compliance as a documentation exercise rather than an operational commitment risk accumulating “governance debt” that will stall AI initiatives not because the technology breaks, but because the organizational infrastructure to govern it was never built.
Analyst Take
The Governance Gap Is Already Here
The AI Act arriving in force is less a starting gun than a deadline many enterprises were not ready for. The regulation’s transparency requirements demand that organizations can explain, monitor, and intervene in the behavior of AI systems, particularly those classified as high-risk. That sounds straightforward until you map it against where most enterprises actually stand. ECI Research’s 2026 Application Development survey found that 58.2% of respondents selected “Moderate increase (10–25%)” when asked how much they will increase AI governance spending, which suggests the majority of organizations are budgeting for incremental improvement rather than the structural overhaul the Act may actually require. Modest budget increases applied to fragile governance foundations will not close the gap McCarthy is describing.
The “governance debt” framing deserves serious attention from ITDMs. Debt in software engineering accumulates when shortcuts taken today create compounding costs tomorrow. The same logic applies to AI governance. Organizations that launch AI agents, automate workflows, and integrate AI into customer-facing processes without building audit trails, intervention mechanisms, and accountability structures are creating obligations they will eventually have to pay down, often under regulatory pressure and on someone else’s timeline.
What the Act Actually Demands from Engineering Teams
For developers and platform engineers, the AI Act’s transparency requirements translate into concrete technical obligations. High-risk AI systems must maintain logs, support human oversight, and allow for meaningful intervention. That means the observability stack is not just a reliability tool anymore; it becomes a compliance artifact. ECI Research’s 2026 Application Development survey found that 61.7% of respondents have AI-driven anomaly detection in place as an observability strategy, which is encouraging. But detection is not the same as governance. Knowing that a model behaved unexpectedly is a prerequisite; having the documented chain of accountability for what happened next is what the AI Act actually requires.
Teams building AI-enabled applications should be auditing their current pipelines now for three specific gaps: whether model behavior is logged at a granular enough level to satisfy a regulator’s review, whether there is a documented human-in-the-loop mechanism that can actually be activated (not just described in a policy), and whether the organization can produce an explanation of a specific AI decision on demand. Most engineering teams have addressed the first gap partially. The second and third remain largely aspirational.
Who Benefits and Who Gets Exposed
Kore.ai’s positioning here is commercially transparent, but McCarthy’s underlying point is not wrong because it comes with a vendor interest attached. The companies that will benefit most from the AI Act’s enforcement are those that have already built AI orchestration and governance infrastructure into their platforms. Vendors offering agent management, workflow oversight, and compliance-ready AI deployment layers are well-positioned to absorb enterprise budget that is now shifting from “AI capability” to “AI accountability.”
The organizations most exposed are those that ran fast on AI adoption in 2023 and 2024 without pausing to build governance guardrails. Many of those deployments were pilots that became production systems by momentum rather than deliberate design. The AI Act will force a reckoning with that legacy. ITDMs should be asking their teams not “what AI do we have running” but “what AI do we have running that we can actually explain, monitor, and shut down on demand.”
Looking Ahead
The AI Act’s transparency rules in force today represent only the early phase of a multi-year regulatory tightening. Enforcement teeth will sharpen as national supervisory authorities mature their review processes and as the first high-profile penalties land. Organizations that treat the current period as a grace window to build genuine governance infrastructure will be in a materially stronger position when enforcement intensifies. Those that continue treating compliance as a documentation exercise will find themselves in exactly the “governance debt” trap McCarthy describes, scrambling to retrofit accountability into systems that were never designed for it.
Kore.ai’s “accountable era” framing is likely to become the dominant lens through which enterprise AI adoption is evaluated over the next two to three years. The competitive question is no longer simply which organization can deploy AI fastest. It is which organization can deploy AI fastest while maintaining the oversight infrastructure that regulators, customers, and boards are increasingly demanding. That shift creates real market opportunity for vendors building AI governance tooling, and real urgency for any enterprise that has so far treated governance as someone else’s problem.
Stay Ahead of Application Development Trends
Get weekly analyst insights, research notes, event coverage, and AppDevANGLE updates delivered directly to your inbox.
Subscribe for Weekly Insights
Join technology leaders, practitioners, and GTM teams following the trends shaping modern software delivery.
Looking for deeper research access?
Explore ECI Research reports, survey insights, and market analysis through the ECI Research Portal.
