As they oversee experimentation with and rollouts of AI agents and guide other leaders on the journey, CAIOs are also navigating through the hype, concerns around security and trust, and interconnectedness (or lack thereof) of these systems. Not to mention having to grapple with the question: What even is an AI agent?

Hype-chasing causes companies to lose focus

No one can seem to agree on what, exactly, the term “AI agent” really means, as Fortune and others have reported. Companies are defining the term differently and often using it to describe varied features and capabilities, including many that were previously described with other terms such as “AI assistants.” For Accenture chief AI officer Lan Guan, who led the build of an AI agent solution called Refinery AI for clients and also works directly with them on their own AI and AI agent deployments, this has caused her to devote a great deal of time to just helping clients sort through the contradictions.

“A year ago, everyone was saying, ‘I need to do gen AI.’ Now everyone is saying, ‘I need to do agentic AI or AI agents.’ And it’s like, at the end of the day, a lot of these things are still the same thing. They’re just getting called different things depending on who you’re talking to,” she said. “And so there’s a ton of confusion in the marketplace with our clients on, ‘What is an AI agent? What am I deploying?’ And so we spend a lot of time on education.”

A runaway effect of this has been companies quickly spinning up so-called AI agents “just for the press release,” says Michelle Bonat, chief AI officer of AI Squared, who also works with companies across regulated industries on their AI development. The pressure to have an answer for the agentic AI moment is causing some companies to rename features or chase AI agents to stay on trend, often merely creating thin layers of agents on top of foundation models.

“I’m totally seeing that. I’m seeing that every day,” Bonat says. “That’s why this space is full of noise.”

Security, errors, and trust dominate the risk analysis

Despite the hype and muddled terminology, the core idea of AI agents—systems designed to autonomously take action to carry out specific tasks—is still generating a lot of justifiable excitement. It’s also key to creating the types of systems technologists and science fiction lovers have always dreamed of, capable of executing sequences of complex tasks across multiple platforms on our behalf. But there are real roadblocks.

Uri Yerushalmi, cofounder and chief AI officer at Fetcherr, which uses AI for predictive pricing in the airline industry, believes the opportunities around AI agents are “enormous” but that unlocking that value depends on addressing real challenges around trust and integration and avoiding failure points. For example, agents must integrate with legacy systems and align with real-world constraints without disrupting existing workflows. And as we give agents more autonomy, we need to build guardrails, monitoring, override systems, and mechanisms for human alignment, he said. 

“Businesses need to trust the agent’s decisions,” he added. “That requires transparency, consistency, and demonstrable ROI.”

One of the most concerning failure points is compounding errors. Google DeepMind CEO Demis Hassabis has compared this issue to compound interest in finances, explaining that even if an agentic model has only a 1% error, it would cause a chain reaction of errors that would, after a few thousand steps, ultimately make the likelihood of a correct result completely “random.” Bonat points to this problem of compounding errors as a severe challenge in terms of trusting AI agents, saying this potential to compound one misstep without humans even being aware of it could “create havoc.”

This is especially true for the sort of multi-agent systems many businesses are contemplating, which Guan said can cause blind spots and get you into trouble very quickly.

“It may not work for you, and may actually introduce a lot of risk,” she said. “Think about it—a lot of the business workflows and transactions or interactions are high stakes. You don’t want agents to just issue a refund for every customer, right?” she said, adding that while her clients have a strong appetite to see impact from AI agents, they’re also wary of surprise high cloud bills and security risks.

Security concerns are certainly top of mind in the AI agent landscape. By 2028, Gartner predicts, 25% of enterprise breaches will be traced back to AI agents, including abuse from both internal and external malicious actors. The dominating factor contributing to security risks is the combination of autonomy and intended interoperability of agent systems, which would have them connect to, exchange data with, and autonomously act across a wide swath of platforms and systems. Put differently, the exact nature of how these systems function and what they’re intended to do is what makes them so risky.

Interoperability dreams struggle to break free from walled gardens

Like all CAIOs, Ali Alkhafaji, chief AI and technology officer at Omnicom Precision Marketing Group, is concerned about data leakage and other security risks. He’s also concerned that many of the companies commercializing agent systems are using security as a convenient excuse to further lock their customers inside their ecosystems, going against the collaborative and decentralized vision many see as intrinsic to an agentic future: “Not because it can’t be solved, but because it’s not in the commercial interest of the vendor to solve it.”

“Every vendor is building their own ‘agent framework,’ but no one is solving for enterprise-level interoperability. Without open frameworks and semantic standards, we’re just building smarter silos,” he said, adding that agent collaboration protocols remain immature and that it’s frustrating to see major vendors and hyperscalers continue to reinforce walled gardens. 

Deloitte U.S. head of AI Jim Rowan is seeing this play out among his clients, noting that they’re mostly sticking with their current providers and using their agent capabilities as they’re released. It’s another iteration of the platform advantage that’s driving growth for providers like OpenAI, Google, and Microsoft as they onboard their current customers into their new AI pipelines and products. 

“There is a definite tension in the marketplace between who wants to own the agent system of record. Like, who’s gonna own the registry, who’s going to orchestrate all the orchestration that’s happening around agents,” said Rowan. “We see that with the hyperscalers and the SaaS providers and the third-party-tool startups that are in the space as well. I think the jury’s still out on who’s owning that.”

Correction, Sept. 3, 2025: An earlier version of this story misstated the name of Accenture’s AI agent solution.