Artificial intelligence (AI) is rapidly reshaping the enterprise landscape, promising a leap in productivity and efficiency. Yet, as organizations rush to deploy these digital agents, they risk unleashing forces they do not fully understand or control. The productivity promise of AI is real, but so is the privacy peril, and the stakes have never been higher.
Discussions that I’ve had with leadership at major financial institutions, including Wells Fargo, reveal a growing anxiety among business leaders. Regulators are no longer content to let technology teams experiment in isolation; they are demanding answers about how AI is governed, secured and, most crucially, how its identity and access are managed. The old paradigms of cybersecurity, built around human users and their digital identities, are crumbling in the face of autonomous AI agents that operate with increasing independence.
The Identity Crisis of Non-Human Agents
The challenge is not theoretical. For decades, identity management meant authenticating people and granting them access to systems and data. Now, organizations must grapple with the question: What does identity mean for a non-human agent? Where should these identities be defined and stored?
The complexity multiplies when AI agents interact, potentially bypassing established access controls and exposing sensitive information in ways that were never anticipated. Consider the scenario where one AI agent requests data from another, which in turn consults a third agent. Each step in this chain carries the risk of privilege escalation, where information meant to be confidential is inadvertently disclosed.
The problem is not the security of the communication channels but the inheritance of rights across agents—a subtle but profound shift in how access is managed. Even the technology giants that build these systems admit they have yet to solve the problem of agentic identity at scale.
Taking a look at a very possible hypothetical, let’s say Dave—a project manager—eagerly asks his company’s newly launched AI assistant, Synthia, to prepare an insightful pre-read for a meeting with the notoriously tough vice president. Synthia dives into every email, document and even HR and finance records, gathering budget overruns and confidential performance notes. It even flags the VP’s dislike of beige, found in chat logs.
Dave, impressed by the thoroughness, sends the report to the VP and team without a second glance. The fallout is immediate: privacy breaches, embarrassed colleagues and a furious VP. Dave’s AI becomes the ultimate office gossip.
Avoiding AI’s Potential as a Privacy Liability
Mapping the landscape of AI tools and their data access is a daunting task. Many enterprises lack a comprehensive inventory of their AI systems, let alone a clear understanding of what data those systems can reach. Meanwhile, regulatory expectations are evolving at breakneck speed. Financial regulators look to the largest banks to set the standard, but the reality is that even these institutions are still searching for best practices.
The risks are not limited to compliance. A single AI-generated report that leaks confidential data can trigger an internal crisis and inflict lasting reputational harm. Mishandled access can violate privacy laws and industry regulations, exposing organizations to legal and financial consequences. The genie, once out of the bottle, is not easily contained.
So what is to be done? The answer lies in treating AI agents with the same rigor as human employees. Their identities must be defined, their roles scoped and their access strictly limited to what is necessary. The principle of least privilege, long a cornerstone of cybersecurity, must now be applied to machines as well as people. Permission inheritance must be tightly controlled, ensuring that AI agents mirror the access rights of their human counterparts and do not inadvertently escalate privileges through agent-to-agent communication.
Data classification becomes paramount. Machine-readable labels and policies must guide AI behavior, ensuring that sensitive information is protected and only accessible to authorized agents. Continuous auditing and monitoring are essential, not only to track what AI agents access and share, but also to detect anomalous behaviors and model drift that could signal a breach or misuse.
Building a security-aware culture is no longer optional. Employees must be trained to understand the risks and responsibilities associated with AI, fostering a climate of vigilance and accountability. Emerging standards, such as SPIFFE, offer promising frameworks for managing machine identities and supporting zero-trust architectures for agentic AI.
Closing Thoughts
Ultimately, leadership is required. Business executives must proactively address the risks of AI identity and access, collaborating with cybersecurity experts and ethicists to shape governance strategies. Continuous evaluation is essential, as both AI capabilities and regulatory requirements evolve. Before deploying AI solutions, organizations must ensure that robust data classification, security controls and incident response plans are in place.
The journey to resilient, secure AI is ongoing. Those who tame the digital genie today will be best positioned to seize tomorrow’s opportunities without falling victim to its unintended consequences.
This article originally appeared on Forbes Technology Council.
Andy Retrum and Ryan McCarthy, managing directors at Protiviti, contributed to this article.
