As the wave of generative AI has not yet receded, a deeper transformation driven by autonomous agents (Agentic AI) is quietly approaching. AI is evolving from a tool that passively responds to commands into a “digital employee” capable of autonomous perception, decision-making, and action. They are rapidly embedded into core business processes and critical systems, but their unique security risks—dynamism, autonomy, and unpredictability—far exceed the protective frameworks of traditional IT security.
In this issue of 【CXO Insights】, Dr. Nicole Nichols, a distinguished software engineer at Palo Alto Networks will provide a profound analysis of the new paradigm of AI Agent security based on the latest report “Building a Secure AI Agent Ecosystem” from cross-institutional collaboration.
▼▼▼
In 2020, when I began researching autonomous network agents, the timeline for real-world deployment was still measured in decades. At that time, these systems were seen as long-term investments, interesting but still marginal improvements for immediate applications.
However, change has quietly arrived.
Generative AI (GenAI) is not an isolated case; it has triggered a series of ongoing waves of progress that continue to compress development cycles at an accelerating pace. This not only means that the goals are changing; the GenAI-driven wave is ruthlessly toppling old benchmarks at an unprecedented speed, redefining the boundaries of possibility. Functions that were once limited to long-term research are now rapidly integrating into real-world environments.
Surprisingly yet predictably, agent systems are being embedded in countless fields, company processes, decision pipelines, and even critical infrastructure, often occurring before we have established governance or protection frameworks. Considering that we are no longer preparing for the arrival of AI Agents but are instead responding to their rapid evolution, 2020 feels like a distant past.
Targeting Moving Goals
The workshop report I co-authored, “Achieving a Secure AI Agent Ecosystem,” is a product of cross-institutional collaboration aimed at elucidating this accelerating trend. The report was completed by RAND, Schmidt Sciences, and leading figures in Agentic AI from industry, academia, and government. It is not a panacea but proposes a new paradigm for thinking about and addressing the security challenges of Agentic AI.
The core of the report articulates the three foundational pillars of AI Agent security, highlighting potential failure points in our current assumptions and infrastructure as systems evolve. The report not only acknowledges the current state but also calls for a profound mindset shift: we must recognize that the era of agent systems has arrived. Therefore, ensuring their security is not a matter for tomorrow but an urgent challenge of today. The relentless pace of innovation, the ongoing expansion of scale, the uneven risks faced by early adopters, and the significant asymmetry between attack capabilities and defense objectives all exacerbate this challenge.
A major difficulty in ensuring AI Agent security is that their forms and behaviors are fundamentally different from traditional software. They evolve dynamically, continuously, and are increasingly capable of making decisions with minimal supervision. Some are built specifically for automating tasks (such as scheduling or email classification); others are gradually moving towards complete autonomy in high-risk environments. In any case, the frameworks we use to secure traditional applications are inadequate. We face not only variants of known vulnerabilities but also entirely new, fundamental issues—the attack surface has fundamentally changed.
The Three Pillars of AI Agent Security
This shift in thinking focuses the security landscape on three core issues:
Protecting AI Agents from third-party intrusions: How to prevent external attackers from taking over or manipulating the AI Agent itself.
Protecting users and businesses from the impacts of the agents themselves: How to ensure that AI Agents (whether operating normally or malfunctioning) do not harm their users or the businesses they serve.
Protecting critical systems from malicious agents: How to safeguard important infrastructure and systems from attacks by malicious AI Agents that are intentionally designed and deployed.
These categories are not fixed; they represent different stages on the spectrum of capability and threat maturity. Currently, most companies deploying agents focus on the first two issues. However, the third category—malicious autonomous adversaries—is increasingly approaching. Nation-states are among the first entities to invest in autonomous network agents, and they are unlikely to remain solitary for long.
Therefore, navigating this powerful and widespread era of autonomous threats cannot be addressed merely by incrementally improving existing defenses. It requires our expert community to fundamentally change the way we collaborate and innovate in the security domain.
Historically, AI researchers and cybersecurity experts have often worked on parallel tracks, holding different assumptions about risks and architectures. However, the complex frontier of Agentic AI security demands collaboration between both sides, as neither can tackle these immense challenges alone; deep and ongoing collaboration is essential. While universal protocols and comprehensive best practices applicable across the field are still being refined, it is frankly outdated to say that “effective one-stop security agent products are exceedingly rare.” Today, mature and deployable solutions are providing critical protection for key agent systems, marking tangible progress. This further underscores the urgent need for adaptive, multi-layered security strategies that encompass model provenance, robust containment measures, and resilient human-machine loop controls, all of which must evolve rapidly in sync with the agents themselves.
Feasible Intervention Directions
While powerful and continuously evolving product solutions are becoming increasingly critical in mitigating the immediate operational risks posed by Agentic AI, achieving comprehensive, long-term security also requires sustained investment from the entire industry in foundational capabilities and consensus building. The following key directions, which complement product innovation, are entirely within our collective capabilities and deserve focused attention:
Establishing an “Agent Bill of Materials” (Agent BOM): Imagine a mechanism similar to a “Software Bill of Materials” (SBOM) aimed at providing transparent visibility into agent components (such as models, training data, tools, and memory). However, its functional feasibility currently faces obstacles, such as the lack of a universal model identifier system, which is a key foundation for achieving such transparency.
Standardizing pre-deployment testing platforms: Supporting scalable, scenario-based assessments before agents are put into production environments.
Strengthening the security of communication protocols: Communication protocols such as MCP (Model Context Protocol) and A2A (Agent-to-Agent) are emerging, but few have built-in security designs from the outset. However, even with integrated security measures, the prevalent “unknowns” in these new agent systems mean that the protocols themselves require rigorous and ongoing evaluation to maintain their integrity and security.
A key challenge discussed in the report is: the memory of agents is essential for their learning, improvement, and (crucially) avoiding repeating mistakes, but it is also a significant vulnerability susceptible to malicious tampering. Proposed strategies include using “clone-on-launch” or task-specific agent instances. In this model, agents designed for specific responsibilities or limited-duration interactions treat their active working memory as ephemeral. Once the task or session is completed, the instance can be discarded, and new operations are taken over by a new instance initialized from a secure and trusted baseline.
This approach aims to significantly reduce the risk of persistent memory corruption or lasting impacts from single-session tampering. However, the key is that such systems must be carefully designed to ensure that the agent’s core foundational knowledge and long-term acquired experience are not only securely maintained and protected from tampering but also effectively and securely accessible to guide these ephemeral operational instances. While this operational state management is not a comprehensive solution to all memory-related threats, it represents the creative, system-level thinking needed to advance agent security and robust containment.
Call for Collective Commitment
Ultimately, ensuring the security of Agentic AI is not derived from a single breakthrough but requires sustained collective effort from multiple parties. This demands interdisciplinary collaboration among researchers, policymakers, practitioners, and industry leaders. The threats are both technical and foundational. We are trying to secure systems that are not yet fully understood. But if the past few years have taught us anything, it is that waiting until we fully understand before taking action is too late.
The evolution of Agentic AI means that our industry is developing critical protective measures while broadly adopting AI. This parallel development is not a crisis in itself but a clear call for collective responsibility. Our success depends on whether the industry can collectively commit to building a trustworthy AI ecosystem through transparency, rigorous standards, and a unified vision.
Add 【Palo Alto Networks】 WeChat for more exclusive services
Click to read more
Click “Read the original”
Enter the 【AI Security Zone】
See more AI security content
Further Reading
▶ From Browser to Agent: Comprehensive Defense Strategies Against AI Security Threats
▶ 78% of Companies Plan to Enable AI Agents in Production, In-Depth Analysis of Their Security Strategies
▶ From Passive Defense to Proactive Control: How AI Completely Simplifies and Reshapes Cybersecurity
Click “Read the original” to enter the AI Security Zone