The shark from Jaws attacked with out warning, exhibiting how an apex predator exploits chaos to create deadly, devastating hurt on its prey. Now, Forrester says, gen AI has develop into that predator within the fingers of attackers: The one which by no means tires or sleeps and executes at scale.
“In Jaws, the shark acts because the chaos agent,” Forrester principal analyst Allie Mellen informed attendees on the IT consultancy agency’s 2025 Safety and Threat Summit. “We have now a chaos agent of our personal in the present day… And that chaos agent is generative AI.”
Mellen offered a quantitative, substantial base of analysis knowledge to again up her declare, illustrating the elemental weaknesses and unreliability of AI methods. As she emphatically put it: “AI is unsuitable. It’s unsuitable not just a bit bit; it is unsuitable a variety of the time.”
Fashions fail 60% of the time
Of the various research Mellen cited in her keynote, one of the damning is predicated on analysis performed by the Tow Middle for Digital Journalism at Columbia College, which analyzed eight completely different AI fashions, together with ChatGPT and Gemini.
The researchers discovered that total, fashions have been unsuitable 60% of the time; their mixed efficiency led to extra failed queries than correct ones.
AI brokers regularly fail at real-world duties
Jeff Pollard, VP, principal analyst at Forrester, additionally drove this residence. “Your crimson teamer is now your AI crimson group orchestrator,” he stated. “Conventional pentesting hunts for infrastructure flaws. AI crimson teaming operates otherwise. It simulates adversarial assaults on the AI mannequin itself.”
Pollard additionally cited a number of research, one of the noteworthy being from Carnegie Mellon researchers who found that AI brokers fail 70 to 90% of the time on real-world company duties.
Practically half (45%) of AI-generated code comprises identified OWASP Prime 10 vulnerabilities. Exacerbating the dangers of gen AI as a chaos agent is how pervasive shadow AI is, with 88% of safety leaders admitting to incorporating unauthorized AI into their every day workflows.
Forrester’s prediction that there might be a $27 billion identification administration market surge by 2029 indicators how pervasive gen AI is turning into in each identification a company has to guard, from human-based to machine-created. Gen AI’s inherent dangers are the chaos agent nobody sees coming in cybersecurity.
Mellen illustrated the stakes with a concrete instance: “AI would not essentially know that sharks do not reside on land,” she defined, referencing an AI-generated map that positioned shark assaults throughout Wyoming, a landlocked state 1,000 miles from the ocean. “It is all high quality and dandy for AI to be unsuitable once we’re simply making a map about shark assaults, nevertheless it’s a completely completely different factor for it to be unsuitable throughout a safety incident. AI is serving us up a brand new kind of false constructive, this time for investigation and response.”
AI confidently positioned shark assaults in Wyoming, 1,000 miles from the ocean. LLMs do not fail quietly. They hallucinate with absolute certainty, then ship to manufacturing. Supply: 2025 Safety & Threat Summit.
When 70-90% incompleteness meets manufacturing velocity
Pollard quoted Carnegie Mellon’s AgentCompany benchmark, which examined main AI fashions towards 175 actual company duties. Claude 3.5 Sonnet, GPT-4 and specialised enterprise brokers all confirmed systemic patterns of failure. Prime performers accomplished solely 24% of duties autonomously.
When researchers added extra complexity, failure charges soared to between 70 and 90%. Pollard additionally led Salesforce’s AI Analysis group, which revealed equally damning outcomes. CRM-oriented brokers failed 62% of baseline enterprise duties. When researchers utilized confidentiality and security guardrails, accuracy dropped by half, pushing failure charges above 90%. Salesforce detailed these findings at Dreamforce 2024’s agentic AI session.
Veracode’s 2025 GenAI Code Safety Report examined 80 coding duties throughout 4 languages (Java, Python, C, JavaScript) and greater than 100 LLMs. Outcomes are stark: 45% of AI-generated code launched OWASP Prime 10 vulnerabilities. Language-specific efficiency varies considerably. Java confirmed the worst outcomes at 28.5% safety move fee, whereas Python (55.3%), C (57.3%) and JavaScript (61.7%) carried out higher. Cross-site scripting and log injection proved catastrophic; solely 12 to 13% safety move charges. SQL injection and cryptographic algorithms scored larger,, at 80 to 86%.
A key perception from the research is how safety efficiency remained flat regardless of dramatic syntactic enhancements. Newer, bigger fashions generate extra compilable code but introduce vulnerabilities. These findings replicate the affect of coaching knowledge on coding high quality and reliability.
Language-by-language safety move charges. Supply: Veracode’s 2025 GenAI Code Safety Report
Each new identification creates a brand new assault floor
Identities are the primary and favourite goal of attackers, and AI’s multiplying impact is escalating the chance exponentially. Merritt Maxim, VP and analysis director at Forrester, delivered a blunt actuality test: “Id safety is present process probably the most vital shift since SSO went mainstream. It is not about innovation anymore; it is about containment failure.”
Maxim additional defined: “Entitlements aren’t static anymore. We have moved towards zero standing privilege; entitlements are actually dynamic, granted simply in time.”
The August 2025 OAuth token breach affecting greater than 700 Salesforce clients offered plain proof. Geoff Cairns, Forrrester principal analyst, underscored the gravity: “OAuth tokens, API keys, certificates … these usually are not configuration artifacts. They’re high-value identities. And when you do not govern them, you lose the enterprise.”
With gen AI increasing identification sprawl, conventional governance collapses at machine speeds. Forrester sees demand for the identification entry administration (IAM) market rising to $27.5 billion by 2029. The highest ten identification safety insights replicate machine identities, creating better complexity and potential chaos that each safety skilled must plan for now.
Supply: 2025 Safety & Threat Summit.
Weaponized gen AI is the apex predator stalking enterprise networks
Forrester’s 2025 Safety and Threat Summit did not merely spotlight threats; it delivered a survival blueprint.
Weaponized gen AI has develop into the apex predator inside enterprise networks, transferring silently, relentlessly and at unprecedented scale.
VentureBeat believes the next are important steps that safety and danger administration professionals have to take as gen AI turns into a extra pervasive risk:
-
Deal with AI brokers as mission-critical identities and understand that having a transparent line on governance of this new class of identification is crucial throughout all areas of the corporate. Forrester VP and principal analyst Andras Cser explicitly highlighted that “AI brokers sit someplace between machines and human identities; excessive quantity, excessive autonomy, excessive affect. Legacy IAM instruments can not govern them successfully.” Specialised governance platforms are important, as they’ll ship real-time visibility, adaptive monitoring and dynamic authorization particularly for AI agent identities.
-
Place a excessive precedence on growing and rising AI crimson group functionality. Pollard warned: “Infrastructure flaws matter, however AI mannequin flaws are what’s going to break you. Conventional pentesting has develop into out of date. AI crimson groups should proactively detect and mitigate AI-specific vulnerabilities, together with immediate injection, bias exploitation, mannequin inversion and cascading failures from autonomous brokers.
-
Function beneath the specific assumption of AI failure. Forrester aimed to ship an emphatic message of how unreliable gen AI is. They succeeded. Mellen’s keynote drove that time residence. AI is “serving us new false positives, particularly throughout investigations and responses,” Mellen famous. With confirmed failure charges round 60%, organizations should function beneath the specific assumption that AI methods will often fail.
-
Design and implement safety controls to allow them to shortly scale to machine velocity. Maxim acknowledged: “Entitlements aren’t static anymore. We have moved towards zero standing privilege; entitlements are actually dynamic, granted simply in time.” Conventional, human-paced controls are insufficient towards gen AI’s velocity.
-
Ruthlessly get rid of blind belief in automation and any legacy infrastructure that’s based mostly on assumed belief. Carnegie Mellon’s AgentCompany benchmark explicitly revealed catastrophic AI agent failure charges (70–90%) amongst top-tier fashions. In one of many strongest statements of the occasion, Pollard expressly warned: “Guardrails do not make brokers secure; they make them fail silently.” Organizations should constantly confirm, audit and problem automated methods with out compromise. Blind belief in automation is a catastrophe ready to occur, and assuming belief relationships with legacy methods is equally dangerous. Each are potential breaches simply ready on an attacker to search out the weak point and exploit it.