AI brokers are actually embedded in actual enterprise workflows, and so they’re nonetheless failing roughly one in three makes an attempt on structured benchmarks. That hole between functionality and reliability is the defining operational problem for IT leaders in 2026, in keeping with Stanford HAI’s ninth annual AI Index report.
This uneven, unpredictable efficiency is what the AI Index calls the “jagged frontier,” a time period coined by AI researcher Ethan Mollick to explain the boundary the place AI excels after which instantly fails.
“AI fashions can win a gold medal on the Worldwide Mathematical Olympiad,” Stanford HAI researchers level out, “however nonetheless can’t reliably inform time.”
How fashions superior in 2025
Enterprise AI adoption has reached 88%. Notable accomplishments in 2025 and early 2026:
-
Frontier fashions improved 30% in only one yr on Humanity’s Final Examination (HLE), which incorporates 2,500 questions throughout math, pure sciences, historic languages, and different specialised subfields. HLE was constructed to be troublesome for AI and favorable to human specialists.
-
Main fashions scored above 87% on MMLU-Professional, which assessments multi-step reasoning based mostly on 12,000 human-reviewed questions throughout greater than a dozen disciplines. This illustrates “how aggressive the frontier has turn into on broad data duties,” the Stanford HAI researchers observe.
-
Prime fashions together with Claude Opus 4.5, GPT-5.2, and Qwen3.5 scored between 62.9% and 70.2% on τ-bench. The benchmark assessments brokers on real-world duties in real looking domains that contain chatting with a person and calling exterior instruments or APIs.
-
Mannequin accuracy on GAIA, which benchmarks common AI assistants, rose from about 20% to 74.5%.
-
Agent efficiency on SWE-bench Verified rose from 60% to close 100% in only one yr. The benchmark evaluates fashions on their potential to resolve real-world software program points.
-
Success charges on WebArena elevated from 15% in 2023 to 74.3% in early 2026. This benchmark presents a sensible internet atmosphere for evaluating autonomous AI brokers, tasking them with data retrieval, website navigation, and content material configuration.
-
Agent efficiency progressed from 17% in 2024 to roughly 65% in early 2026 on MLE-bench, which evaluates machine studying (ML) engineering capabilities.
AI brokers are displaying functionality positive aspects in cybersecurity. As an example, frontier fashions solved 93% of issues on Cybench, a benchmark that features 40 professional-level duties throughout six capture-the-flag classes, together with cryptography, internet safety, reverse engineering, forensics, and exploitation.
That is in comparison with 15% in 2024 and represents the “steepest enchancment fee,” indicating that cybersecurity duties are a “good match for present agent capabilities.”
Video era has additionally advanced considerably over the past yr; fashions can now seize how objects behave. As an example, Google DeepMind’s Veo 3 was examined throughout greater than 18,000 generated movies, and demonstrated the flexibility to simulate buoyancy and solved mazes with out having been skilled on these duties.
“Video era fashions are not simply producing realistic-looking content material,” the researchers write. “Some are starting to find out how the bodily world truly works.”
Total, AI is getting used throughout plenty of areas in enterprise — data administration, software program engineering and IT, advertising and marketing and gross sales — and increasing into specialised domains like tax, mortgage processing, company finance, and authorized reasoning, the place accuracy ranges from 60 to 90%.
“AI functionality shouldn’t be plateauing,” Stanford HAI says. “It’s accelerating and reaching extra folks than ever.”
AI functionality surges, however reliability lags
Multimodal fashions now meet or exceed human baselines on PhD-level science questions, multimodal reasoning, and competitors arithmetic. For instance, Gemini Deep Suppose earned a gold medal on the 2025 Worldwide Mathematical Olympiad (IMO), fixing 5 of six issues end-to-end in pure language throughout the 4.5-hour time restrict — a notable enchancment from a silver-level rating in 2024.
But these similar AI methods nonetheless fail in roughly one in three makes an attempt, and have bother with fundamental notion duties, in keeping with Stanford HAI. On ClockBench — a check masking 180 clock designs and 720 questions — Gemini Deep Suppose achieved solely 50.1% accuracy, in comparison with roughly 90% for people. GPT-4.5 Excessive reached an nearly an identical rating of fifty.6%.
“Many multimodal fashions nonetheless wrestle with one thing most people discover routine: Telling the time,” the Stanford HAI report factors out. The seemingly easy job combines visible notion with easy arithmetic, identification of clock palms and their positions, and conversion of these right into a time worth. In the end, errors at any of those steps can cascade, resulting in incorrect outcomes, in keeping with researchers.
In evaluation, fashions have been proven a spread of clock types: commonplace analog, clocks with out a second hand, these with arrows as palms, others with black dials or Roman numerals. However even after fine-tuning on 5,000 artificial photographs, fashions improved solely on acquainted codecs and didn’t generalize to real-world variations (like distorted dials or thinner palms).
Researchers extrapolated that, when fashions confused hour and minute palms, their potential to interpret course deteriorated, suggesting that the problem lies not simply in knowledge, however in integrating a number of visible cues.
“Whilst fashions shut the hole with human specialists on knowledge-intensive duties, this sort of visible reasoning stays a persistent problem,” Stanford HAI notes.
Hallucination and multi-step reasoning stay main gaps
Whilst fashions proceed to speed up of their reasoning, hallucinations stay a significant concern.
In a single benchmark, for example, hallucination charges throughout 26 main fashions ranged from 22% to 94%. Accuracy for some fashions dropped sharply when put beneath scrutiny —for instance, GPT-4o’s accuracy slid from 98.2% to 64.4%, and DeepSeek R1 plummeted from greater than 90% to 14.4%.
Alternatively, Grok 4.20 Beta, Claude 4.5 Haiku, and MiMo-V2-Professional confirmed the bottom charges.
Additional, fashions proceed to wrestle with multi-step workflows, at the same time as they’re tasked with extra of them. For instance, on the τ-bench benchmark — which evaluates software use and multi-turn reasoning — no mannequin exceeded 71%, suggesting that “managing multiturn conversations whereas appropriately utilizing instruments and following coverage constraints stays troublesome even for frontier fashions,” in keeping with the Stanford HAI report.
Fashions have gotten opaque
Main fashions are actually “almost indistinguishable” from one another on the subject of efficiency, the Stanford HAI report notes. Open-weight fashions are extra aggressive than ever, however they’re converging.
As functionality is not a “clear differentiator,” aggressive strain is shifting towards price, reliability, and real-world usefulness.
Frontier labs are disclosing much less details about their fashions, analysis strategies are rapidly dropping relevance, and unbiased testing can’t all the time corroborate developer-reported metrics.
As Stanford HAI factors out: “Essentially the most succesful methods are actually the least clear.”
Coaching code, parameter counts, dataset sizes, and durations are sometimes being withheld — by corporations together with OpenAI, Anthropic and Google. And transparency is declining extra broadly: In 2025, 80 out of 95 fashions have been launched with out corresponding coaching code, whereas solely 4 made their code totally open supply.
Additional, after rising between 2023 and 2024, scores on the Basis Mannequin Transparency Index — which ranks main basis builders on 100 transparency indicators — have since dropped. The typical rating is now 40, representing a 17 level lower.
“Main gaps persist in disclosure round coaching knowledge, compute sources, and post-deployment influence,” in keeping with the report.
Benchmarking AI is getting tougher — and fewer dependable
The benchmarks used to measure AI progress are dealing with rising reliability points, with error charges reaching as excessive as 42% on widely-used evaluations. “AI is being examined extra ambitiously throughout reasoning, security, and real-world job execution,” the Stanford report notes, but “these measurements are more and more troublesome to depend on.”
Key challenges embody:
-
“Sparse and declining” reporting on bias from builders
-
Benchmark contamination, or when fashions are uncovered to check knowledge; this could result in “falsely inflated” scores
-
Discrepancies between developer-reported outcomes and unbiased testing
-
“Poorly constructed” evals missing documentation, particulars on statistical significance and reproducible scripts
-
“Rising opacity and non-standard prompting” that make model-to-model comparisons unreliable
“Even when benchmark scores are technically legitimate, robust benchmark efficiency doesn’t all the time translate to real-world utility,” in keeping with the report. Additional, “AI functionality is outpacing the benchmarks designed to measure it.”
That is resulting in “benchmark saturation,” the place fashions obtain scores so excessive that assessments can not differentiate between them. Extra complicated, interactive types of intelligence have gotten more and more troublesome to benchmark. Some are calling for evals that measure human-AI collaboration, reasonably than AI efficiency in isolation, however this system is early in growth.
“Evaluations supposed to be difficult for years are saturated in months, compressing the window through which benchmarks stay helpful for monitoring progress,” in keeping with Stanford HAI.
Are we at “peak knowledge”?
As builders transfer into extra data-intensive inference, there may be rising concern about knowledge bottlenecks and scaling sustainability. Main researchers are warning that the out there pool of high-quality human textual content and internet knowledge has been “exhausted” — a state known as “peak knowledge.”
Hybrid approaches combining actual and artificial knowledge can “considerably speed up coaching” — typically by an element of 5 to 10 — and smaller fashions skilled on purely artificial knowledge have proven promise for narrowly outlined duties like classification or code era, in keeping with Stanford HAI.
Synthetically generated knowledge could be efficient for enhancing mannequin efficiency in post-training settings, together with fine-tuning, alignment, instruction tuning, and reinforcement studying (RL), the report notes. Nonetheless, “these positive aspects haven’t generalized to massive, general-purpose language fashions.”
Slightly than scaling knowledge “indiscriminately,” researchers are turning to pruning, curating, and refining inputs, and are enhancing efficiency by cleansing labels, deduplicating samples, and developing general higher-quality datasets.
“Discussions on knowledge availability typically overlook an essential shift in current AI analysis,” in keeping with the report. “Efficiency positive aspects are more and more pushed by enhancing the standard of present datasets, not by buying extra.”
Accountable AI is falling behind
Whereas the infrastructure for accountable AI is rising, progress has been “uneven” and is unable to maintain tempo with speedy functionality positive aspects, in keeping with Stanford HAI.
Whereas nearly all main frontier AI mannequin builders report outcomes on functionality benchmarks, corresponding reporting on security and duty is inconsistent and “spotty.”
Documented AI incidents rose considerably yr over yr — 362 in 2025 in comparison with 233 in 2024. And, whereas a number of frontier fashions obtained “Very Good” or “Good” security scores beneath commonplace use (per the AILuminate benchmark, which assesses generative AI throughout 12 “hazard” classes), security efficiency dropped throughout all fashions when examined towards jailbreak makes an attempt utilizing adversarial prompts.
“AI fashions carry out nicely on security assessments beneath regular situations, however their defenses weaken beneath deliberate assault,” Stanford HAI notes.
Including to this problem, builders have reported that enhancing one dimension, similar to security, can degrade one other, like accuracy. “The infrastructure for accountable AI is rising, however progress has been uneven, and it’s not protecting tempo with the pace of AI deployment,” in keeping with Stanford researchers.
The Stanford knowledge makes one factor clear: the hole that issues in 2026 is not between AI and human efficiency. It is between what AI can do in a demo and what it does reliably in manufacturing. Proper now — with much less transparency from the labs and benchmarks that saturate earlier than they’re helpful — that hole is tougher to measure than ever.