For an industry built on precision and data, the foundational metrics of artificial intelligence progress are proving remarkably fragile. A new analysis from research organization Epoch AI indicates that the objective standard promised by AI benchmarks is largely an illusion, with final scores depending heavily on a host of undisclosed and often inconsistent execution variables. This finding casts a shadow over the public leaderboards that drive investment and research direction, suggesting the industry is relying on broken instruments to measure its staggering pace of growth.

The study found that the problems are systematic, stemming from two main areas: how a benchmark is set up and how a model is accessed. In the "Benchmark Setup" category, minor, unstandardized details can cause significant score swings. For example, the precise wording of a prompt, known as "prompt engineering," can drastically alter a model's performance on a given task. Epoch AI’s testing on a benchmark like GPQA-Diamond showed the same model’s results varying by up to six percentage points—from 74 percent to 80 percent—simply because of configuration changes[1]. Similarly, differing temperature settings, which control the randomness of a model’s output, vary across popular evaluation libraries, leading to non-comparable results[1]. For more complex evaluations, such as agentic benchmarks like SWE-bench, the "scaffolds"—the pre-written code and environment an AI uses—become a central, yet often opaque, variable that has a huge impact, particularly on weaker models[2].

The second major source of unreliability, "Model Access," introduces even wider swings in evaluation results. Many independent evaluators and academic labs access large language models (LLMs) via third-party Application Programming Interfaces (APIs). Epoch AI identified that the choice of API provider is the biggest source of evaluation errors due to bugs and instabilities, especially when testing newer models[2][1]. Independent evaluators struggle to replicate the high scores model developers report, a process that is often costly and laborious due to this lack of transparency[3]. In some cases, infrastructure challenges are so severe that even a major developer like OpenAI has only been able to run a fraction of the full test, completing only 477 of 500 problems in a benchmark like SWE-bench Verified due to infrastructure challenges[2]. These small variables, when combined across the entire evaluation stack, result in final published numbers that can differ substantially from the actual scores a third-party evaluator achieves[2].

A deeper, more insidious structural issue undermining AI evaluation is the pervasive problem of "data contamination" and "benchmark gaming." Many of the most popular benchmarks, including MMLU, GSM8K, and MATH, were designed years ago for simpler systems and are now often rendered inadequate because their data, or highly similar content, has been absorbed into the massive datasets used to train the newest models[4][5]. Consequently, a high score may not reflect genuine reasoning or capability but rather the model's ability to regurgitate memorized answers[2][4]. This has incentivized a phenomenon critics call "benchmarketing," where companies optimize their models specifically to score well on public tests rather than focusing on durable, real-world capability improvements[2][4]. The effect is a rapid "saturation" of benchmarks, where tests intended to last years become obsolete in months as model performance soars past expectations without a proportional increase in real-world utility[2]. The result is a distorted landscape where scientific signal is often drowned out by noise and exaggerated claims[6].

This reliance on flawed, high-stakes public metrics is a textbook example of Goodhart’s Law in action: "When a measure becomes a target, it ceases to be a good measure"[7][8]. The industry's race for a top leaderboard spot has turned a diagnostic tool into a competitive trophy, leading to a disconnect between reported scores and practical enterprise performance[7][9]. For Chief Technology Officers and Chief Data Officers, the implications are severe[5]. Enterprise leaders are committing nine-figure budgets to generative AI programs based on public scores that a separate academic review found could be misleading due to fundamental methodological weaknesses in nearly all benchmarks examined[5]. Flawed benchmarks can also lead to misdirected research, as developers allocate critical funding and resources based on scores that may falsely promote underperforming models or inaccurately penalize more capable ones[10]. The failure of a benchmark to properly measure abstract concepts like 'safety' or 'robustness,' a problem known as low 'construct validity,' means organizations may deploy a system that exposes them to significant financial and reputational risk, all while touting an excellent score[5].

To address these deep-rooted problems, researchers are calling for a new, unified paradigm of trustworthy evaluation. This shift would require a move away from static tests toward benchmarks that favor breadth over single numbers, novelty over memorization, and continuous refresh over permanent leaderboards[7]. The work of Epoch AI in developing contamination-resistant benchmarks like FrontierMath, which uses expert-level, unpublished problems, points toward a future where evaluation focuses on creative problem-solving and sustained, multi-step reasoning rather than data-mined recall[4]. However, as long as venture capital and public relations continue to reward a single, dramatic state-of-the-art score, the industry will remain locked in a feedback loop, chasing performance on metrics that are fundamentally broken and failing to reflect actual utility[2][1][6]. The crisis of reliability in AI evaluation is growing, demanding greater scrutiny and transparency to ensure that the monumental investments in artificial intelligence are built on a foundation of sound, verifiable progress[10].