Benchmarkless LLM Safety: The Audit Contract That...

A new arXiv paper from May 2026 has formalized the nightmare scenario for every AI safety team: comparing LLM safety without a pre-existing labeled benchmark. The authors propose a 'benchmarkless comparative safety scoring' framework that imposes a strict contract on what counts as valid evidence, exposing most current safety claims as methodologically unsound.

What happened: Researchers on arXiv (May 7, 2026) published a formal framework for comparative LLM safety scoring when no labeled benchmark exists, specifying the exact conditions under which scenario-based audits produce valid deployment evidence.
Why it matters: Most organizations currently compare LLM safety using ad-hoc methods or vendor-provided scores that lack methodological transparency. This paper provides the mathematical contract that makes such comparisons statistically valid.
Key tension resolved: The framework replaces the impossible demand for ground-truth labels with a fixed, auditable configuration of scenario pack, rubric, auditor, judge, sampling configuration, and rerun budget — but only if all six components are explicitly disclosed and held constant.

What Exactly Does the 'Audit Contract' Require for Valid Safety Scores?

According to the arXiv paper published on May 7, 2026, the core innovation is the specification of a "contract" under which a scenario-based audit can be interpreted as deployment evidence. The paper states explicitly: "Scores are valid only under a fixed scenario pack, rubric, auditor, judge, sampling configuration, and rerun budget." This is not a suggestion — it is the formal condition for statistical validity. Any comparison that changes even one of these six variables produces scores that cannot be meaningfully compared. The paper replaces the need for ground-truth labels with this fixed configuration, but in doing so, it creates a new burden: every safety score must now come with a full methodological disclosure. Anthropic's public safety taxonomy, for example, provides scenario-based evaluations but has not historically disclosed all six components of this contract, making cross-provider comparisons methodologically suspect under this framework.

Why Can't We Just Use Existing Benchmarks for Cross-Model Safety Comparisons?

Benchmarkless LLM Safety: The Audit Contract That Changes Everything

The paper's title itself answers this question: "When No Benchmark Exists." The authors identify that many deployment contexts — new languages, emerging sectors like healthcare or legal AI, or novel regulatory regimes such as the EU AI Act's specific requirements — have no pre-existing labeled benchmark. Even where benchmarks exist, they often cover different safety dimensions or use different evaluation rubrics. According to the arXiv authors, the fundamental problem is that "because no labels are available, we replace ground-truth" with the fixed audit contract. This means that even if two models score well on different benchmarks, those scores are incommensurable under this framework. The practical implication is stark: an organization evaluating three candidate LLMs for a French-language customer service deployment cannot rely on any existing benchmark — they must construct their own scenario pack, rubric, and evaluation protocol from scratch, and then never change those parameters during the comparison.

Component of Audit Contract	Traditional Benchmark Approach	Benchmarkless Framework Requirement
Scenario Pack	Fixed public dataset	Custom-built, must be fixed across comparisons
Rubric	Implicit in benchmark design	Must be explicitly documented
Auditor	Often automated	Specific human or automated entity named
Judge	Implicit accuracy metric	Specific evaluation procedure fixed
Sampling Configuration	Fixed test split	Must be specified and held constant
Rerun Budget	Not typically tracked	Must be explicitly stated and capped
Verdict	Comparable only within same benchmark	Comparable only within same contract

What Happens When Organizations Ignore This Contract?

The paper does not mince words: scores produced outside this contract "cannot be interpreted as deployment evidence." This is a devastating conclusion for the current practice of LLM safety evaluation. Most organizations today compare models using different prompt templates, different evaluation rubrics (sometimes designed by the model vendor themselves), different judges (human vs. automated), and different sampling strategies. The arXiv authors explicitly formalize that such comparisons produce statistically invalid results. The implications for regulatory compliance are severe. Under the EU AI Act, for example, providers of high-risk AI systems must demonstrate comparative safety. According to the paper's framework, any such demonstration that does not disclose all six components of the audit contract — and keep them fixed across all compared models — would be methodologically unsound. This creates a clear pathway for regulatory challenge: a provider could be asked to produce their full audit contract, and if any component is missing or changed between evaluations, the safety claim becomes unverifiable.

Who Benefits Most From This Formalization?

The clear winners under this framework are specialist third-party auditing firms that can offer reproducible, contractually bounded evaluations. Companies like Scale AI's safety division or government-accredited testing labs gain a massive advantage because they can standardize all six components of the audit contract across evaluations. The losers are model vendors who market absolute safety scores without disclosing their full methodology. According to Anthropic's published safety research, their evaluations use scenario-based testing, but the arXiv paper would demand that Anthropic disclose the exact scenario pack, rubric, auditor identity, judge procedure, sampling configuration, and rerun budget for every safety score they publish. Without this disclosure, any claim that Claude is "safer than GPT-4" becomes methodologically unsupported. The framework also benefits procurement teams in regulated industries — healthcare, finance, legal — who can now demand a complete audit contract from vendors and reject any safety comparison that does not meet the formal validity conditions.

My analysis: This paper is the most important methodological contribution to LLM safety evaluation since the advent of red-teaming frameworks, precisely because it names the invisible assumptions that have made safety comparisons meaningless. In the short term, expect widespread resistance from vendors who have built marketing around absolute safety scores — they will need to either disclose their full audit contracts or abandon comparative safety claims. The long-term consequence is a market bifurcation: one tier of providers who invest in reproducible, contractually bounded evaluations and can substantiate their safety claims, and another tier who continue making unsupported comparisons and face regulatory or procurement rejection. The concrete prediction: by Q2 2027, the EU AI Office will issue guidance requiring all high-risk AI system providers to disclose the six-component audit contract for any comparative safety claim made in regulatory submissions, directly citing this arXiv paper as the methodological basis.

Prediction 1: By December 2026, at least three major LLM providers (likely Anthropic, Google DeepMind, and Mistral) will publish their full audit contracts for at least one deployment context, responding to procurement pressure from regulated industries.
Prediction 2: By Q2 2027, the EU AI Office will issue formal guidance requiring disclosure of all six audit contract components for any comparative safety claim in regulatory filings, citing this arXiv paper as the methodological standard.
Prediction 3: By Q4 2027, at least two third-party auditing firms will launch "contract-verified safety scoring" services that guarantee all six components are fixed and disclosed, capturing a premium market segment in healthcare and financial services AI procurement.

May 2026
arXiv publication of benchmarkless safety scoring framework
Authors formalize conditions for valid comparative LLM safety scoring without ground-truth labels, specifying the six-component audit contract.
Q4 2026 (predicted)
Major LLM providers disclose audit contracts
Expected response from Anthropic, Google DeepMind, and Mistral to procurement pressure from regulated industries.
Q2 2027 (predicted)
EU AI Office guidance on audit contract disclosure
Expected regulatory guidance requiring disclosure of all six audit contract components for comparative safety claims.

Insight 1: The paper's most disruptive implication is that most existing safety comparisons between LLMs are statistically invalid — not because the models are unsafe, but because the evaluation methodology was never designed to meet the formal contract conditions.
Insight 2: The framework creates a natural monopoly for auditing firms that can standardize all six contract components across evaluations, since any change in any component breaks comparability.
Insight 3: Regulatory bodies can now demand a specific, falsifiable methodological standard for safety claims, moving from vague "best practices" to auditable contractual conditions.
Insight 4: Organizations evaluating models for deployment now have a clear checklist: demand the full six-component audit contract from every vendor, and reject any safety comparison that cannot provide it.
Insight 5: The paper inadvertently reveals that the most important safety evaluation decision is the selection of the scenario pack — since all other components can be standardized, but the scenario pack defines what "safety" means in the specific deployment context.