ObviousBenchGPQA.ai

How the clean
leaderboard is built

A deterministic evidence pipeline turns public evaluation artifacts and carefully labelled independent runs into one auditable ranking.

Results 0.1.0Data through Aug 17, 2026

Two linked methods

The paper decides what belongs. This site decides what can be ranked.

PaperAudit the answer key

Determines which questions remain in the benchmark membership.

Read the current preprint (opens in a new tab)
LeaderboardBuild the ranking

Admits evidence, applies Clean membership, and computes paired scores and uncertainty.

Continue to the pipeline ↓

1 Pipeline

  1. 1Resolve source roster
  2. 2Validate artifacts
  3. 3Recover only when rules permit
  4. 4Add complete native evidence
  5. 5Apply Clean membership
  6. 6Compute exact scores
  7. 7Quantify uncertainty
  8. 8Export safe release bundle

Public export boundary: raw evidence stays outside the website bundle.

2 Ranking contract

Included

  • Every fully accepted configuration with all 189 retained questions
  • All accepted mean@N protocols in one combined ranking
  • Exact rational scores; exact ties share standard competition rank and skip the occupied places
  • Repeat count and evidence class remain visible

Never ranked

  • Missing retained-question coverage
  • Incomplete attempts without a valid uniform subset
  • Unresolved score-source conflict
  • Evidence that fails privacy or provenance validation
Clean accuracy = correct attempts across retained questions / (189 × attempts per question)

Combined rank is not normalized for inference compute.

3 Paired cleaning comparison

Originalsame evaluated attempt universe
−
9 removed questionssame source and route
=
Cleanpaired delta

Never subtract a reconstructed Clean score from an unrelated published aggregate.

4 Uncertainty

  • Resample Record IDs, not attempts.
  • One synchronized retained-ID draw for every configuration.
  • One shared retained plus excluded draw for paired deltas.
  • Fixed seed 20260817.
  • 20,000 replicates.
  • Clean, paired-delta, and rank 95% intervals.
Configuration Asame Record-ID drawscore + interval
Configuration Bsame Record-ID drawscore + interval
Configuration Csame Record-ID drawscore + interval
Configuration Dsame Record-ID drawscore + interval

Independent per-configuration resampling is rejected because it destroys paired covariance.

5 Evidence admission

  1. Complete 198-question evidence?No → unranked gap
  2. Exact source/configuration/protocol match?No → compatibility or recovery review
  3. Complete uniform repeat subset?Yes → derived, separately labelled
  4. Exact failed-slot recovery?Yes → targeted composite
  5. Complete native all-198 run?Yes → complete native primary or native transport composite
  6. Otherwisewithheld

6 Reproducibility and privacy

Reproduce

  • Locked Python and Node environments
  • JSON Schemas for public artifacts
  • Immutable release manifest
  • Deterministic timestamps and ordering
  • Checksums for every exported file

Protected

Question text, options, keys, completions, reasoning, raw evaluation artifacts, private evidence locations, and per-configuration-by-question matrices.

Continue exploring

Leaderboard Data & downloads