90% threshold
25 Original 198 configurations
45 Clean 189 configurations
+20 cross upward
What cleaning changes
Nine answer-key and eligibility defects were removed from GPQA Diamond.
The effect is measurable, configuration-specific, and large enough to change interpretation.
Results 0.1.0Data through Aug 17, 2026
235of 243 improved
Median+2.43 pp
Middle 50%+1.05 pp to +2.98 pp
8decreased
Clean score versus original score
Points above the diagonal improved after the nine exclusions.
Chart loads from the release bundle when visible.
View chart data table
| Configuration | Original 198 | Clean 189 | Change |
|---|---|---|---|
| gpt-5.5-pro-pre-release_xhigh | 93.94% | 97.35% | +3.42 pp |
| Gemini 3.6 Flash (high) | 93.94% | 97.35% | +3.42 pp |
| gpt-5.5-pre-release_xhigh | 94.00% | 97.22% | +3.22 pp |
| gemini-3.1-pro-preview | 94.51% | 97.02% | +2.52 pp |
| GPT-5.6 Sol (max) | 93.43% | 96.83% | +3.39 pp |
| gpt-5.4-2026-03-05_xhigh | 93.24% | 96.69% | +3.45 pp |
| gemini-3.1-pro-preview_high | 94.44% | 96.30% | +1.85 pp |
| gemini-3.5-flash_high | 92.80% | 96.10% | +3.29 pp |
| Grok 4.6 (high) | 92.42% | 95.77% | +3.34 pp |
| DeepSeek V4 Flash (max) | 92.93% | 95.77% | +2.84 pp |
| gemini-3-pro-preview | 92.61% | 95.50% | +2.89 pp |
| DeepSeek V4 Pro (max) | 92.42% | 95.24% | +2.81 pp |
| claude-opus-5 | 92.93% | 95.24% | +2.31 pp |
| gpt-5.2-2025-12-11_xhigh | 91.41% | 95.04% | +3.63 pp |
| GPT-5.6 Luna (max) | 92.42% | 94.71% | +2.28 pp |
| qwen3.7-max | 90.91% | 94.71% | +3.80 pp |
| kimi-k3_high | 91.92% | 94.71% | +2.79 pp |
| gpt-5.5_low | 90.66% | 93.85% | +3.19 pp |
| kimi-k2.6 | 90.91% | 93.83% | +2.92 pp |
| GPT-5.6 Terra (max) | 90.91% | 93.65% | +2.74 pp |
| MiniMax-M3 | 90.91% | 93.65% | +2.74 pp |
| gpt-5.4-2026-03-05_high | 89.90% | 93.65% | +3.75 pp |
| DeepSeek V4 Flash (high) | 90.40% | 93.65% | +3.25 pp |
| claude-opus-4-7_xhigh | 90.15% | 93.65% | +3.50 pp |
| gpt-5.6-sol_low | 89.90% | 93.65% | +3.75 pp |
| claude-opus-4-6_32K | 90.53% | 93.65% | +3.12 pp |
| glm-5.1 | 89.90% | 93.12% | +3.22 pp |
| deepseek-v4-pro_high | 90.91% | 93.12% | +2.21 pp |
| gemini-3.5-flash_low | 88.89% | 92.59% | +3.70 pp |
| gpt-5.4-2026-03-05_medium | 88.89% | 92.06% | +3.17 pp |
| gemini-3-flash-preview_high | 89.39% | 92.06% | +2.67 pp |
| claude-opus-4-6_64K | 88.53% | 91.69% | +3.16 pp |
| claude-opus-4-6_max | 88.38% | 91.53% | +3.15 pp |
| gpt-5.2-2025-12-11_high | 88.19% | 91.40% | +3.21 pp |
| claude-opus-5_low | 87.88% | 91.01% | +3.13 pp |
| glm-5 | 87.88% | 91.01% | +3.13 pp |
| qwen3.6-plus | 88.38% | 91.01% | +2.62 pp |
| kimi-k2.7-code | 87.88% | 91.01% | +3.13 pp |
| claude-opus-4-8_low | 88.38% | 91.01% | +2.62 pp |
| gpt-5.2-2025-12-11_medium | 87.88% | 90.94% | +3.06 pp |
| fireworks/kimi-k2p5 | 87.56% | 90.48% | +2.91 pp |
| qwen3.7-plus | 87.88% | 90.48% | +2.60 pp |
| gpt-5.6-terra_low | 87.37% | 90.48% | +3.10 pp |
| qwen3.6-max-preview | 87.37% | 90.48% | +3.10 pp |
| gpt-5.1-2025-11-13_high | 87.63% | 90.41% | +2.78 pp |
| gemini-3.6-flash_low | 86.36% | 89.95% | +3.58 pp |
| gpt-5.4-mini-2026-03-17_xhigh | 86.87% | 89.95% | +3.08 pp |
| gemini-3.5-flash_minimal | 86.36% | 89.95% | +3.58 pp |
| Grok 4.3 (high) | 86.87% | 89.95% | +3.08 pp |
| claude-sonnet-4-6_32K | 87.37% | 89.68% | +2.31 pp |
| gpt-5-2025-08-07_high | 86.17% | 89.55% | +3.38 pp |
| claude-opus-4-7_max | 86.36% | 89.42% | +3.05 pp |
| claude-fable-5_max | 85.86% | 89.42% | +3.56 pp |
| qwen3.5-397b-a17b | 85.86% | 89.42% | +3.56 pp |
| qwen3.5-397b-a17b_none | 86.36% | 89.42% | +3.05 pp |
| qwen3.6-27b | 85.86% | 88.89% | +3.03 pp |
| claude-opus-4-5-20251101_32K | 86.05% | 88.76% | +2.71 pp |
| gpt-5-2025-08-07_medium | 85.43% | 88.74% | +3.31 pp |
| claude-opus-4-5-20251101_16K | 85.48% | 88.69% | +3.21 pp |
| gemini-3.6-flash_minimal | 85.86% | 88.36% | +2.50 pp |
| qwen3.5-plus | 84.85% | 88.36% | +3.51 pp |
| kimi-k3_low | 84.85% | 88.36% | +3.51 pp |
| gemini-2.5-pro | 85.29% | 88.29% | +3.00 pp |
| gemini-2.5-pro-preview-06-05 | 84.85% | 87.83% | +2.98 pp |
| claude-opus-4-8_none | 85.35% | 87.83% | +2.48 pp |
| qwen3.6-27b_none | 84.85% | 87.83% | +2.98 pp |
| gpt-5.4-2026-03-05_low | 84.85% | 87.83% | +2.98 pp |
| qwen3.6-35b-a3b_none | 84.85% | 87.83% | +2.98 pp |
| gpt-5.1-2025-11-13_medium | 85.04% | 87.76% | +2.73 pp |
| gemini-2.5-pro-exp-03-25 | 83.84% | 87.30% | +3.46 pp |
| qwen3.6-35b-a3b | 83.84% | 86.77% | +2.93 pp |
| claude-fable-5_high | 83.33% | 86.77% | +3.44 pp |
| claude-sonnet-4-6_high | 83.33% | 86.77% | +3.44 pp |
| gpt-5.4-mini-2026-03-17_high | 83.59% | 86.77% | +3.19 pp |
| kimi-k2-thinking-turbo | 84.22% | 86.51% | +2.29 pp |
| claude-sonnet-4-6_medium | 83.33% | 86.24% | +2.91 pp |
| gpt-5.6-sol_none | 82.83% | 86.24% | +3.42 pp |
| deepseek-reasoner | 83.33% | 86.17% | +2.83 pp |
| gemini-3-flash-preview | 83.21% | 85.91% | +2.71 pp |
| claude-sonnet-4-5-20250929_59K | 82.32% | 85.71% | +3.39 pp |
| qwen3.5-flash | 82.32% | 85.71% | +3.39 pp |
| qwen3.6-flash | 83.33% | 85.71% | +2.38 pp |
| gemini-3.5-flash-lite_high | 83.33% | 85.71% | +2.38 pp |
| gpt-5.2-2025-12-11_low | 82.70% | 85.65% | +2.95 pp |
| gpt-5.6-luna_low | 82.32% | 85.19% | +2.86 pp |
| qwen3.7-flash | 82.32% | 85.19% | +2.86 pp |
| o3-2025-04-16_high | 81.82% | 85.05% | +3.23 pp |
| claude-sonnet-4-5-20250929_32K | 81.82% | 84.66% | +2.84 pp |
| gemini-3.1-flash-lite_high | 81.82% | 84.66% | +2.84 pp |
| qwen3.7-flash_none | 80.81% | 84.13% | +3.32 pp |
| qwen3.7-plus_none | 81.82% | 84.13% | +2.31 pp |
| claude-sonnet-5_max | 80.30% | 83.60% | +3.29 pp |
| o3-2025-04-16_medium | 80.81% | 83.60% | +2.79 pp |
| glm-4.7 | 80.30% | 83.07% | +2.77 pp |
| o4-mini-2025-04-16_high | 79.61% | 82.94% | +3.33 pp |
| claude-opus-4-5-20251101 | 80.68% | 82.74% | +2.06 pp |
| o3-2025-04-16_low | 79.80% | 82.54% | +2.74 pp |
| claude-sonnet-4-5-20250929_16K | 78.79% | 82.54% | +3.75 pp |
| Qwen3-235B-A22B-Thinking-2507 | 80.05% | 82.28% | +2.22 pp |
| claude-fable-5_low | 78.79% | 82.01% | +3.22 pp |
| claude-sonnet-4-6_max | 78.79% | 81.48% | +2.69 pp |
| gpt-5.4-nano-2026-03-17_high | 78.47% | 81.02% | +2.55 pp |
| claude-sonnet-4-20250514_32K | 78.28% | 80.42% | +2.14 pp |
| claude-opus-4-1-20250805_16K | 77.27% | 80.42% | +3.15 pp |
| o4-mini-2025-04-16_medium | 77.78% | 80.42% | +2.65 pp |
| claude-3-7-sonnet-20250219_16K | 76.77% | 79.89% | +3.13 pp |
| claude-3-7-sonnet-20250219_32K | 76.77% | 79.89% | +3.13 pp |
| claude-opus-4-1-20250805_27K | 76.77% | 79.89% | +3.13 pp |
| gpt-5.5_none | 77.27% | 79.89% | +2.62 pp |
| o3-mini-2025-01-31_high | 77.02% | 79.56% | +2.54 pp |
| o1-2024-12-17_high | 76.77% | 79.37% | +2.60 pp |
| grok-3-mini-beta_low | 76.26% | 79.37% | +3.10 pp |
| gpt-5.6-terra_none | 77.27% | 79.37% | +2.09 pp |
| DeepSeek-R1-0528 | 76.33% | 78.84% | +2.51 pp |
| claude-opus-4-20250514_16K | 76.26% | 78.84% | +2.57 pp |
| claude-sonnet-4-20250514_16K | 75.76% | 78.31% | +2.55 pp |
| openai/gpt-oss-120b_high | 75.76% | 78.31% | +2.55 pp |
| gemma-4-31b-it_minimal | 75.76% | 78.31% | +2.55 pp |
| gemini-3.5-flash-lite_low | 75.76% | 78.31% | +2.55 pp |
| gpt-5-mini-2025-08-07_high | 75.00% | 77.84% | +2.84 pp |
| o4-mini-2025-04-16_low | 75.25% | 77.78% | +2.53 pp |
| o1-2024-12-17_medium | 75.76% | 77.25% | +1.49 pp |
| gpt-5.4-2026-03-05_none | 74.75% | 77.25% | +2.50 pp |
| o1-2024-12-17_low | 74.24% | 76.72% | +2.48 pp |
| gemini-3.1-flash-lite_low | 74.24% | 76.72% | +2.48 pp |
| gemini-3.5-flash-lite_minimal | 74.24% | 76.72% | +2.48 pp |
| o3-mini-2025-01-31_medium | 74.27% | 76.52% | +2.25 pp |
| gpt-5.2-2025-12-11_none | 73.23% | 76.19% | +2.96 pp |
| claude-opus-4-1-20250805 | 73.23% | 75.66% | +2.43 pp |
| gemini-3.1-flash-lite_minimal | 73.74% | 75.66% | +1.92 pp |
| deepseek-v4-pro_none | 73.23% | 75.66% | +2.43 pp |
| claude-sonnet-4-5-20250929 | 73.74% | 75.13% | +1.39 pp |
| qwen3-max-2025-09-23 | 72.60% | 75.13% | +2.53 pp |
| gpt-5-2025-08-07_minimal | 71.72% | 74.60% | +2.89 pp |
| gpt-5-mini-2025-08-07_medium | 71.65% | 74.54% | +2.88 pp |
| gpt-5.4-nano-2026-03-17_low | 72.22% | 74.07% | +1.85 pp |
| gpt-5-mini-2025-08-07_minimal | 71.72% | 74.07% | +2.36 pp |
| deepseek-chat | 71.21% | 73.54% | +2.33 pp |
| claude-haiku-4-5-20251001_32K | 71.21% | 73.54% | +2.33 pp |
| qwen3-235b-a22b | 70.71% | 73.35% | +2.64 pp |
| glm-5.2_none | 71.21% | 72.49% | +1.27 pp |
| gpt-5-nano-2025-08-07_high | 69.44% | 72.16% | +2.71 pp |
| claude-opus-4-20250514 | 69.19% | 71.43% | +2.24 pp |
| o3-mini-2025-01-31_low | 68.18% | 70.90% | +2.72 pp |
| gpt-4.5-preview-2025-02-27 | 68.69% | 70.90% | +2.21 pp |
| gpt-5-nano-2025-08-07_medium | 67.42% | 70.17% | +2.75 pp |
| DeepSeek-V3-0324 | 67.61% | 69.64% | +2.03 pp |
| gpt-4.1-2025-04-14 | 66.92% | 69.31% | +2.39 pp |
| gpt-5.1-2025-11-13_none | 66.67% | 69.31% | +2.65 pp |
| claude-sonnet-4-20250514 | 66.67% | 68.78% | +2.12 pp |
| gemini-2.5-pro-preview-05-06 | 66.67% | 68.78% | +2.12 pp |
| Llama-4-Maverick-17B-128E-Instruct-FP8 | 66.98% | 68.25% | +1.27 pp |
| gpt-4.1-mini-2025-04-14 | 65.85% | 68.19% | +2.34 pp |
| claude-3-7-sonnet-20250219 | 66.04% | 68.19% | +2.15 pp |
| qwq-plus | 65.40% | 67.26% | +1.86 pp |
| gemini-2.0-pro-exp-02-05 | 65.66% | 67.20% | +1.54 pp |
| gpt-5.4-mini-2026-03-17_none | 64.14% | 66.14% | +2.00 pp |
| gpt-5.6-luna_none | 63.64% | 65.61% | +1.97 pp |
| gemini-2.0-flash-001 | 64.14% | 65.54% | +1.40 pp |
| o1-mini-2024-09-12_high | 62.37% | 64.29% | +1.91 pp |
| claude-haiku-4-5-20251001 | 60.48% | 61.77% | +1.29 pp |
| o1-mini-2024-09-12_medium | 59.50% | 61.64% | +2.14 pp |
| mistral-medium-2505 | 59.53% | 60.98% | +1.45 pp |
| gpt-5-nano-2025-08-07_low | 57.58% | 59.26% | +1.68 pp |
| gemini-2.0-flash-thinking-exp-01-21 | 57.07% | 59.26% | +2.19 pp |
| gemini-1.5-pro-002 | 57.23% | 58.63% | +1.40 pp |
| DeepSeek-V3 | 56.53% | 57.84% | +1.30 pp |
| DeepSeek-R1-Distill-Llama-70B | 55.74% | 57.80% | +2.06 pp |
| magistral-small-2506 | 56.06% | 57.14% | +1.08 pp |
| gpt-5.4-nano-2026-03-17_none | 55.56% | 57.14% | +1.59 pp |
| qwen-max-2025-01-25 | 56.12% | 57.08% | +0.95 pp |
| phi-4 | 56.06% | 56.94% | +0.88 pp |
| claude-3-5-sonnet-20241022 | 55.30% | 56.45% | +1.15 pp |
| claude-3-5-sonnet-20240620 | 54.04% | 55.69% | +1.65 pp |
| grok-2-1212 | 53.79% | 54.07% | +0.28 pp |
| Llama-4-Scout-17B-16E-Instruct | 51.83% | 52.65% | +0.81 pp |
| o1-preview-2024-09-12 | 50.32% | 52.55% | +2.23 pp |
| Llama-3.1-405B-Instruct | 50.92% | 52.51% | +1.60 pp |
| mistral-large-2411 | 51.33% | 51.98% | +0.66 pp |
| gpt-4o-2024-08-06 | 49.21% | 50.46% | +1.25 pp |
| qwen2.5-72b-instruct | 49.15% | 50.03% | +0.89 pp |
| gpt-4o-2024-05-13 | 48.90% | 49.97% | +1.07 pp |
| gpt-4.1-nano-2025-04-14 | 48.93% | 49.93% | +1.01 pp |
| mistral-large-2407 | 49.02% | 49.80% | +0.78 pp |
| gpt-4o-2024-11-20 | 47.89% | 49.14% | +1.26 pp |
| gpt-5-nano-2025-08-07_minimal | 48.48% | 48.68% | +0.19 pp |
| qwen-plus-2025-01-25 | 48.11% | 48.35% | +0.24 pp |
| Llama-3.3-70B-Instruct | 47.44% | 48.15% | +0.70 pp |
| gpt-oss-20b_high | 45.96% | 48.15% | +2.19 pp |
| gemini-1.5-flash-002 | 47.32% | 48.05% | +0.73 pp |
| mistral-small-2503 | 47.47% | 48.02% | +0.54 pp |
| claude-3-opus-20240229 | 47.16% | 47.88% | +0.72 pp |
| gpt-4-turbo-2024-04-09 | 46.59% | 46.96% | +0.37 pp |
| Llama-3.1-Tulu-3-70B-DPO | 46.28% | 46.96% | +0.68 pp |
| qwen2.5-32b-instruct | 46.09% | 46.63% | +0.54 pp |
| DeepSeek-R1-Distill-Qwen-14B | 44.70% | 46.49% | +1.80 pp |
| gemini-1.5-pro-001 | 45.86% | 46.43% | +0.56 pp |
| mistral-small-2501 | 45.30% | 46.06% | +0.77 pp |
| gemma-3-27b-it | 43.94% | 44.97% | +1.03 pp |
| Llama-3.1-70B-Instruct | 44.19% | 44.87% | +0.68 pp |
| WizardLM-2-8x22B | 43.43% | 44.28% | +0.84 pp |
| gpt-4-1106-preview | 42.36% | 43.42% | +1.06 pp |
| gpt-4-0125-preview | 42.27% | 43.39% | +1.12 pp |
| qwen-turbo-2024-11-01 | 41.79% | 41.80% | +0.01 pp |
| Llama-3.2-90B-Vision-Instruct | 41.04% | 41.63% | +0.60 pp |
| Meta-Llama-3-70B-Instruct | 40.56% | 41.57% | +1.01 pp |
| claude-3-sonnet-20240229 | 40.59% | 41.30% | +0.71 pp |
| qwen2-72b-instruct | 40.78% | 40.91% | +0.12 pp |
| gemini-1.5-flash-001 | 40.37% | 40.38% | 0.00 pp |
| mistral-large-2402 | 38.76% | 39.42% | +0.66 pp |
| claude-3-5-haiku-20241022 | 38.13% | 38.96% | +0.82 pp |
| gpt-4o-mini-2024-07-18 | 37.72% | 38.82% | +1.10 pp |
| Hermes-2-Theta-Llama-3-70B | 37.47% | 38.26% | +0.79 pp |
| claude-3-haiku-20240307 | 36.30% | 36.38% | +0.08 pp |
| gemma-2-27b-it | 36.49% | 36.24% | -0.25 pp |
| gpt-4-0314 | 35.73% | 36.04% | +0.31 pp |
| gemini-1.0-pro-001 | 33.96% | 34.59% | +0.63 pp |
| open-mixtral-8x22b | 34.06% | 34.26% | +0.20 pp |
| claude-2.0 | 34.66% | 34.19% | -0.47 pp |
| Eurus-2-7B-PRIME | 33.90% | 34.16% | +0.26 pp |
| claude-2.1 | 32.95% | 33.47% | +0.51 pp |
| dbrx-instruct | 32.89% | 33.40% | +0.51 pp |
| gemini-1.5-flash-8b-001 | 32.95% | 33.00% | +0.05 pp |
| Yi-1.5-34B-Chat | 31.98% | 32.24% | +0.27 pp |
| gpt-4-0613 | 30.65% | 31.32% | +0.67 pp |
| Mixtral-8x7B-Instruct-v0.1 | 30.59% | 31.25% | +0.66 pp |
| qwen1.5-32b-chat | 30.74% | 31.02% | +0.27 pp |
| open-mixtral-8x7b | 29.83% | 30.29% | +0.46 pp |
| open-mistral-nemo-2407 | 29.89% | 29.99% | +0.10 pp |
| qwen1.5-72b-chat | 28.82% | 29.50% | +0.68 pp |
| gpt-3.5-turbo-1106 | 28.03% | 27.88% | -0.15 pp |
| Phi-3-medium-128k-instruct | 27.59% | 27.58% | -0.01 pp |
| gemma-2-9b-it | 27.46% | 27.58% | +0.12 pp |
| gpt-3.5-turbo-0125 | 27.18% | 27.31% | +0.14 pp |
| Llama-2-70b-chat-hf | 26.33% | 27.08% | +0.76 pp |
| ministral-8b-2410 | 27.15% | 26.46% | -0.69 pp |
| Meta-Llama-3-8B-Instruct | 26.07% | 26.36% | +0.28 pp |
| Llama-3.1-8B-Instruct | 25.95% | 26.22% | +0.28 pp |
| ministral-3b-2410 | 25.25% | 24.60% | -0.65 pp |
| deepseek-llm-67b-chat | 24.62% | 24.54% | -0.08 pp |
| Mistral-7B-Instruct-v0.3 | 15.18% | 15.44% | +0.26 pp |
| Yi-34B-Chat | 14.74% | 15.21% | +0.47 pp |
| open-mistral-7b | 13.23% | 13.10% | -0.13 pp |
Distribution of score changes
The median and middle 50% come from all 243 accepted configurations.
Chart loads from the release bundle when visible.
8 decreased; 235 improved. Range -0.69 to +3.80 percentage points.
Threshold interpretation changes
These are benchmark-definition changes, not new model progress.
95% threshold
0 Original 198 configurations
14 Clean 189 configurations
+14 cross upward
Where ranks remain fragile
| Observed rank | Configuration | 95% rank interval | Change |
|---|---|---|---|
| 1 | gpt-5.5-pro-pre-release_xhigh mean@7 | 1–10 | +3.42 pp |
| 1 | Gemini 3.6 Flash (high) mean@1 | 1–14 | +3.42 pp |
| 3 | gpt-5.5-pre-release_xhigh mean@8 | 1–10 | +3.22 pp |
| 4 | gemini-3.1-pro-preview mean@8 | 1–12 | +2.52 pp |
| 5 | GPT-5.6 Sol (max) mean@1 | 1–17 | +3.39 pp |
| 6 | gpt-5.4-2026-03-05_xhigh mean@8 | 1–14 | +3.45 pp |
| 7 | gemini-3.1-pro-preview_high mean@1 | 1–23 | +1.85 pp |
| 8 | gemini-3.5-flash_high mean@8 | 2–20 | +3.29 pp |
Why people should care
- Leaderboard claims can hinge on defective items.Small benchmark flaws can invert or exaggerate differences.
- A Clean score is paired evidence.It is recomputed from the same evaluated attempts after the exclusions.
- Provenance makes disagreement inspectable.Every accepted path remains explicitly labelled.
What this does not claim
- Cleaning does not make GPQA contamination-free.
- Combined rank does not erase protocol differences.
- Targeted recoveries remain labelled composites.
- No private prompts, answers, or reasoning traces are published.