Scientific Question Answering on GPQA Diamond
84.4AccuracyGemini 2.5 Pro
Evaluation Results
| Method | Links | |
|---|---|---|
| Gemini 2.5 Pro2026.02 | 84.4 | |
| Interfaze-Beta2026.02 | 81.31 | |
| Claude Opus 4Reasoning=Thinking2026.02 | 79.6 | |
| Claude Sonnet 4Reasoning=Thinking2026.02 | 77.7 | |
| Claude Sonnet 42026.02 | 68.3 | |
| Gemini 2.5 Flash2026.02 | 68.3 | |
| GPT-5Reasoning=Minimal Reasoning2026.02 | 67.3 | |
| GPT-4.1(2025-04-14)Maximum output tokens=32,7682025.07 | 67.17 | |
| GPT-o3-mini(2025-01-31)Maximum output tokens=32,7682025.07 | 66.67 | |
| GPT-4.12026.02 | 66.3 | |
| SMCSMaximum output tokens=32,7682025.07 | 66.16 | |
| Qwen3-32BMaximum output tokens=32,7682025.07 | 64.65 | |
| Claude-3.7-Sonnet(2025-02-19)Maximum output tokens=32,7682025.07 | 63.64 | |
| EXAONE-Deep-32BMaximum output tokens=32,7682025.07 | 63.13 | |
| QwQ-32BMaximum output tokens=32,7682025.07 | 62.63 | |
| GLM-Z1-32B-0414Maximum output tokens=32,7682025.07 | 62.12 | |
| Claude-3.5-Sonnet(2024-06-20)Maximum output tokens=32,7682025.07 | 61.62 | |
| DeepSeek-R1-Distill-Llama-70BMaximum output tokens=32,7682025.07 | 60.6 | |
| DeepSeek-V3Starting Checkpoint=Base, Reasoning Strategy=Self-Contained Reasoning, Backbone Architecture Family=DeepSeek-V32026.02 | 59.1 | |
| ELPO-4BStarting Checkpoint=Inst, Backbone Architecture Family=Qwen3-4B2026.02 | 59.1 | |
| Qwen-3-30B-A3BOpenness=Open-weights, Regional Origin=Non-European, Post-training=Instruction-tuned2026.02 | 58.6 | |
| DemyAgent-4BStarting Checkpoint=Inst, Backbone Architecture Family=Qwen3-4B2026.02 | 58.5 | |
| DeepSeek-R1-Distill-Qwen-32BMaximum output tokens=32,7682025.07 | 57.58 | |
| Qwen-3-14BOpenness=Open-weights, Regional Origin=Non-European, Post-training=Instruction-tuned2026.02 | 56.6 | |
| Qwen-3-32BOpenness=Open-weights, Regional Origin=Non-European, Post-training=Instruction-tuned2026.02 | 54.7 | |
| Base(instruct)Architecture=Qwen3-30B-A3B2026.05 | 54.04 | |
| ARPOStarting Checkpoint=Inst, Backbone Architecture Family=Qwen2.5-7B2026.02 | 53 | |
| AEPOStarting Checkpoint=Inst, Backbone Architecture Family=Qwen2.5-7B2026.02 | 53 | |
| ELPO-7BStarting Checkpoint=Inst, Backbone Architecture Family=Qwen2.5-7B2026.02 | 52.7 | |
| GPT-4o(2024-08-06)Maximum output tokens=32,7682025.07 | 52.53 | |
| HuatuoGPT-o1-72BMaximum output tokens=32,7682025.07 | 52.53 | |
| GIGPOStarting Checkpoint=Inst, Backbone Architecture Family=Qwen2.5-7B2026.02 | 52.5 | |
| GRPOArchitecture=Qwen3-30B-A3B2026.05 | 52.02 | |
| Qwen3-4BStarting Checkpoint=Inst, Reasoning Strategy=Self-Contained Reasoning, Backbone Architecture Family=Qwen3-4B2026.02 | 52 | |
| DemyAgentStarting Checkpoint=Inst, Backbone Architecture Family=Qwen2.5-7B2026.02 | 52 | |
| ToRLStarting Checkpoint=Math-Inst, Backbone Architecture Family=Qwen2.5-7B2026.02 | 51.5 | |
| Gemma-3-27b-itMaximum output tokens=32,7682025.07 | 50.51 | |
| CIRStarting Checkpoint=Math, Backbone Architecture Family=Qwen2.5-7B2026.02 | 49.5 | |
| MESAArchitecture=Qwen3-30B-A3B2026.05 | 49.49 | |
| Stair-SFTArchitecture=Qwen3-30B-A3B2026.05 | 48.98 | |
| Llama-3.3-Nemotron-Super-49B-v1Maximum output tokens=32,7682025.07 | 48.48 | |
| OPDTeacher-Student Setting=DeepSeek-R1 -> DeepSeek-R1-Distill-Qwen-7B2026.06 | 48.1 | |
| TIPSetting=Qwen3-14B -> Qwen3-4B, Supervised-token budget=10%2026.05 | 47.98 | |
| BaseSetting=Qwen3-8B (GRPO) -> Qwen3-4B, Supervised-token budget=10%2026.05 | 47.97 | |
| BaseSetting=Qwen3-14B -> Qwen3-4B, Supervised-token budget=10%2026.05 | 47.97 | |
| EntropyModel=Qwen3-14B-Base, Evaluation Samples=8 samples2026.01 | 47.85 | |
| Gemma-3-27BOpenness=Open-weights, Regional Origin=Non-European, Post-training=Instruction-tuned2026.02 | 47.6 | |
| ALMTeacher-Student Setting=DeepSeek-R1 -> DeepSeek-R1-Distill-Qwen-7B2026.06 | 47.6 | |
| Mistral-3.2-24BOpenness=Open-weights, Regional Origin=European, Post-training=Instruction-tuned2026.02 | 47.5 | |
| EntropySetting=Qwen3-8B (GRPO) -> Qwen3-4B, Supervised-token budget=10%2026.05 | 47.47 | |
| TIPSetting=Qwen3-8B (GRPO) -> Qwen3-4B, Supervised-token budget=10%2026.05 | 47.47 | |
| TA-OPDSetting=Qwen3-8B (GRPO) -> Qwen3-4B, Supervised-token budget=10%2026.05 | 47.47 | |
| Stair-DPOArchitecture=Qwen3-30B-A3B2026.05 | 47.47 | |
| CDMTeacher-Student Setting=DeepSeek-R1 -> DeepSeek-R1-Distill-Qwen-7B2026.06 | 47.4 | |
| Pure OPDSetting=Qwen3-8B (GRPO) -> Qwen3-4B, Supervised-token budget=10%2026.05 | 47.22 | |
| Qwen-2.5-72B-InstructMaximum output tokens=32,7682025.07 | 46.97 | |
| Llama-3.3-70B-InstructMaximum output tokens=32,7682025.07 | 46.97 | |
| SFTTeacher-Student Setting=DeepSeek-R1 -> DeepSeek-R1-Distill-Qwen-7B2026.06 | 46.9 | |
| Llama-3.3-70BOpenness=Open-weights, Regional Origin=Non-European, Post-training=Instruction-tuned2026.02 | 46.6 | |
| ProFitModel=Qwen3-14B-Base, Evaluation Samples=8 samples2026.01 | 46.53 | |
| EntropySetting=Qwen3-14B -> Qwen3-4B, Supervised-token budget=10%2026.05 | 46.46 | |
| SFTModel=Qwen3-14B-Base, Evaluation Samples=8 samples2026.01 | 46.02 | |
| TA-OPD+Ent.Setting=Qwen3-8B (GRPO) -> Qwen3-4B, Supervised-token budget=10%2026.05 | 45.7 | |
| TA-OPDSetting=Qwen3-14B -> Qwen3-4B, Supervised-token budget=10%2026.05 | 45.2 | |
| SafeXArchitecture=Qwen3-30B-A3B2026.05 | 44.44 | |
| Qwen3-4BStarting Checkpoint=Inst, Reasoning Strategy=TIR Reasoning, Backbone Architecture Family=Qwen3-4B2026.02 | 44.3 | |
| DFTModel=Qwen3-14B-Base, Evaluation Samples=8 samples2026.01 | 43.81 | |
| Pure OPDSetting=Qwen3-14B -> Qwen3-4B, Supervised-token budget=10%2026.05 | 42.17 | |
| TA-OPD+Ent.Setting=Qwen3-14B -> Qwen3-4B, Supervised-token budget=10%2026.05 | 42.17 | |
| Qwen2.5-Coder-32B-InstructMaximum output tokens=32,7682025.07 | 41.92 | |
| Qwen2.5-32b-InstructMaximum output tokens=32,7682025.07 | 40.91 | |
| SFTArchitecture=Qwen3-30B-A3B2026.05 | 40.4 | |
| VanillaModel=Qwen3-14B-Base, Evaluation Samples=8 samples2026.01 | 37.69 | |
| Gemma-3-12BOpenness=Open-weights, Regional Origin=Non-European, Post-training=Instruction-tuned2026.02 | 37.2 | |
| OLMo-3.1-32BOpenness=Fully-open, Regional Origin=Non-European, Post-training=Instruction-tuned2026.02 | 36 | |
| EntropyModel=Qwen3-4B-Base, Evaluation Samples=8 samples2026.01 | 34.91 | |
| InternLM2.5-20B-ChatMaximum output tokens=32,7682025.07 | 34.85 | |
| TA-OPDSetting=Qwen3-4B -> Qwen3-1.7B, Supervised-token budget=10%2026.05 | 34.59 | |
| ProFitModel=Qwen3-4B-Base, Evaluation Samples=8 samples2026.01 | 34.34 | |
| SFTModel=Qwen3-4B-Base, Evaluation Samples=8 samples2026.01 | 34.15 | |
| DAPOStarting Checkpoint=Instruct, Backbone Architecture Family=Qwen2.5-7B2026.02 | 33.5 | |
| TeleChat2-35B-32KMaximum output tokens=32,7682025.07 | 33.33 | |
| OLMo-3-7BOpenness=Fully-open, Regional Origin=Non-European, Post-training=Instruction-tuned2026.02 | 33.2 | |
| GRPOStarting Checkpoint=Inst, Backbone Architecture Family=Qwen2.5-7B2026.02 | 32.7 | |
| EntropySetting=Qwen3-4B -> Qwen3-1.7B, Supervised-token budget=10%2026.05 | 32.58 | |
| TA-OPD+Ent.Setting=Qwen3-4B -> Qwen3-1.7B, Supervised-token budget=10%2026.05 | 32.58 | |
| DFTModel=Qwen3-4B-Base, Evaluation Samples=8 samples2026.01 | 31.69 | |
| TA-OPD+Ent.Setting=DeepSeek-R1-Distill-Qwen-14B -> Qwen2.5-3B, Supervised-token budget=10%2026.05 | 31.63 | |
| BaseSetting=Qwen3-4B -> Qwen3-1.7B, Supervised-token budget=10%2026.05 | 31.56 | |
| Qwen2.5-7BStarting Checkpoint=Inst, Reasoning Strategy=Self-Contained Reasoning, Backbone Architecture Family=Qwen2.5-7B2026.02 | 31.3 | |
| Reinforce++Starting Checkpoint=Instruct, Backbone Architecture Family=Qwen2.5-7B2026.02 | 31.3 | |
| SFTArchitecture=DeepSeek-v2-Lite2026.05 | 30.81 | |
| Pure OPDSetting=Qwen3-4B -> Qwen3-1.7B, Supervised-token budget=10%2026.05 | 30.55 | |
| TA-OPDSetting=DeepSeek-R1-Distill-Qwen-14B -> Qwen2.5-3B, Supervised-token budget=10%2026.05 | 29.59 | |
| TIPSetting=Qwen3-4B -> Qwen3-1.7B, Supervised-token budget=10%2026.05 | 29.29 | |
| Stair-SFTArchitecture=DeepSeek-v2-Lite2026.05 | 28.28 | |
| BaseSetting=DeepSeek-R1-Distill-Qwen-14B -> Qwen2.5-3B, Supervised-token budget=10%2026.05 | 27.81 | |
| Pure OPDSetting=DeepSeek-R1-Distill-Qwen-14B -> Qwen2.5-3B, Supervised-token budget=10%2026.05 | 27.78 | |
| TIPSetting=DeepSeek-R1-Distill-Qwen-14B -> Qwen2.5-3B, Supervised-token budget=10%2026.05 | 27.55 | |
| EuroLLM-22BOpenness=Fully-open, Regional Origin=European, Post-training=Instruction-tuned2026.02 | 26.8 |