Scientific Reasoning on GPQA Diamond
73.2ScoreQwen3-30B-A3B-Thinking-2507
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Qwen3-30B-A3B-Thinking-2507Sampling=Bo∞ (Best-of-infinity sampling)2025.09 | 73.2 | — | |
| Phi-4-reasoningSampling=Bo∞ (Best-of-infinity sampling)2025.09 | 72.7 | — | |
| GPT-OSS-20BSampling=Bo∞ (Best-of-infinity sampling), reasoning mode=medium2025.09 | 72.2 | — | |
| Qwen3-30B-A3B-Thinking-2507Sampling=Bo1 (Best-of-1 sampling)2025.09 | 72 | — | |
| MetaStone-S1-32BSampling=Bo∞ (Best-of-infinity sampling)2025.09 | 70.7 | — | |
| EXAONE-Deep-32BSampling=Bo∞ (Best-of-infinity sampling)2025.09 | 69.2 | — | |
| MetaStone-S1-32BSampling=Bo1 (Best-of-1 sampling)2025.09 | 67 | — | |
| EXAONE-Deep-32BSampling=Bo1 (Best-of-1 sampling)2025.09 | 66.1 | — | |
| Phi-4-reasoningSampling=Bo1 (Best-of-1 sampling)2025.09 | 65.8 | — | |
| GPT-OSS-20BSampling=Bo1 (Best-of-1 sampling), reasoning mode=medium2025.09 | 64.2 | — | |
| NVIDIA-Nemotron-Nano-9B-v2Sampling=Bo∞ (Best-of-infinity sampling)2025.09 | 62.6 | — | |
| NVIDIA-Nemotron-Nano-9B-v2Sampling=Bo1 (Best-of-1 sampling)2025.09 | 58.4 | — | |
| SPARDBackbone=Qwen3-8B2026.04 | 49.49 | — | |
| RLCERScale=8B2026.02 | 48.77 | — | |
| + GRPOavgBackbone=Qwen3-8B2026.04 | 46.97 | — | |
| RLVRScale=8B2026.02 | 46.56 | — | |
| + GRPOimpBackbone=Qwen3-8B2026.04 | 45.96 | — | |
| RLCERScale=4B2026.02 | 44.91 | — | |
| RLVRScale=4B2026.02 | 44.16 | — | |
| + DPOBackbone=Qwen3-8B2026.04 | 43.94 | — | |
| + GRPOrmBackbone=Qwen3-8B2026.04 | 43.94 | — | |
| BaseBackbone=Qwen3-8B2026.04 | 42.93 | — | |
| Mistral 24B2026.02 | 41.41 | — | |
| Llama 70B2026.02 | 41.41 | — | |
| FiMI Instruct2026.02 | 39.9 | — | |
| GPT-4Parameter Scale=API2024.01 | 39.7 | — | |
| Gemma 2 27B2026.02 | 38.88 | — | |
| SPARDBackbone=Qwen2.5-7B-Instruct2026.04 | 38.38 | — | |
| + GRPOavgBackbone=Qwen2.5-7B-Instruct2026.04 | 35.35 | — | |
| + GRPOrmBackbone=Qwen2.5-7B-Instruct2026.04 | 34.85 | — | |
| NeMo 12B IT2026.02 | 34.34 | — | |
| Granite-4 Micro (3B)Variant=PRISM2026.03 | 34.34 | — | |
| BaseBackbone=Qwen2.5-7B-Instruct2026.04 | 33.84 | — | |
| + DPOBackbone=Qwen2.5-7B-Instruct2026.04 | 33.54 | — | |
| + SFTBackbone=Qwen3-8B2026.04 | 33.33 | — | |
| + SFTBackbone=Qwen2.5-7B-Instruct2026.04 | 32.83 | — | |
| + GRPOimpBackbone=Qwen2.5-7B-Instruct2026.04 | 32.83 | — | |
| Granite-4-H Micro (3B)Variant=PRISM2026.03 | 32.66 | — | |
| Nemotron-H-8k (8B)Variant=PRISM2026.03 | 31.98 | — | |
| SFTScale=8B2026.02 | 31.72 | — | |
| Llama3-8B-InstructEvaluation Protocol=few-shot, Parameter Scale=6B~7B2024.01 | 31.31 | — | |
| Mistral 7B2026.02 | 31.31 | — | |
| Llama3-8B-Instruct + SciInstructFine-tuning=SciInstruct, Parameter Scale=6B~7B2024.01 | 29.29 | — | |
| Mistral-7B: MetaMATHEvaluation Protocol=few-shot, Parameter Scale=6B~7B2024.01 | 29.29 | — | |
| Granite-3.3 (8B)Variant=PRISM2026.03 | 29.12 | — | |
| ChatGLM3-6BParameter Scale=6B~7B2024.01 | 28.7 | — | |
| Llama 3.1 8B2026.02 | 28.28 | — | |
| Mistral-7B: MetaMATHEvaluation Protocol=zero-shot, Parameter Scale=6B~7B2024.01 | 27.78 | — | |
| Llama3-8B-InstructEvaluation Protocol=zero-shot, Parameter Scale=6B~7B2024.01 | 27.27 | — | |
| Mistral-7B: MetaMATH + SciInstructFine-tuning=SciInstruct, Parameter Scale=6B~7B2024.01 | 27.27 | — | |
| Mistral-7BVariant=Base2026.03 | 26.94 | — | |
| LLaMA-2-13BParameter Scale=12B~13B2024.01 | 26.2 | — | |
| SciGLMBackbone=ChatGLM3-32B-Base, Parameter Scale=30B~32B2024.01 | 25.76 | — | |
| SciGLMBackbone=ChatGLM3-6B-Base, Parameter Scale=6B~7B2024.01 | 25.25 | — | |
| ChatGLM3-6B-BaseParameter Scale=6B~7B2024.01 | 24.75 | — | |
| SFTScale=4B2026.02 | 24.43 | — | |
| BaseScale=8B2026.02 | 24.37 | — | |
| Mistral-7BVariant=PRISM2026.03 | 24.07 | — | |
| Granite-3.3 (8B)Variant=Base2026.03 | 22.56 | — | |
| Mistral-Small (24B)Variant=Base2026.03 | 22.55 | — | |
| ChatGLM3-32B-BaseParameter Scale=30B~32B2024.01 | 22.22 | — | |
| Mistral-Small (24B)Variant=PRISM2026.03 | 22.05 | — | |
| Granite-4 Micro (3B)Variant=Base2026.03 | 21.55 | — | |
| LLaMA-3.1 (8B)Variant=PRISM2026.03 | 21.04 | — | |
| Granite-4-H Micro (3B)Variant=Base2026.03 | 20.88 | — | |
| LLaMA-3.1 (8B)Variant=Base2026.03 | 20.2 | — | |
| BaseScale=4B2026.02 | 7.77 | — | |
| Nemotron-H-8k (8B)Variant=Base2026.03 | 4.21 | — | |
| Contrastive Causality (CC)Perturbation=none2026.01 | — | 35.87 | |
| Contrastive Causality (CC)Perturbation=shuffling2026.01 | — | 32.47 | |
| Contrastive Causality (CC)Perturbation=small-eval2026.01 | — | 26.4 | |
| Self-Certainty (SC)Perturbation=none2026.01 | — | 34.33 | |
| Self-Certainty (SC)Perturbation=shuffling2026.01 | — | 32.13 | |
| Self-Certainty (SC)Perturbation=small-eval2026.01 | — | 31.23 |