General Reasoning on MMLU Pro, Super GPQA, GPQA Diamond, and BBEH Suite
84MMLU ProDeepSeek-R1
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| DeepSeek-R1Model Group=Baselines2026.03 | 84 | 59.9 | 71.5 | 34.9 | — | 62.6 | |
| GPT-4oModel Group=Baselines2026.03 | 74.6 | 46.3 | 50 | 22.3 | — | 48.3 | |
| OpenSIRBackbone=Qwen3-8B, Training Method=OpenSIR2025.11 | 69.05 | 39.84 | — | 18.3 | 42.4 | — | |
| Webinstruct-V (SFT only)Model Group=4B Models (Webinstruct-verified), Training Protocol=SFT only2026.03 | 68.6 | 38.4 | 46.8 | 13.5 | — | 41.8 | |
| Webinstruct-V (SFT then selected SFT)Model Group=4B Models (Webinstruct-verified), Training Protocol=SFT then selected SFT2026.03 | 68.6 | 38.2 | 45.6 | 13 | — | 41.4 | |
| Webinstruct-V (SFT then random RL)Model Group=4B Models (Webinstruct-verified), Training Protocol=SFT then random RL2026.03 | 68.6 | 39.4 | 47.8 | 15.8 | — | 42.9 | |
| Webinstruct-V (SFT easy + RL hard)Model Group=4B Models (Webinstruct-verified), Training Protocol=SFT easy + RL hard2026.03 | 68.4 | 40.2 | 50 | 16.7 | — | 43.8 | |
| GRPOmathBackbone=Qwen3-8B, Training Method=GRPOmath2025.11 | 65.34 | 35.43 | — | 13.49 | 38.09 | — | |
| General-ReasonerBackbone=Qwen3-8B-Base, Training Strategy=Supervised Reference2026.01 | 65.1 | 35.3 | 42.9 | 10.8 | 38.5 | 56 | |
| R-ZeroBackbone=Qwen3-8B, Training Method=R-Zero2025.11 | 64.68 | 33.19 | — | 12.53 | 36.8 | — | |
| BaseBackbone=Qwen3-8B, Training Method=Base2025.11 | 64.57 | 34.06 | — | 15.35 | 37.99 | — | |
| General-ReasonerBackbone=Qwen3-4B-Base, Training Strategy=Supervised Reference2026.01 | 62.8 | 32.5 | 42.9 | 12.2 | 37.6 | 54.3 | |
| R-Few (1%)Backbone=Qwen3-8B-Base2026.01 | 62.8 | 32.7 | 40.4 | 11.8 | 36.9 | 55.1 | |
| Webinstruct-V (RL only)Model Group=4B Models (Webinstruct-verified), Training Protocol=RL only2026.03 | 62.8 | 32.5 | 42.9 | 12.2 | — | 37.6 | |
| GRPOgsm8kBackbone=Qwen3-8B, Training Method=GRPOgsm8k2025.11 | 62.53 | 34.03 | — | 14.31 | 36.96 | — | |
| Absolute ZeroBackbone=Qwen3-8B, Training Method=Absolute Zero2025.11 | 62.51 | 33.96 | — | 15.41 | 37.29 | — | |
| Absolute ZeroBackbone=Qwen3-8B-Base2026.01 | 62.5 | 33.5 | 36.8 | 10.8 | 35.9 | 52.8 | |
| DARCBackbone=Qwen3-8B-Base2026.01 | 62.3 | 32.8 | 44.4 | 11.8 | 37.8 | 56.3 | |
| R-ZeroBackbone=Qwen3-8B-Base2026.01 | 61.6 | 31.8 | 40.5 | 11.3 | 36.3 | 53.7 | |
| SPICEBackbone=Qwen3-8B-Base2026.01 | 61 | 32.4 | 40.4 | 12.1 | 36.5 | 53.9 | |
| Webscale (SFT only)Model Group=4B Models (Webscale-RL), Training Protocol=SFT only2026.03 | 60.7 | 37.3 | 39.2 | 13.4 | — | 37.7 | |
| Webscale (SFT easy + RL hard)Model Group=4B Models (Webscale-RL), Training Protocol=SFT easy + RL hard2026.03 | 60.3 | 38.8 | 43.7 | 15.7 | — | 39.6 | |
| Open-Reasoner-ZeroModel Group=Baselines2026.03 | 59.4 | 32.8 | 36.6 | 12.2 | — | 35.3 | |
| Base ModelBackbone=Qwen3-8B-Base2026.01 | 58 | 30.4 | 33.3 | 10.5 | 33.1 | 49.9 | |
| Qwen2.5-7B-InstructModel Group=Baselines2026.03 | 57 | 30.7 | 33.8 | 12.2 | — | 33.4 | |
| DARCBackbone=Qwen3-4B-Base2026.01 | 56.9 | 29.2 | 38.9 | 11.2 | 34.1 | 52.2 | |
| SPICEBackbone=Qwen3-4B-Base2026.01 | 56.5 | 28.3 | 37.9 | 11.3 | 33.5 | 50.2 | |
| R-Few (1%)Backbone=Qwen3-4B-Base2026.01 | 55.9 | 29.4 | 35.4 | 11.2 | 33 | 49.9 | |
| Webscale (RL only)Model Group=4B Models (Webscale-RL), Training Protocol=RL only2026.03 | 55.4 | 30.9 | 34 | 10.1 | — | 32.6 | |
| R-ZeroBackbone=Qwen3-4B-Base2026.01 | 54.2 | 27.8 | 36.4 | 10.4 | 32.2 | 48.2 | |
| Absolute ZeroBackbone=Qwen3-4B-Base2026.01 | 52.6 | 27.1 | 35.3 | 8.3 | 30.8 | 46.9 | |
| QwQ-32BModel Group=Baselines2026.03 | 52 | 43.6 | 54.5 | 22.6 | — | 43.2 | |
| Base ModelBackbone=Qwen3-4B-Base2026.01 | 51.6 | 25.4 | 26.3 | 8.1 | 27.9 | 41.9 | |
| Qwen3-4B-BaseModel Group=4B Models (Webinstruct-verified)2026.03 | 51.6 | 25.4 | 26.5 | 8.1 | — | 27.9 | |
| SimpleRL-Qwen2.5-7B-ZooModel Group=Baselines2026.03 | 51.5 | 29.9 | 24.2 | 11.9 | — | 29.4 | |
| Qwen2.5-7B-BaseModel Group=Baselines2026.03 | 47.7 | 26.7 | 29.3 | 8 | — | 27.9 | |
| OpenSIRBackbone=DeepSeek-R1-Distill-Llama-8B, Training Method=OpenSIR2025.11 | 44.18 | 23.59 | — | 11.45 | 26.41 | — | |
| DARCBackbone=OctoThinker-8B-Hybrid-Base2026.01 | 43.8 | 22.3 | 32.3 | 10.8 | 27.3 | 40.6 | |
| GRPOgsm8kBackbone=DeepSeek-R1-Distill-Llama-8B, Training Method=GRPOgsm8k2025.11 | 41.35 | 18.72 | — | 7.17 | 22.41 | — | |
| SPICEBackbone=OctoThinker-8B-Hybrid-Base2026.01 | 41.3 | 19.9 | 29.8 | 7.2 | 24.5 | 38.4 | |
| R-ZeroBackbone=DeepSeek-R1-Distill-Llama-8B, Training Method=R-Zero2025.11 | 40.45 | 15.45 | — | 7.29 | 21.06 | — | |
| GRPOmathBackbone=DeepSeek-R1-Distill-Llama-8B, Training Method=GRPOmath2025.11 | 40.42 | 19.24 | — | 8.32 | 22.66 | — | |
| BaseBackbone=DeepSeek-R1-Distill-Llama-8B, Training Method=Base2025.11 | 40.33 | 17.38 | — | 7.15 | 21.62 | — | |
| Absolute ZeroBackbone=DeepSeek-R1-Distill-Llama-8B, Training Method=Absolute Zero2025.11 | 39.37 | 16.4 | — | 6.04 | 20.6 | — | |
| R-ZeroBackbone=OctoThinker-8B-Hybrid-Base2026.01 | 37.4 | 17.9 | 21.7 | 7.8 | 21.2 | 35.2 | |
| Absolute ZeroBackbone=OctoThinker-8B-Hybrid-Base2026.01 | 31.4 | 18.8 | 27.8 | 5 | 20.8 | 35.5 | |
| Base ModelBackbone=OctoThinker-8B-Hybrid-Base2026.01 | 14.7 | 11.4 | 15.7 | 0.6 | 10.6 | 24.6 |