Expert-level Question Answering on GPQA Diamond
66.65Pass@1DART
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| DARTBackbone=Qwen3-4B-Thinking-2507, Paradigm=RL, CoT Type=tool-integrated long2026.01 | 66.65 | 83.75 | 3.81 | — | |
| UTIRBackbone=Qwen3-4B-Thinking-2507, Paradigm=RL, CoT Type=tool-integrated long2026.01 | 66.08 | 81.9 | 2.11 | — | |
| Original checkpointBackbone=Qwen3-4B-Thinking-2507, CoT Type=pure-text long2026.01 | 65.98 | 83.32 | 0 | — | |
| DAPOBackbone=Qwen3-4B-Thinking-2507, Paradigm=RL, CoT Type=pure-text long2026.01 | 65.53 | 82.82 | 0 | — | |
| ToRL-ThinkBackbone=Qwen3-4B-Thinking-2507, Paradigm=RL, CoT Type=pure-text long2026.01 | 65.15 | 74.99 | 0 | — | |
| DARTBackbone=Qwen3-8B, Paradigm=RL, CoT Type=tool-integrated long2026.01 | 60.03 | 81.57 | 4.14 | — | |
| UTIRBackbone=Qwen3-8B, Paradigm=RL, CoT Type=tool-integrated long2026.01 | 59.7 | 80.71 | 2.74 | — | |
| DAPOBackbone=Qwen3-8B, Paradigm=RL, CoT Type=pure-text long2026.01 | 59.41 | 80.29 | 0 | — | |
| Original checkpointBackbone=Qwen3-8B, CoT Type=pure-text long2026.01 | 58.88 | 79.53 | 0 | — | |
| STARTBackbone=Qwen3-4B-Thinking-2507, Paradigm=SFT, CoT Type=tool-integrated long2026.01 | 55.82 | 82.22 | 4.24 | — | |
| ToRL-ThinkBackbone=Qwen3-8B, Paradigm=RL, CoT Type=pure-text long2026.01 | 55.24 | 67.11 | 0 | — | |
| ToRL-BaseBackbone=Qwen3-8B, Paradigm=RL, CoT Type=tool-integrated short2026.01 | 41.56 | 56.62 | 1.06 | — | |
| SCR-Stage IBackbone=Qwen2.5-3B-Instruct2026.01 | 39.39 | — | — | — | |
| SCR (Ours)Backbone=Qwen2.5-3B-Instruct2026.01 | 39.39 | — | — | — | |
| SCR (Ours)Backbone=Qwen2.5-7B-Instruct2026.01 | 38.38 | — | — | — | |
| GRPOBackbone=Qwen2.5-7B-Instruct2026.01 | 37.88 | — | — | — | |
| GRPOBackbone=Qwen2.5-3B-Instruct2026.01 | 36.36 | — | — | — | |
| SFT + GRPOBackbone=Qwen2.5-3B-Instruct2026.01 | 36.36 | — | — | — | |
| STARTBackbone=Qwen3-8B, Paradigm=SFT, CoT Type=tool-integrated long2026.01 | 35.89 | 67.61 | 5.79 | — | |
| SFT + GRPOBackbone=Qwen2.5-7B-Instruct2026.01 | 35.86 | — | — | — | |
| SCR-Stage IBackbone=Qwen2.5-7B-Instruct2026.01 | 35.35 | — | — | — | |
| Self-RefineBackbone=Qwen2.5-7B-Instruct2026.01 | 34.85 | — | — | — | |
| BaseBackbone=Qwen2.5-7B-Instruct2026.01 | 33.84 | — | — | — | |
| SCR-SFTBackbone=Qwen2.5-7B-Instruct2026.01 | 33.84 | — | — | — | |
| Self-RefineBackbone=Qwen2.5-3B-Instruct2026.01 | 32.32 | — | — | — | |
| SCR-Stage IBackbone=Llama3.1-8B-Instruct2026.01 | 31.82 | — | — | — | |
| ToRL-BaseBackbone=Qwen3-4B-Thinking-2507, Paradigm=RL, CoT Type=tool-integrated short2026.01 | 31.67 | 43.65 | 1.01 | — | |
| GRPOBackbone=Llama3.1-8B-Instruct2026.01 | 31.31 | — | — | — | |
| BaseBackbone=Qwen2.5-3B-Instruct2026.01 | 29.8 | — | — | — | |
| SCR-SFTBackbone=Qwen2.5-3B-Instruct2026.01 | 27.27 | — | — | — | |
| SCR (Ours)Backbone=Llama3.1-8B-Instruct2026.01 | 27.27 | — | — | — | |
| BaseBackbone=Llama3.1-8B-Instruct2026.01 | 24.75 | — | — | — | |
| SCR-SFTBackbone=Llama3.1-8B-Instruct2026.01 | 24.75 | — | — | — | |
| SFT + GRPOBackbone=Llama3.1-8B-Instruct2026.01 | 24.24 | — | — | — | |
| Self-RefineBackbone=Llama3.1-8B-Instruct2026.01 | 22.73 | — | — | — | |
| Retool (only SFT)Backbone=Qwen3-8B, Paradigm=SFT, CoT Type=tool-integrated short2026.01 | 21.23 | 46.32 | 1.16 | — | |
| Retool (only SFT)Backbone=Qwen3-4B-Thinking-2507, Paradigm=SFT, CoT Type=tool-integrated short2026.01 | 20.12 | 42.06 | 1.12 | — | |
| RetoolBackbone=Qwen3-8B, Paradigm=SFT+RL, CoT Type=tool-integrated short2026.01 | 18.27 | 36.04 | 0.92 | — | |
| RetoolBackbone=Qwen3-4B-Thinking-2507, Paradigm=SFT+RL, CoT Type=tool-integrated short2026.01 | 13.26 | 20.95 | 0.98 | — | |
| Mellum 2Parameters=2.5B/12B2026.05 | — | — | — | 31.3 | |
| OLMo-3-7BParameters=7B2026.05 | — | — | — | 28.8 | |
| Qwen2.5-7BParameters=7B2026.05 | — | — | — | 32.8 | |
| Qwen3-4BParameters=4B2026.05 | — | — | — | 36.9 | |
| Qwen3.5-4BParameters=4B2026.05 | — | — | — | 41.4 |