General Reasoning on GPQA Diamond (pass@1 accuracy)
86.4Pass@1 Accuracygemini-2.5-pro
Evaluation Results
| Method | Links | |
|---|---|---|
| gemini-2.5-pro# Params=–, Scale Category=Large Scale (> 70B)2026.03 | 86.4 | |
| o4-mini (high)# Params=–, Scale Category=Large Scale (> 70B)2026.03 | 81.4 | |
| Qwen3-235B-A22B-Thinking-2507# Params=235B, Scale Category=Large Scale (> 70B)2026.03 | 81.1 | |
| DeepSeek-R1-0528# Params=671B, Scale Category=Large Scale (> 70B)2026.03 | 81 | |
| o3-mini (medium)# Params=–, Scale Category=Large Scale (> 70B)2026.03 | 76.8 | |
| DeepSeek-R1# Params=671B, Scale Category=Large Scale (> 70B)2026.03 | 71.5 | |
| Qwen3-235B-A22B# Params=235B, Scale Category=Large Scale (> 70B)2026.03 | 71.1 | |
| Qwen3-4B-Thinking-2507 + CHIMERA# Params=4B, Scale Category=Small to Medium Scale (≤ 70B)2026.03 | 70.1 | |
| Qwen3-32B# Params=32B, Scale Category=Small to Medium Scale (≤ 70B)2026.03 | 68.4 | |
| Qwen3-4B-Thinking-2507# Params=4B, Scale Category=Small to Medium Scale (≤ 70B)2026.03 | 65.8 | |
| DeepSeek-R1-Distill-Llama-70B# Params=70B, Scale Category=Small to Medium Scale (≤ 70B)2026.03 | 65.2 | |
| DeepSeek-R1-0528-Qwen3-8B# Params=8B, Scale Category=Small to Medium Scale (≤ 70B)2026.03 | 61.1 | |
| CLPOOptimization Policy=Restructuring-Aware RL, Base Model=Qwen3-8B2025.09 | 56.06 | |
| Qwen3-4B-Thinking-2507 + OpenScience# Params=4B, Scale Category=Small to Medium Scale (≤ 70B)2026.03 | 53.5 | |
| ACTTraining Data=ALFWorld agentic data2026.03 | 53.37 | |
| RLTraining Data=ALFWorld agentic data2026.03 | 52.36 | |
| Early Experience (Self-Reflection)Training Data=ALFWorld agentic data with self-reflection2026.03 | 51.85 | |
| Prompt w/ CoT thinkingReasoning Mode=Chain of Thought2026.03 | 51.52 | |
| Critique-GRPO (CoT)Optimization Policy=Critique-Driven RL, Base Model=Qwen3-8B2025.09 | 50.5 | |
| LUFFYOptimization Policy=Off-Policy Imitation, Base Model=Qwen3-8B2025.09 | 49.49 | |
| Critique-GRPO (Simple)Optimization Policy=Critique-Driven RL, Base Model=Qwen3-8B2025.09 | 48.63 | |
| DAPOOptimization Policy=Dynamic Sampling, Base Model=Qwen3-8B2025.09 | 48.48 | |
| GRPOOptimization Policy=Group-Based RL, Base Model=Qwen3-8B2025.09 | 47.8 | |
| Imitation LearningTraining Data=ALFWorld agentic data2026.03 | 44.61 | |
| Prompt w/o CoT thinkingReasoning Mode=Direct Answer2026.03 | 42.93 | |
| TTSRBackbone Model=Qwen3-8B-Base2026.02 | 42.6 | |
| R-ZeroBackbone Model=Qwen3-8B-Base2026.02 | 40.5 | |
| AEROBackbone Model=Qwen3-8B-Base, Evolutionary Round=R42026.02 | 38.4 | |
| TTRLBackbone Model=Qwen3-8B-Base2026.02 | 38.4 | |
| AEROBackbone Model=Qwen3-8B-Base, Evolutionary Round=R52026.02 | 36.9 | |
| Absolute ZeroBackbone Model=Qwen3-8B-Base2026.02 | 36.8 | |
| RAFTOptimization Policy=Ranking-Based Imitation, Base Model=Qwen3-8B2025.09 | 36.76 | |
| R-ZeroBackbone Model=Qwen3-8B-Base2026.02 | 36.7 | |
| R-ZeroBackbone Model=Qwen3-4B-Base2026.02 | 36.4 | |
| CITL-FTOptimization Policy=Mixed-Data SFT, Base Model=Qwen3-8B2025.09 | 36.36 | |
| AEROBackbone Model=Qwen3-8B-Base, Evolutionary Round=R22026.02 | 35.9 | |
| Absolute ZeroBackbone Model=Qwen3-4B-Base2026.02 | 35.3 | |
| Critique-FTOptimization Policy=Learning to Critique, Base Model=Qwen3-8B2025.09 | 34.84 | |
| Refinement-FTOptimization Policy=Guided Refinement, Base Model=Qwen3-8B2025.09 | 34.47 | |
| AEROBackbone Model=Qwen3-4B-Base, Evolutionary Round=R42026.02 | 34.3 | |
| AEROBackbone Model=Qwen3-4B-Base, Evolutionary Round=R52026.02 | 34.3 | |
| AEROBackbone Model=Qwen3-8B-Base, Evolutionary Round=R32026.02 | 34.3 | |
| TTSRBackbone Model=Qwen3-4B-Base2026.02 | 34.2 | |
| AEROBackbone Model=Qwen3-8B-Base, Evolutionary Round=R12026.02 | 33.8 | |
| AEROBackbone Model=Qwen3-4B-Base, Evolutionary Round=R32026.02 | 33.3 | |
| Qwen3-8B-BaseBackbone Model=Qwen3-8B-Base2026.02 | 33.3 | |
| Base ModelBackbone Model=Qwen3-8B-Base2026.02 | 33.1 | |
| AEROBackbone Model=Qwen3-4B-Base, Evolutionary Round=R22026.02 | 32.3 | |
| TTRLBackbone Model=Qwen3-4B-Base2026.02 | 29.1 | |
| R-ZeroBackbone Model=Qwen3-4B-Base2026.02 | 28.4 | |
| TTSRBackbone Model=OctoThinker-8B-Hybrid-Base2026.02 | 27.9 | |
| Qwen3-4B-BaseBackbone Model=Qwen3-4B-Base2026.02 | 26.3 | |
| AEROBackbone Model=Qwen3-4B-Base, Evolutionary Round=R12026.02 | 26.3 | |
| Base ModelBackbone Model=Qwen3-4B-Base2026.02 | 25.7 | |
| TTRLBackbone Model=OctoThinker-8B-Hybrid-Base2026.02 | 21.5 | |
| R-ZeroBackbone Model=OctoThinker-8B-Hybrid-Base2026.02 | 19.7 | |
| Base ModelBackbone Model=OctoThinker-8B-Hybrid-Base2026.02 | 15.2 |