General Reasoning on Super GPQA
71.1AccuracyGemini 2.5 Pro
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Gemini 2.5 ProInput Modality=Text, LLM-as-a-Judge=GPT-4o2025.11 | 71.1 | — | |
| Claude Sonnet 4.5Input Modality=Text, LLM-as-a-Judge=GPT-4o2025.11 | 69 | — | |
| GPT-5 highInput Modality=Text, LLM-as-a-Judge=GPT-4o2025.11 | 68.3 | — | |
| Sora-2 Last FrameInput Modality=Last Frame, LLM-as-a-Judge=GPT-4o2025.11 | 53.2 | — | |
| ROSA2Model=Qwen3-8B2026.03 | 52.4 | — | |
| ROSA2Model=Qwen2.5-7B-Base2026.03 | 48.8 | — | |
| ROSAModel=Qwen2.5-7B-Base2026.03 | 47.8 | — | |
| TextGradModel=Qwen2.5-7B-Base2026.03 | 46.4 | — | |
| Sora-2 AudioInput Modality=Audio, LLM-as-a-Judge=GPT-4o2025.11 | 44.5 | — | |
| ROSAModel=Qwen2.5-7B-Instruct, Update Location=LM Head, Reward Model=Model-based2025.09 | 40.67 | — | |
| ROSAModel=Qwen3-8B, Update Location=LM Head, Reward Model=Model-based2025.09 | 40.34 | — | |
| TextGradModel=Qwen3-8B2026.03 | 40 | — | |
| ROSAModel=Qwen3-8B2026.03 | 38.6 | — | |
| ROSAModel=Qwen3-8B, Update Location=Hidden States, Reward Model=Rule-based2025.09 | 37.73 | — | |
| WISTBackbone=Qwen3-14B-Base2026.03 | 37.4 | — | |
| ROSAModel=Qwen2.5-7B-Instruct, Update Location=LM Head, Reward Model=Rule-based2025.09 | 37.26 | — | |
| R-ZeroBackbone=Qwen3-14B-Base2026.03 | 37.1 | — | |
| SPICEBackbone=Qwen3-14B-Base2026.03 | 37.1 | — | |
| ROSAModel=Qwen2.5-7B-Instruct, Update Location=Hidden States, Reward Model=Rule-based2025.09 | 36.27 | — | |
| ROSAModel=Qwen3-8B, Update Location=LM Head, Reward Model=Rule-based2025.09 | 36.11 | — | |
| SPICEBackbone=Qwen3-8B-Base, Methodology=SPICE, Supervision Ratio=None2025.12 | 35.7 | — | |
| General-ReasonerBackbone=Qwen3-8B-Base, Methodology=General-Reasoner, Supervision Ratio=232k WebInstruct2025.12 | 35.3 | — | |
| Base ModelBackbone=Qwen3-14B-Base2026.03 | 35.2 | — | |
| ROSA2Model=DeepSeek-R1-Distill-Llama-8B2026.03 | 35 | — | |
| Absolute ZeroBackbone=Qwen3-8B-Base, Methodology=Absolute Zero, Supervision Ratio=None2025.12 | 33.5 | — | |
| R-Few (5%)Backbone=Qwen3-8B-Base, Methodology=R-Few, Supervision Ratio=5%2025.12 | 33.5 | — | |
| R-Few (1%)Backbone=Qwen3-8B-Base, Methodology=R-Few, Supervision Ratio=1%2025.12 | 32.7 | — | |
| General-ReasonerBackbone=Qwen3-4B-Base, Methodology=General-Reasoner, Supervision Ratio=232k WebInstruct2025.12 | 32.5 | — | |
| WISTBackbone=Qwen3-8B-Base2026.03 | 32.5 | — | |
| R-ZeroBackbone=Qwen3-8B-Base, Methodology=R-Zero, Supervision Ratio=None2025.12 | 31.8 | — | |
| R-ZeroBackbone=Qwen3-8B-Base2026.03 | 31.8 | — | |
| CoT Cold-Start + Solver FeedbackModel=Qwen3-4B-Base2025.11 | 31.33 | — | |
| SPICEBackbone=Qwen3-8B-Base2026.03 | 31.3 | — | |
| Self-Instruct + Solver FeedbackModel=Qwen3-4B-Base2025.11 | 31.29 | — | |
| Self-Instruct + R-ZeroModel=Qwen3-4B-Base2025.11 | 31.27 | — | |
| CoT Cold-StartModel=Qwen3-4B-Base2025.11 | 30.65 | — | |
| Base ModelBackbone=Qwen3-8B-Base, Methodology=Base, Supervision Ratio=None2025.12 | 30.4 | — | |
| Self-Instruct + CoT-Self-InstructModel=Qwen3-4B-Base2025.11 | 30.35 | — | |
| SPICEBackbone=Qwen3-4B-Base, Methodology=SPICE, Supervision Ratio=None2025.12 | 30.2 | — | |
| Self-InstructModel=Qwen3-4B-Base2025.11 | 30.07 | — | |
| Seed SetModel=Qwen3-4B-Base2025.11 | 29.74 | — | |
| CoT Cold-Start + RLMTModel=Qwen3-4B-Base2025.11 | 29.7 | — | |
| WISTBackbone=Qwen3-4B-Base2026.03 | 29.6 | — | |
| R-Few (1%)Backbone=Qwen3-4B-Base, Methodology=R-Few, Supervision Ratio=1%2025.12 | 29.4 | — | |
| R-Few (5%)Backbone=Qwen3-4B-Base, Methodology=R-Few, Supervision Ratio=5%2025.12 | 29.4 | — | |
| NEMOTRON-CROSSTHINKCategory=Data Source, Blend=Bgpr↑2025.04 | 29.16 | — | |
| CoT Cold-Start + R-ZeroModel=Qwen3-4B-Base2025.11 | 29.15 | — | |
| Base ModelBackbone=Qwen3-8B-Base2026.03 | 28.3 | — | |
| NEMOTRON-CROSSTHINKCategory=Question Types, Blend=Bmcq↑2025.04 | 28.05 | — | |
| R-ZeroBackbone=Qwen3-4B-Base, Methodology=R-Zero, Supervision Ratio=None2025.12 | 27.8 | — | |
| SPICEBackbone=Qwen3-4B-Base2026.03 | 27.8 | — | |
| NEMOTRON-CROSSTHINKCategory=Single Source, Blend=Bonly_mr2025.04 | 27.69 | — | |
| BaselineModel=Qwen3-8B2025.09 | 27.61 | — | |
| ORZModel=ORZ2025.04 | 27.6 | — | |
| NEMOTRON-CROSSTHINKCategory=Single Source, Blend=Bonly_gpr2025.04 | 27.44 | — | |
| NEMOTRON-CROSSTHINKCategory=Data Usefulness, Blend=Bscore2025.04 | 27.37 | — | |
| Absolute ZeroBackbone=Qwen3-4B-Base, Methodology=Absolute Zero, Supervision Ratio=None2025.12 | 27.1 | — | |
| NEMOTRON-CROSSTHINKCategory=Question Types, Blend=Bopen↑2025.04 | 26.82 | — | |
| NEMOTRON-CROSSTHINKCategory=Data Source, Blend=Bmr↑2025.04 | 26.81 | — | |
| R-ZeroBackbone=Qwen3-4B-Base2026.03 | 26.7 | — | |
| NEMOTRON-CROSSTHINKBlend=Bnd2025.04 | 26.54 | — | |
| Base ModelBackbone=Qwen3-4B-Base, Methodology=Base, Supervision Ratio=None2025.12 | 25.4 | — | |
| WISTBackbone=OctoThinker-8B-Hybrid-Base2026.03 | 25.4 | — | |
| MModel=M2025.04 | 25.36 | — | |
| ROSAModel=DeepSeek-R1-Distill-Llama-8B, Update Location=LM Head, Reward Model=Model-based2025.09 | 24.97 | — | |
| BaselineModel=Qwen3-8B2026.03 | 24.2 | — | |
| R-ZeroBackbone=OctoThinker-8B-Hybrid-Base2026.03 | 24.1 | — | |
| SPICEBackbone=OctoThinker-8B-Hybrid-Base2026.03 | 23.3 | — | |
| ROSAModel=DeepSeek-R1-Distill-Llama-8B2026.03 | 21.4 | — | |
| ROSAModel=DeepSeek-R1-Distill-Llama-8B, Update Location=Hidden States, Reward Model=Rule-based2025.09 | 21.34 | — | |
| Base ModelModel=Qwen3-4B-Base2025.11 | 20.88 | — | |
| TextGradModel=DeepSeek-R1-Distill-Llama-8B2026.03 | 20.8 | — | |
| ROSAModel=DeepSeek-R1-Distill-Llama-8B, Update Location=LM Head, Reward Model=Rule-based2025.09 | 20.49 | — | |
| BaselineModel=Qwen2.5-7B-Instruct2025.09 | 19.31 | — | |
| SPICEBackbone=OctoThinker-3B-Hybrid-Base2026.03 | 18.4 | — | |
| Base ModelBackbone=Qwen3-4B-Base2026.03 | 18 | — | |
| BaselineModel=Qwen2.5-7B-Base2026.03 | 17.8 | — | |
| WISTBackbone=OctoThinker-3B-Hybrid-Base2026.03 | 17.8 | — | |
| Base ModelBackbone=OctoThinker-8B-Hybrid-Base2026.03 | 16.4 | — | |
| ROSAModel=Qwen3-0.6B, Update Location=LM Head, Reward Model=Model-based2025.09 | 15.73 | — | |
| R-ZeroBackbone=OctoThinker-3B-Hybrid-Base2026.03 | 12.6 | — | |
| BaselineModel=DeepSeek-R1-Distill-Llama-8B2026.03 | 10.4 | — | |
| Base ModelBackbone=OctoThinker-3B-Hybrid-Base2026.03 | 10.4 | — | |
| BaselineModel=DeepSeek-R1-Distill-Llama-8B2025.09 | 10.37 | — | |
| ROSAModel=Qwen3-0.6B, Update Location=Hidden States, Reward Model=Rule-based2025.09 | 9.7 | — | |
| ROSAModel=Qwen3-0.6B, Update Location=LM Head, Reward Model=Rule-based2025.09 | 9.13 | — | |
| ROSAModel=Qwen2.5-0.5B-Instruct, Update Location=LM Head, Reward Model=Model-based2025.09 | 6.57 | — | |
| ROSA2Model=Qwen2.5-0.5B-Instruct2026.03 | 6.4 | — | |
| ROSA2Model=Qwen3-0.6B-Instruct2026.03 | 6.4 | — | |
| ROSAModel=Qwen2.5-0.5B-Instruct, Update Location=LM Head, Reward Model=Rule-based2025.09 | 5.63 | — | |
| ROSAModel=Qwen2.5-0.5B-Instruct, Update Location=Hidden States, Reward Model=Rule-based2025.09 | 5 | — | |
| BaselineModel=Qwen3-0.6B2025.09 | 4.2 | — | |
| ROSAModel=Qwen2.5-0.5B-Instruct2026.03 | 4 | — | |
| ROSAModel=Qwen3-0.6B-Instruct2026.03 | 4 | — | |
| BaselineModel=Qwen2.5-0.5B-Instruct2026.03 | 3.8 | — | |
| TextGradModel=Qwen2.5-0.5B-Instruct2026.03 | 3.8 | — | |
| BaselineModel=Qwen3-0.6B-Instruct2026.03 | 3.8 | — | |
| TextGradModel=Qwen3-0.6B-Instruct2026.03 | 3.8 | — | |
| BaselineModel=Qwen2.5-0.5B-Instruct2025.09 | 1.9 | — | |
| Absolute ZeroBackbone Model=Qwen3-4B-Base2026.02 | — | 27.1 |