Graduate-level Science Reasoning on GPQA
82.34Accuracyphi-balancing
Evaluation results
| Method | Links | ||
|---|---|---|---|
| phi-balancingModel=DeepSeek-MoE-Chat2026.05 | 82.34 | — | |
| phi-balancingModel=Moonlight-16B-A3B-Instruct2026.05 | 81.92 | — | |
| ST-MoEModel=DeepSeek-MoE-Chat2026.05 | 79.7 | — | |
| ST-MoEModel=Moonlight-16B-A3B-Instruct2026.05 | 78.59 | — | |
| NanoV3-30B*Model Scale=30B, Active Parameters=3.6A, Teacher Model Status=True2026.05 | 73.11 | — | |
| phi-balancingModel=DeepSeek-V2-Lite2026.05 | 72.91 | — | |
| NanoV3 Elastic-30BModel Scale=30B, Active Parameters=3.6A, Teacher Model Status=False2026.05 | 72.1 | — | |
| Qwen3-30B-A3BModel Scale=30B, Active Parameters=3.3A, Teacher Model Status=False2026.05 | 70.83 | — | |
| NanoV3 Elastic-23BModel Scale=23B, Active Parameters=2.8A, Teacher Model Status=False2026.05 | 69.82 | — | |
| Apriel-Reasoner (Ours)Size=15B2026.04 | 69.8 | 5,800 | |
| In-placeModel=Qwen, Number of turns=92025.10 | 69 | — | |
| Apriel-Base + RLVR w/ LPSize=15B, Length Penalty (LP)=true2026.04 | 68.9 | 4,500 | |
| Apriel-BaseSize=15B2026.04 | 68.8 | 10,500 | |
| ST-MoEModel=DeepSeek-V2-Lite2026.05 | 68.67 | — | |
| Nemotron-CascadeSize=14B2026.04 | 68.4 | 10,600 | |
| In-placeModel=Qwen, Number of turns=82025.10 | 66.7 | — | |
| Phi-4-reasoningSize=14B2026.04 | 64.8 | 3,500 | |
| Qwen3Size=14B2026.04 | 64.3 | 6,700 | |
| In-placeModel=Qwen, Number of turns=72025.10 | 64.3 | — | |
| In-placeModel=Qwen, Number of turns=62025.10 | 63.5 | — | |
| In-placeModel=Llama, Number of turns=92025.10 | 61.9 | — | |
| In-placeModel=Llama, Number of turns=82025.10 | 61.1 | — | |
| In-placeModel=Qwen, Number of turns=52025.10 | 57.9 | — | |
| NanoV3 Elastic-12BModel Scale=12B, Active Parameters=2.0A, Teacher Model Status=False2026.05 | 57.39 | — | |
| In-placeModel=Llama, Number of turns=72025.10 | 55.6 | — | |
| In-placeModel=Gemma, Number of turns=92025.10 | 53.2 | — | |
| In-placeModel=Llama, Number of turns=62025.10 | 53.2 | — | |
| In-placeModel=Qwen, Number of turns=42025.10 | 51.6 | — | |
| Bf16Backbone=DeepSeek-R1-Distill-Llama-8B, Quantization=Bf162025.02 | 51.52 | — | |
| BaselineModel=Qwen3-8B2026.05 | 51.3 | — | |
| KIVI-4Backbone=DeepSeek-R1-Distill-Llama-8B, Quantization=KIVI-42025.02 | 51.01 | — | |
| PolarQuant44Backbone=DeepSeek-R1-Distill-Llama-8B, Quantization=PolarQuant442025.02 | 50 | — | |
| ScaleSearchModel=Qwen3-8B2026.05 | 49.9 | — | |
| ZipCache-4Backbone=DeepSeek-R1-Distill-Llama-8B, Quantization=ZipCache-42025.02 | 48.48 | — | |
| In-placeModel=Qwen, Number of turns=32025.10 | 48.4 | — | |
| In-placeModel=Gemma, Number of turns=82025.10 | 47.6 | — | |
| In-placeModel=Llama, Number of turns=52025.10 | 47.6 | — | |
| Multi-turnModel=Qwen, Number of turns=92025.10 | 46 | — | |
| In-placeModel=Gemma, Number of turns=62025.10 | 46 | — | |
| In-placeModel=Gemma, Number of turns=72025.10 | 46 | — | |
| Multi-turnModel=Qwen, Number of turns=62025.10 | 43.7 | — | |
| Multi-turnModel=Qwen, Number of turns=72025.10 | 43.7 | — | |
| Multi-turnModel=Qwen, Number of turns=82025.10 | 43.7 | — | |
| Multi-turnModel=Llama, Number of turns=92025.10 | 43.7 | — | |
| In-placeModel=Llama, Number of turns=42025.10 | 43.7 | — | |
| Multi-turnModel=Qwen, Number of turns=52025.10 | 42.9 | — | |
| In-placeModel=Gemma, Number of turns=52025.10 | 42.9 | — | |
| NVFP4Model=Qwen3-8B2026.05 | 42.4 | — | |
| Multi-turnModel=Llama, Number of turns=82025.10 | 42.1 | — | |
| Multi-turnModel=Qwen, Number of turns=42025.10 | 40.5 | — | |
| MemoryModel=Llama, Number of turns=92025.10 | 40.5 | — | |
| Bf16Backbone=DeepSeek-R1-Distill-Qwen-1.5B, Quantization=Bf162025.02 | 39.9 | — | |
| MemoryModel=Qwen, Number of turns=92025.10 | 38.9 | — | |
| In-placeModel=Gemma, Number of turns=42025.10 | 38.9 | — | |
| Multi-turnModel=Qwen, Number of turns=32025.10 | 38.1 | — | |
| MemoryModel=Qwen, Number of turns=72025.10 | 38.1 | — | |
| MemoryModel=Qwen, Number of turns=82025.10 | 38.1 | — | |
| Multi-turnModel=Llama, Number of turns=72025.10 | 38.1 | — | |
| PolarQuant44Backbone=DeepSeek-R1-Distill-Qwen-1.5B, Quantization=PolarQuant442025.02 | 37.88 | — | |
| MemoryModel=Qwen, Number of turns=62025.10 | 37.3 | — | |
| MemoryModel=Llama, Number of turns=82025.10 | 37.3 | — | |
| MemoryModel=Qwen, Number of turns=52025.10 | 36.5 | — | |
| In-placeModel=Qwen, Number of turns=22025.10 | 36.5 | — | |
| In-placeModel=Llama, Number of turns=32025.10 | 35.7 | — | |
| MemoryModel=Qwen, Number of turns=42025.10 | 34.9 | — | |
| Multi-turnModel=Llama, Number of turns=62025.10 | 34.1 | — | |
| MemoryModel=Llama, Number of turns=72025.10 | 34.1 | — | |
| KIVI-4Backbone=DeepSeek-R1-Distill-Qwen-1.5B, Quantization=KIVI-42025.02 | 33.84 | — | |
| In-placeModel=Gemma, Number of turns=32025.10 | 33.3 | — | |
| BaselineModel=DeepSeek-R1-Distill-Qwen-1.5B2026.05 | 32.6 | — | |
| Majority ConsensusBackbone=Llama 8B2026.07 | 32.3 | — | |
| MemoryModel=Llama, Number of turns=62025.10 | 31.7 | — | |
| ScaleSearchModel=DeepSeek-R1-Distill-Qwen-1.5B2026.05 | 31.3 | — | |
| Cumulative VotingBackbone=Llama 8B2026.07 | 31.3 | — | |
| Approval VotingBackbone=Llama 8B2026.07 | 31.3 | — | |
| Multi-turnModel=Llama, Number of turns=52025.10 | 31 | — | |
| Supermajority Con.Backbone=Llama 8B2026.07 | 30.7 | — | |
| Simple VotingBackbone=Llama 8B2026.07 | 30.5 | — | |
| NVFP4Model=DeepSeek-R1-Distill-Qwen-1.5B2026.05 | 30.4 | — | |
| Unanimity Con.Backbone=Llama 8B2026.07 | 30 | — | |
| Baseline with CoTBackbone=Llama 8B2026.07 | 29.9 | — | |
| COACTTrained on=WebInstruct, Evaluation=Out-of-domain2026.04 | 29.51 | — | |
| MemoryModel=Qwen, Number of turns=32025.10 | 29.4 | — | |
| Frozen checkpointModel=DeepSeek-MoE-Chat2026.05 | 28.91 | — | |
| BaselineBackbone=Llama 8B2026.07 | 28.9 | — | |
| Pref + EntTrained on=WebInstruct, Evaluation=Out-of-domain2026.04 | 28.76 | — | |
| EntropyTrained on=WebInstruct, Evaluation=Out-of-domain2026.04 | 28.12 | — | |
| Frozen checkpointModel=Moonlight-16B-A3B-Instruct2026.05 | 27.98 | — | |
| Multi-turnModel=Qwen, Number of turns=22025.10 | 27.8 | — | |
| In-placeModel=Gemma, Number of turns=22025.10 | 27.8 | — | |
| MemoryModel=Llama, Number of turns=52025.10 | 27.8 | — | |
| JudgeBackbone=Llama 8B2026.07 | 27.6 | — | |
| Pref CertaintyTrained on=WebInstruct, Evaluation=Out-of-domain2026.04 | 27.45 | — | |
| Ranked VotingBackbone=Llama 8B2026.07 | 27.3 | — | |
| MemoryModel=Qwen, Number of turns=22025.10 | 27 | — | |
| Multi-turnModel=Gemma, Number of turns=92025.10 | 27 | — | |
| Multi-turnModel=Llama, Number of turns=42025.10 | 27 | — | |
| RandomTrained on=WebInstruct, Evaluation=Out-of-domain2026.04 | 26.83 | — | |
| In-placeModel=Llama, Number of turns=22025.10 | 26.2 | — | |
| In-placeModel=Qwen, Number of turns=12025.10 | 25.4 | — |