Question Answering on ARC-c (Acc, Len)
96.82AccuracyZero-shot-EI
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Zero-shot-EIModel=Gemini-1.5-Pro, Model Role=Teacher Model2026.02 | 96.82 | — | |
| Zero-shotModel=Gemini-1.5-Pro, Model Role=Teacher Model2026.02 | 92 | — | |
| ExGRPOModel=Qwen2.5-7B-Instruct, Model Role=Student Model, Adapter=On-Policy Adapt.2026.02 | 91.78 | — | |
| ExGRPOModel=Qwen2.5-7B-Instruct, Model Role=Student Model2026.02 | 91.45 | — | |
| On-Policy DistillationModel=Qwen2.5-7B-Instruct, Model Role=Student Model2026.02 | 91.08 | — | |
| RevThinkModel=Qwen2.5-7B-Instruct, Model Role=Student Model2026.02 | 90.55 | — | |
| Divide-or-ConquerModel=Qwen2.5-7B-Instruct, Model Role=Student Model2026.02 | 90.12 | — | |
| Answer AugModel=Qwen2.5-7B-Instruct, Model Role=Student Model2026.02 | 90.1 | — | |
| Question AugModel=Qwen2.5-7B-Instruct, Model Role=Student Model2026.02 | 89.64 | — | |
| Rephrase QuestionModel=Qwen2.5-7B-Instruct, Model Role=Student Model2026.02 | 89.05 | — | |
| Distill Step-by-StepModel=Qwen2.5-7B-Instruct, Model Role=Student Model2026.02 | 88.4 | — | |
| SKDModel=Qwen2.5-7B-Instruct, Model Role=Student Model2026.02 | 87.26 | — | |
| Zero-shotModel=Qwen2.5-7B-Instruct, Model Role=Student Model2026.02 | 85.36 | — | |
| ReLIFT2025.06 | 81.6 | 1,321 | |
| RL2025.06 | 80.8 | 639 | |
| ExGRPOModel=Gemma-7B-it, Model Role=Student Model, Adapter=On-Policy Adapt.2026.02 | 80.67 | — | |
| LUFFY2025.06 | 80.4 | 1,168 | |
| ExGRPOModel=Gemma-7B-it, Model Role=Student Model2026.02 | 79.94 | — | |
| On-Policy DistillationModel=Gemma-7B-it, Model Role=Student Model2026.02 | 78.15 | — | |
| RevThinkModel=Gemma-7B-it, Model Role=Student Model2026.02 | 75.09 | — | |
| SFT2025.06 | 75 | 1,515 | |
| Divide-or-ConquerModel=Gemma-7B-it, Model Role=Student Model2026.02 | 74.23 | — | |
| Answer AugModel=Gemma-7B-it, Model Role=Student Model2026.02 | 73.92 | — | |
| SKDModel=Gemma-7B-it, Model Role=Student Model2026.02 | 73.29 | — | |
| Distill Step-by-StepModel=Gemma-7B-it, Model Role=Student Model2026.02 | 72.92 | — | |
| SFT then RL(v2)2025.06 | 72.8 | 980 | |
| Question AugModel=Gemma-7B-it, Model Role=Student Model2026.02 | 72.74 | — | |
| Rephrase QuestionModel=Gemma-7B-it, Model Role=Student Model2026.02 | 72.37 | — | |
| SFT then RL(v1)2025.06 | 72.3 | 879 | |
| Zero-shotModel=Gemma-7B-it, Model Role=Student Model2026.02 | 68.34 | — | |
| Qwen-Math-Instruct2025.06 | 65.7 | 2,881 | |
| RL w/ SFT Loss2025.06 | 63.5 | 1,814 | |
| Qwen-Math2025.06 | 50.3 | 504 | |
| FullBits=16, Size=30B, Evaluation Protocol=zero-shot2026.04 | 45.48 | — | |
| SEPTQBits=3, Size=30B, Evaluation Protocol=zero-shot2026.04 | 44.8 | — | |
| FullBits=16, Size=13B, Evaluation Protocol=zero-shot2026.04 | 44.53 | — | |
| QuIPBits=3, Size=30B, Evaluation Protocol=zero-shot2026.04 | 42.91 | — | |
| SEPTQBits=3, Size=13B, Evaluation Protocol=zero-shot2026.04 | 42.06 | — | |
| QuIPBits=3, Size=13B, Evaluation Protocol=zero-shot2026.04 | 41.55 | — | |
| FullBits=16, Size=7B, Evaluation Protocol=zero-shot2026.04 | 41.38 | — | |
| SEPTQBits=2, Size=30B, Evaluation Protocol=zero-shot2026.04 | 41.38 | — | |
| GPTQBits=3, Size=30B, Evaluation Protocol=zero-shot2026.04 | 40.44 | — | |
| SEPTQBits=3, Size=7B, Evaluation Protocol=zero-shot2026.04 | 39.33 | — | |
| GPTQBits=3, Size=13B, Evaluation Protocol=zero-shot2026.04 | 39.16 | — | |
| CARVE + SWAModel Category=Hybrid, Parameter Scale=1.3B, Token Scale=100B2026.06 | 38.84 | — | |
| CARVEModel Category=Recurrent, Parameter Scale=1.3B, Token Scale=100B2026.06 | 38.74 | — | |
| Mamba-3 MIMO + SWAModel Category=Hybrid, Parameter Scale=1.3B, Token Scale=100B2026.06 | 38.48 | — | |
| GDN-2Model Category=Recurrent, Parameter Scale=1.3B, Token Scale=100B2026.06 | 38.23 | — | |
| Mamba-3 MIMOModel Category=Recurrent, Parameter Scale=1.3B, Token Scale=100B2026.06 | 38.07 | — | |
| QuIPBits=3, Size=7B, Evaluation Protocol=zero-shot2026.04 | 37.63 | — | |
| SEPTQBits=2, Size=13B, Evaluation Protocol=zero-shot2026.04 | 37.12 | — | |
| GDN-2 + SWAModel Category=Hybrid, Parameter Scale=1.3B, Token Scale=100B2026.06 | 36.69 | — | |
| GPTQBits=3, Size=7B, Evaluation Protocol=zero-shot2026.04 | 36.43 | — | |
| Mamba-3 SISO + SWAModel Category=Hybrid, Parameter Scale=1.3B, Token Scale=100B2026.06 | 36.35 | — | |
| KDAModel Category=Recurrent, Parameter Scale=1.3B, Token Scale=100B2026.06 | 35.92 | — | |
| Mamba-2Model Category=Recurrent, Parameter Scale=1.3B, Token Scale=100B2026.06 | 35.26 | — | |
| Gated DeltaNetModel Category=Recurrent, Parameter Scale=1.3B, Token Scale=100B2026.06 | 35.15 | — | |
| Gated DeltaNet + SWAModel Category=Hybrid, Parameter Scale=1.3B, Token Scale=100B2026.06 | 35.15 | — | |
| KDA + SWAModel Category=Hybrid, Parameter Scale=1.3B, Token Scale=100B2026.06 | 35.07 | — | |
| Mamba-2 + SWAModel Category=Hybrid, Parameter Scale=1.3B, Token Scale=100B2026.06 | 34.73 | — | |
| Mamba-3 SISOModel Category=Recurrent, Parameter Scale=1.3B, Token Scale=100B2026.06 | 34.56 | — | |
| QuIPBits=2, Size=30B, Evaluation Protocol=zero-shot2026.04 | 34.22 | — | |
| TransformerModel Category=Hybrid, Parameter Scale=1.3B, Token Scale=100B2026.06 | 33.84 | — | |
| QuIPBits=2, Size=13B, Evaluation Protocol=zero-shot2026.04 | 31.74 | — | |
| SEPTQBits=2, Size=7B, Evaluation Protocol=zero-shot2026.04 | 28.49 | — | |
| GPTQBits=2, Size=7B, Evaluation Protocol=zero-shot2026.04 | 27.9 | — | |
| GPTQBits=2, Size=30B, Evaluation Protocol=zero-shot2026.04 | 27.73 | — | |
| QuIPBits=2, Size=7B, Evaluation Protocol=zero-shot2026.04 | 27.56 | — | |
| GPTQBits=2, Size=13B, Evaluation Protocol=zero-shot2026.04 | 27.56 | — |