Logical Reasoning on ARC Challenge (test)
95.7AccuracyEAGLE3
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| EAGLE3Base Model=Qwen3-4B-Thinking-25072026.02 | 95.7 | 1,822 | 164.2 | |
| ThinkBase Model=Qwen3-4B-Thinking-25072026.02 | 95.6 | 1,812 | 156.5 | |
| NoThink*Base Model=Qwen3-4B-Thinking-25072026.02 | 95.1 | 1,889 | 159.8 | |
| DEERBase Model=Qwen3-4B-Thinking-25072026.02 | 94.6 | 1,011 | 200.3 | |
| SpecExitBase Model=Qwen3-4B-Thinking-25072026.02 | 94.5 | 588 | 71.4 | |
| EAGLE3Base Model=DeepSeek-R1-Distill-Llama-8B2026.02 | 59.2 | 1,378 | 496.4 | |
| SpecExitBase Model=DeepSeek-R1-Distill-Llama-8B2026.02 | 50.3 | 500 | 253.7 | |
| VanillaBase Model=DeepSeek-R1-Distill-Llama-8B2026.02 | 49.9 | 1,917 | 628.5 | |
| DEERBase Model=DeepSeek-R1-Distill-Llama-8B2026.02 | 47.5 | 1,029 | 531.3 | |
| NoThinkBase Model=DeepSeek-R1-Distill-Llama-8B2026.02 | 12.6 | 135 | 13.6 |