Logical Reasoning on ARC-AGI
4.5Pass@1Random
Evaluation results
| Method | Links | |
|---|---|---|
| RandomBackbone=Qwen3-4B-Base, Seeds=12026.06 | 4.5 | |
| SECBackbone=Qwen3-4B-Base, Seeds=12026.06 | 4.2 | |
| TACBackbone=Qwen3-4B-Base, Seeds=12026.06 | 2.6 | |
| TACBackbone=Qwen3-0.6B-Base, Seeds=12026.06 | 0.5 | |
| RandomBackbone=Qwen3-0.6B-Base, Seeds=12026.06 | 0.3 | |
| SECBackbone=Qwen3-0.6B-Base, Seeds=12026.06 | 0.3 | |
| TACBackbone=Qwen3-1.7B2026.06 | 0.014 | |
| TACBackbone=Llama3.2-3B2026.06 | 0.012 | |
| RandomBackbone=Qwen3-1.7B2026.06 | 0.01 | |
| BaseBackbone=Llama3.2-3B2026.06 | 0.01 | |
| RandomBackbone=Llama3.2-3B2026.06 | 0.008 | |
| M2OBackbone=Qwen3-1.7B2026.06 | 0.006 | |
| SECBackbone=Qwen3-1.7B2026.06 | 0.005 | |
| M2OBackbone=Llama3.2-3B2026.06 | 0.004 | |
| BaseBackbone=Qwen3-1.7B2026.06 | 0.001 | |
| SECBackbone=Llama3.2-3B2026.06 | 0.001 |