Abstraction and Reasoning on 1D-ARC
397AccuracyIO-Prompt
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| IO-PromptBase model=gpt-4o-2024-, Temperature=0.72026.04 | 397 | — | — | |
| Auto CoT-PromptBase model=gpt-4o-2024-, Temperature=0.72026.04 | 346 | — | — | |
| Minitron + DIARC#Params=6.9B2026.06 | 58.82 | — | — | |
| Minitron + SFT only#Params=6.9B2026.06 | 57.05 | — | — | |
| Qwen-2.5#Params=7B2026.06 | 55 | — | — | |
| Llama-3.2 + DIARC#Params=2.6B2026.06 | 52.39 | — | — | |
| GPT-4#Params=N/A2026.06 | 52 | — | — | |
| Llama-3.2 + SFT only#Params=2.6B2026.06 | 51.72 | — | — | |
| Qwen3 + DIARC#Params=3.6B2026.06 | 45.95 | — | — | |
| Qwen3 + SFT only#Params=3.6B2026.06 | 42.84 | — | — | |
| Minitron#Params=8B2026.06 | 3.55 | — | — | |
| Llama-3.2#Params=3B2026.06 | 0.11 | — | — | |
| Qwen3#Params=4B2026.06 | 0.11 | — | — |