Abstract Reasoning on ARC-AGI v1 (test)
98AccuracyBest Human
Evaluation results
| Method | Links | |
|---|---|---|
| Best Human2026.05 | 98 | |
| Gemini 3 Pro#Params=N/A2026.05 | 75 | |
| Grok-4-thinking#Params=1.7T2026.05 | 66.7 | |
| Avg. Human2026.05 | 60.2 | |
| GPT 5.2 (low)#Params=N/A2026.05 | 55.7 | |
| GRAM#Params=10M, Supervision steps=162026.05 | 52 | |
| TRM#Params=7M, Supervision steps=162026.05 | 44.6 | |
| SePO-GeneralistTask Agent=DeepSeek-V3.2, Prompt Agent=Gemini 3.1 Pro Preview2026.06 | 43.39 | |
| HRM#Params=27M, Supervision steps=162026.05 | 40.3 | |
| SePO-SpecialistTask Agent=DeepSeek-V3.2, Prompt Agent=Gemini 3.1 Pro Preview2026.06 | 37.46 | |
| Manual-CoTTask Agent=DeepSeek-V3.2, Prompt Agent=Gemini 3.1 Pro Preview2026.06 | 37.3 | |
| MetaSPOTask Agent=DeepSeek-V3.2, Prompt Agent=Gemini 3.1 Pro Preview2026.06 | 37.27 | |
| TextGradTask Agent=DeepSeek-V3.2, Prompt Agent=Gemini 3.1 Pro Preview2026.06 | 34.75 | |
| o3-mini-high#Params=N/A2026.05 | 34.5 | |
| Claude 3.7 16k#Params=N/A2026.05 | 28.6 | |
| Direct Pred#Params=27M, Supervision steps=162026.05 | 21 | |
| Deepseek-R1#Params=671B2026.05 | 15.8 |