Reasoning on ARC-AGI 2 (test)
24.9Pass@2 Exact Match AccuracyDRM (Ours)
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| DRM (Ours)#Params=14M, Pretraining Data=+NVARC Train2026.04 | 24.9 | — | — | — | |
| LLM (NVARC)#Params=4B, Pretraining Data=+NVARC Train2026.04 | 22.6 | — | — | — | |
| SPRM (Ours)#Params=14M, Pretraining Data=+NVARC Train2026.04 | 22.2 | — | — | — | |
| TRM (Baseline)#Params=14M, Pretraining Data=+NVARC Train2026.04 | 21.8 | — | — | — | |
| DRM (Ours)#Params=14M, Pretraining Data=+reARC2026.04 | 21 | — | — | — | |
| TRM (Baseline)#Params=14M, Pretraining Data=+reARC2026.04 | 18.8 | — | — | — | |
| SPRM (Ours)#Params=14M, Pretraining Data=+reARC2026.04 | 17.4 | — | — | — | |
| DRM (Ours)#Params=7M, Pretraining Data=+NVARC Train2026.04 | 16.7 | — | — | — | |
| LLM (NVARC)#Params=4B, Pretraining Data=+reARC2026.04 | 15.6 | — | — | — | |
| DRM (Ours)#Params=7M, Pretraining Data=+reARC2026.04 | 14.7 | — | — | — | |
| SPRM (Ours)#Params=7M, Pretraining Data=+NVARC Train2026.04 | 14.4 | — | — | — | |
| LLM (NVARC)#Params=4B, Pretraining Data=+ARC-Easy2026.04 | 13.8 | — | — | — | |
| DRM (Ours)#Params=14M, Pretraining Data=+ARC-Easy2026.04 | 13.3 | — | — | — | |
| TRM (Baseline)#Params=7M, Pretraining Data=+NVARC Train2026.04 | 12.5 | — | — | — | |
| TRM (Baseline)#Params=7M, Pretraining Data=+reARC2026.04 | 12.4 | — | — | — | |
| SPRM (Ours)#Params=7M, Pretraining Data=+reARC2026.04 | 11.8 | — | — | — | |
| SPRM (Ours)#Params=14M, Pretraining Data=+ARC-Easy2026.04 | 11.8 | — | — | — | |
| TRM (Baseline)#Params=14M, Pretraining Data=+ARC-Easy2026.04 | 10.6 | — | — | — | |
| SPRM (Ours)#Params=7M, Pretraining Data=+ARC-Easy2026.04 | 10.1 | — | — | — | |
| DRM (Ours)#Params=7M, Pretraining Data=+ARC-Easy2026.04 | 9.6 | — | — | — | |
| ViARC#Params=18M, Pretraining Data=+reARC2026.04 | 8.3 | — | — | — | |
| TRM (Baseline)#Params=7M, Pretraining Data=+ARC-Easy2026.04 | 6.3 | — | — | — | |
| TRM w/o deep sup#Params=7M, Pretraining Data=+ARC-Easy2026.04 | 3.3 | — | — | — | |
| TRM w/o inner loop#Params=34M, Pretraining Data=+ARC-Easy2026.04 | 3 | — | — | — | |
| Transformer w/ deep sup#Params=70M, Pretraining Data=+ARC-Easy2026.04 | 2.5 | — | — | — | |
| Transformer#Params=70M, Pretraining Data=+ARC-Easy2026.04 | 1.7 | — | — | — | |
| Masked Diffusion (Standard)#Params=7M, Pretraining Data=+ARC-Easy2026.04 | 0 | — | — | — | |
| Masked Diffusion (Standard)#Params=70M, Pretraining Data=+ARC-Easy2026.04 | 0 | — | — | — | |
| Claude Sonnet 4.52026.02 | — | 13.6 | 0.76 | — | |
| DeepSeek-R1-05282026.02 | — | 1.1 | 0.05 | — | |
| Empirical-MCTSBase Model=Gemini 3 Flash2026.02 | — | 38.33 | 0.97 | — | |
| Gemini 2.5 Pro2026.02 | — | 4.9 | 0.76 | — | |
| Gemini 3 Flash2026.02 | — | 26.75 | 0.21 | — | |
| Gemini 3 Pro2026.02 | — | 31.1 | 0.81 | — | |
| GPT-5.1 (High)2026.02 | — | 17.6 | 1.17 | — | |
| GPT-5.2 (High)2026.02 | — | 43.3 | 1.39 | — | |
| Grok 42026.02 | — | 16 | 2.17 | — | |
| Grok 4 Fast (Reasoning)2026.02 | — | 5.3 | 0.06 | — | |
| HRM2026.03 | — | — | — | 5 | |
| SE-RRM2026.03 | — | — | — | 7.1 | |
| TRM2026.03 | — | — | — | 7.8 |