Abstract Reasoning on ARC-AGI 2 (pass@k)
77.1Pass@2Gemini 3.1-Pro
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Gemini 3.1-Pro2026.06 | 77.1 | — | — | |
| Claude Opus-4.72026.06 | 75.8 | — | — | |
| OpenAI GPT-5.42026.06 | 74 | — | — | |
| Claude Opus-4.62026.06 | 68.8 | — | — | |
| Ring-2.6-1Tvariant=xhigh2026.06 | 66.18 | — | — | |
| DS-V4-Pro2026.06 | 62.43 | — | — | |
| Bespoke (Grok-4)Regime=Chain-of-thought, pretrained, Params=1.7T2025.11 | 29.4 | — | — | |
| Kimi-K2.62026.06 | 29.38 | — | — | |
| GLM-5.12026.06 | 21.74 | — | — | |
| Grok-4-thinkingRegime=Chain-of-thought, pretrained, Params=1.7T2025.11 | 16 | — | — | |
| Standard TRM, our reproduction# Params=7M, augmentation-voting pipeline=true, architecture=TRM-Att2026.05 | 9.72 | 7.36 | 14.31 | |
| PTRM# Params=7M, K (stochastic rollouts)=25, D (depth)=16, sigma (exploration parameter)=0.2, augmentation-voting pipeline=true, architecture=TRM-Att2026.05 | 9.72 | 8.47 | 15.97 | |
| TRM# Params=7M, architecture=TRM-Att2026.05 | 7.8 | — | — | |
| DISRegime=Small-sample training, Params=7M2025.11 | 6 | — | — | |
| HRM# Params=27M2026.05 | 5 | — | — | |
| Gemini 2.5 Pro 32KRegime=Chain-of-thought, pretrained, Params=?2025.11 | 4.9 | — | — | |
| TRMRegime=Small-sample training, Params=7M2025.11 | 3.3 | — | — | |
| o3-mini-highRegime=Chain-of-thought, pretrained, Params=?2025.11 | 3 | — | — | |
| DIS-mediumRegime=Small-sample training, Params=7M2025.11 | 2.5 | — | — | |
| Deepseek R1Regime=Chain-of-thought, pretrained, Params=671B2025.11 | 1.3 | — | — | |
| Claude 3.7 16KRegime=Chain-of-thought, pretrained, Params=?2025.11 | 0.7 | — | — | |
| TRM-compactRegime=Small-sample training, Params=0.8M2025.11 | 0 | — | — | |
| DIS-compactRegime=Small-sample training, Params=0.8M2025.11 | 0 | — | — | |
| TRM-mediumRegime=Small-sample training, Params=7M2025.11 | 0 | — | — |