Abstract Reasoning on ARC 1
79.6Pass@2Bespoke (Grok-4)
Evaluation Results
| Method | Links | |
|---|---|---|
| Bespoke (Grok-4)Regime=Chain-of-thought, pretrained, Params=1.7T2025.11 | 79.6 | |
| Grok-4-thinkingRegime=Chain-of-thought, pretrained, Params=1.7T2025.11 | 66.7 | |
| DISRegime=Small-sample training, Params=7M2025.11 | 41.3 | |
| TRMRegime=Small-sample training, Params=7M2025.11 | 40.4 | |
| DIS-mediumRegime=Small-sample training, Params=7M2025.11 | 40 | |
| Gemini 2.5 Pro 32KRegime=Chain-of-thought, pretrained, Params=?2025.11 | 37 | |
| o3-mini-highRegime=Chain-of-thought, pretrained, Params=?2025.11 | 34.5 | |
| Claude 3.7 16KRegime=Chain-of-thought, pretrained, Params=?2025.11 | 28.6 | |
| TRM-mediumRegime=Small-sample training, Params=7M2025.11 | 27.1 | |
| DIS-compactRegime=Small-sample training, Params=0.8M2025.11 | 24 | |
| Deepseek R1Regime=Chain-of-thought, pretrained, Params=671B2025.11 | 15.8 | |
| TRM-compactRegime=Small-sample training, Params=0.8M2025.11 | 12 |