Numerical Reasoning on Countdown-4
98.9CD4Self-Aware Markov Models
Evaluation results
| Method | Links | |
|---|---|---|
| Self-Aware Markov ModelsParams=34M, Training steps=7 · 10^6, Model architecture type=Diffusion2026.03 | 98.9 | |
| Self-Aware Markov ModelsParams=34M, Model architecture type=Diffusion2026.03 | 95.9 | |
| Self-Aware Markov Models (Small variant)Params=11M, Model architecture type=Diffusion2026.03 | 92.1 | |
| MGDMParams=85M, Model architecture type=Diffusion2026.03 | 91.5 | |
| DFMParams=34M, Model architecture type=Diffusion2026.03 | 87.5 | |
| RDMParams=85M, Model architecture type=Diffusion2026.03 | 87 | |
| D3PMParams=85M, Model architecture type=Diffusion2026.03 | 83.1 | |
| VDMParams=85M, Model architecture type=Diffusion2026.03 | 73.4 | |
| Stream-of-SearchParams=250M, Model architecture type=Autoregressive2026.03 | 54.2 | |
| LLaMAParams=13B, Model architecture type=Autoregressive2026.03 | 51.1 | |
| GPT-2 ScratchParams=85M, Model architecture type=Autoregressive2026.03 | 45.8 | |
| GPT-2 ScratchParams=303M, Model architecture type=Autoregressive2026.03 | 41.3 | |
| LLaMAParams=7B, Model architecture type=Autoregressive2026.03 | 41.1 |