Math & Code Reasoning on ARC Easy
81.1ScoreLlama 3-8B
Evaluation Results
| Method | Links | |
|---|---|---|
| Llama 3-8BParadigm=AR, Training Tokens=15T, Training Data=Not Released, Evaluation protocol=Reported by prior work, Shots=02026.06 | 81.1 | |
| Llama 2-7BParadigm=AR, Training Tokens=2T, Training Data=Not Released, Evaluation protocol=Evaluated under our protocol, Shots=02026.06 | 73.8 | |
| LLaDA-8BParadigm=Masked Diffusion, Training Tokens=2.3T, Training Data=Not Released, Evaluation protocol=Reported by prior work, Shots=02026.06 | 71.8 | |
| Falcon-7BParadigm=AR, Training Tokens=1.5T, Training Data=Partially Released, Evaluation protocol=Evaluated under our protocol, Shots=02026.06 | 70.8 | |
| Sumi-7BParadigm=Uniform Diffusion, Training Tokens=1.5T, Training Data=Fully Released, Evaluation protocol=Evaluated under our protocol, Shots=02026.06 | 70 | |
| OLMo-7BParadigm=AR, Training Tokens=2.5T, Training Data=Fully Released, Evaluation protocol=Evaluated under our protocol, Shots=02026.06 | 68.8 | |
| DiReCTBackbone=Llama-1.1B2026.05 | 47 | |
| GradNorm (IS)Backbone=Llama-1.1B2026.05 | 44.3 | |
| InfoBatchBackbone=Llama-1.1B2026.05 | 43.7 | |
| Perplexity-basedBackbone=Llama-1.1B2026.05 | 42.4 | |
| Uniform SamplingBackbone=Llama-1.1B2026.05 | 41.3 | |
| Loss-basedBackbone=Llama-1.1B2026.05 | 41 | |
| DiReCTBackbone=GPT-2-Medium (355M)2026.05 | 34 | |
| InfoBatchBackbone=GPT-2-Medium (355M)2026.05 | 31.7 | |
| GradNorm (IS)Backbone=GPT-2-Medium (355M)2026.05 | 30.6 | |
| Perplexity-basedBackbone=GPT-2-Medium (355M)2026.05 | 29.8 | |
| Loss-basedBackbone=GPT-2-Medium (355M)2026.05 | 28.4 | |
| Uniform SamplingBackbone=GPT-2-Medium (355M)2026.05 | 27.2 |