Zero-shot Commonsense Reasoning Suite (HellaSwag, ARC-E, ARC-C, PIQA, Lambada) (test)
55.4HellaSwag AccuracySmolLM2-360M
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| SmolLM2-360MParams=360M, Tokens=4T2026.01 | 55.4 | 69.7 | 41.1 | 73.6 | 53.2 | 58.6 | |
| SmolLM-360MParams=360M, Tokens=600B2026.01 | 54 | 67.6 | 35.1 | 72.6 | 50.9 | 56 | |
| IMU-1Params=430M, Tokens=72B, Training Stage=Stage 3, EMA=true2026.01 | 51.1 | 71.4 | 42.8 | 70.2 | 51.3 | 57.4 | |
| IMU-1Params=430M, Tokens=72B, Training Stage=Stage 3, Iterations=265k2026.01 | 50.3 | 69.7 | 41.3 | 70.3 | 48.2 | 56 | |
| IMU-1Params=430M, Tokens=57B, Training Stage=Stage 2, Iterations=200k2026.01 | 45.1 | 68.9 | 38.1 | 68.2 | 40.5 | 52.2 | |
| SmolLM-135MParams=135M, Tokens=600B2026.01 | 42.2 | 57.6 | 27.6 | 69.3 | 30.4 | 45.4 | |
| IMU-1Params=430M, Tokens=29B, Training Stage=Stage 1, Iterations=100k2026.01 | 39.3 | 61.5 | 33.3 | 64.7 | 31.7 | 46.1 |