Language Understanding on HellaSwag, PIQA, BoolQ, ARC-Challenge, and OpenBookQA
42.9HellaSwag AccuracySmolLM2-135M-Instruct
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| SmolLM2-135M-InstructSize (M)=1352026.04 | 42.9 | 67.1 | 43.1 | 28 | 33.4 | 42.9 | |
| pythia-160mSize (M)=1602026.04 | 31.3 | 62.2 | 48.6 | 24.1 | 28.2 | 38.9 | |
| LaMini-GPT-124MSize (M)=1242026.04 | 30.8 | 63 | 53.3 | 22.9 | 26.6 | 39.3 | |
| LaMini-Neo-125MSize (M)=1252026.04 | 30.6 | 62.5 | 58.2 | 23.5 | 28.4 | 40.6 | |
| µLM-512-8Size (M)=28.852026.04 | 28.5 | 57.2 | 59.8 | 25.2 | 24.2 | 39 | |
| µLM-384-16Size (M)=29.502026.04 | 28.5 | 53.5 | 61.3 | 23.3 | 25.6 | 38.4 | |
| LaMini-Cerebras-256MSize (M)=2562026.04 | 28.5 | 61.5 | 60.1 | 24.2 | 28.6 | 40.6 | |
| µLM-384-8Size (M)=17.112026.04 | 28.2 | 54.6 | 62.1 | 23.8 | 25 | 38.7 | |
| pythia-70mSize (M)=702026.04 | 27.9 | 58 | 60.9 | 21.2 | 25.8 | 38.8 | |
| µLM-256-8Size (M)=8.792026.04 | 27.6 | 52.9 | 43.4 | 24.8 | 23.6 | 34.5 | |
| µLM-256-16Size (M)=14.432026.04 | 26.8 | 54.2 | 60.5 | 26.1 | 25.2 | 38.6 |