Zero-shot Evaluation on German LLM Evaluation Suite (MMLU, ARC, Hellaswag, LAMBADA, OBQA)
31.06MMLUDENSE-CORE 1B
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| DENSE-CORE 1BToken Count=28B (7.2x)2026.04 | 31.06 | 35.99 | 57.3 | 48.69 | 42.8 | 48.48 | 44.05 | |
| Gemma-3-1BToken Count=2T*2026.04 | 30.01 | 30.55 | 47.89 | 43.43 | 41.71 | 45.05 | 39.77 | |
| DENSE-CORE 1BToken Count=28B (3.6x)2026.04 | 29.88 | 32.99 | 54.64 | 45.51 | 40.29 | 48.49 | 41.97 | |
| LLäMmlein-1BToken Count=1T2026.04 | 29.26 | 30.27 | 48.19 | 44.8 | 44.89 | 47.27 | 40.78 | |
| DENSE-CORE 350MToken Count=28B (3.6x)2026.04 | 28.97 | 31.4 | 50.55 | 41.1 | 37.55 | 45.86 | 39.24 | |
| Llama-3.2-1BToken Count=9T*2026.04 | 28.58 | 29.9 | 40.51 | 40.07 | 44.31 | 44.04 | 37.9 | |
| RANDOM 1BToken Count=100B (1.0x)2026.04 | 27.28 | 27.46 | 43.97 | 41.6 | 38.84 | 41.82 | 36.83 | |
| RANDOM 350MToken Count=100B (1.0x)2026.04 | 27.13 | 26.15 | 41.1 | 37.18 | 33.52 | 41.01 | 34.35 |