Topic Classification on SIB200
79.95AccuracySpectrum 6B-3B
Evaluation results
| Method | Links | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Spectrum 6B-3BZero-shot=true2026.03 | 79.95 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Llama 3.1 8B InstructZero-shot=true2026.03 | 66.07 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Llama 3.2 3B InstructZero-shot=true2026.03 | 65.69 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SmolLM2-1.7b-Instructtuning=Instruction-tuned, prompt_language=English2025.12 | 61.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Gemma-3-1b-ittuning=Instruction-tuned, prompt_language=English2025.12 | 55.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MiniLingua-1bprompting=Zero-shot, data_tokens=1.5T, gpu_hours=75K2025.12 | 24.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SmolLM2-1.7bprompting=Zero-shot, data_tokens=11T, gpu_hours=not reported2025.12 | 24 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| EuroLLM-1.7bprompting=Zero-shot, data_tokens=4T, gpu_hours=500K2025.12 | 23 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Gemma-3-1b-ptprompting=Zero-shot, data_tokens=2T, gpu_hours=∼300K2025.12 | 23 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MiniLingua-1b-Instructtuning=Instruction-tuned, prompt_language=English2025.12 | 14.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| EuroLLM-1.7b-Instructtuning=Instruction-tuned, prompt_language=English2025.12 | 12.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GPT-2Params=124M, Patch Size=NA2026.04 | — | — | — | — | — | — | 20.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | 24.6 | — | 19.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 16.9 | — | — | — | — | — | — | — | 42.2 | 44.6 | 22.3 | 36.4 | 39.4 | 79.4 | 38.4 | 28.6 | 50.2 | 47.2 | — | 17.3 | 20.3 | 33.3 | — | 17 | 20.5 | — | 19.5 | 20.6 | — | 16.5 | 16.8 | 17 | — | 20 | 26 | |
| Granite 4 MicroVariant=Base2026.01 | — | 26.4 | 84.6 | 81.1 | 85.1 | 78.1 | 85.6 | 77.1 | 64.2 | 60.2 | 80.6 | 72.6 | 85.6 | 69.2 | 73.6 | 62.2 | 68.2 | 67.2 | 72.6 | 51.2 | 75.1 | 63.2 | 27.9 | 83.1 | 81.1 | 49.3 | 65.7 | 63.2 | 83.1 | 65.2 | 59.7 | 75.1 | 64.2 | 49.8 | 51.7 | 59.7 | 57.2 | 78.6 | 58.2 | 77.1 | 69.2 | 57.2 | 78.6 | 73.6 | 71.6 | 79.1 | 86.6 | 23.4 | 71.1 | 75.1 | 71.6 | 60.7 | 62.2 | 55.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| KakugoVariant=GenReas+Tran (full)2026.01 | — | 51.2 | 83.6 | 85.1 | 85.6 | 80.6 | 85.6 | 80.6 | 78.1 | 76.6 | 82.1 | 78.6 | 87.1 | 77.1 | 83.6 | 72.6 | 75.1 | 73.6 | 83.1 | 73.6 | 81.1 | 67.7 | 47.3 | 83.1 | 88.1 | 68.2 | 69.7 | 81.1 | 84.1 | 74.6 | 72.6 | 79.6 | 74.1 | 75.6 | 75.1 | 64.7 | 79.6 | 82.6 | 73.6 | 76.1 | 77.1 | 68.7 | 81.6 | 74.6 | 78.1 | 83.6 | 86.1 | 47.8 | 83.1 | 81.6 | 76.1 | 68.7 | 67.2 | 71.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MGPT-2Params=124M, Patch Size=NA2026.04 | — | — | — | — | — | — | 17.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | 23.4 | — | 20.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 18.2 | — | — | — | — | — | — | — | 65.9 | 81.1 | 61.1 | 69.4 | 81.7 | 87.7 | 81.7 | 81.6 | 81.2 | 82.8 | 77.8 | 44.9 | 45.2 | 40.5 | — | 23 | 26.3 | — | 25.3 | 25.4 | — | 17.6 | 24.1 | 14.9 | — | 26.2 | 42.9 | |
| MIXAR stage2Params=116M, Patch Size=32×322026.04 | — | — | — | — | — | — | 21.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | 22.6 | — | 24.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 22.7 | — | — | — | — | — | — | — | 20.4 | 31.9 | 22.1 | 24.8 | 89.9 | 83 | 84.1 | 88.7 | 86.5 | 86.4 | 63.3 | 42.6 | 40.7 | 45.3 | — | 24 | 21.6 | — | 18.2 | 16 | — | 23.8 | 15.2 | 22.8 | — | 25.8 | 37.4 | |
| PIXAR stage2Params=113M, Patch Size=8×82026.04 | — | — | — | — | — | — | 15.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | 25.4 | — | 12 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 21.9 | — | — | — | — | — | — | — | 11 | 25.9 | 16.6 | 17.8 | 40.6 | 81.5 | 52.3 | 53.8 | 60.5 | 57.7 | — | 29.2 | 39.7 | 42.9 | — | 24.9 | 22 | — | 20.7 | 16.8 | — | 23.1 | 24.4 | 19.9 | — | 24.2 | 28.5 |