Multilingual NLP on HPLT Evaluation Set (da, de, en, fr, ga, hu, ur)
92.4Performance (da)XLM-R-Large
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| XLM-R-LargeParameters (M)=560, Layers=24, Latency Improvement (x)=1, Throughput Improvement (x)=1, Optim Energy (J/sample)=0.82, Energy Efficiency Improvement (x)=1.0x2025.12 | 92.4 | 90.5 | 92.4 | 95.1 | 83.1 | 81.6 | 89 | 89.2 | |
| XLM-R-BaseParameters (M)=278, Layers=12, Latency Improvement (x)=1.9, Throughput Improvement (x)=3.2, Optim Energy (J/sample)=0.25, Energy Efficiency Improvement (x)=3.3x2025.12 | 91 | 89.5 | 91.5 | 94.2 | 81 | 80 | 87.9 | 87.9 | |
| TiME-mParameters (M)=236, Layers=6, Latency Improvement (x)=3.9, Throughput Improvement (x)=5.4, Optim Energy (J/sample)=0.12, Energy Efficiency Improvement (x)=6.6x2025.12 | 89.9 | 88.4 | 91.1 | 93.2 | 82.4 | 80.8 | 88.1 | 87.7 | |
| HPLT (our eval)Parameters (M)=150, Layers=12, Latency Improvement (x)=1, Throughput Improvement (x)=1.6, Optim Energy (J/sample)=0.61, Energy Efficiency Improvement (x)=1.3x2025.12 | 88.8 | 89.4 | 91.1 | 93.7 | 80.3 | 70.9 | 75.6 | 84.3 | |
| TiME-sParameters (M)=107, Layers=6, Latency Improvement (x)=3.9, Throughput Improvement (x)=15.9, Optim Energy (J/sample)=0.04, Energy Efficiency Improvement (x)=18.7x2025.12 | 88.1 | 86.8 | 88.7 | 91.9 | 77.5 | 76.9 | 86.3 | 85.2 | |
| *-hplt-mParameters (M)=69, Layers=6, Latency Improvement (x)=3.8, Throughput Improvement (x)=4.1, Optim Energy (J/sample)=0.18, Energy Efficiency Improvement (x)=4.5x2025.12 | 88.1 | 87 | 90 | 91.8 | 80.2 | 77.8 | 74.5 | 84.2 | |
| TiME-xsParameters (M)=103, Layers=4, Latency Improvement (x)=5.8, Throughput Improvement (x)=25.2, Optim Energy (J/sample)=0.02, Energy Efficiency Improvement (x)=30.2x2025.12 | 86.2 | 84.8 | 87.6 | 91.1 | 75.8 | 76 | 86 | 83.9 | |
| mMiniLMParameters (M)=107, Layers=6, Variant=L6-H384 version, Latency Improvement (x)=3.5, Throughput Improvement (x)=16.9, Optim Energy (J/sample)=0.04, Energy Efficiency Improvement (x)=19.4x2025.12 | 86.1 | 84.4 | 88.5 | 91.4 | 70.2 | 75.1 | 85.3 | 83.5 |