Language Modeling on peS2o (Diagnostic Suite)
8.52PPLMoEUT
Evaluation Results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| MoEUTnumber of parameters=244M, zero-shot=true2024.05 | 8.52 | 19.4 | 73.5 | 77.4 | 30.1 | 56.3 | 35.6 | 48.7 | |
| Baselinenumber of parameters=244M, zero-shot=true2024.05 | 8.55 | 18.7 | 72.8 | 78 | 30.4 | 56.3 | 35 | 48.5 | |
| MoEUTnumber of parameters=44M, zero-shot=true2024.05 | 11.09 | 13.1 | 68.7 | 69.6 | 28.3 | 55.1 | 31.4 | 44.4 | |
| Baselinenumber of parameters=44M, zero-shot=true2024.05 | 11.46 | 13.2 | 66.5 | 68.6 | 28.5 | 56.3 | 32 | 44.2 |