General Language Understanding on 8 Sub-Tasks (test)
62.3Performance on 8 Sub-TasksLoRA
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| LoRARatio=0%, # Params=0.96B2026.03 | 62.3 | 52 | |
| OriginalRatio=0%, # Params=-2026.03 | 61.8 | 26.2 | |
| MoBERatio=30%, # Params=0.96B2026.03 | 59.7 | 53.8 | |
| LightMoERatio=30%, # Params=0.96B2026.03 | 59.2 | 51.4 | |
| HC-SMoERatio=30%, # Params=0.96B2026.03 | 57.8 | 49.8 | |
| MoBERatio=40%, # Params=0.96B2026.03 | 57.6 | 49 | |
| LightMoERatio=40%, # Params=0.96B2026.03 | 57.3 | 50.9 | |
| MC-SMoERatio=30%, # Params=0.96B2026.03 | 57 | 49.3 | |
| HC-SMoERatio=40%, # Params=0.96B2026.03 | 55.1 | 48.9 | |
| MC-SMoERatio=40%, # Params=0.96B2026.03 | 54.7 | 48.4 | |
| LightMoERatio=50%, # Params=0.96B2026.03 | 54.3 | 48.4 | |
| HC-SMoERatio=50%, # Params=0.96B2026.03 | 53 | 45.8 | |
| MC-SMoERatio=50%, # Params=0.96B2026.03 | 51.3 | 40.4 | |
| MoBERatio=50%, # Params=0.96B2026.03 | 50.7 | 38.6 |