Language Understanding on MMLU (LIS and Accuracy)
4.0509LISDeepSeek V2 Lite
Evaluation results
| Method | Links | ||
|---|---|---|---|
| DeepSeek V2 LiteStrategy=Raw2026.02 | 4.0509 | 0.55 | |
| DeepSeek-MoEStrategy=Raw2026.02 | 3.7359 | 0.447 | |
| DeepSeek V2 LiteStrategy=R&Q2026.02 | 3.0119 | 0.58 | |
| DeepSeek-MoEStrategy=R&Q2026.02 | 2.8222 | 0.447 | |
| Switch Transformer (16 experts)Strategy=Raw2026.02 | 1.9964 | 0.23 | |
| Switch Transformer (16 experts)Strategy=R&Q2026.02 | 1.7067 | 0.25 | |
| Switch Transformer (8 experts)Strategy=Raw2026.02 | 1.5405 | 0.23 | |
| LLaMa-MoEStrategy=Raw2026.02 | 1.3289 | 0.28 | |
| Switch Transformer (8 experts)Strategy=R&Q2026.02 | 1.2962 | 0.252 | |
| LLaMa-MoEStrategy=R&Q2026.02 | 1.1964 | 0.28 |