Multi-task Language Understanding on MMLU-Pro (Accuracy, Latency, and SLO)
61AccuracyINFRAMIND
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| INFRAMINDLoad level=10 req/min2026.06 | 61 | 44 | 99.8 | |
| INFRAMINDLoad level=50 req/min2026.06 | 60 | 319 | 59.8 | |
| INFRAMINDLoad level=100 req/min2026.06 | 59.8 | 372 | 56 | |
| MASRouterLoad level=50 req/min2026.06 | 58.9 | 821 | 14.7 | |
| MASRouterLoad level=10 req/min2026.06 | 58.5 | 83 | 98.8 | |
| GPTSwarmLoad level=10 req/min2026.06 | 57.8 | 36 | 100 | |
| GPTSwarmLoad level=100 req/min2026.06 | 57.4 | 753 | 11 | |
| MASRouterLoad level=100 req/min2026.06 | 57 | 908 | 13.4 | |
| GPTSwarmLoad level=50 req/min2026.06 | 55.4 | 563 | 18.4 | |
| MoALoad level=100 req/min2026.06 | 47.8 | 1,171 | 0.8 | |
| MoALoad level=50 req/min2026.06 | 46.2 | 1,240 | 2 | |
| MoALoad level=10 req/min2026.06 | 41.6 | 124 | 84.2 |