Code Generation on HumanEval (Accuracy, Latency, SLO)
100Accuracy (%)INFRAMIND
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| INFRAMINDLoad level=10 req/min2026.06 | 100 | 5 | 100 | |
| GPTSwarmLoad level=10 req/min2026.06 | 100 | 42 | 100 | |
| INFRAMINDLoad level=50 req/min2026.06 | 100 | 8 | 100 | |
| GPTSwarmLoad level=50 req/min2026.06 | 100 | 134 | 100 | |
| MoALoad level=50 req/min2026.06 | 100 | 155 | 100 | |
| INFRAMINDLoad level=100 req/min2026.06 | 100 | 33 | 99.9 | |
| GPTSwarmLoad level=100 req/min2026.06 | 100 | 153 | 100 | |
| MoALoad level=100 req/min2026.06 | 100 | 177 | 95.4 | |
| MoALoad level=10 req/min2026.06 | 99.2 | 49 | 100 | |
| MASRouterLoad level=10 req/min2026.06 | 93.1 | 70 | 99.2 | |
| MASRouterLoad level=50 req/min2026.06 | 93.1 | 195 | 77.1 | |
| MASRouterLoad level=100 req/min2026.06 | 89.3 | 205 | 71 |