LLM Throughput on AIME (Mathematical Reasoning)
998Throughput (token/s)FlashEvolve
Evaluation results
| Method | Links | ||
|---|---|---|---|
| FlashEvolveDeployment=vLLM, Model=Qwen3-8B2026.05 | 998 | 11.4 | |
| CombeeDeployment=vLLM, Model=Qwen3-8B, K (parallelization size)=102026.05 | 994 | 6.2 | |
| CombeeDeployment=vLLM, Model=Qwen3-8B, K (parallelization size)=402026.05 | 977 | 1.6 | |
| FlashEvolveDeployment=API, Model=GPT-4o-mini2026.05 | 485 | 6.6 | |
| CombeeDeployment=API, Model=GPT-4o-mini, K (parallelization size)=402026.05 | 336 | 0.6 | |
| CombeeDeployment=API, Model=GPT-4o-mini, K (parallelization size)=102026.05 | 211 | 1 | |
| GEPADeployment=vLLM, Model=Qwen3-8B2026.05 | 200 | 2.2 | |
| GEPADeployment=API, Model=GPT-4o-mini2026.05 | 103 | 1.3 |