Instruction Following on IFBench (Throughput Metrics)
2,688LLM Throughput (tokens/s)FlashEvolve
Evaluation results
| Method | Links | ||
|---|---|---|---|
| FlashEvolveDeployment=vLLM, Model=Qwen3-8B2026.05 | 2,688 | 0.1483 | |
| GEPADeployment=vLLM, Model=Qwen3-8B2026.05 | 963 | 0.0317 | |
| CombeeDeployment=vLLM, Model=Qwen3-8B, K (parallelization size)=402026.05 | 900 | 0.0117 | |
| FlashEvolveDeployment=API, Model=GPT-4o-mini2026.05 | 791 | 0.1683 | |
| CombeeDeployment=vLLM, Model=Qwen3-8B, K (parallelization size)=102026.05 | 696 | 0.02 | |
| CombeeDeployment=API, Model=GPT-4o-mini, K (parallelization size)=102026.05 | 397 | 0.0167 | |
| CombeeDeployment=API, Model=GPT-4o-mini, K (parallelization size)=402026.05 | 389 | 0.0133 | |
| GEPADeployment=API, Model=GPT-4o-mini2026.05 | 361 | 0.0283 |