VRP optimization-modeling on VRP Optimization-Modeling Benchmark 4
96.4Pass@1Gemini-3.1-Pro Preview
Evaluation Results
| Method | Links | |
|---|---|---|
| Gemini-3.1-Pro PreviewModel type=Closed-source frontier LLM2026.06 | 96.4 | |
| VRPCoder-GRPOModel scale=8B, Stage=GRPO2026.06 | 88 | |
| VRPCoder-SFTModel scale=8B, Stage=SFT2026.06 | 85.2 | |
| Claude-Sonnet-4.5Model type=Closed-source frontier LLM2026.06 | 58.8 | |
| DeepSeek-V3.2Model type=Open-source general-purpose LLM2026.06 | 45.6 | |
| MiniMax-M2.5Model type=Open-source general-purpose LLM2026.06 | 32.8 | |
| SIRL-Gurobi-32BModel type=OR-LLM2026.06 | 30 | |
| Qwen2.5-72B-InstructModel type=Open-source general-purpose LLM2026.06 | 27.2 | |
| SIRL-Gurobi-8BModel type=OR-LLM2026.06 | 13.2 | |
| OptMATH-Qwen2.5-7BModel type=OR-LLM2026.06 | 8.4 | |
| ORLM-LLaMA-3-8BModel type=OR-LLM2026.06 | 7.6 | |
| Qwen3-8BModel type=Open-source general-purpose LLM2026.06 | 1.2 |