Operations Research Problem Solving on IndustryOR (test)
75Success Rate (SA)ORPilot
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| ORPilotBase LLM=Claude Snonnet 4.6, Solver Backend=pulp2026.05 | 75 | 31 | |
| ORPilotBase LLM=Deepseek-Reasoner, Solver Backend=pulp2026.05 | 74 | 421.8 | |
| LLMOPTBase LLM=Qwen1.5-14B2026.05 | 46 | 8.35 | |
| OptiMUS0.3Base LLM=o12026.05 | 46 | — | |
| ORPilotBase LLM=GPT-4o, Solver Backend=pulp2026.05 | 46 | 16.2 | |
| ORLMBase LLM=LLaMa3-8B2026.05 | 38 | — | |
| OptiMUS0.3Base LLM=GPT-4o2026.05 | 37 | — | |
| ORLMBase LLM=Deepseek-Math2026.05 | 33 | — |