Operations Research Modeling and Solving on NLP4LP
97.2Accuracy (pass@1)EvoOptiGraph
Evaluation results
| Method | Links | |
|---|---|---|
| EvoOptiGraphParams=8B, Category=Fine-tuned LLMs2026.06 | 97.2 | |
| DeepSeek-V3Params=671B, Category=Zero-shot LLMs2026.06 | 94.9 | |
| DCM-AgentBackbone=GPT5.12026.04 | 93.39 | |
| Qwen3-32BParams=32B, Category=Zero-shot LLMs2026.06 | 90.5 | |
| OptiTreeBackbone=GPT5.12026.04 | 90.08 | |
| Qwen2.5-72B-InstParams=72B, Category=Zero-shot LLMs2026.06 | 89.9 | |
| OptiMUS-v0.3Params=Closed, Category=Agentic Methods2026.06 | 89.8 | |
| AF-MCTSBackbone=GPT5.12026.04 | 88.84 | |
| GPT-4oParams=Closed, Category=Zero-shot LLMs2026.06 | 87.6 | |
| CoEParams=Closed, Category=Agentic Methods2026.06 | 87.4 | |
| OptiMUSBackbone=GPT5.12026.04 | 86.78 | |
| DCM-AgentBackbone=Qwen3-235B2026.04 | 84.71 | |
| BaselineBackbone=GPT5.12026.04 | 84.3 | |
| LLMOPTType=Fine-tuning2026.04 | 83.8 | |
| DCM-AgentBackbone=Deepseek-V3.22026.04 | 83.06 | |
| ORllmAgentType=Prompt-based2026.04 | 81.4 | |
| NEDTreeType=Context-based2026.04 | 80.16 | |
| OptiTreeBackbone=Qwen3-235B2026.04 | 80.16 | |
| OptiTreeBackbone=Deepseek-V3.22026.04 | 79.34 | |
| AF-MCTSBackbone=Qwen3-235B2026.04 | 78.93 | |
| AF-MCTSBackbone=Deepseek-V3.22026.04 | 78.1 | |
| OptiMUSBackbone=Qwen3-235B2026.04 | 77.27 | |
| GPT-o4-miniType=Reasoning2026.04 | 76.85 | |
| Deepseek-R1Type=Reasoning2026.04 | 76.44 | |
| ORLMParams=8B, Category=Fine-tuned LLMs2026.06 | 76.4 | |
| GPT-4oType=Non-reasoning2026.04 | 76.03 | |
| DCM-AgentBackbone=Qwen3-30B2026.04 | 76.03 | |
| OptiMUSBackbone=Deepseek-V3.22026.04 | 75.21 | |
| CoTParams=Closed, Category=Agentic Methods2026.06 | 74.7 | |
| BaselineBackbone=Qwen3-235B2026.04 | 74.38 | |
| Deepseek-V3Type=Non-reasoning2026.04 | 73.96 | |
| LLMOPT (origin)Params=14B, Category=Fine-tuned LLMs2026.06 | 73.4 | |
| OptimusType=Prompt-based2026.04 | 72 | |
| BaselineBackbone=Deepseek-V3.22026.04 | 71.73 | |
| OptiTreeBackbone=Qwen3-30B2026.04 | 70.25 | |
| ORLM(best)Type=Fine-tuning2026.04 | 68.6 | |
| AF-MCTSBackbone=Qwen3-30B2026.04 | 68.18 | |
| OptiMUSBackbone=Qwen3-30B2026.04 | 63.64 | |
| DCM-AgentBackbone=Qwen3-8B2026.04 | 62.4 | |
| Chain-of-ExpertsType=Prompt-based2026.04 | 53.1 | |
| BaselineBackbone=Qwen3-30B2026.04 | 52.07 | |
| OptiTreeBackbone=Qwen3-8B2026.04 | 46.28 | |
| OptiMUSBackbone=Qwen3-8B2026.04 | 43.39 | |
| AF-MCTSBackbone=Qwen3-8B2026.04 | 39.26 | |
| BaselineBackbone=Qwen3-8B2026.04 | 35.12 |