Optimization Modeling on NL4Opt
98Accuracy (pass@1)SIRL
Evaluation results
| Method | Links | |
|---|---|---|
| SIRLCategory=Learning-based methods, Model Scale=32B2026.02 | 98 | |
| DCM-AgentBackbone=GPT5.12026.04 | 97.83 | |
| OptMATHType=Fine-tuning2026.04 | 95.9 | |
| OptiTreeBackbone=GPT5.12026.04 | 95.65 | |
| DCM-AgentBackbone=Deepseek-V3.22026.04 | 95.22 | |
| AF-MCTSBackbone=GPT5.12026.04 | 94.47 | |
| NEDTreeType=Context-based2026.04 | 94.35 | |
| DCM-AgentBackbone=Qwen3-235B2026.04 | 93.48 | |
| LLMOPTType=Fine-tuning2026.04 | 93 | |
| MiniOptCategory=Learning-based methods, Model Scale=14B2026.02 | 92.17 | |
| OptiTreeBackbone=Deepseek-V3.22026.04 | 90.87 | |
| OptiMUSBackbone=GPT5.12026.04 | 90.43 | |
| Deepseek-R1Type=Reasoning2026.04 | 90 | |
| OptiTreeBackbone=Qwen3-235B2026.04 | 89.56 | |
| AF-MCTSBackbone=Deepseek-V3.22026.04 | 89.13 | |
| GPT-o4-miniType=Reasoning2026.04 | 88.26 | |
| ORllmAgentType=Prompt-based2026.04 | 88.26 | |
| Deepseek-V3Type=Non-reasoning2026.04 | 87.82 | |
| AF-MCTSBackbone=Qwen3-235B2026.04 | 87.39 | |
| BaselineBackbone=GPT5.12026.04 | 87.39 | |
| MIRRORCategory=Ours2026.02 | 86.5 | |
| ORLM(best)Type=Fine-tuning2026.04 | 86.5 | |
| OptiMUSBackbone=Deepseek-V3.22026.04 | 86.09 | |
| ORLMCategory=Learning-based methods, Model Scale=8B2026.02 | 85.7 | |
| COECategory=Agent-based methods2026.02 | 84.9 | |
| OptiMUSBackbone=Qwen3-235B2026.04 | 83.48 | |
| BaselineBackbone=Deepseek-V3.22026.04 | 82.61 | |
| MIRRORCategory=Ours, Model Scale=30B2026.02 | 82.4 | |
| GPT-4oType=Non-reasoning2026.04 | 81.3 | |
| Gemini 3 Proevaluation_protocol=pass@12026.04 | 80.95 | |
| LLMOPTCategory=Learning-based methods, Model Scale=14B2026.02 | 80.28 | |
| OptimusType=Prompt-based2026.04 | 78.8 | |
| OptMATHCategory=Learning-based methods, Model Scale=7B2026.02 | 78.7 | |
| AutoOR 8Bevaluation_protocol=pass@12026.04 | 78.57 | |
| BaselineBackbone=Qwen3-235B2026.04 | 78.13 | |
| ORMindCategory=Agent-based methods2026.02 | 77.55 | |
| DCM-AgentBackbone=Qwen3-30B2026.04 | 77.39 | |
| Deepseek-v3Category=Traditional prompting2026.02 | 73.88 | |
| Backbone modelCategory=Traditional prompting2026.02 | 72.24 | |
| OptiTreeBackbone=Qwen3-30B2026.04 | 70.88 | |
| Gemini 2.5 Proevaluation_protocol=pass@12026.04 | 69.84 | |
| qwen3-30BCategory=Traditional prompting, Model Scale=30B2026.02 | 68.57 | |
| AF-MCTSBackbone=Qwen3-30B2026.04 | 65.65 | |
| DCM-AgentBackbone=Qwen3-8B2026.04 | 64.35 | |
| Chain-of-ExpertsType=Prompt-based2026.04 | 64.2 | |
| OptiMUSBackbone=Qwen3-30B2026.04 | 63.48 | |
| Qwen3-8Bevaluation_protocol=pass@12026.04 | 59.52 | |
| OptiMUSCategory=Agent-based methods2026.02 | 57.96 | |
| OptiTreeBackbone=Qwen3-8B2026.04 | 55.22 | |
| BaselineBackbone=Qwen3-30B2026.04 | 55.22 | |
| OptiMUSBackbone=Qwen3-8B2026.04 | 53.48 | |
| AF-MCTSBackbone=Qwen3-8B2026.04 | 47.39 | |
| BaselineBackbone=Qwen3-8B2026.04 | 41.74 |