Optimization Modeling on ComplexOR
83.33Accuracy (pass@1)NEDTree
Evaluation results
| Method | Links | ||
|---|---|---|---|
| NEDTreeType=Context-based2026.04 | 83.33 | — | |
| LLMOPTType=Fine-tuning2026.04 | 72.7 | — | |
| GPT-o4-miniType=Reasoning2026.04 | 72.22 | — | |
| Deepseek-V3Type=Non-reasoning2026.04 | 66.67 | — | |
| GPT-4oType=Non-reasoning2026.04 | 66.67 | — | |
| Deepseek-R1Type=Reasoning2026.04 | 66.67 | — | |
| OptimusType=Prompt-based2026.04 | 66.67 | — | |
| ORllmAgentType=Prompt-based2026.04 | 66.67 | — | |
| MiniOptCategory=Learning-based methods, Model Scale=14B2026.02 | 61.11 | — | |
| MIRRORCategory=Ours2026.02 | 61.11 | — | |
| Deepseek-v3Category=Traditional prompting2026.02 | 55.56 | — | |
| COECategory=Agent-based methods2026.02 | 55.56 | — | |
| ORLM(best)Type=Fine-tuning2026.04 | 55.56 | — | |
| Backbone modelCategory=Traditional prompting2026.02 | 44.44 | — | |
| ORMindCategory=Agent-based methods2026.02 | 44.44 | — | |
| MIRRORCategory=Ours, Model Scale=30B2026.02 | 44.44 | — | |
| LLMOPTCategory=Learning-based methods, Model Scale=14B2026.02 | 35.29 | — | |
| qwen3-30BCategory=Traditional prompting, Model Scale=30B2026.02 | 33.33 | — | |
| OptMATHCategory=Learning-based methods, Model Scale=7B2026.02 | 33.33 | — | |
| OptiMUSCategory=Agent-based methods2026.02 | 27.78 | — | |
| Chain-of-ExpertsType=Prompt-based2026.04 | 25.9 | — | |
| AlphaOPTCategory=Experience-Enhanced Methods2026.05 | — | 61.11 | |
| Chain-of-ExpertsCategory=Agent-Based Methods2026.05 | — | 16.67 | |
| DeepSeek-V3.2Category=General Models2026.05 | — | 55.56 | |
| Gemini-3.1-ProCategory=General Models2026.05 | — | 66.67 | |
| GPT-5.4Category=General Models2026.05 | — | 66.67 | |
| LEAN-LLM-OPTCategory=Experience-Enhanced Methods2026.05 | — | 50 | |
| OptiMUSCategory=Agent-Based Methods2026.05 | — | 33.33 | |
| OptSkills-DCategory=Skill-Based Methods, Base Model=DeepSeek-V3.22026.05 | — | 72.22 | |
| OptSkills-QCategory=Skill-Based Methods, Base Model=Qwen3-235B-A22b-instruct-25072026.05 | — | 66.67 | |
| ORMindCategory=Agent-Based Methods2026.05 | — | 33.33 | |
| ORThoughtCategory=Agent-Based Methods2026.05 | — | 55.56 | |
| Qwen3-235BCategory=General Models2026.05 | — | 38.89 | |
| Trace2SkillCategory=Skill-Based Methods2026.05 | — | 72.22 |