Optimization Modeling on ComplexOR (Com.OR) (ER)
5.56Error RateQwen2.5-3B-Instruct
Evaluation results
| Method | Links | |
|---|---|---|
| Qwen2.5-3B-InstructCategory=General Models, Avg.=16.57, Rank=18, Rank*=82026.06 | 5.56 | |
| Qwen3-4BCategory=General Models (Thinking), Avg.=14.02, Rank=19, Rank*=92026.06 | 11.11 | |
| Qwen2.5-7B-InstructCategory=General Models, Avg.=41.86, Rank=15, Rank*=62026.06 | 16.67 | |
| Qwen3-8BCategory=General Models (Thinking), Avg.=25.43, Rank=17, Rank*=72026.06 | 22.22 | |
| Qwen2.5-14B-InstructCategory=General Models, Avg.=60.64, Rank=11, Rank*=-2026.06 | 38.89 | |
| OptiMUSCategory=Prompt-based Methods, Avg.=49.43, Rank=14, Rank*=-2026.06 | 44.44 | |
| Chain-of-ExpertsCategory=Prompt-based Methods, Avg.=60.33, Rank=12, Rank*=-2026.06 | 55.56 | |
| Qwen3-14BCategory=General Models (Thinking), Avg.=30.04, Rank=16, Rank*=-2026.06 | 61.11 | |
| DeepSeek-R1Category=General Models (Thinking), Avg.=82.24, Rank=7, Rank*=-2026.06 | 61.11 | |
| Step-OPT-Qwen2.5-3BCategory=Learning-based Models, Avg.=54.65, Rank=13, Rank*=52026.06 | 61.11 | |
| MiniOpt-3BCategory=Ours, Avg.=87.92, Rank=6, Rank*=32026.06 | 61.11 | |
| OptMATH-7BCategory=Learning-based Models, Avg.=83.39, Rank=6, Rank*=22026.06 | 66.67 | |
| DeepSeek-V3Category=General Models, Avg.=81.86, Rank=8, Rank*=-2026.06 | 72.22 | |
| Step-OPT-Qwen2.5-7BCategory=Learning-based Models, Avg.=69.76, Rank=10, Rank*=42026.06 | 72.22 | |
| Gemini-2.5-ProCategory=General Models (Thinking), Avg.=88.87, Rank=3, Rank*=-2026.06 | 77.78 | |
| ReflexionCategory=Prompt-based Methods, Avg.=78.28, Rank=9, Rank*=-2026.06 | 83.33 | |
| MiniOpt-7BCategory=Ours, Avg.=91.17, Rank=1, Rank*=12026.06 | 88.39 | |
| GPT-5Category=General Models (Thinking), Avg.=84.73, Rank=5, Rank*=-2026.06 | 88.89 | |
| LLMOPT-14BCategory=Learning-based Models, Avg.=89.75, Rank=2, Rank*=-2026.06 | 88.89 |