Optimization Modeling on OptMATH
87.5Accuracy Rate (AR)TriVAL
Evaluation Results
| Method | Links | |
|---|---|---|
| TriVALModel=DeepSeek-V3.2, Method category=Prompt-based2026.05 | 87.5 | |
| TriVALModel=GPT-5.1, Method category=Prompt-based2026.05 | 87.5 | |
| OR-LLM-AgentModel=DeepSeek-V3.2, Method category=Prompt-based2026.05 | 85.9 | |
| OR-LLM-AgentModel=GPT-5.1, Method category=Prompt-based2026.05 | 85.2 | |
| OptiTreeModel=GPT-5.1, Method category=Prompt-based2026.05 | 83.6 | |
| OptiTreeModel=DeepSeek-V3.2, Method category=Prompt-based2026.05 | 74.2 | |
| SIRLMethod category=Learning-based2026.05 | 71.1 | |
| DCM-AgentBackbone=GPT5.12026.04 | 55.42 | |
| OptiTreeBackbone=GPT5.12026.04 | 53.61 | |
| DCM-AgentBackbone=Deepseek-V3.22026.04 | 52.73 | |
| R2C with reflectionReflection mechanism=Enabled, Maximum reflection iterations=32026.02 | 51.8 | |
| AF-MCTSBackbone=GPT5.12026.04 | 51.2 | |
| OptiTreeBackbone=Deepseek-V3.22026.04 | 47.59 | |
| Grok-42026.02 | 47 | |
| Grok42026.02 | 47 | |
| AutoFormulationModel=GPT-5.1, Method category=Prompt-based2026.05 | 46.9 | |
| DCM-AgentBackbone=Qwen3-235B2026.04 | 46.39 | |
| GPT-52026.02 | 45.8 | |
| GPT52026.02 | 45.8 | |
| OptiMUSBackbone=GPT5.12026.04 | 45.78 | |
| R2CReflection mechanism=Disabled2026.02 | 44.6 | |
| R2CBackbone=deepseek-v3.22026.02 | 44.6 | |
| Standard PromptBackbone=deepseek-v3.22026.02 | 44 | |
| OpenAI-o3Method Category=Model-based Methods2026.06 | 44 | |
| AF-MCTSBackbone=Deepseek-V3.22026.04 | 43.98 | |
| DeepSeek-V3.1Method Category=Model-based Methods2026.06 | 43.9 | |
| BaselineBackbone=GPT5.12026.04 | 43.38 | |
| OptiTreeBackbone=Qwen3-235B2026.04 | 41.57 | |
| AutoFormulationModel=DeepSeek-V3.2, Method category=Prompt-based2026.05 | 40.6 | |
| DeepSeek-R1Method Category=Model-based Methods2026.06 | 40.4 | |
| AF-MCTSBackbone=Qwen3-235B2026.04 | 40.36 | |
| OptiMUSBackbone=Deepseek-V3.22026.04 | 39.76 | |
| OptiMUSBackbone=Qwen3-235B2026.04 | 37.95 | |
| BaselineBackbone=Deepseek-V3.22026.04 | 36.75 | |
| OptMATHBackbone=Qwen2.5-32B2026.02 | 34.7 | |
| DCM-AgentBackbone=Qwen3-30B2026.04 | 32.53 | |
| StarORMethod Category=Test-Time Methods, Base model=Qwen3-4B-Instruct-25072026.06 | 32.5 | |
| BaselineBackbone=Qwen3-235B2026.04 | 31.33 | |
| SIRL-Qwen2.5-7BMethod Category=Learning-based Methods2026.06 | 30.5 | |
| OptiTreeBackbone=Qwen3-30B2026.04 | 27.71 | |
| AF-MCTSBackbone=Qwen3-30B2026.04 | 25.9 | |
| OptiMUSBackbone=Qwen3-30B2026.04 | 24.7 | |
| OptMATH-Qwen2.5-7BMethod Category=Learning-based Methods2026.06 | 24.4 | |
| LLMOPTMethod category=Learning-based2026.05 | 21.9 | |
| DCM-AgentBackbone=Qwen3-8B2026.04 | 21.69 | |
| Best-of-NMethod Category=Test-Time Methods, Base model=Qwen3-4B-Instruct-2507, N=162026.06 | 21.6 | |
| BaselineBackbone=Qwen3-30B2026.04 | 16.87 | |
| GPT-42026.02 | 16.6 | |
| GPT-4Method Category=Model-based Methods2026.06 | 16.6 | |
| OptiTreeBackbone=Qwen3-8B2026.04 | 16.26 | |
| OptiMUSBackbone=Qwen3-8B2026.04 | 15.06 | |
| ORLMMethod category=Learning-based2026.05 | 14.1 | |
| OptiTreeMethod Category=Test-Time Methods, Base model=Qwen3-4B-Instruct-25072026.06 | 12.7 | |
| AF-MCTSBackbone=Qwen3-8B2026.04 | 12.65 | |
| LLMOPT-Qwen2.5-14BMethod Category=Learning-based Methods2026.06 | 12.5 | |
| ReflexionMethod Category=Test-Time Methods, Base model=Qwen3-4B-Instruct-25072026.06 | 11.4 | |
| AutoFormulatorMethod Category=Test-Time Methods, Base model=Qwen3-4B-Instruct-25072026.06 | 11.4 | |
| Zero-shotMethod Category=Test-Time Methods, Base model=Qwen3-4B-Instruct-25072026.06 | 10.8 | |
| BaselineBackbone=Qwen3-8B2026.04 | 10.24 | |
| ORLM-LLaMA3-8BMethod Category=Learning-based Methods2026.06 | 2.6 |