Text-to-Optimization on Text2Opt-Bench eval-subset 1.0
89.9Resource Allocation ScoreOpus 3.5
Evaluation Results
| Method | Links | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Opus 3.5Model Family=Frontier, pass@1=true2026.05 | 89.9 | — | 98 | 96 | 100 | 38 | 96 | 100 | 88 | 92 | 62 | 88 | 96 | 86.7 | |
| GPT-4oModel Family=Frontier, pass@1=true2026.05 | 87.9 | — | 100 | 86 | 90 | 70 | 88 | 100 | 98 | 80 | 70 | 70 | 96 | 86.2 | |
| Sonnet 3.5Model Family=Frontier, pass@1=true2026.05 | 84.7 | — | 100 | 96 | 98 | 50 | 100 | 98 | 64 | 98 | 62 | 98 | 100 | 87.6 | |
| DS-R1Model Family=Reasoning, pass@1=true2026.05 | 80.6 | — | 100 | 78 | 96 | 34 | 34 | 94 | 64 | 70 | 60 | 76 | 96 | 72.9 | |
| o1-miniModel Family=Reasoning, pass@1=true2026.05 | 80.2 | — | 100 | 94 | 96 | 34 | 82 | 98 | 70 | 76 | 66 | 68 | 100 | 80.4 | |
| DS-V3Model Family=Open-Large, pass@1=true2026.05 | 79 | — | 100 | 90 | 96 | 22 | 62 | 98 | 54 | 66 | 18 | 86 | 100 | 72 | |
| Llama3.3-70BModel Family=Open-Other, pass@1=true2026.05 | 49.6 | — | 88 | 30 | 0 | 0 | 0 | 98 | 16 | 10 | 6 | 42 | 96 | 35.1 | |
| GPT-4o-miniModel Family=Close-Mid, pass@1=true2026.05 | 49.2 | — | 100 | 62 | 82 | 2 | 26 | 90 | 50 | 56 | 32 | 60 | 90 | 59.1 | |
| Qwen2.5-7BModel Family=Open-Other, pass@1=true2026.05 | 13.3 | — | 38 | 0 | 0 | 0 | 0 | 6 | 0 | 0 | 0 | 4 | 2 | 4.5 |