Operations Research Reasoning on OR-BIAS-BENCH OOD
100Rationalitygpt-4.1
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| gpt-4.12026.01 | 100 | 14.6 | 3 | |
| claude-haiku-4.52026.01 | 99.9 | 3.6 | 3.6 | |
| gpt-4.1-mini2026.01 | 99.9 | 12.1 | 8 | |
| DeepSeek-V3.22026.01 | 99.9 | 11.3 | -6.9 | |
| qwen2.5-14bParameters=14b2026.01 | 99.8 | 10.7 | 4.8 | |
| qwen2.5-32bParameters=32b2026.01 | 99.8 | 5 | -10.4 | |
| gpt-5-mini2026.01 | 99.7 | 53.3 | 52.1 | |
| Qwen3-8B-OM-SFTParameters=8B, Training=SFT2026.01 | 99.6 | 11.5 | 6.6 | |
| Qwen3-8B-OM-CurriculumParameters=8B, Training=Curriculum2026.01 | 99.6 | 10.4 | -9.6 | |
| DeepSeek-R12026.01 | 99.6 | 40.9 | -3.2 | |
| gemini-2.5-pro2026.01 | 99.6 | 100 | 2.1 | |
| o4-mini2026.01 | 99.4 | 7.7 | 1 | |
| claude-3.7-sonnet2026.01 | 99.4 | 18.1 | 6.8 | |
| qwen2.5-7bParameters=7b2026.01 | 99.3 | 8.2 | 0.8 | |
| gemini-2.0-flash2026.01 | 98.7 | 0 | -5.9 | |
| o12026.01 | 98.6 | 43.6 | 20.3 | |
| qwen2.5-max2026.01 | 98.5 | 25 | 24.5 | |
| Qwen3-8B-OM-GRPOParameters=8B, Training=GRPO2026.01 | 98.4 | 33.8 | -14.2 | |
| gemini-2.5-flash2026.01 | 98.1 | 71.2 | 53 | |
| o32026.01 | 97.7 | 24.5 | 24.1 | |
| kimi-k22026.01 | 97.6 | 2.3 | -6.6 | |
| Llama-3.3-70BParameters=70B2026.01 | 96.9 | 12.5 | -6.7 | |
| claude-opus-42026.01 | 94.8 | 15.9 | -3.1 | |
| claude-sonnet-42026.01 | 93.5 | 7.7 | 6.2 |