Data Analysis on DABStep hard
37.04AccuracyClaude Sonnet 4.5
Evaluation results
| Method | Links | |
|---|---|---|
| Claude Sonnet 4.5Model Type=Proprietary2026.01 | 37.04 | |
| Claude Sonnet 4Model Type=Proprietary2026.01 | 31.75 | |
| Kimi K2 InstructModel Type=Open-sourced2026.01 | 28.84 | |
| GPT-5Reasoning effort=medium, Model Type=Proprietary2026.01 | 28.31 | |
| Deepseek-v3.1Model Type=Open-sourced2026.01 | 21.96 | |
| Qwen3 235B InstructModel Type=Open-sourced2026.01 | 17.46 | |
| Qwen3-Coder 480BModel Type=Open-sourced2026.01 | 14.29 | |
| GPT-5.1Reasoning effort=high, Model Type=Proprietary2026.01 | 13.23 | |
| GPT-5.1Reasoning effort=none, Model Type=Proprietary2026.01 | 11.9 | |
| GPT-OSS-120BModel Type=Open-sourced2026.01 | 7.94 | |
| GPT-4oModel Type=Proprietary2026.01 | 7.41 | |
| Qwen3-4B-InstructModel Type=Open-sourced2026.01 | 2.9 | |
| Qwen2.5-7B-InstructModel Type=Open-sourced2026.01 | 2.38 |