Software Engineering on PaperBench
66.8ScoreClaude Sonnet 4.5
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Claude Sonnet 4.5SDK=v1.11.0, Agent Configuration=SINGLE-AGENT + CAID2026.03 | 66.8 | 3,883.9 | 12.6 | |
| Claude Sonnet 4.5SDK=v1.11.0, Agent Configuration=CAID, Number of Engineers=22026.03 | 63.3 | 2,080.4 | 9.3 | |
| Claude Sonnet 4.5SDK=v1.11.0, Agent Configuration=Single-Agent2026.03 | 57.2 | 1,803.5 | 3.3 | |
| GLM 4.7SDK=v1.11.0, Agent Configuration=SINGLE-AGENT + CAID2026.03 | 48.5 | 2,626.9 | 7.5 | |
| GLM 4.7SDK=v1.11.0, Agent Configuration=CAID, Number of Engineers=22026.03 | 45.4 | 1,449.4 | 4.7 | |
| GLM 4.7SDK=v1.11.0, Agent Configuration=Single-Agent2026.03 | 38 | 1,177.6 | 2.8 | |
| MiniMax 2.5SDK=v1.11.0, Agent Configuration=CAID, Number of Engineers=22026.03 | 36.7 | 2,999.4 | 2.6 | |
| MiniMax 2.5SDK=v1.11.0, Agent Configuration=SINGLE-AGENT + CAID2026.03 | 36.7 | 5,524.7 | 3.7 | |
| MiniMax 2.5SDK=v1.11.0, Agent Configuration=Single-Agent2026.03 | 10.4 | 2,525.3 | 1.1 |