Software Engineering on PaperBench Code (dev)
78.2ScoreSTORM-Combined
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| STORM-CombinedModel=Claude Sonnet 4.6, Protocol=STORM-Combined2026.05 | 78.2 | 17.3 | 31.4 | |
| GitWorktree-CombinedModel=Claude Sonnet 4.6, Protocol=GitWorktree-Combined2026.05 | 76.6 | 22.7 | 30.8 | |
| STORMModel=Claude Sonnet 4.6, Protocol=STORM2026.05 | 74.1 | 12.6 | 24.1 | |
| GitWorktreeModel=Claude Sonnet 4.6, Protocol=GitWorktree2026.05 | 72.7 | 17.2 | 22.5 | |
| Single-AgentModel=Claude Sonnet 4.6, Protocol=Single-Agent2026.05 | 68.7 | 12.5 | 11.6 | |
| STORM-CombinedModel=DeepSeek V4 Pro, Protocol=STORM-Combined2026.05 | 68.3 | 10.9 | 41.7 | |
| STORMModel=DeepSeek V4 Pro, Protocol=STORM2026.05 | 66.5 | 9.7 | 35.5 | |
| Single-AgentModel=DeepSeek V4 Pro, Protocol=Single-Agent2026.05 | 62.9 | 4.1 | 16.5 | |
| GitWorktree-CombinedModel=DeepSeek V4 Pro, Protocol=GitWorktree-Combined2026.05 | 60.8 | 9.1 | 41.3 | |
| STORM-CombinedModel=Qwen 3.6 Plus, Protocol=STORM-Combined2026.05 | 57 | 10.7 | 29.6 | |
| GitWorktreeModel=DeepSeek V4 Pro, Protocol=GitWorktree2026.05 | 55.8 | 8.8 | 35.7 | |
| GitWorktree-CombinedModel=Qwen 3.6 Plus, Protocol=GitWorktree-Combined2026.05 | 55.4 | 12.1 | 43.1 | |
| STORMModel=Qwen 3.6 Plus, Protocol=STORM2026.05 | 55 | 8.2 | 21 | |
| GitWorktreeModel=Qwen 3.6 Plus, Protocol=GitWorktree2026.05 | 51.6 | 13.4 | 36 | |
| Single-AgentModel=Qwen 3.6 Plus, Protocol=Single-Agent2026.05 | 47.7 | 4.9 | 16.1 |