Software Engineering on Commit0-Lite
88.2ScoreSTORM-Combined
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| STORM-CombinedModel=Qwen 3.6 Plus, Protocol=STORM-Combined2026.05 | 88.2 | 13.2 | 2.1 | 76.2 | |
| STORM-CombinedModel=Claude Sonnet 4.6, Protocol=STORM-Combined2026.05 | 87.6 | 21.1 | 6.3 | 49.2 | |
| GitWorktree-CombinedModel=Qwen 3.6 Plus, Protocol=GitWorktree-Combined2026.05 | 83 | 31.1 | 3.7 | 36.3 | |
| STORMModel=Claude Sonnet 4.6, Protocol=STORM2026.05 | 82.5 | 16.8 | 6.3 | 46.2 | |
| GitWorktree-CombinedModel=Claude Sonnet 4.6, Protocol=GitWorktree-Combined2026.05 | 78.6 | 25.7 | 8.1 | 31.4 | |
| STORM-CombinedModel=DeepSeek V4 Pro, Protocol=STORM-Combined2026.05 | 77.7 | 49 | 3.4 | 41.3 | |
| Single-AgentModel=Qwen 3.6 Plus, Protocol=Single-Agent2026.05 | 75.3 | 13 | 1.3 | 34 | |
| GitWorktree-CombinedModel=DeepSeek V4 Pro, Protocol=GitWorktree-Combined2026.05 | 75.2 | 53.9 | 3.8 | 30.9 | |
| STORMModel=Qwen 3.6 Plus, Protocol=STORM2026.05 | 70.5 | 12.5 | 2.5 | 61.4 | |
| Single-AgentModel=Claude Sonnet 4.6, Protocol=Single-Agent2026.05 | 66.4 | 18.8 | 3.2 | 20.7 | |
| Single-AgentModel=DeepSeek V4 Pro, Protocol=Single-Agent2026.05 | 65.2 | 31.3 | 1.8 | 26.8 | |
| GitWorktreeModel=Claude Sonnet 4.6, Protocol=GitWorktree2026.05 | 63.8 | 20.9 | 8.6 | 24.6 | |
| STORMModel=DeepSeek V4 Pro, Protocol=STORM2026.05 | 63.2 | 39 | 3 | 32.3 | |
| Claude Sonnet 4.5SDK=v1.11.0, Agent Configuration=SINGLE-AGENT + CAID2026.03 | 59.5 | 2,275.8 | 10.1 | — | |
| Claude Sonnet 4.5SDK=v1.11.0, Agent Configuration=CAID, Number of Engineers=42026.03 | 59.1 | 1,583.2 | 8.1 | — | |
| GitWorktreeModel=Qwen 3.6 Plus, Protocol=GitWorktree2026.05 | 57.4 | 56 | 6.9 | 16.7 | |
| MiniMax 2.5SDK=v1.11.0, Agent Configuration=CAID, Number of Engineers=42026.03 | 57 | 1,908.6 | 4.5 | — | |
| MiniMax 2.5SDK=v1.11.0, Agent Configuration=SINGLE-AGENT + CAID2026.03 | 57 | 2,660.7 | 6.1 | — | |
| Claude Sonnet 4.5SDK=v1.11.0, Agent Configuration=Single-Agent2026.03 | 53.1 | 692.6 | 1.9 | — | |
| GLM 4.7SDK=v1.11.0, Agent Configuration=CAID, Number of Engineers=42026.03 | 46.5 | 1,387.8 | 7.3 | — | |
| GLM 4.7SDK=v1.11.0, Agent Configuration=SINGLE-AGENT + CAID2026.03 | 46.5 | 2,257.8 | 9.8 | — | |
| GitWorktreeModel=DeepSeek V4 Pro, Protocol=GitWorktree2026.05 | 44 | 45.1 | 3.8 | 18.5 | |
| GLM 4.7SDK=v1.11.0, Agent Configuration=Single-Agent2026.03 | 42.8 | 871 | 2.5 | — | |
| MiniMax 2.5SDK=v1.11.0, Agent Configuration=Single-Agent2026.03 | 42.3 | 752.1 | 1.6 | — |