Software Issue Resolution on SWE-bench Lite (test)
185Resolved Issues CountCodex CLI GPT-5.4
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Codex CLI GPT-5.4Config=default, Effort=high2026.05 | 185 | 236.73 | 220.67 | |
| Codex CLI GPT-5.4Config=default, Effort=medium2026.05 | 172 | 161.82 | 155.96 | |
| Spell GPT-5.4Config=default, Effort=medium2026.05 | 171 | 102.12 | 51.37 | |
| Spell GPT-5.4Config=default, Effort=high2026.05 | 171 | 177.38 | 49.26 | |
| Codex CLI GPT-5.4Config=default, Effort=low2026.05 | 170 | 136.79 | 134.09 | |
| Spell GPT-5.4Config=default, Effort=low2026.05 | 153 | 68.94 | 74.3 |