Scaling Law Discovery on SLDBench extrapolation (test)
1Parallel Scaling FitTerminus-2
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Terminus-2Model=GPT-52026.04 | 1 | 0.502 | -0.754 | -0.604 | 0.036 | |
| OpenCodeModel=GPT-52026.04 | 1 | 0.96 | -0.368 | -0.48 | 0.278 | |
| OpenHandsModel=GPT-52026.04 | 1 | 0.899 | -0.909 | -0.278 | 0.178 | |
| GooseModel=GPT-52026.04 | 1 | 0.944 | 0.28 | -0.232 | 0.498 | |
| SLDAgentModel=GPT-52026.04 | 1 | 0.988 | 0.604 | -0.305 | 0.572 | |
| HumanModel=–2026.04 | 1 | 0.671 | -0.076 | -1 | 0.149 | |
| SLDAgentModel=Gemini-2.5-Flash2026.04 | 1 | 0.991 | -0.871 | -0.758 | 0.09 | |
| SLDAgentModel=Gemini-3-Pro-Preview2026.04 | 1 | 0.984 | 0.513 | -1 | 0.374 | |
| Claude CodeModel=Claude-Haiku-4.52026.04 | 1 | 0.905 | -0.511 | -1 | 0.099 | |
| SLDAgentModel=Claude-Haiku-4.52026.04 | 1 | 0.98 | -0.657 | -0.754 | 0.142 | |
| SLDAgentModel=Claude-Sonnet-4.52026.04 | 1 | 0.985 | -0.514 | -0.522 | 0.237 | |
| CodeXModel=o4-mini2026.04 | 1 | 0.553 | -0.773 | -1 | -0.055 | |
| SLDAgentModel=o4-mini2026.04 | 1 | 0.989 | 0.611 | -1 | 0.4 | |
| SIMPLETESModel=gpt-oss-120b2026.04 | 1 | 0.991 | 0.712 | -0.008 | 0.674 | |
| CodeXModel=GPT-52026.04 | 0.999 | 0.933 | -0.039 | -0.74 | 0.288 | |
| Claude CodeModel=Claude-Sonnet-4.52026.04 | 0.998 | 0.971 | -0.846 | -1 | 0.031 | |
| Mini-SWE-AgentModel=GPT-52026.04 | 0.997 | 0.873 | -0.269 | -0.491 | 0.277 | |
| AiderModel=GPT-52026.04 | 0.991 | 0.514 | -0.659 | -0.474 | 0.093 | |
| Gemini-CLIModel=Gemini-3-Pro-Preview2026.04 | 0.6 | 0.978 | -0.332 | -0.847 | 0.1 | |
| Gemini-CLIModel=Gemini-2.5-Flash2026.04 | 0.2 | 0.53 | -0.873 | -0.794 | -0.234 |