Agentic Coding on Project dev (test)
0.881Tau^2Claude-Sonnet-4.5
Evaluation Results
| Method | Links | |
|---|---|---|
| Claude-Sonnet-4.5Model Type=Proprietary2025.12 | 0.881 | |
| GPT-5Model Type=Proprietary2025.12 | 0.842 | |
| DeepSeek-V3.1-Nex-N1Model Type=Open Source, Parameter Scale=> 100B2025.12 | 0.802 | |
| Minimax-M2Model Type=Open Source, Parameter Scale=> 100B2025.12 | 0.762 | |
| GLM-4.6Model Type=Open Source, Parameter Scale=> 100B2025.12 | 0.759 | |
| Kimi-K2-thinkingModel Type=Open Source, Parameter Scale=> 100B2025.12 | 0.752 | |
| Qwen3-32B-Nex-N1Model Type=Open Source, Parameter Scale=< 100B2025.12 | 0.721 | |
| Gemini-2.5-proModel Type=Proprietary2025.12 | 0.654 | |
| Qwen3-30B-A3B-Nex-N1Model Type=Open Source, Parameter Scale=< 100B2025.12 | 0.653 | |
| InternLM3-8B-Nex-N1Model Type=Open Source, Parameter Scale=< 100B2025.12 | 0.63 | |
| DeepSeek-V3.1Model Type=Open Source, Parameter Scale=> 100B2025.12 | 0.428 | |
| Qwen3-32BModel Type=Open Source, Parameter Scale=< 100B2025.12 | 0.415 | |
| Qwen3-30B-A3BModel Type=Open Source, Parameter Scale=< 100B2025.12 | 0.285 |