Template Following on In-house IDE/CLI Agentic Coding Scaffolds
98Scaffold 1 RateDeepseek-v3.2
Evaluation Results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Deepseek-v3.22026.02 | 98 | 87 | 100 | 92.5 | 91 | 93.7 | |
| Qwen3-Coder-Next2026.02 | 98 | 83 | 98 | 91.5 | 93 | 92.7 | |
| Gemini-3-pro2026.02 | 92.4 | 57 | 98 | 93.5 | 94 | 87 | |
| GLM-4.72026.02 | 91 | 64 | 100 | 94.7 | 0 | 69.9 | |
| Claude-sonnet-4-52026.02 | 86.8 | 61 | 100 | 88 | 91 | 85.4 | |
| GLM-4.62026.02 | 85.8 | 86 | 100 | 82.9 | 24 | 75.7 | |
| GPT-5-22026.02 | 84 | 14 | 41.8 | 29.8 | 77 | 49.3 | |
| Kimi-K2-thinking2026.02 | 71.5 | 70 | 91.8 | 83 | 80 | 79.3 | |
| MiniMax-M2.12026.02 | 68.3 | 48 | 93.8 | 86.2 | 90 | 77.3 | |
| Kimi-K22026.02 | 59 | 59 | 100 | 57.4 | 81 | 71.3 |