Program Repair on Cangjie-bench
90.63ACCILA-agent
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| ILA-agentModels=Claude-Sonnet-4.52026.01 | 90.63 | 100 | |
| ILA-agentModels=Qwen3-Max2026.01 | 81.25 | 100 | |
| ILA-agentModels=DeepSeek-V3.22026.01 | 71.88 | 87.5 | |
| Iterative RAGModels=Claude-Sonnet-4.52026.01 | 71.88 | 84.38 | |
| Iterative RAGModels=DeepSeek-V3.22026.01 | 68.75 | 87.5 | |
| Iterative RAGModels=Qwen3-Max2026.01 | 68.75 | 87.5 | |
| Zero-shotModels=Claude-Sonnet-4.52026.01 | 65.63 | 81.25 | |
| Zero-shotModels=Qwen3-Max2026.01 | 62.5 | 71.88 | |
| Single-time RAGModels=Qwen3-Max2026.01 | 62.5 | 78.13 | |
| Single-time RAGModels=Claude-Sonnet-4.52026.01 | 62.5 | 81.25 | |
| Zero-shotModels=DeepSeek-V3.22026.01 | 56.25 | 71.88 | |
| Single-time RAGModels=DeepSeek-V3.22026.01 | 56.25 | 78.13 |