Code Generation on LoCoBench Agent
70.4AccuracyECHO
Evaluation results
| Method | Links | |
|---|---|---|
| ECHOBackbone=Qwen3-32B-Instruct, Zero-shot protocol=true2026.06 | 70.4 | |
| ECHO w/o Traceable CA & Turn-level ISBackbone=Qwen3-32B-Instruct, Zero-shot protocol=true2026.06 | 70.1 | |
| ECHO w/ Top-K retrieval & w/o turn summaryBackbone=Qwen3-32B-Instruct, Zero-shot protocol=true2026.06 | 69.5 | |
| ECHO w/ Top-K retrievalBackbone=Qwen3-32B-Instruct, Zero-shot protocol=true2026.06 | 69.3 | |
| ECHO w/o traceable CABackbone=Qwen3-32B-Instruct, Zero-shot protocol=true2026.06 | 68.2 | |
| SUPOBackbone=Qwen3-32B-Instruct, Zero-shot protocol=true2026.06 | 68.1 | |
| GRPOBackbone=Qwen3-32B-Instruct, Zero-shot protocol=true2026.06 | 67.7 | |
| ECHOBackbone=Qwen3-30B-A3B-Instruct, Zero-shot protocol=true2026.06 | 66.8 | |
| GRPOBackbone=Qwen3-30B-A3B-Instruct, Zero-shot protocol=true2026.06 | 65.7 | |
| SUPOBackbone=Qwen3-30B-A3B-Instruct, Zero-shot protocol=true2026.06 | 65.1 |