Code Generation on LIVECODE (held-out)
25.7DBSEXPERT
Evaluation results
| Method | Links | ||
|---|---|---|---|
| EXPERTModel=Qwen3-1.7B optimized2026.02 | 25.7 | — | |
| SOURCEbest2026.02 | 10.3 | — | |
| DataChef-32BOracle Upper Bound=true, Samples=322026.02 | 10.3 | — | |
| Kimi-K22026.02 | 9.7 | 19.3 | |
| Gemini-3-Pro2026.02 | 9.1 | 53.6 | |
| DataChef-32B2026.02 | 9.1 | 45.8 | |
| Qwen3-32B2026.02 | 8 | 24.3 | |
| Qwen3-Next ⊕ Kimi-K2Reasoning backbone=Qwen3-Next-80B, Coding backbone=Kimi-K2-Instruct2026.02 | 7.4 | 39.2 | |
| SOURCEavg2026.02 | 6.3 | — |