Code Generation on Live Code Generation
24.75Pass@1DOMINO
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| DOMINOBackbone=QWEN2.5-CODER-14B-Instruct2026.05 | 24.75 | 30.45 | 33.24 | |
| QWEN2.5-CODER-14B-InstructBackbone=QWEN2.5-CODER-14B-Instruct2026.05 | 23.35 | 30.13 | 31.73 | |
| DOMINO-Direct DomainBackbone=QWEN2.5-CODER-14B-Instruct2026.05 | 21.62 | 28.97 | 31.73 | |
| MAGPIE-HumanBackbone=QWEN2.5-CODER-14B-Instruct2026.05 | 20.78 | 27.17 | 28.74 | |
| MAGPIE-Few ShotBackbone=QWEN2.5-CODER-14B-Instruct2026.05 | 20.12 | 25.27 | 27.45 | |
| Reference SFTBackbone=QWEN2.5-CODER-14B-Instruct2026.05 | 19.64 | 24.86 | 26.33 | |
| QWEN2.5-CODER-14B-BaseBackbone=QWEN2.5-CODER-14B-Instruct2026.05 | 19.46 | 26.42 | 27.4 | |
| DOMINOBackbone=QWEN2.5-CODER-7B-Instruct2026.05 | 17.31 | 26.81 | 29.94 | |
| QWEN2.5-CODER-7B-InstructBackbone=QWEN2.5-CODER-7B-Instruct2026.05 | 16.88 | 21.73 | 23.35 | |
| DOMINO-Direct DomainBackbone=QWEN2.5-CODER-7B-Instruct2026.05 | 15.74 | 24.66 | 28.14 | |
| MAGPIE-Few ShotBackbone=QWEN2.5-CODER-7B-Instruct2026.05 | 14.36 | 20.98 | 24.23 | |
| Reference SFTBackbone=QWEN2.5-CODER-7B-Instruct2026.05 | 13.47 | 21.59 | 23.95 | |
| DOMINOBackbone=OPENCODER-8B-Instruct2026.05 | 12.63 | 16.81 | 20.44 | |
| MAGPIE-HumanBackbone=QWEN2.5-CODER-7B-Instruct2026.05 | 12.45 | 20.85 | 22.72 | |
| DOMINO-Direct DomainBackbone=OPENCODER-8B-Instruct2026.05 | 10.15 | 16.24 | 19.39 | |
| OPENCODER-8B-InstructBackbone=OPENCODER-8B-Instruct2026.05 | 10 | 17.23 | 19.76 | |
| MAGPIE-Few ShotBackbone=OPENCODER-8B-Instruct2026.05 | 9.81 | 15.32 | 17.49 | |
| Reference SFTBackbone=OPENCODER-8B-Instruct2026.05 | 9.34 | 14.13 | 16.88 | |
| QWEN2.5-CODER-7B-BaseBackbone=QWEN2.5-CODER-7B-Instruct2026.05 | 9.04 | 15.35 | 17.96 | |
| MAGPIE-HumanBackbone=OPENCODER-8B-Instruct2026.05 | 8.77 | 13.83 | 15.88 | |
| OPENCODER-8B-BaseBackbone=OPENCODER-8B-Instruct2026.05 | 7.96 | 14.56 | 17.36 |