Code Execution on Live Code Execution
62.04Pass@1DOMINO
Evaluation results
| Method | Links | |
|---|---|---|
| DOMINOBackbone=QWEN2.5-CODER-14B-Instruct2026.05 | 62.04 | |
| DOMINO-Direct DomainBackbone=QWEN2.5-CODER-14B-Instruct2026.05 | 60.19 | |
| QWEN2.5-CODER-14B-InstructBackbone=QWEN2.5-CODER-14B-Instruct2026.05 | 59.26 | |
| MAGPIE-HumanBackbone=QWEN2.5-CODER-14B-Instruct2026.05 | 57.41 | |
| MAGPIE-Few ShotBackbone=QWEN2.5-CODER-14B-Instruct2026.05 | 57.41 | |
| DOMINOBackbone=QWEN2.5-CODER-7B-Instruct2026.05 | 56.48 | |
| QWEN2.5-CODER-7B-InstructBackbone=QWEN2.5-CODER-7B-Instruct2026.05 | 55.55 | |
| DOMINO-Direct DomainBackbone=QWEN2.5-CODER-7B-Instruct2026.05 | 50.92 | |
| MAGPIE-Few ShotBackbone=QWEN2.5-CODER-7B-Instruct2026.05 | 48.15 | |
| QWEN2.5-CODER-7B-BaseBackbone=QWEN2.5-CODER-7B-Instruct2026.05 | 45.37 | |
| QWEN2.5-CODER-14B-BaseBackbone=QWEN2.5-CODER-14B-Instruct2026.05 | 45.37 | |
| DOMINOBackbone=OPENCODER-8B-Instruct2026.05 | 42.59 | |
| MAGPIE-HumanBackbone=QWEN2.5-CODER-7B-Instruct2026.05 | 42.59 | |
| DOMINO-Direct DomainBackbone=OPENCODER-8B-Instruct2026.05 | 38.88 | |
| Reference SFTBackbone=QWEN2.5-CODER-14B-Instruct2026.05 | 38.88 | |
| OPENCODER-8B-InstructBackbone=OPENCODER-8B-Instruct2026.05 | 37.96 | |
| Reference SFTBackbone=QWEN2.5-CODER-7B-Instruct2026.05 | 36.11 | |
| MAGPIE-Few ShotBackbone=OPENCODER-8B-Instruct2026.05 | 34.26 | |
| MAGPIE-HumanBackbone=OPENCODER-8B-Instruct2026.05 | 32.41 | |
| Reference SFTBackbone=OPENCODER-8B-Instruct2026.05 | 27.78 | |
| OPENCODER-8B-BaseBackbone=OPENCODER-8B-Instruct2026.05 | 25.93 |