Fill-in-the-Middle Code Completion on HE-Inf multi-line
27.1Exact Match (EM)base
Evaluation results
| Method | Links | ||
|---|---|---|---|
| baseModel Size=3B, Truncation Strategy=untruncated2026.06 | 27.1 | 0.73 | |
| baseModel Size=3B, Truncation Strategy=+trunc2026.06 | 27.1 | 0.761 | |
| ORPOModel Size=7B, Truncation Strategy=untruncated2026.06 | 23.7 | 0.694 | |
| ORPOModel Size=7B, Truncation Strategy=+trunc2026.06 | 23.7 | 0.691 | |
| SFTModel Size=7B, Truncation Strategy=untruncated2026.06 | 22.9 | 0.69 | |
| SFTModel Size=7B, Truncation Strategy=+trunc2026.06 | 22.9 | 0.687 | |
| DPOModel Size=3B, Truncation Strategy=+trunc2026.06 | 21.2 | 0.743 | |
| DPOModel Size=3B, Truncation Strategy=untruncated2026.06 | 19.5 | 0.707 | |
| baseModel Size=7B, Truncation Strategy=+trunc2026.06 | 19.5 | 0.715 | |
| baseModel Size=7B, Truncation Strategy=untruncated2026.06 | 16.1 | 0.487 | |
| SFTModel Size=3B, Truncation Strategy=untruncated2026.06 | 11 | 0.581 | |
| SFTModel Size=3B, Truncation Strategy=+trunc2026.06 | 11 | 0.601 | |
| ORPOModel Size=3B, Truncation Strategy=untruncated2026.06 | 9.3 | 0.55 | |
| ORPOModel Size=3B, Truncation Strategy=+trunc2026.06 | 9.3 | 0.564 | |
| DPOModel Size=7B, Truncation Strategy=+trunc2026.06 | 5.1 | 0.647 | |
| DPOModel Size=7B, Truncation Strategy=untruncated2026.06 | 3.4 | 0.172 |