Fill-in-the-Middle Code Completion on HE-Inf single-line
57.3Exact Match (EM)SFT
Evaluation results
| Method | Links | ||
|---|---|---|---|
| SFTModel Size=7B, Truncation Strategy=untruncated2026.06 | 57.3 | 84.9 | |
| SFTModel Size=7B, Truncation Strategy=+trunc2026.06 | 57.3 | 84.7 | |
| ORPOModel Size=7B, Truncation Strategy=untruncated2026.06 | 56.7 | 84.8 | |
| ORPOModel Size=7B, Truncation Strategy=+trunc2026.06 | 56.7 | 84.7 | |
| ORPOModel Size=3B, Truncation Strategy=untruncated2026.06 | 54.9 | 84.2 | |
| ORPOModel Size=3B, Truncation Strategy=+trunc2026.06 | 54.9 | 84.2 | |
| baseModel Size=3B, Truncation Strategy=untruncated2026.06 | 52.4 | 81.5 | |
| baseModel Size=3B, Truncation Strategy=+trunc2026.06 | 52.4 | 83 | |
| DPOModel Size=3B, Truncation Strategy=untruncated2026.06 | 52.4 | 74 | |
| SFTModel Size=3B, Truncation Strategy=untruncated2026.06 | 51.8 | 80.8 | |
| SFTModel Size=3B, Truncation Strategy=+trunc2026.06 | 51.8 | 81.1 | |
| DPOModel Size=3B, Truncation Strategy=+trunc2026.06 | 48.8 | 63.3 | |
| baseModel Size=7B, Truncation Strategy=+trunc2026.06 | 43.3 | 76.7 | |
| baseModel Size=7B, Truncation Strategy=untruncated2026.06 | 40.2 | 66 | |
| DPOModel Size=7B, Truncation Strategy=+trunc2026.06 | 12.8 | 59.6 | |
| DPOModel Size=7B, Truncation Strategy=untruncated2026.06 | 8.5 | 18.3 |