Fill-in-the-Middle Code Completion on HE-Inf random-span
50.4Exact Match (EM)base
Evaluation results
| Method | Links | ||
|---|---|---|---|
| baseModel Size=3B, Truncation Strategy=+trunc2026.06 | 50.4 | 81 | |
| DPOModel Size=3B, Truncation Strategy=untruncated2026.06 | 47.2 | 74.2 | |
| SFTModel Size=7B, Truncation Strategy=+trunc2026.06 | 47.2 | 76.5 | |
| ORPOModel Size=7B, Truncation Strategy=+trunc2026.06 | 47.2 | 77.4 | |
| ORPOModel Size=7B, Truncation Strategy=untruncated2026.06 | 46.5 | 77 | |
| SFTModel Size=7B, Truncation Strategy=untruncated2026.06 | 44.9 | 74.5 | |
| SFTModel Size=3B, Truncation Strategy=+trunc2026.06 | 42.5 | 72.9 | |
| baseModel Size=3B, Truncation Strategy=untruncated2026.06 | 41.7 | 70.2 | |
| SFTModel Size=3B, Truncation Strategy=untruncated2026.06 | 41.7 | 72.3 | |
| ORPOModel Size=3B, Truncation Strategy=untruncated2026.06 | 40.2 | 71 | |
| baseModel Size=7B, Truncation Strategy=+trunc2026.06 | 40.2 | 75.7 | |
| DPOModel Size=3B, Truncation Strategy=+trunc2026.06 | 39.4 | 68.4 | |
| ORPOModel Size=3B, Truncation Strategy=+trunc2026.06 | 39.4 | 71.2 | |
| baseModel Size=7B, Truncation Strategy=untruncated2026.06 | 34.6 | 60.1 | |
| DPOModel Size=7B, Truncation Strategy=+trunc2026.06 | 21.3 | 64.1 | |
| DPOModel Size=7B, Truncation Strategy=untruncated2026.06 | 8.7 | 22.2 |