Fill-in-the-Middle Code Completion on Real-FIM-Eval
4.4Exact Match (EM)SFT
Evaluation results
| Method | Links | ||
|---|---|---|---|
| SFTModel Size=7B, Truncation Strategy=+trunc2026.06 | 4.4 | 33.9 | |
| SFTModel Size=7B, Truncation Strategy=untruncated2026.06 | 4.2 | 33.1 | |
| ORPOModel Size=7B, Truncation Strategy=+trunc2026.06 | 3.8 | 34.3 | |
| SFTModel Size=3B, Truncation Strategy=+trunc2026.06 | 3.7 | 32.5 | |
| ORPOModel Size=7B, Truncation Strategy=untruncated2026.06 | 3.7 | 33.8 | |
| SFTModel Size=3B, Truncation Strategy=untruncated2026.06 | 3.6 | 31.8 | |
| ORPOModel Size=3B, Truncation Strategy=untruncated2026.06 | 3.3 | 32.9 | |
| ORPOModel Size=3B, Truncation Strategy=+trunc2026.06 | 3.3 | 33.3 | |
| DPOModel Size=7B, Truncation Strategy=+trunc2026.06 | 2.4 | 34 | |
| DPOModel Size=3B, Truncation Strategy=+trunc2026.06 | 2.3 | 33.6 | |
| DPOModel Size=7B, Truncation Strategy=untruncated2026.06 | 1.7 | 19.3 | |
| DPOModel Size=3B, Truncation Strategy=untruncated2026.06 | 1.2 | 18.9 | |
| baseModel Size=3B, Truncation Strategy=+trunc2026.06 | 0.4 | 18.5 | |
| baseModel Size=7B, Truncation Strategy=+trunc2026.06 | 0.4 | 18.2 | |
| baseModel Size=3B, Truncation Strategy=untruncated2026.06 | 0.2 | 14.3 | |
| baseModel Size=7B, Truncation Strategy=untruncated2026.06 | 0.2 | 14 |