Fill-in-the-Middle Code Completion on SAFIM block v2
37.6Exact Match (EM)base
Evaluation results
| Method | Links | ||
|---|---|---|---|
| baseModel Size=7B, Truncation Strategy=+trunc2026.06 | 37.6 | 65.5 | |
| baseModel Size=3B, Truncation Strategy=+trunc2026.06 | 37.5 | 66.3 | |
| SFTModel Size=7B, Truncation Strategy=untruncated2026.06 | 32.9 | 58 | |
| SFTModel Size=7B, Truncation Strategy=+trunc2026.06 | 32.9 | 58.1 | |
| ORPOModel Size=7B, Truncation Strategy=+trunc2026.06 | 32.7 | 56 | |
| ORPOModel Size=7B, Truncation Strategy=untruncated2026.06 | 32.5 | 56 | |
| SFTModel Size=3B, Truncation Strategy=untruncated2026.06 | 31.1 | 56.6 | |
| SFTModel Size=3B, Truncation Strategy=+trunc2026.06 | 31.1 | 56.8 | |
| ORPOModel Size=3B, Truncation Strategy=untruncated2026.06 | 31.1 | 54.8 | |
| ORPOModel Size=3B, Truncation Strategy=+trunc2026.06 | 31.1 | 54.8 | |
| DPOModel Size=7B, Truncation Strategy=+trunc2026.06 | 30.2 | 59.7 | |
| baseModel Size=3B, Truncation Strategy=untruncated2026.06 | 27.3 | 51.4 | |
| DPOModel Size=3B, Truncation Strategy=+trunc2026.06 | 24.5 | 55.1 | |
| DPOModel Size=7B, Truncation Strategy=untruncated2026.06 | 22.2 | 41.3 | |
| DPOModel Size=3B, Truncation Strategy=untruncated2026.06 | 15.3 | 34.8 | |
| baseModel Size=7B, Truncation Strategy=untruncated2026.06 | 13.4 | 28.2 |