Fill-in-the-Middle Code Completion on SAFIM block
30.9Exact Match (EM)base
Evaluation results
| Method | Links | ||
|---|---|---|---|
| baseModel Size=3B, Truncation Strategy=+trunc2026.06 | 30.9 | 63.9 | |
| baseModel Size=7B, Truncation Strategy=+trunc2026.06 | 30.1 | 62.1 | |
| SFTModel Size=7B, Truncation Strategy=untruncated2026.06 | 26.3 | 52.3 | |
| SFTModel Size=7B, Truncation Strategy=+trunc2026.06 | 26.3 | 52.5 | |
| ORPOModel Size=7B, Truncation Strategy=+trunc2026.06 | 25.8 | 49.7 | |
| ORPOModel Size=7B, Truncation Strategy=+trunc2026.06 | 25.7 | 49.6 | |
| DPOModel Size=7B, Truncation Strategy=+trunc2026.06 | 24.7 | 56.7 | |
| baseModel Size=3B, Truncation Strategy=untruncated2026.06 | 24.6 | 50.8 | |
| SFTModel Size=3B, Truncation Strategy=untruncated2026.06 | 24.5 | 51.5 | |
| SFTModel Size=3B, Truncation Strategy=+trunc2026.06 | 24.5 | 51.9 | |
| ORPOModel Size=3B, Truncation Strategy=untruncated2026.06 | 23.9 | 48.8 | |
| ORPOModel Size=3B, Truncation Strategy=+trunc2026.06 | 23.9 | 49 | |
| DPOModel Size=7B, Truncation Strategy=untruncated2026.06 | 19.6 | 40.5 | |
| DPOModel Size=3B, Truncation Strategy=+trunc2026.06 | 19.3 | 52.4 | |
| DPOModel Size=3B, Truncation Strategy=untruncated2026.06 | 14.6 | 35.1 | |
| baseModel Size=7B, Truncation Strategy=untruncated2026.06 | 12.4 | 28.5 |