Fill-in-the-Middle Code Completion on SAFIM control
41Exact Match (EM)ORPO
Evaluation results
| Method | Links | ||
|---|---|---|---|
| ORPOModel Size=7B, Truncation Strategy=untruncated2026.06 | 41 | 72.7 | |
| ORPOModel Size=7B, Truncation Strategy=+trunc2026.06 | 41 | 72.7 | |
| SFTModel Size=7B, Truncation Strategy=+trunc2026.06 | 39.3 | 72.3 | |
| SFTModel Size=7B, Truncation Strategy=untruncated2026.06 | 39.2 | 72.2 | |
| SFTModel Size=3B, Truncation Strategy=untruncated2026.06 | 36.4 | 69.6 | |
| SFTModel Size=3B, Truncation Strategy=+trunc2026.06 | 36.4 | 69.8 | |
| baseModel Size=3B, Truncation Strategy=+trunc2026.06 | 35.3 | 66.7 | |
| baseModel Size=3B, Truncation Strategy=untruncated2026.06 | 34.7 | 57.7 | |
| ORPOModel Size=3B, Truncation Strategy=untruncated2026.06 | 34.3 | 69.5 | |
| ORPOModel Size=3B, Truncation Strategy=+trunc2026.06 | 34.3 | 69.7 | |
| DPOModel Size=3B, Truncation Strategy=+trunc2026.06 | 15 | 53.2 | |
| DPOModel Size=3B, Truncation Strategy=untruncated2026.06 | 14 | 29.1 | |
| DPOModel Size=7B, Truncation Strategy=+trunc2026.06 | 12 | 51.8 | |
| DPOModel Size=7B, Truncation Strategy=untruncated2026.06 | 8.9 | 16.3 | |
| baseModel Size=7B, Truncation Strategy=+trunc2026.06 | 7.6 | 51 | |
| baseModel Size=7B, Truncation Strategy=untruncated2026.06 | 4.8 | 9.9 |