Fill-in-the-Middle Code Completion on SAFIM API
72.6Exact Match (%)SFT
Evaluation results
| Method | Links | ||
|---|---|---|---|
| SFTModel Size=7B, Truncation Strategy=untruncated2026.06 | 72.6 | 91.4 | |
| SFTModel Size=7B, Truncation Strategy=+trunc2026.06 | 72.6 | 91.4 | |
| ORPOModel Size=7B, Truncation Strategy=untruncated2026.06 | 71 | 90.8 | |
| ORPOModel Size=7B, Truncation Strategy=+trunc2026.06 | 71 | 90.8 | |
| SFTModel Size=3B, Truncation Strategy=+trunc2026.06 | 69.4 | 88.4 | |
| SFTModel Size=3B, Truncation Strategy=untruncated2026.06 | 69 | 88.5 | |
| ORPOModel Size=3B, Truncation Strategy=untruncated2026.06 | 66.8 | 87.7 | |
| ORPOModel Size=3B, Truncation Strategy=+trunc2026.06 | 66.8 | 87.7 | |
| baseModel Size=7B, Truncation Strategy=+trunc2026.06 | 58.7 | 83.8 | |
| baseModel Size=3B, Truncation Strategy=+trunc2026.06 | 55.5 | 83.2 | |
| baseModel Size=7B, Truncation Strategy=untruncated2026.06 | 53.9 | 68.7 | |
| baseModel Size=3B, Truncation Strategy=untruncated2026.06 | 50.3 | 66.7 | |
| DPOModel Size=7B, Truncation Strategy=+trunc2026.06 | 39.7 | 71.5 | |
| DPOModel Size=7B, Truncation Strategy=untruncated2026.06 | 38.1 | 47.4 | |
| DPOModel Size=3B, Truncation Strategy=+trunc2026.06 | 26.5 | 64.9 | |
| DPOModel Size=3B, Truncation Strategy=untruncated2026.06 | 25.5 | 37 |