Model Alignment Evaluation on Re:Form SFT Evaluation Set (val)
94.3Validation Rate14B SFT
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| 14B SFTModel Size=14B, Training Protocol=SFT2025.07 | 94.3 | 62.9 | 34.2 | |
| 7B SFTModel Size=7B, Training Protocol=SFT2025.07 | 90.8 | 53.3 | 27.9 | |
| 3B SFTModel Size=3B, Training Protocol=SFT2025.07 | 88.7 | 48 | 26.6 | |
| 1.5B SFTModel Size=1.5B, Training Protocol=SFT2025.07 | 84.2 | 41.6 | 22.1 | |
| 0.5B SFTModel Size=0.5B, Training Protocol=SFT2025.07 | 80.1 | 33.6 | 18 | |
| GPT-4o2025.07 | 47.7 | 12.1 | 7 | |
| Qwen-Coder-7BModel Size=7B, Training Protocol=Base2025.07 | 17.6 | 3.7 | 0 | |
| Qwen-Coder-3BModel Size=3B, Training Protocol=Base2025.07 | 6.6 | 2.3 | 0.2 | |
| Qwen-Coder-14BModel Size=14B, Training Protocol=Base2025.07 | 5.9 | 2.5 | 0.4 | |
| Qwen-Coder-1.5BModel Size=1.5B, Training Protocol=Base2025.07 | 5.5 | 1.2 | 0 | |
| Qwen-Coder-0.5BModel Size=0.5B, Training Protocol=Base2025.07 | 3.5 | 1.6 | 0 |