Student Simulation on FalconCode 1.0 (test)
98.2CoverageDPO
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| DPOModel=Qwen3-4B2026.04 | 98.2 | 79.7 | 70 | -0.024 | -0.079 | -0.137 | |
| DPOModel=Qwen3-8B2026.04 | 95.7 | 79.6 | 71.4 | -0.061 | -0.074 | -0.129 | |
| SFTModel=Qwen3-4B2026.04 | 92.2 | 78.2 | 70.4 | -0.107 | -0.06 | -0.138 | |
| PARAModel=Qwen3-8B2026.04 | 92.1 | 78.5 | 71.4 | -0.112 | -0.059 | -0.121 | |
| SFTModel=Qwen3-8B2026.04 | 92.1 | 79.3 | 71.8 | -0.11 | -0.062 | -0.126 | |
| DAPOModel=Qwen3-8B2026.04 | 91.9 | 79.6 | 70.9 | -0.116 | -0.078 | -0.133 | |
| PARAModel=Qwen3-4B2026.04 | 91.8 | 76.3 | 69.5 | -0.118 | -0.071 | -0.137 | |
| DAPOModel=Qwen3-4B2026.04 | 85.9 | 78.1 | 70 | -0.203 | -0.068 | -0.138 | |
| BASEModel=Qwen3-8B2026.04 | 57 | 73.3 | 58.1 | -0.581 | 0.287 | -0.046 | |
| BASEModel=Qwen3-4B2026.04 | 52.8 | 69 | 49.1 | -0.64 | 0.278 | -0.063 | |
| BASEModel=GPT-5-mini2026.04 | 34 | 54.9 | 43.3 | -0.91 | 0.184 | -0.078 |