Code Generation on LCB v5 (pass@1)
49.5pass@1Lightning OPD
Evaluation Results
| Method | Links | |
|---|---|---|
| Lightning OPDStudent=Qwen3-8B-Base, Teacher=Qwen3-32B2026.04 | 49.5 | |
| OPDStudent=Qwen3-8B-Base, Teacher=Qwen3-32B2026.04 | 47.3 | |
| SFTStudent=Qwen3-8B-Base, Teacher=Qwen3-32B2026.04 | 44.7 | |
| OPDStudent=Qwen3-4B-Base, Teacher=Qwen3-8B2026.04 | 44.2 | |
| Lightning OPDStudent=Qwen3-4B-Base, Teacher=Qwen3-8B2026.04 | 42.8 | |
| SFTStudent=Qwen3-4B-Base, Teacher=Qwen3-8B2026.04 | 33.8 | |
| PriFT-massBackbone=Qwen2.5-Coder-3B2026.06 | 22.27 | |
| OriginalBackbone=Qwen2.5-Coder-3B2026.06 | 21.25 | |
| ASFTBackbone=Qwen2.5-Coder-3B2026.06 | 20.8 | |
| GASP + Real-data RLBase Model=Qwen2.5-Coder-7B, Decoding=Greedy2026.03 | 20.52 | |
| GASPBase Model=Qwen2.5-Coder-7B, Decoding=Greedy2026.03 | 19.75 | |
| Real-data RLBase Model=Qwen2.5-Coder-7B, Decoding=Greedy2026.03 | 18.21 | |
| PriFT-probBackbone=Qwen2.5-Coder-3B2026.06 | 17.73 | |
| EAFTBackbone=Qwen2.5-Coder-3B2026.06 | 17.61 | |
| AZRBase Model=Qwen2.5-Coder-7B, Decoding=Greedy2026.03 | 17.13 | |
| SFTBackbone=Qwen2.5-Coder-3B2026.06 | 17.05 | |
| IDFTBackbone=Qwen2.5-Coder-3B2026.06 | 16.36 | |
| Qwen2.5-Coder-7BDecoding=Greedy2026.03 | 13.89 | |
| TALRBackbone=Qwen2.5-Coder-3B2026.06 | 12.84 | |
| DFTBackbone=Qwen2.5-Coder-3B2026.06 | 12.73 |