Code Generation on LiveCodeBench v5 (2024.08.01-2025.02.01)
44.8Average@8ASPO-Code-4B
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| ASPO-Code-4BBase Model=Qwen3-4B2025.10 | 44.8 | 58.1 | 41.6 | |
| Archer-Code-4B2025.07 | 42.1 | 57 | 39.8 | |
| Qwen3-4B (GRPO)training_algorithm=GRPO2025.07 | 40.8 | 55.2 | 38.7 | |
| GRPOBase Model=Qwen3-4B2025.10 | 40.8 | 55.2 | 38.7 | |
| ASPO-Code-1.5BBase Model=DeepSeek-R1-Distill-Qwen-1.5B2025.10 | 31.5 | 47 | 31 | |
| Archer-Code-1.5B2025.07 | 29.4 | 43.7 | 29.8 | |
| Nemotron-1.5B2025.07 | 26.1 | 35.5 | 27.8 | |
| Nemotron-1.5BBase Model=DeepSeek-R1-Distill-Qwen-1.5B2025.10 | 26.1 | 35.5 | 27.8 | |
| DeepSeek-R1-1.5B (GRPO)training_algorithm=GRPO2025.07 | 26 | 40.5 | 26.8 | |
| GRPOBase Model=DeepSeek-R1-Distill-Qwen-1.5B2025.10 | 26 | 40.5 | 26.8 | |
| Qwen3-4B2025.07 | 23.8 | 35.8 | 23.9 | |
| Qwen3-4BBase Model=Qwen3-4B2025.10 | 23.8 | 35.8 | 23.9 | |
| DeepCoder-1.5B2025.07 | 23.3 | 39.1 | 23 | |
| DeepCoder-1.5BBase Model=DeepSeek-R1-Distill-Qwen-1.5B2025.10 | 23.3 | 39.1 | 23 | |
| DeepSeek-R1-1.5B2025.07 | 16.7 | 29 | 17 | |
| DeepSeek-R1-1.5BBase Model=DeepSeek-R1-Distill-Qwen-1.5B2025.10 | 16.7 | 29 | 17 |