Code Generation on CodeContests (Avg@8)
39.33Avg@8CodeScaler
Evaluation Results
| Method | Links | |
|---|---|---|
| CodeScalerBase Model=Qwen3-14B-Base, Reward Feedback Source=CodeScaler2026.02 | 39.33 | |
| RLVR (Binary)Base Model=Qwen3-14B-Base, Reward Feedback Source=RLVR (Binary)2026.02 | 34.3 | |
| CodeScalerBase Model=Qwen3-8B-Base, Reward Feedback Source=CodeScaler2026.02 | 33.94 | |
| RLVR (Binary)Training Dataset=rStarCoder, Backbone Model=Qwen3-14B, Training Protocol=RLVR2026.02 | 33.26 | |
| CodeScalerTraining Dataset=KodCode, Backbone Model=Qwen3-14B, Training Protocol=CodeScaler2026.02 | 32.89 | |
| CodeScalerTraining Dataset=rStarCoder, Backbone Model=Qwen3-14B, Training Protocol=CodeScaler2026.02 | 32.63 | |
| RLVR (Binary)Training Dataset=KodCode, Backbone Model=Qwen3-14B, Training Protocol=RLVR2026.02 | 32.32 | |
| AceCodeRMBase Model=Qwen3-14B-Base, Reward Feedback Source=AceCodeRM2026.02 | 32.11 | |
| AEROBackbone=Qwen2.5-7B-Instruct-1M, Time (min)=6.95, FLOPs (PetaFLOP)=2295, p-value=—2026.02 | 32.1 | |
| RLVR (Binary)Base Model=Qwen3-8B-Base, Reward Feedback Source=RLVR (Binary)2026.02 | 32 | |
| SkyworkRMBase Model=Qwen3-14B-Base, Reward Feedback Source=SkyworkRM2026.02 | 31.64 | |
| DAPOBackbone=Qwen2.5-7B-Instruct-1M, Time (min)=7.75, FLOPs (PetaFLOP)=2941, p-value== 0.0342026.02 | 31.3 | |
| CodeScalerTraining Dataset=rStarCoder, Backbone Model=Qwen3-8B, Training Protocol=CodeScaler2026.02 | 30.75 | |
| GRPOBackbone=Qwen2.5-7B-Instruct-1M, Time (min)=11.66, FLOPs (PetaFLOP)=4488, p-value=< 0.0052026.02 | 29.6 | |
| RLVR (Binary)Training Dataset=rStarCoder, Backbone Model=Qwen3-8B, Training Protocol=RLVR2026.02 | 29.44 | |
| CodeScalerTraining Dataset=KodCode, Backbone Model=Qwen3-8B, Training Protocol=CodeScaler2026.02 | 29.34 | |
| AceCodeRMBase Model=Qwen3-8B-Base, Reward Feedback Source=AceCodeRM2026.02 | 28.34 | |
| RLVR (Binary)Training Dataset=KodCode, Backbone Model=Qwen3-8B, Training Protocol=RLVR2026.02 | 27.87 | |
| Qwen3-14B-BaseTraining Dataset=KodCode, Backbone Model=Qwen3-14B, Training Protocol=Base2026.02 | 26.25 | |
| Qwen3-14B-BaseTraining Dataset=rStarCoder, Backbone Model=Qwen3-14B, Training Protocol=Base2026.02 | 26.25 | |
| Qwen3-14B-BaseBase Model=Qwen3-14B-Base, Reward Feedback Source=None (Base)2026.02 | 26.25 | |
| SkyworkRMBase Model=Qwen3-8B-Base, Reward Feedback Source=SkyworkRM2026.02 | 23.22 | |
| Base ModelBackbone=Qwen2.5-7B-Instruct-1M, Time (min)=—, FLOPs (PetaFLOP)=—, p-value=< 1e-52026.02 | 19.5 | |
| Qwen3-8B-BaseTraining Dataset=KodCode, Backbone Model=Qwen3-8B, Training Protocol=Base2026.02 | 19.03 | |
| Qwen3-8B-BaseTraining Dataset=rStarCoder, Backbone Model=Qwen3-8B, Training Protocol=Base2026.02 | 19.03 | |
| Qwen3-8B-BaseBase Model=Qwen3-8B-Base, Reward Feedback Source=None (Base)2026.02 | 19.03 |