Code Generation on Code Category Average (test)
77.43AccuracyInstruct Model
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Instruct ModelModel Size=7B, Backbone=Qwen2.5-Instruct2026.03 | 77.43 | 0.57 | 0.22 | 0.23 | |
| CoCAModel Size=7B, Backbone=Qwen2.5-Instruct2026.03 | 76.6 | 0.72 | 0.16 | 0.19 | |
| RLVR+QuestionProbModel Size=7B, Backbone=Qwen2.5-Instruct2026.03 | 76.58 | 0.47 | 0.7 | 0.68 | |
| RLVR+Assessor ModelModel Size=7B, Backbone=Qwen2.5-Instruct2026.03 | 76.58 | 0.55 | 0.29 | 0.3 | |
| RLVR+ProbeModel Size=7B, Backbone=Qwen2.5-Instruct2026.03 | 76.58 | 0.62 | 0.71 | 0.68 | |
| RLVRModel Size=7B, Backbone=Qwen2.5-Instruct2026.03 | 76.1 | 0.55 | 0.23 | 0.23 | |
| RLVR+QuestionProbModel Size=3B, Backbone=Qwen2.5-Instruct2026.03 | 69.03 | 0.42 | 0.65 | 0.63 | |
| RLVR+Assessor ModelModel Size=3B, Backbone=Qwen2.5-Instruct2026.03 | 69.03 | 0.64 | 0.29 | 0.33 | |
| RLVR+ProbeModel Size=3B, Backbone=Qwen2.5-Instruct2026.03 | 69.03 | 0.59 | 0.37 | 0.39 | |
| CoCAModel Size=3B, Backbone=Qwen2.5-Instruct2026.03 | 67.38 | 0.67 | 0.28 | 0.3 | |
| Instruct ModelModel Size=3B, Backbone=Qwen2.5-Instruct2026.03 | 63.78 | 0.52 | 0.3 | 0.33 | |
| RLVR+QuestionProbModel Size=1.5B, Backbone=Qwen2.5-Instruct2026.03 | 57.15 | 0.46 | 0.48 | 0.48 | |
| RLVR+Assessor ModelModel Size=1.5B, Backbone=Qwen2.5-Instruct2026.03 | 57.15 | 0.63 | 0.25 | 0.35 | |
| RLVR+ProbeModel Size=1.5B, Backbone=Qwen2.5-Instruct2026.03 | 57.15 | 0.64 | 0.27 | 0.34 | |
| Instruct ModelModel Size=1.5B, Backbone=Qwen2.5-Instruct2026.03 | 55.1 | 0.53 | 0.41 | 0.42 | |
| CoCAModel Size=1.5B, Backbone=Qwen2.5-Instruct2026.03 | 46.72 | 0.72 | 0.09 | 0.22 | |
| RLVRModel Size=1.5B, Backbone=Qwen2.5-Instruct2026.03 | 39 | 0.58 | 0.56 | 0.55 | |
| RLVRModel Size=3B, Backbone=Qwen2.5-Instruct2026.03 | 34.58 | 0.48 | 0.56 | 0.54 |