Code Generation on Average dataset-weighted (test)
34.1Sig. ScoreCURE-14B + CoSPlay w/ Cluster
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| CURE-14B + CoSPlay w/ ClusterBackbone=Ins., #GT Data=4.5k2026.05 | 34.1 | 80.1 | 36 | 46.2 | |
| Qwen2.5-14B-Ins. + CoSPlay w/ ClusterBackbone=Ins., #GT Data=02026.05 | 34 | 77.6 | 30.8 | 42 | |
| Qwen2.5-14B-Ins. + CoSPlayBackbone=Ins., #GT Data=02026.05 | 33.8 | 77.6 | 30.8 | 41.2 | |
| CURE-14B + CoSPlayBackbone=Ins., #GT Data=4.5k2026.05 | 33.8 | 80.1 | 36 | 45.9 | |
| CURE-14BBackbone=Ins., #GT Data=4.5k2026.05 | 33.6 | 82.4 | 30.1 | 41.8 | |
| CURE-7B + CoSPlay w/ ClusterBackbone=Ins., #GT Data=4.5k2026.05 | 33.4 | 78.3 | 27.7 | 38.6 | |
| CURE-7B + CoSPlayBackbone=Ins., #GT Data=4.5k2026.05 | 33 | 78.3 | 27.7 | 38.4 | |
| Qwen2.5-7B-Ins. + CoSPlay w/ ClusterBackbone=Ins., #GT Data=02026.05 | 31.9 | 78.3 | 23.3 | 33.2 | |
| Qwen2.5-7B-Ins. + CoSPlayBackbone=Ins., #GT Data=02026.05 | 31.3 | 78.3 | 23.3 | 32.6 | |
| CURE-7BBackbone=Ins., #GT Data=4.5k2026.05 | 31 | 54.9 | 22.5 | 32.9 | |
| DeepSeek-Coder-V2-16BBackbone=-, #GT Data=-2026.05 | 30.1 | 41 | 19.5 | 29.6 | |
| Qwen2.5-14B-Instruct + CodeTBackbone=Ins., #GT Data=-2026.05 | 29.7 | 32.4 | 23.9 | 34.3 | |
| Qwen2.5-14B-InstructBackbone=Ins., #GT Data=-2026.05 | 29.1 | 32.4 | 23.9 | 33.2 | |
| Qwen2.5-14B-Instruct + ClusterBackbone=Ins., #GT Data=-2026.05 | 29 | 32.4 | 23.9 | 33.8 | |
| Qwen2.5-14B-Coder-Ins.Backbone=Coder-Ins., #GT Data=-2026.05 | 27.2 | 31.5 | 26.5 | 32.8 | |
| Qwen2.5-14B-CoderBackbone=Coder, #GT Data=-2026.05 | 26.3 | 19.5 | 6.7 | 20.1 | |
| AZR-7B-CoderBackbone=Coder, #GT Data=02026.05 | 26.1 | 27 | 10.2 | 21.9 | |
| Seed-Coder-8B-InstructBackbone=-, #GT Data=-2026.05 | 25.2 | 25.5 | 21.4 | 28.2 | |
| Qwen2.5-7B-Instruct + ClusterBackbone=Ins., #GT Data=-2026.05 | 24.4 | 14.6 | 19 | 24.4 | |
| AZR-14B-CoderBackbone=Coder, #GT Data=02026.05 | 24.3 | 26.2 | 25.1 | 30.3 | |
| AceCoder-7B-RuleBackbone=Coder-Ins., #GT Data=22k2026.05 | 23.6 | 13.7 | 20.5 | 24.8 | |
| AceCoder-7B-RMBackbone=Coder-Ins., #GT Data=329k2026.05 | 23.6 | 15.1 | 20.6 | 25.8 | |
| Qwen2.5-7B-Coder-Ins.Backbone=Coder-Ins., #GT Data=-2026.05 | 23.5 | 10.4 | 20 | 24.5 | |
| Qwen2.5-7B-CoderBackbone=Coder, #GT Data=-2026.05 | 23.4 | 13.3 | 3 | 12 | |
| Qwen2.5-7B-Instruct + CodeTBackbone=Ins., #GT Data=-2026.05 | 22.9 | 14.6 | 19 | 22.9 | |
| Qwen2.5-7B-InstructBackbone=Ins., #GT Data=-2026.05 | 22.1 | 14.6 | 19 | 22.1 |