Code Generation on MBPP+ (avg@16 accuracy)
73.8Avg@16 AccuracyTRD
Evaluation results
| Method | Links | |
|---|---|---|
| TRDModel Backbone=Qwen3-4B-Instruct-2507, Trajectory Type (Traj.)=yr, Teacher Model=Qwen3-8B2026.06 | 73.8 | |
| BaseModel Backbone=Qwen3-4B-Instruct-2507, Trajectory Type (Traj.)=-, Teacher Model=Qwen3-8B2026.06 | 73.5 | |
| Forward KL w/ ClipModel Backbone=Qwen3-4B-Instruct-2507, Trajectory Type (Traj.)=yo, Teacher Model=Qwen3-8B2026.06 | 73.3 | |
| Reverse KLModel Backbone=Qwen3-4B-Instruct-2507, Trajectory Type (Traj.)=yo, Teacher Model=Qwen3-8B2026.06 | 73.3 | |
| Forward KLModel Backbone=Qwen3-4B-Instruct-2507, Trajectory Type (Traj.)=yo, Teacher Model=Qwen3-8B2026.06 | 72.6 | |
| Reverse KL w/ Top-KModel Backbone=Qwen3-4B-Instruct-2507, Trajectory Type (Traj.)=yo, Teacher Model=Qwen3-8B2026.06 | 72.2 | |
| Qwen3-8B-Base + PASC-GRPOmax_completion_tokens=8K2026.01 | 65.24 | |
| TRDModel=Qwen3-4B-Instruct-2507, Traj.=yr2026.06 | 65.2 | |
| Reverse KLModel=Qwen3-4B-Instruct-2507, Traj.=yo2026.06 | 64.8 | |
| Qwen3-8B-Base + GRP-SFTmax_completion_tokens=8K2026.01 | 64.71 | |
| BaseModel=Qwen3-4B-Instruct-2507, Traj.=–2026.06 | 64.6 | |
| Forward KL w/ ClipModel=Qwen3-4B-Instruct-2507, Traj.=yo2026.06 | 64.6 | |
| Reverse KL w/ Top-KModel=Qwen3-4B-Instruct-2507, Traj.=yo2026.06 | 64.3 | |
| Forward KLModel=Qwen3-4B-Instruct-2507, Traj.=yo2026.06 | 64.2 | |
| Gemma-3-27B-ITmax_completion_tokens=8K2026.01 | 63 | |
| TRDModel Backbone=Qwen3-1.7B (w/ thinking), Trajectory Type (Traj.)=yr, Teacher Model=Qwen3-8B2026.06 | 62.7 | |
| Reverse KL w/ Top-KModel Backbone=Qwen3-1.7B (w/ thinking), Trajectory Type (Traj.)=yo, Teacher Model=Qwen3-8B2026.06 | 62.2 | |
| Gemma-3-12B-ITmax_completion_tokens=8K2026.01 | 62.1 | |
| Forward KL w/ ClipModel Backbone=Qwen3-1.7B (w/ thinking), Trajectory Type (Traj.)=yo, Teacher Model=Qwen3-8B2026.06 | 61.9 | |
| Qwen3-8B-Basemax_completion_tokens=8K2026.01 | 61.73 | |
| Reverse KLModel Backbone=Qwen3-1.7B (w/ thinking), Trajectory Type (Traj.)=yo, Teacher Model=Qwen3-8B2026.06 | 61.4 | |
| BaseModel Backbone=Qwen3-1.7B (w/ thinking), Trajectory Type (Traj.)=-, Teacher Model=Qwen3-8B2026.06 | 60.8 | |
| Forward KLModel Backbone=Qwen3-1.7B (w/ thinking), Trajectory Type (Traj.)=yo, Teacher Model=Qwen3-8B2026.06 | 58.2 | |
| Qwen3-4B-Base + PASC-GRPOmax_completion_tokens=8K2026.01 | 55.43 | |
| Qwen3-4B-Base + GRP-SFTmax_completion_tokens=8K2026.01 | 54.2 | |
| LLaMA-3.1-8B-Instructmax_completion_tokens=8K2026.01 | 52.3 | |
| Qwen3-4B-Basemax_completion_tokens=8K2026.01 | 51.3 | |
| TRDModel=Qwen3-1.7B (w/ thinking), Traj.=yr2026.06 | 51.2 | |
| Forward KLModel=Qwen3-1.7B (w/ thinking), Traj.=yo2026.06 | 50.9 | |
| Forward KL w/ ClipModel=Qwen3-1.7B (w/ thinking), Traj.=yo2026.06 | 50.3 | |
| Reverse KL w/ Top-KModel=Qwen3-1.7B (w/ thinking), Traj.=yo2026.06 | 50.3 | |
| Reverse KLModel=Qwen3-1.7B (w/ thinking), Traj.=yo2026.06 | 50.2 | |
| BaseModel=Qwen3-1.7B (w/ thinking), Traj.=–2026.06 | 50.1 |