Code Generation on MBPP (avg@16 accuracy)
77.49MBPP avg@16 AccuracyQwen3-8B-Base + PASC-GRPO
Evaluation results
| Method | Links | |
|---|---|---|
| Qwen3-8B-Base + PASC-GRPOmax_completion_tokens=8K2026.01 | 77.49 | |
| Qwen3-8B-Base + GRP-SFTmax_completion_tokens=8K2026.01 | 75.45 | |
| Gemma-3-27B-ITmax_completion_tokens=8K2026.01 | 74.4 | |
| Qwen3-8B-Basemax_completion_tokens=8K2026.01 | 73.27 | |
| Gemma-3-12B-ITmax_completion_tokens=8K2026.01 | 73 | |
| Qwen3-4B-Base + PASC-GRPOmax_completion_tokens=8K2026.01 | 67.33 | |
| Qwen3-4B-Base + GRP-SFTmax_completion_tokens=8K2026.01 | 65.43 | |
| Qwen3-4B-Basemax_completion_tokens=8K2026.01 | 62.4 | |
| LLaMA-3.1-8B-Instructmax_completion_tokens=8K2026.01 | 61.2 |