Code Generation on MBPP (P@1, TTFR)
56.8Pass@1 AccuracyPlan-Answer
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Plan-AnswerModel=Qwen3-8B, Decoding Strategy=Plan-Answer2025.12 | 56.8 | 554 | |
| Think-AnswerModel=Qwen3-8B, Decoding Strategy=Think-Answer2025.12 | 54.5 | 1,417 | |
| Plan-AnswerModel=Qwen3-4B, Decoding Strategy=Plan-Answer2025.12 | 52.6 | 523 | |
| No ThinkingModel=Qwen3-8B, Decoding Strategy=No Thinking2025.12 | 51.4 | 0 | |
| Think-AnswerModel=Qwen3-4B, Decoding Strategy=Think-Answer2025.12 | 50.8 | 1,298 | |
| No ThinkingModel=Qwen3-4B, Decoding Strategy=No Thinking2025.12 | 48.2 | 0 |