Code Generation on MBPP Plus 3-shot (test)
42.6Pass@1ESPO
Evaluation results
| Method | Links | ||
|---|---|---|---|
| ESPOSequence Length=512, Few-shot=32025.12 | 42.6 | — | |
| ESPOSequence Length=256, Few-shot=32025.12 | 41.6 | — | |
| LLaDA-1.5Sequence Length=256, Few-shot=32025.12 | 40.4 | — | |
| diffu-GRPO(d1)Sequence Length=512, Few-shot=32025.12 | 39.4 | — | |
| ESPOSequence Length=128, Few-shot=32025.12 | 38.9 | — | |
| LLaDA-1.5Sequence Length=128, Few-shot=32025.12 | 38.8 | — | |
| LLaDASequence Length=512, Few-shot=32025.12 | 37.8 | — | |
| LLaDA-1.5Sequence Length=512, Few-shot=32025.12 | 37.3 | — | |
| LLaDASequence Length=128, Few-shot=32025.12 | 36.8 | — | |
| LLaDASequence Length=256, Few-shot=32025.12 | 36.8 | — | |
| wd1Sequence Length=512, Few-shot=32025.12 | 35.6 | — | |
| diffu-GRPO(d1)Sequence Length=256, Few-shot=32025.12 | 34.8 | — | |
| wd1Sequence Length=256, Few-shot=32025.12 | 32.8 | — | |
| diffu-GRPO(d1)Sequence Length=128, Few-shot=32025.12 | 31.8 | — | |
| wd1Sequence Length=128, Few-shot=32025.12 | 31.8 | — | |
| diffu-GRPO(d1)Few-shot=32025.12 | — | 35.3 | |
| ESPOFew-shot=32025.12 | — | 41 | |
| LLaDAFew-shot=32025.12 | — | 37.1 | |
| LLaDA-1.5Few-shot=32025.12 | — | 38.8 | |
| wd1Few-shot=32025.12 | — | 33.4 |