Code Generation on MBPP (Pass@k, JDiv, Vendi)
93.2Pass@1JPlag-RLVR
Evaluation Results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| JPlag-RLVRModel=QWEN3-4B2026.05 | 93.2 | 98.9 | 99.3 | 0.822 | 0.446 | 1.82 | |
| JPlag-RLVRModel=QWEN3-8B2026.05 | 90 | 98.5 | 99.2 | 0.927 | 0.598 | 2.71 | |
| PKPOModel=QWEN3-8B2026.05 | 79.3 | 97.1 | 99 | 0.583 | 0.342 | 1.78 | |
| Base-RLVRModel=QWEN3-8B2026.05 | 79.1 | 88 | 90.8 | 0.305 | 0.143 | 1.24 | |
| Pass@k-RLVRModel=QWEN3-8B2026.05 | 74.5 | 89.5 | 93.6 | 0.51 | 0.326 | 1.77 | |
| Pass@k-RLVRModel=QWEN3-4B2026.05 | 73.6 | 87.8 | 92.9 | 0.389 | 0.218 | 1.3 | |
| PKPOModel=QWEN3-4B2026.05 | 72.5 | 86.5 | 92.2 | 0.462 | 0.279 | 1.47 | |
| Base-RLVRModel=QWEN3-4B2026.05 | 72.4 | 82.4 | 86.8 | 0.301 | 0.153 | 1.23 | |
| JPlag-RLVRModel=OLMO3-7B2026.05 | 71.6 | 95 | 98.2 | 0.872 | 0.598 | 2.71 | |
| Base-RLVRModel=OLMO3-7B2026.05 | 64.8 | 84.1 | 91.9 | 0.555 | 0.395 | 1.81 | |
| PKPOModel=OLMO3-7B2026.05 | 64.7 | 86.7 | 94 | 0.569 | 0.373 | 1.85 | |
| Pass@k-RLVRModel=OLMO3-7B2026.05 | 60.9 | 83.7 | 92.5 | 0.632 | 0.446 | 2.21 |