Code Generation on MBPP (Pass@1, MBPP+)
38.5Pass@1GRIP 16B
Evaluation results
| Method | Links | ||
|---|---|---|---|
| GRIP 16BModel Size=16B, Total Budget=300B tokens2026.02 | 38.5 | 44.8 | |
| GRIP 8BModel Size=8B, Total Budget=300B tokens2026.02 | 36 | 42.6 | |
| 6. GRIP (Full)Model=GRIP-8B, Total Budget=100B tokens2026.02 | 32.5 | 38.2 | |
| 4. + Loss ReplayModel=GRIP-8B, Total Budget=100B tokens2026.02 | 30.8 | 34.9 | |
| 5. + DiversityModel=GRIP-8B, Total Budget=100B tokens2026.02 | 30.5 | 34.5 | |
| Random 16BModel Size=16B, Total Budget=300B tokens2026.02 | 29.1 | 32.5 | |
| 3. + Static ReplayModel=GRIP-8B, Total Budget=100B tokens2026.02 | 28.1 | 32.2 | |
| Random 8BModel Size=8B, Total Budget=300B tokens2026.02 | 27.4 | 30.1 | |
| 2. + Static BudgetModel=GRIP-8B, Total Budget=100B tokens2026.02 | 26.2 | 30.4 | |
| 1. RandomModel=GRIP-8B, Total Budget=100B tokens2026.02 | 24.5 | 28.1 |