Code Generation on MBPP, MBPP+, BCB-f, BCB-h, and LCB
88.9MBPP ScoreMIRA-Group
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| MIRA-GroupMid-training Protocol=Subset Selection, Token Budget=25B2026.05 | 88.9 | 73.8 | 55.26 | 33.11 | 21.56 | 54.53 | |
| MIRA-GlobalMid-training Protocol=Subset Selection, Token Budget=25B2026.05 | 88.4 | 73.8 | 52.19 | 29.05 | 22.16 | 53.12 | |
| DataManMid-training Protocol=Subset Selection, Token Budget=25B2026.05 | 87.8 | 72.5 | 55.79 | 29.05 | 23.95 | 53.82 | |
| Raw MixtureMid-training Protocol=Full Corpus, Token Budget=50B2026.05 | 87.6 | 72.5 | 53.51 | 30.41 | 24.55 | 53.71 | |
| MIRA-SourceMid-training Protocol=Subset Selection, Token Budget=25B2026.05 | 87.3 | 72.2 | 55.09 | 31.76 | 24.55 | 54.18 | |
| +SFTMid-training Protocol=No Mid-training, Token Budget=02026.05 | 86.8 | 72.5 | 56.84 | 32.43 | 20.96 | 53.91 | |
| DSIRMid-training Protocol=Subset Selection, Token Budget=25B2026.05 | 86.8 | 72.2 | 40.18 | 22.97 | 21.56 | 48.74 | |
| RandomMid-training Protocol=Subset Selection, Token Budget=25B2026.05 | 86.8 | 70.9 | 55 | 25.68 | 25.15 | 52.71 | |
| Base ModelMid-training Protocol=No Mid-training, Token Budget=02026.05 | 84.1 | 69.3 | 51.4 | 23.65 | 5.99 | 46.89 | |
| PPLMid-training Protocol=Subset Selection, Token Budget=25B2026.05 | 83.6 | 70.1 | 52.37 | 26.76 | 19.76 | 50.52 |