Code Generation on MBPP+ (MB+)
54.2MB+ Score75% CodeEnhance-gemma-4b + 15% CodeQA + 10% Raw
Evaluation results
| Method | Links | |
|---|---|---|
| 75% CodeEnhance-gemma-4b + 15% CodeQA + 10% RawTokens (B)=102026.06 | 54.2 | |
| 85% CodeEnhance-gemma-4b + 10% CodeQA + 5% RawTokens (B)=102026.06 | 54 | |
| 80% CodeEnhance-gemma-4b + 15% CodeQA + 5% RawTokens (B)=102026.06 | 54 | |
| 90% (CodeEnhance-gemma-4b+CodeQA) + 10% RawTokens (B)=102026.06 | 53.7 | |
| 95% (CodeEnhance-gemma-4b+CodeQA) + 5% RawTokens (B)=102026.06 | 53.4 | |
| CodeEnhance-gemma-4bTokens (B)=102026.06 | 52.1 | |
| 95% SwallowCode + 5% RawTokens (B)=102026.06 | 48.1 | |
| Nemotron-RWTokens (B)=102026.06 | 46.6 | |
| 95% Nemotron-RW + 5% RawTokens (B)=102026.06 | 46.6 | |
| CodeEnhance-gemma-1bTokens (B)=102026.06 | 46.3 | |
| 50% CodeEnhance + 50% Nemotron-RWTokens (B)=102026.06 | 39.9 | |
| 90% CodeEnhance+CodeQA + 10% RawTokens (B)=102026.06 | 39.7 | |
| Olmo3 BaseTuning Protocol=Freezing param.2026.07 | 37.6 | |
| SwallowCodeTokens (B)=102026.06 | 36.8 | |
| LMK token tuningTuning Protocol=Freezing param.2026.07 | 33.8 | |
| HiLS-Attn RoPE-Q-CalTuning Protocol=Full param. CPT2026.07 | 33.3 | |
| HiLS-Attn HoPE-Q-CalTuning Protocol=Full param. CPT2026.07 | 32.6 | |
| Olmo3 512swaTuning Protocol=Full param. CPT2026.07 | 32.3 | |
| HiLS-Attn NoPE-Q-CalTuning Protocol=Full param. CPT2026.07 | 31.1 | |
| 95% CodeEnhance+CodeQA + 5% RawTokens (B)=102026.06 | 28 | |
| CodeEnhance+Nemotron-RW+SwallowCodeTokens (B)=102026.06 | 13.8 | |
| CodeEnhance+CodeQATokens (B)=102026.06 | 13 | |
| CodeEnhanceTokens (B)=102026.06 | 5 |