Code Generation on MBPP (MB)
65MB Score90% (CodeEnhance-gemma-4b+CodeQA) + 10% Raw
Evaluation results
| Method | Links | |
|---|---|---|
| 90% (CodeEnhance-gemma-4b+CodeQA) + 10% RawTokens (B)=102026.06 | 65 | |
| 80% CodeEnhance-gemma-4b + 15% CodeQA + 5% RawTokens (B)=102026.06 | 65 | |
| 95% (CodeEnhance-gemma-4b+CodeQA) + 5% RawTokens (B)=102026.06 | 64.5 | |
| 85% CodeEnhance-gemma-4b + 10% CodeQA + 5% RawTokens (B)=102026.06 | 64.5 | |
| Nemotron-RWTokens (B)=102026.06 | 64 | |
| 75% CodeEnhance-gemma-4b + 15% CodeQA + 10% RawTokens (B)=102026.06 | 64 | |
| CodeEnhance-gemma-4bTokens (B)=102026.06 | 63 | |
| 95% SwallowCode + 5% RawTokens (B)=102026.06 | 58.2 | |
| 50% CodeEnhance + 50% Nemotron-RWTokens (B)=102026.06 | 58.2 | |
| CodeEnhance-gemma-1bTokens (B)=102026.06 | 57.7 | |
| 95% Nemotron-RW + 5% RawTokens (B)=102026.06 | 57.7 | |
| 90% CodeEnhance+CodeQA + 10% RawTokens (B)=102026.06 | 48.2 | |
| SwallowCodeTokens (B)=102026.06 | 43.4 | |
| 95% CodeEnhance+CodeQA + 5% RawTokens (B)=102026.06 | 36 | |
| CodeEnhance+Nemotron-RW+SwallowCodeTokens (B)=102026.06 | 19 | |
| CodeEnhance+CodeQATokens (B)=102026.06 | 16.4 | |
| CodeEnhanceTokens (B)=102026.06 | 5.6 |