Code Generation on HumanEval (HE)
48.7HumanEval Score95% (CodeEnhance-gemma-4b+CodeQA) + 5% Raw
Evaluation results
| Method | Links | |
|---|---|---|
| 95% (CodeEnhance-gemma-4b+CodeQA) + 5% RawTokens (B)=102026.06 | 48.7 | |
| Nemotron-RWTokens (B)=102026.06 | 48.2 | |
| 80% CodeEnhance-gemma-4b + 15% CodeQA + 5% RawTokens (B)=102026.06 | 46.3 | |
| 90% (CodeEnhance-gemma-4b+CodeQA) + 10% RawTokens (B)=102026.06 | 45.7 | |
| 50% CodeEnhance + 50% Nemotron-RWTokens (B)=102026.06 | 45.7 | |
| CodeEnhance+Nemotron-RW+SwallowCodeTokens (B)=102026.06 | 45.1 | |
| CodeEnhance+CodeQATokens (B)=102026.06 | 45.1 | |
| SwallowCodeTokens (B)=102026.06 | 43.9 | |
| 85% CodeEnhance-gemma-4b + 10% CodeQA + 5% RawTokens (B)=102026.06 | 43.3 | |
| 75% CodeEnhance-gemma-4b + 15% CodeQA + 10% RawTokens (B)=102026.06 | 43.3 | |
| 95% CodeEnhance+CodeQA + 5% RawTokens (B)=102026.06 | 42.7 | |
| 95% Nemotron-RW + 5% RawTokens (B)=102026.06 | 41.5 | |
| CodeEnhance-gemma-4bTokens (B)=102026.06 | 38.4 | |
| 90% CodeEnhance+CodeQA + 10% RawTokens (B)=102026.06 | 37.8 | |
| 95% SwallowCode + 5% RawTokens (B)=102026.06 | 37.2 | |
| CodeEnhanceTokens (B)=102026.06 | 34.1 | |
| CodeEnhance-gemma-1bTokens (B)=102026.06 | 32.9 |