Code Generation on HumanEval (Accuracy, General Capability Average Accuracy)
35.4AccuracyPre-trained
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Pre-trainedModel=Gemma-3-4B2026.06 | 35.4 | 50.95 | |
| AlphaTokenModel=Gemma-3-4B2026.06 | 33.27 | 49.51 | |
| STMModel=Gemma-3-4B2026.06 | 31.66 | 48.32 | |
| ssTOKENModel=Gemma-3-4B2026.06 | 31.16 | 48.05 | |
| XTFModel=Gemma-3-4B2026.06 | 30.8 | 47.3 | |
| Token CleaningModel=Gemma-3-4B2026.06 | 30.58 | 47.94 | |
| LESSModel=Gemma-3-4B2026.06 | 30.42 | 47.92 | |
| LoRAModel=Gemma-3-4B2026.06 | 29.93 | 47.42 | |
| Standard FTModel=Gemma-3-4B2026.06 | 28.9 | 44.2 |