Code Generation on HumanEvalPack
42.68Pass@1w/o Merging
Evaluation Results
| Method | Links | |
|---|---|---|
| w/o MergingLM=false, Math=false, Code=true2025.02 | 42.68 | |
| Model StockLM=true, Math=false, Code=true2025.02 | 39.02 | |
| LED-MergingLM=true, Math=false, Code=true2025.02 | 37.8 | |
| Model StockLM=true, Math=true, Code=true2025.02 | 36.59 | |
| BreadcrumbsLM=true, Math=true, Code=true2025.02 | 36.59 | |
| LED-MergingLM=true, Math=true, Code=true2025.02 | 36.59 | |
| BreadcrumbsLM=true, Math=false, Code=true2025.02 | 36.58 | |
| Ties-MergingLM=true, Math=false, Code=true2025.02 | 33.53 | |
| BreadcrumbsWizardLM-13B=true, WizardMath-13B=false, LLama-2-13B-Code-Alpaca=true2025.02 | 32.93 | |
| Task ArithmeticWizardLM-13B=true, WizardMath-13B=false, LLama-2-13B-Code-Alpaca=true2025.02 | 31.09 | |
| Ties-MergingLM=true, Math=true, Code=true2025.02 | 30.34 | |
| Ties-MergingWizardLM-13B=true, WizardMath-13B=false, LLama-2-13B-Code-Alpaca=true2025.02 | 28.66 | |
| Ties-MergingWizardLM-13B=true, WizardMath-13B=true, LLama-2-13B-Code-Alpaca=true2025.02 | 26.83 | |
| w/o MergingWizardLM-13B=true, WizardMath-13B=false, LLama-2-13B-Code-Alpaca=false2025.02 | 26.22 | |
| w/o MergingWizardLM-13B=false, WizardMath-13B=false, LLama-2-13B-Code-Alpaca=true2025.02 | 22.56 | |
| BreadcrumbsWizardLM-13B=true, WizardMath-13B=true, LLama-2-13B-Code-Alpaca=true2025.02 | 20.73 | |
| Task ArithmeticLM=true, Math=false, Code=true2025.02 | 18.9 | |
| LED-MergingWizardLM-13B=true, WizardMath-13B=true, LLama-2-13B-Code-Alpaca=true2025.02 | 17.68 | |
| Task ArithmeticWizardLM-13B=true, WizardMath-13B=true, LLama-2-13B-Code-Alpaca=true2025.02 | 16.46 | |
| Model StockWizardLM-13B=true, WizardMath-13B=true, LLama-2-13B-Code-Alpaca=true2025.02 | 15.85 | |
| LED-MergingWizardLM-13B=true, WizardMath-13B=false, LLama-2-13B-Code-Alpaca=true2025.02 | 15.85 | |
| Model StockWizardLM-13B=true, WizardMath-13B=false, LLama-2-13B-Code-Alpaca=true2025.02 | 14.02 | |
| Task ArithmeticLM=true, Math=true, Code=true2025.02 | 9.15 | |
| w/o MergingLM=true, Math=false, Code=false2025.02 | 3.65 |