List your company
ResearchFormerly WizWand

Code Generation on HumanEvalPack

42.68Pass@1w/o Merging
2.088812.626923.16533.7031Feb 24, 2025
Updated 8mo ago

Evaluation results

MethodLinks
w/o MergingLM=false, Math=false, Code=true2025.0242.68
Model StockLM=true, Math=false, Code=true2025.0239.02
LED-MergingLM=true, Math=false, Code=true2025.0237.8
Model StockLM=true, Math=true, Code=true2025.0236.59
BreadcrumbsLM=true, Math=true, Code=true2025.0236.59
LED-MergingLM=true, Math=true, Code=true2025.0236.59
BreadcrumbsLM=true, Math=false, Code=true2025.0236.58
Ties-MergingLM=true, Math=false, Code=true2025.0233.53
BreadcrumbsWizardLM-13B=true, WizardMath-13B=false, LLama-2-13B-Code-Alpaca=true2025.0232.93
Task ArithmeticWizardLM-13B=true, WizardMath-13B=false, LLama-2-13B-Code-Alpaca=true2025.0231.09
Ties-MergingLM=true, Math=true, Code=true2025.0230.34
Ties-MergingWizardLM-13B=true, WizardMath-13B=false, LLama-2-13B-Code-Alpaca=true2025.0228.66
Ties-MergingWizardLM-13B=true, WizardMath-13B=true, LLama-2-13B-Code-Alpaca=true2025.0226.83
w/o MergingWizardLM-13B=true, WizardMath-13B=false, LLama-2-13B-Code-Alpaca=false2025.0226.22
w/o MergingWizardLM-13B=false, WizardMath-13B=false, LLama-2-13B-Code-Alpaca=true2025.0222.56
BreadcrumbsWizardLM-13B=true, WizardMath-13B=true, LLama-2-13B-Code-Alpaca=true2025.0220.73
Task ArithmeticLM=true, Math=false, Code=true2025.0218.9
LED-MergingWizardLM-13B=true, WizardMath-13B=true, LLama-2-13B-Code-Alpaca=true2025.0217.68
Task ArithmeticWizardLM-13B=true, WizardMath-13B=true, LLama-2-13B-Code-Alpaca=true2025.0216.46
Model StockWizardLM-13B=true, WizardMath-13B=true, LLama-2-13B-Code-Alpaca=true2025.0215.85
LED-MergingWizardLM-13B=true, WizardMath-13B=false, LLama-2-13B-Code-Alpaca=true2025.0215.85
Model StockWizardLM-13B=true, WizardMath-13B=false, LLama-2-13B-Code-Alpaca=true2025.0214.02
Task ArithmeticLM=true, Math=true, Code=true2025.029.15
w/o MergingLM=true, Math=false, Code=false2025.023.65