List your company
ResearchFormerly WizWand

Code Generation on MBPP subset of 500 (test)

64.6Pass@1Magicoder-S-DS
40.47246.7365359.264Dec 20, 2023
Updated 8mo ago

Evaluation results

MethodLinks
Magicoder-S-DSParams=6.7B, Base Model=DeepseekCoder, InsT Data=185K, Decoding Strategy=greedy2023.1264.6
WaveCoder-Ultra-6.7BParams=6.7B, Base Model=DeepseekCoder, InsT Data=130K, Decoding Strategy=greedy2023.1264.4
WaveCoder-Pro-6.7BParams=6.7B, Base Model=DeepseekCoder, InsT Data=20K, Decoding Strategy=greedy2023.1263.6
WaveCoder-DS-6.7BParams=6.7B, Base Model=DeepseekCoder, InsT Data=20K, Decoding Strategy=greedy2023.1262.8
DeepseekCoder-instruct*Params=6.7B, Base Model=DeepseekCoder, Decoding Strategy=greedy2023.1262.8
DeepseekCoderParams=6.7B, InsT Data=None, Decoding Strategy=greedy2023.1260.6
Magicoder-DSParams=6.7B, Base Model=DeepseekCoder, InsT Data=75K, Decoding Strategy=greedy2023.1260.4
ChatGPTDecoding Strategy=n=2002023.1252.2
WizardCoderParams=15B, Base Model=StarCoder, InsT Data=78K, Decoding Strategy=n=2002023.1251.8
WaveCoder-SC-15BParams=15B, Base Model=StarCoder, InsT Data=20K, Decoding Strategy=n=2002023.1251
WaveCoder-CL-13BParams=13B, Base Model=CodeLLaMa, InsT Data=20K, Decoding Strategy=n=2002023.1249.6
CodeLLaMa-instructParams=13B, Base Model=CodeLLaMa, InsT Data=14K, Decoding Strategy=n=2002023.1249.4
WaveCoder-CL-7BParams=7B, Base Model=CodeLLaMa, InsT Data=20K, Decoding Strategy=n=2002023.1247.2
CodeLLaMaParams=13B, InsT Data=None, Decoding Strategy=n=2002023.1247
CodeLLaMa-instructParams=7B, Base Model=CodeLLaMa, InsT Data=14K, Decoding Strategy=n=2002023.1244.4
OctoCoderParams=15B, Base Model=StarCoder, InsT Data=13K, Decoding Strategy=n=2002023.1243.5
StarCoderParams=15B, InsT Data=None, Decoding Strategy=n=2002023.1243.3
CodeLLaMaParams=7B, InsT Data=None, Decoding Strategy=n=2002023.1241.4