List your company
ResearchFormerly WizWand

Code Completion on MBPP (test)

69.43-Shot Pass RateYi-Coder-9B
29.46439.83250.260.568Nov 7, 2024
Updated 8mo ago

Evaluation results

MethodLinks
Yi-Coder-9BSize=9B2024.1169.4
Qwen2.5-Coder-7BSize=7B2024.1168.8
DS-Coder-V2-Lite-Base (MoE)Size=16B2024.1162.6
CodeQwen1.5-7BSize=7B2024.1161.8
DS-Coder-6.7B-BaseSize=6.7B2024.1160.6
OpenCoder-8B-BaseSize=8B2024.1160.6
Qwen2.5-Coder-1.5BSize=1.5B2024.1159.2
StarCoder2-7BSize=7B2024.1155.2
CodeGemma-7BSize=7B2024.1155
OpenCoder-1.5B-BaseSize=1.5B2024.1151.8
Yi-Coder-1.5BSize=1.5B2024.1151.6
StarCoder2-3BSize=3B2024.1146.4
DeepSeek-Coder-1.3B-BaseSize=1.3B2024.1146.2
CodeGemma-2BSize=2B2024.1145.4
CodeLlama-7BSize=7B2024.1141.4
Crystal-7BSize=7B2024.1131