List your company
ResearchFormerly WizWand

Python Coding on MBPP standard (test)

85.25Pass@1 AccuracyCoT
-3.4119.607542.62565.6425Sep 11, 2023Feb 7, 2024Jul 5, 2024Dec 1, 2024Apr 29, 2025Sep 25, 2025Feb 21, 2026
Updated 8mo ago

Evaluation results

MethodLinks
CoTModel=Qwen-3-8B, Prompting Strategy=Chain-of-Thought2026.0285.25
Ann BrownModel=Qwen-3-8B, Prompting Strategy=Ann Brown Prompting2026.0279.39
StdModel=Qwen-3-8B, Prompting Strategy=Standard2026.0274.94
MPModel=Qwen-3-8B, Prompting Strategy=Metacognitive Prompting2026.0273.07
CoTModel=Llama-3-8B, Prompting Strategy=Chain-of-Thought2026.0269.79
MPModel=Llama-3-8B, Prompting Strategy=Metacognitive Prompting2026.0267.92
Ann BrownModel=Llama-3-8B, Prompting Strategy=Ann Brown Prompting2026.0266.51
InternLM2-Chat-20BShots=5-shot, Model Size Group=13~20B Models2024.0365.8
StdModel=Llama-3-8B, Prompting Strategy=Standard2026.0264.64
InternLM2-Chat-20B-SFTShots=5-shot, Model Size Group=13~20B Models2024.0363.4
Mixtral-8x7B-Instruct-v0.1Shots=5-shot, Model Size Group=13~20B Models2024.0359.5
ChatGLM3-6BShots=5-shot, Model Size Group=< 7B Models2024.0354.9
InternLM2-Chat-7BShots=5-shot, Model Size Group=< 7B Models2024.0352.5
InternLM2-Chat-7B-SFTShots=5-shot, Model Size Group=< 7B Models2024.0347.5
phi-1.5-webzero-shot=true, number of parameters=1.3B2023.0943.5
Baichuan2-13B-ChatShots=5-shot, Model Size Group=13~20B Models2024.0340.9
Llama-65Bzero-shot=true, number of parameters=65B2023.0937.7
phi-1.5zero-shot=true, number of parameters=1.3B2023.0937.7
Baichuan2-7B-ChatShots=5-shot, Model Size Group=< 7B Models2024.0337.7
Qwen-7B-ChatShots=5-shot, Model Size Group=< 7B Models2024.0333.9
Llama2-7B-ChatShots=5-shot, Model Size Group=< 7B Models2024.0330.4
Qwen-14B-ChatShots=5-shot, Model Size Group=13~20B Models2024.0329.2
phi-1.5-web-onlyzero-shot=true, number of parameters=1.3B2023.0927.3
Mistral-7B-Instruct-v0.2Shots=5-shot, Model Size Group=< 7B Models2024.0323.7
MPT-7Bzero-shot=true, number of parameters=7B2023.0922.6
Llama2-7Bzero-shot=true, number of parameters=7B2023.0920.8
Llama2-13B-ChatShots=5-shot, Model Size Group=13~20B Models2024.0319.8
Llama-7Bzero-shot=true, number of parameters=7B2023.0917.7
Falcon 7Bzero-shot=true, number of parameters=7B2023.0911.7
Falcon-rw-1.3Bzero-shot=true, number of parameters=1.3B2023.090
OPT-1.3Bzero-shot=true, number of parameters=1.3B2023.090
GPT2 XL 1.5Bzero-shot=true, number of parameters=1.5B2023.090