List your company
ResearchFormerly WizWand

Code Generation on HumanEval (Pass@10)

93.1Pass@10Qwen-2.5-7B-Instruct
26.12443.51260.978.288Nov 14, 2025
Updated 5mo ago

Evaluation results

MethodLinks
Qwen-2.5-7B-InstructDecoding Method=tree-based typical decoding, Samples=20, Temperature=0.82025.1193.1
Ministral-8B-InstructDecoding Method=tree-based typical decoding, Samples=20, Temperature=0.82025.1191
Llama-3.1-8B-InstructDecoding Method=tree-based typical decoding, Samples=20, Temperature=0.82025.1186.3
Tulu 3 8BDecoding Method=tree-based typical decoding, Samples=20, Temperature=0.82025.1183.9
EvaByte-SFTDecoding Method=tree-based typical decoding, Samples=20, Temperature=0.82025.1173.7
MAP-Neo-7B-InstructDecoding Method=tree-based typical decoding, Samples=20, Temperature=0.82025.1172.1
Gemma-2-9B-itDecoding Method=tree-based typical decoding, Samples=20, Temperature=0.82025.1171.7
OLMo-2-7B-1124-InstructDecoding Method=tree-based typical decoding, Samples=20, Temperature=0.82025.1167.6
OLMo-2-7B-SFTDecoding Method=tree-based typical decoding, Samples=20, Temperature=0.82025.1167
OLMoE-1B-7B-0924-InstructDecoding Method=tree-based typical decoding, Samples=20, Temperature=0.82025.1154.8
OLMo-v1.7-7B-InstructDecoding Method=tree-based typical decoding, Samples=20, Temperature=0.82025.1149.7
OLMo-7B-InstructDecoding Method=tree-based typical decoding, Samples=20, Temperature=0.82025.1128.7