List your company
ResearchFormerly WizWand

Code Generation on HumanEval (Acc, AMSR)

98.2Accuracy (HumanEval)Dense
74.69680.79886.993.002Jul 29, 2025
Updated 5mo ago

Evaluation results

MethodLinks
DenseModel=DeepSeek R1 Distill Qwen 32B2025.0798.20
ReasonCacheModel=QwQ 32B2025.0797.222.1
ReasonCacheModel=DeepSeek R1 Distill Qwen 32B2025.0796.921.5
DenseModel=QwQ 32B2025.0796.80
ReasonCacheModel=Phi 4 reasoning plus2025.0794.516.8
DenseModel=Phi 4 reasoning plus2025.0793.30
QuestModel=QwQ 32B2025.0791.520.8
StreamingLLMModel=DeepSeek R1 Distill Qwen 32B2025.0788.420.1
SnapKVModel=DeepSeek R1 Distill Qwen 32B2025.0787.29.6
QuestModel=DeepSeek R1 Distill Qwen 32B2025.0786.620.4
QuestModel=Phi 4 reasoning plus2025.0785.415.3
StreamingLLMModel=Phi 4 reasoning plus2025.0781.214.8
StreamingLLMModel=QwQ 32B2025.0779.921.7
SnapKVModel=Phi 4 reasoning plus2025.0779.311.5
SnapKVModel=QwQ 32B2025.0775.66.2