List your company
ResearchFormerly WizWand

Code Generation on MBPP (Performance Score)

71.98MBPP Performance ScoreGenius
-2.879216.555435.9955.4246Apr 11, 2025Jun 5, 2025Jul 31, 2025Sep 24, 2025Nov 19, 2025Jan 13, 2026Mar 10, 2026
Updated 7mo ago

Evaluation results

MethodLinks
GeniusTraining Corpus=OpenHermes2025.0471.98
GeniusTraining Corpus=Magpie2025.0471.6
C32A2Base Model=Qwen2.5-7B, #P/B=184.42, #AP/B=13.332026.0371.6
LLaMA3.1-8B-Instruct2025.0469.65
FineRMoEBase Model=Qwen2.5-7B, #P/B=26.65, #AP/B=7.942026.0369.4
CTBase Model=Qwen2.5-7B, #P/B=7.62, #AP/B=7.622026.0364
PTBase Model=Qwen2.5-7B, #P/B=7.62, #AP/B=7.622026.0363.6
DreamBackbone=Dream-7B-Instruct, TPS=21.68, Speedup=1.0×2026.0360.4
ES-dLLM*Backbone=Dream-7B-Instruct, TPS=130.49, Speedup=6.0×, Note=Frequent KV cache refreshment2026.0359
ES-dLLMBackbone=Dream-7B-Instruct, TPS=276.12, Speedup=12.7×2026.0357
DualCacheBackbone=Dream-7B-Instruct, TPS=214.14, Speedup=9.9×2026.0356.8
NVShardBase Model=Qwen2.5-7B, #P/B=47.55, #AP/B=7.632026.0353.2
C32A2Base Model=Qwen2.5-1.5B, #P/B=37.62, #AP/B=2.942026.0350.6
FineRMoEBase Model=Qwen2.5-1.5B, #P/B=5.64, #AP/B=1.852026.0350.4
PTBase Model=Qwen2.5-1.5B, #P/B=1.54, #AP/B=1.542026.0343.4
CTBase Model=Qwen2.5-1.5B, #P/B=1.54, #AP/B=1.542026.0341.2
C32A2Base Model=Qwen2.5-0.5B, #P/B=10.36, #AP/B=0.952026.0333.8
NVShardBase Model=Qwen2.5-1.5B, #P/B=9.88, #AP/B=1.782026.0333.8
FineRMoEBase Model=Qwen2.5-0.5B, #P/B=1.68, #AP/B=0.652026.0333.4
PTBase Model=Qwen2.5-0.5B, #P/B=0.49, #AP/B=0.492026.0329.4
CTBase Model=Qwen2.5-0.5B, #P/B=0.49, #AP/B=0.492026.0328
S16A4Base Model=Qwen2.5-7B, #P/B=7.62, #AP/B=3.342026.0317.2
NVShardBase Model=Qwen2.5-0.5B, #P/B=2.83, #AP/B=0.632026.0316.8
S16A4Base Model=Qwen2.5-1.5B, #P/B=1.78, #AP/B=0.912026.031
S16A4Base Model=Qwen2.5-0.5B, #P/B=0.63, #AP/B=0.402026.030
DUBase Model=Qwen2.5-0.5B, #P/B=2.83, #AP/B=0.942026.030
DUBase Model=Qwen2.5-1.5B, #P/B=9.87, #AP/B=2.932026.030
DUBase Model=Qwen2.5-7B, #P/B=47.54, #AP/B=13.322026.030