Code Generation on MBPP (Performance Score)
71.98MBPP Performance ScoreGenius
Evaluation Results
| Method | Links | |
|---|---|---|
| GeniusTraining Corpus=OpenHermes2025.04 | 71.98 | |
| GeniusTraining Corpus=Magpie2025.04 | 71.6 | |
| C32A2Base Model=Qwen2.5-7B, #P/B=184.42, #AP/B=13.332026.03 | 71.6 | |
| LLaMA3.1-8B-Instruct2025.04 | 69.65 | |
| FineRMoEBase Model=Qwen2.5-7B, #P/B=26.65, #AP/B=7.942026.03 | 69.4 | |
| CTBase Model=Qwen2.5-7B, #P/B=7.62, #AP/B=7.622026.03 | 64 | |
| PTBase Model=Qwen2.5-7B, #P/B=7.62, #AP/B=7.622026.03 | 63.6 | |
| DreamBackbone=Dream-7B-Instruct, TPS=21.68, Speedup=1.0×2026.03 | 60.4 | |
| ES-dLLM*Backbone=Dream-7B-Instruct, TPS=130.49, Speedup=6.0×, Note=Frequent KV cache refreshment2026.03 | 59 | |
| ES-dLLMBackbone=Dream-7B-Instruct, TPS=276.12, Speedup=12.7×2026.03 | 57 | |
| DualCacheBackbone=Dream-7B-Instruct, TPS=214.14, Speedup=9.9×2026.03 | 56.8 | |
| NVShardBase Model=Qwen2.5-7B, #P/B=47.55, #AP/B=7.632026.03 | 53.2 | |
| C32A2Base Model=Qwen2.5-1.5B, #P/B=37.62, #AP/B=2.942026.03 | 50.6 | |
| FineRMoEBase Model=Qwen2.5-1.5B, #P/B=5.64, #AP/B=1.852026.03 | 50.4 | |
| PTBase Model=Qwen2.5-1.5B, #P/B=1.54, #AP/B=1.542026.03 | 43.4 | |
| CTBase Model=Qwen2.5-1.5B, #P/B=1.54, #AP/B=1.542026.03 | 41.2 | |
| C32A2Base Model=Qwen2.5-0.5B, #P/B=10.36, #AP/B=0.952026.03 | 33.8 | |
| NVShardBase Model=Qwen2.5-1.5B, #P/B=9.88, #AP/B=1.782026.03 | 33.8 | |
| FineRMoEBase Model=Qwen2.5-0.5B, #P/B=1.68, #AP/B=0.652026.03 | 33.4 | |
| PTBase Model=Qwen2.5-0.5B, #P/B=0.49, #AP/B=0.492026.03 | 29.4 | |
| CTBase Model=Qwen2.5-0.5B, #P/B=0.49, #AP/B=0.492026.03 | 28 | |
| S16A4Base Model=Qwen2.5-7B, #P/B=7.62, #AP/B=3.342026.03 | 17.2 | |
| NVShardBase Model=Qwen2.5-0.5B, #P/B=2.83, #AP/B=0.632026.03 | 16.8 | |
| S16A4Base Model=Qwen2.5-1.5B, #P/B=1.78, #AP/B=0.912026.03 | 1 | |
| S16A4Base Model=Qwen2.5-0.5B, #P/B=0.63, #AP/B=0.402026.03 | 0 | |
| DUBase Model=Qwen2.5-0.5B, #P/B=2.83, #AP/B=0.942026.03 | 0 | |
| DUBase Model=Qwen2.5-1.5B, #P/B=9.87, #AP/B=2.932026.03 | 0 | |
| DUBase Model=Qwen2.5-7B, #P/B=47.54, #AP/B=13.322026.03 | 0 |