List your company
ResearchFormerly WizWand

Code Generation on MBPP (Acc (%))

66.7MBPP Accuracy (%)Oracle
39.97246.91153.8560.789Apr 26, 2026
Updated 4mo ago

Evaluation results

MethodLinks
OracleBackbone=Dream-7B-Base, Shots=4-shot2026.0466.7
VanillaBackbone=Dream-7B-Base, Shots=4-shot, Latency (s)=4.582026.0465.2
OracleBackbone=LLaDA-8B-Base, Shots=4-shot2026.0463.1
Auto-ICLBackbone=Dream-7B-Base, Shots=4-shot, Latency (s)=4.602026.0457.3
VanillaBackbone=LLaDA-8B-Base, Shots=4-shot, Latency (s)=5.382026.0448.8
RandomBackbone=Dream-7B-Base, Shots=4-shot, Latency (s)=4.592026.0447.2
PrefixBackbone=Dream-7B-Base, Shots=4-shot, Latency (s)=4.602026.0446.3
Auto-ICLBackbone=LLaDA-8B-Base, Shots=4-shot, Latency (s)=5.402026.0446.2
RandomBackbone=LLaDA-8B-Base, Shots=4-shot, Latency (s)=5.342026.0443.3
PrefixBackbone=LLaDA-8B-Base, Shots=4-shot, Latency (s)=5.352026.0441