Code Generation on MBPP (Acc (%))
66.7MBPP Accuracy (%)Oracle
Evaluation Results
| Method | Links | |
|---|---|---|
| OracleBackbone=Dream-7B-Base, Shots=4-shot2026.04 | 66.7 | |
| VanillaBackbone=Dream-7B-Base, Shots=4-shot, Latency (s)=4.582026.04 | 65.2 | |
| OracleBackbone=LLaDA-8B-Base, Shots=4-shot2026.04 | 63.1 | |
| Auto-ICLBackbone=Dream-7B-Base, Shots=4-shot, Latency (s)=4.602026.04 | 57.3 | |
| VanillaBackbone=LLaDA-8B-Base, Shots=4-shot, Latency (s)=5.382026.04 | 48.8 | |
| RandomBackbone=Dream-7B-Base, Shots=4-shot, Latency (s)=4.592026.04 | 47.2 | |
| PrefixBackbone=Dream-7B-Base, Shots=4-shot, Latency (s)=4.602026.04 | 46.3 | |
| Auto-ICLBackbone=LLaDA-8B-Base, Shots=4-shot, Latency (s)=5.402026.04 | 46.2 | |
| RandomBackbone=LLaDA-8B-Base, Shots=4-shot, Latency (s)=5.342026.04 | 43.3 | |
| PrefixBackbone=LLaDA-8B-Base, Shots=4-shot, Latency (s)=5.352026.04 | 41 |