Code Generation on HumanEval (Accuracy)
54.27AccuracyLlama3.1-8B-Instruct
Evaluation results
| Method | Links | |
|---|---|---|
| Llama3.1-8B-InstructType=Autoregressive2026.01 | 54.27 | |
| Qwen2.5-7B-InstructType=Autoregressive2026.01 | 52.44 | |
| LLaDA1.5-8BDecoding Strategy=FourierSampler2026.01 | 43.29 | |
| LLaDA1.5-8BDecoding Strategy=Vanilla2026.01 | 40.85 | |
| LLaDA-8B-InstructDecoding Strategy=FourierSampler2026.01 | 40.85 | |
| LLaDA1.5-8BDecoding Strategy=RWS2026.01 | 39.63 | |
| LLaDA-8B-InstructDecoding Strategy=Vanilla2026.01 | 39.63 | |
| LLaDA-8B-InstructDecoding Strategy=PC-Sampler2026.01 | 39.63 | |
| LLaDA1.5-8BDecoding Strategy=PC-Sampler2026.01 | 39.02 | |
| LLaDA-8B-InstructDecoding Strategy=RWS2026.01 | 39.02 |