Code Generation on HumanEval (Pass rate)
42.7Pass RatePRISM
Evaluation results
| Method | Links | |
|---|---|---|
| PRISMSampling steps=1024, Zero-shot=true, Backbone=LLaDA-8B-Instruct, Trainable parameters=≈ 250M2025.10 | 42.7 | |
| ReMDMSampling steps=1024, Zero-shot=true, Backbone=LLaDA-8B-Instruct2025.10 | 42.5 | |
| PRISMSampling steps=512, Zero-shot=true, Backbone=LLaDA-8B-Instruct, Trainable parameters=≈ 250M2025.10 | 39 | |
| ReMDM-confSampling steps=1024, Zero-shot=true, Backbone=LLaDA-8B-Instruct2025.10 | 36.6 | |
| MDMSampling steps=1024, Zero-shot=true, Backbone=LLaDA-8B-Instruct2025.10 | 36.6 | |
| ReMDMSampling steps=512, Zero-shot=true, Backbone=LLaDA-8B-Instruct2025.10 | 34.8 | |
| ReMDM-confSampling steps=512, Zero-shot=true, Backbone=LLaDA-8B-Instruct2025.10 | 34.1 | |
| MDMSampling steps=512, Zero-shot=true, Backbone=LLaDA-8B-Instruct2025.10 | 34.1 | |
| PRISMSampling steps=256, Zero-shot=true, Backbone=LLaDA-8B-Instruct, Trainable parameters=≈ 250M2025.10 | 28 | |
| ReMDMSampling steps=256, Zero-shot=true, Backbone=LLaDA-8B-Instruct2025.10 | 26.2 | |
| ReMDM-confSampling steps=256, Zero-shot=true, Backbone=LLaDA-8B-Instruct2025.10 | 25.6 | |
| MDMSampling steps=256, Zero-shot=true, Backbone=LLaDA-8B-Instruct2025.10 | 25.6 |