Code Generation on HumanEval pass@1 (test)
48.2Pass@1 (HumanEval)CORA
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| CORABackbone=LLaMA-3-8B, Rank=r=16, Params=10.3 M2026.06 | 48.2 | — | — | — | |
| CORABackbone=LLaMA-3-8B, Rank=r=64, Params=43.3 M2026.06 | 47 | — | — | — | |
| CORABackbone=LLaMA-3-8B, Rank=r=32, Params=21.3 M2026.06 | 45.1 | — | — | — | |
| CORABackbone=LLaMA-3-8B, Rank=r=8, Params=4.8 M2026.06 | 45.1 | — | — | — | |
| LoRIBackbone=LLaMA-3-8B, Rank=r=32, Params=45 M2026.06 | 43.2 | — | — | — | |
| DiaBloBackbone=LLaMA-3-8B, Rank=N=64, Params=121 M2026.06 | 43.2 | — | — | — | |
| CORABackbone=Mistral-7B, Rank=r=64, Params=43.3 M2026.06 | 40.9 | — | — | — | |
| CORABackbone=Mistral-7B, Rank=r=32, Params=21.3 M2026.06 | 40.9 | — | — | — | |
| CORABackbone=Mistral-7B, Rank=r=16, Params=10.3 M2026.06 | 40.2 | — | — | — | |
| CORABackbone=Mistral-7B, Rank=r=8, Params=4.8 M2026.06 | 40.2 | — | — | — | |
| DiaBloBackbone=LLaMA-3-8B, Rank=N=128, Params=61 M2026.06 | 39.4 | — | — | — | |
| LoRABackbone=LLaMA-3-8B, Rank=r=32, Params=90 M2026.06 | 34.7 | — | — | — | |
| DiaBloBackbone=Mistral-7B, Rank=N=64, Params=122 M2026.06 | 34.4 | — | — | — | |
| DiaBloBackbone=Mistral-7B, Rank=N=128, Params=61 M2026.06 | 34 | — | — | — | |
| LoRABackbone=Mistral-7B, Rank=r=32, Params=91 M2026.06 | 33.8 | — | — | — | |
| LoRIBackbone=Mistral-7B, Rank=r=32, Params=46 M2026.06 | 33.8 | — | — | — | |
| DoRABackbone=Mistral-7B, Rank=r=32, Params=91 M2026.06 | 33.7 | — | — | — | |
| DoRABackbone=LLaMA-3-8B, Rank=r=32, Params=90 M2026.06 | 33.1 | — | — | — | |
| Original LLaDAStrategy=std unmask2026.05 | — | 14 | 26.8 | 32.9 | |
| Original LLaDAStrategy=D3IM2026.05 | — | 5.5 | 4.3 | 7.3 | |
| SCOPEStrategy=std unmask2026.05 | — | 6.7 | 20.1 | 31.1 | |
| SCOPEStrategy=D3IM2026.05 | — | 29.3 | 30.5 | 34.8 |