Code Generation on HumanEval (Pass@1, Pass@k)
83.5Pass@1GAC + Token-φ
Evaluation results
| Method | Links | ||
|---|---|---|---|
| GAC + Token-φseeds=32026.05 | 83.5 | — | |
| GAC w/o φseeds=32026.05 | 82.7 | — | |
| HPTseeds=32026.05 | 81.2 | — | |
| LUFFYseeds=32026.05 | 80.9 | — | |
| KL-ctrlseeds=32026.05 | 80.8 | — | |
| CHORDseeds=32026.05 | 80.5 | — | |
| Nash-MTLseeds=32026.05 | 80.2 | — | |
| SRFTseeds=32026.05 | 79.8 | — | |
| SFT-best + RLseeds=32026.05 | 78 | — | |
| SFT-bestseeds=32026.05 | 75.6 | — | |
| IMHModel=WeDLM-8B2026.04 | 74.5 | 96.2 | |
| EntropyModel=WeDLM-8B, Default decoding=true2026.04 | 73.2 | 95 | |
| ConfidenceModel=WeDLM-8B2026.04 | 72.5 | 90.5 | |
| Qwen2.5-7B-Inst.seeds=32026.05 | 72 | — | |
| MarginModel=WeDLM-8B2026.04 | 58.4 | 90.3 | |
| FullBackbone=Llama-3.1-8B-Base2026.06 | 51.63 | — | |
| NA-LoRABackbone=Llama-3.1-8B-Base, Rank (r)=1282026.06 | 47.76 | — | |
| NA-LoRABackbone=Llama-3.1-8B-Base, Rank (r)=322026.06 | 47.13 | — | |
| NA-LoRABackbone=Llama-3.1-8B-Base, Rank (r)=82026.06 | 46.34 | — | |
| GoRABackbone=Llama-3.1-8B-Base2026.06 | 45.32 | — | |
| DoRABackbone=Llama-3.1-8B-Base2026.06 | 45.12 | — | |
| LoRA-GABackbone=Llama-3.1-8B-Base2026.06 | 44.1 | — | |
| PiSSABackbone=Llama-3.1-8B-Base2026.06 | 43.7 | — | |
| LoRA-ProBackbone=Llama-3.1-8B-Base2026.06 | 43.63 | — | |
| AdaLoRABackbone=Llama-3.1-8B-Base2026.06 | 43.49 | — | |
| LoRA+Backbone=Llama-3.1-8B-Base2026.06 | 43.09 | — | |
| rsLoRABackbone=Llama-3.1-8B-Base2026.06 | 42.27 | — | |
| LoRABackbone=Llama-3.1-8B-Base2026.06 | 41.05 | — | |
| ConfidenceModel=LLaDA-8B, Default decoding=true2026.04 | 41 | 57.3 | |
| EntropyModel=LLaDA-8B2026.04 | 38.7 | 67 | |
| IMHModel=LLaDA-8B2026.04 | 38.5 | 69.5 | |
| FullBackbone=Llama-2-7B-Base2026.06 | 35.31 | — | |
| MarginModel=LLaDA-8B2026.04 | 35 | 61 | |
| StelLAModel=LLaMA2-7B, Params (%)=4.5812025.10 | 33.73 | — | |
| Full FTCost=×1, Backbone=Llama 2-7B, Evaluation Protocol=Fine-tuning2025.08 | 33.12 | — | |
| PiSSAModel=LLaMA2-7B, Params (%)=4.5312025.10 | 31.1 | — | |
| LoRAModel=LLaMA2-7B, Params (%)=4.5312025.10 | 30.07 | — | |
| NA-LoRABackbone=Llama-2-7B-Base, Rank (r)=1282026.06 | 28.66 | — | |
| NA-LoRABackbone=Llama-2-7B-Base, Rank (r)=322026.06 | 27.64 | — | |
| Bi-LoRACost=×1, Backbone=Llama 2-7B, Evaluation Protocol=Fine-tuning2025.08 | 27.2 | — | |
| DoRAModel=LLaMA2-7B, Params (%)=4.5502025.10 | 27 | — | |
| Flat-LoRACost=×1, Backbone=Llama 2-7B, Evaluation Protocol=Fine-tuning2025.08 | 26.67 | — | |
| LoRA-SAMCost=×2, Backbone=Llama 2-7B, Evaluation Protocol=Fine-tuning2025.08 | 26.59 | — | |
| LoRA-nBARCost=×1, Backbone=Llama 2-7B, Evaluation Protocol=Fine-tuning2025.08 | 26.5 | — | |
| LoRA-oBARCost=×1, Backbone=Llama 2-7B, Evaluation Protocol=Fine-tuning2025.08 | 26.3 | — | |
| RWP_fullCost=×1, Backbone=Llama 2-7B, Evaluation Protocol=Fine-tuning2025.08 | 26.26 | — | |
| NA-LoRABackbone=Llama-2-7B-Base, Rank (r)=82026.06 | 26.02 | — | |
| ROPCost=×1, Backbone=Llama 2-7B, Evaluation Protocol=Fine-tuning2025.08 | 25.41 | — | |
| LoRA-LookSAMCost=×1.x, Backbone=Llama 2-7B, Evaluation Protocol=Fine-tuning2025.08 | 25.28 | — | |
| RWP_LoRACost=×1, Backbone=Llama 2-7B, Evaluation Protocol=Fine-tuning2025.08 | 24.92 | — | |
| LoRA-ESAMCost=×1.x, Backbone=Llama 2-7B, Evaluation Protocol=Fine-tuning2025.08 | 24.84 | — | |
| GoRABackbone=Llama-2-7B-Base2026.06 | 24.8 | — | |
| LoRACost=×1, Backbone=Llama 2-7B, Evaluation Protocol=Fine-tuning2025.08 | 24.75 | — | |
| LoRA-ProBackbone=Llama-2-7B-Base2026.06 | 22.97 | — | |
| LoRA-GABackbone=Llama-2-7B-Base2026.06 | 19.81 | — | |
| DoRABackbone=Llama-2-7B-Base2026.06 | 19.75 | — | |
| RandomModel=LLaDA-8B2026.04 | 19.6 | 54.9 | |
| LoRA+Backbone=Llama-2-7B-Base2026.06 | 18.17 | — | |
| AdaLoRABackbone=Llama-2-7B-Base2026.06 | 17.8 | — | |
| PiSSABackbone=Llama-2-7B-Base2026.06 | 16.02 | — | |
| rsLoRABackbone=Llama-2-7B-Base2026.06 | 16.01 | — | |
| LoRABackbone=Llama-2-7B-Base2026.06 | 14.76 | — |