Code Generation on MBPP (Avg # Correct, Diversity, Efficiency, Top-1 Acc.)
90.5Top-1 Acc.Original CLM-CQ
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Original CLM-CQModel=Original2026.05 | 90.5 | — | — | — | |
| PwSPoisoned dataset=PCS-TRN-222026.05 | 84.4 | — | — | — | |
| PwSPoisoned dataset=PCS-TRN-792026.05 | 83.9 | — | — | — | |
| PwSPoisoned dataset=PCS-TRN-202026.05 | 83.6 | — | — | — | |
| PwSPoisoned dataset=PCS-TRN-892026.05 | 82 | — | — | — | |
| PwSPoisoned dataset=PCS-TRN-782026.05 | 80.7 | — | — | — | |
| Qwen2.5-Coder-1.5B-InstructModel scale=1.5B, Evaluation mode=0-shot2026.06 | 53.6 | — | — | — | |
| RouterDCModel scale=1.5B, Evaluation mode=0-shot2026.06 | 53.4 | — | — | — | |
| Pack of LLMsModel scale=1.5B, Evaluation mode=0-shot2026.06 | 52.6 | — | — | — | |
| DLLGModel scale=1.5B, Evaluation mode=0-shot2026.06 | 52.6 | — | — | — | |
| Token Maj-VotingModel scale=1.5B, Evaluation mode=0-shot2026.06 | 52 | — | — | — | |
| Entropy WeightingModel scale=1.5B, Evaluation mode=0-shot2026.06 | 51.6 | — | — | — | |
| SLERPModel scale=1.5B, Evaluation mode=0-shot2026.06 | 50.8 | — | — | — | |
| GaCModel scale=1.5B, Evaluation mode=0-shot2026.06 | 50.6 | — | — | — | |
| UniTeModel scale=1.5B, Evaluation mode=0-shot2026.06 | 50 | — | — | — | |
| Qwen2.5-1.5B-InstructModel scale=1.5B, Evaluation mode=0-shot2026.06 | 49.8 | — | — | — | |
| EmbedLLMModel scale=1.5B, Evaluation mode=0-shot2026.06 | 49.2 | — | — | — | |
| Multi-Answer RLVRLoss type=Multi Loss, Prompting strategy=Multi Prompt, k=32026.03 | 49 | 1.35 | 2.98 | 235.49 | |
| Multi-Answer RLCRLoss type=Multi Loss, Prompting strategy=Multi Prompt, k=32026.03 | 48 | 1.38 | 2.94 | 250.94 | |
| LinearModel scale=1.5B, Evaluation mode=0-shot2026.06 | 45.8 | — | — | — | |
| Task ArithmeticModel scale=1.5B, Evaluation mode=0-shot2026.06 | 42 | — | — | — | |
| Full Fine-TuningRank (r)=N/A, Backbone Model=Llama 2-7B2026.06 | 39.27 | — | — | — | |
| RLCREvaluation protocol=Zero-Shot, Prompting strategy=Multi Prompt, k=32026.03 | 37 | 0.97 | 2.02 | 677.93 | |
| RLVRLoss type=Single Loss, Prompting strategy=Multi Prompt, k=32026.03 | 35 | 0.95 | 1.93 | 695.13 | |
| RLCRLoss type=Single Loss, Prompting strategy=Multi Prompt, k=32026.03 | 35 | 0.92 | 1.9 | 724.89 | |
| RLVREvaluation protocol=Zero-Shot, Prompting strategy=Multi Prompt, k=32026.03 | 34 | 0.91 | 1.94 | 664.78 | |
| LoRA-αRank (r)=128, Backbone Model=Llama 2-7B2026.06 | 33.9 | — | — | — | |
| LLaDAModel=LLaDA 8B-Instruct, Decoding Method=Top Margin + Confidence Modulation2026.05 | 31.8 | — | — | — | |
| LoRAMRank (r)=128, Backbone Model=Llama 2-7B2026.06 | 31.53 | — | — | — | |
| PiSSARank (r)=128, Backbone Model=Llama 2-7B2026.06 | 31.07 | — | — | — | |
| RsLoRARank (r)=128, Backbone Model=Llama 2-7B2026.06 | 30.73 | — | — | — | |
| Qwen2.5-Math-1.5B-InstructModel scale=1.5B, Evaluation mode=0-shot2026.06 | 30.4 | — | — | — | |
| LoRA+Rank (r)=128, Backbone Model=Llama 2-7B2026.06 | 29.13 | — | — | — | |
| RLVRLoss type=Single Loss, Prompting strategy=Single Prompt, k=32026.03 | 29 | 0.98 | 2.09 | 511.73 | |
| LoRARank (r)=128, Backbone Model=Llama 2-7B2026.06 | 28.84 | — | — | — | |
| LoRAMRank (r)=16, Backbone Model=Llama 2-7B2026.06 | 28.83 | — | — | — | |
| LLaDAModel=LLaDA 8B-Instruct, Decoding Method=Top Margin + Suffix Anchor2026.05 | 28.8 | — | — | — | |
| LoRARank (r)=16, Backbone Model=Llama 2-7B2026.06 | 28.65 | — | — | — | |
| PiSSARank (r)=16, Backbone Model=Llama 2-7B2026.06 | 28.62 | — | — | — | |
| LLaDAModel=LLaDA 8B-Instruct, Decoding Method=Top Probability + Confidence Modulation2026.05 | 28.4 | — | — | — | |
| LoRA-αRank (r)=16, Backbone Model=Llama 2-7B2026.06 | 28.3 | — | — | — | |
| RsLoRARank (r)=16, Backbone Model=Llama 2-7B2026.06 | 28.1 | — | — | — | |
| LoRA+Rank (r)=16, Backbone Model=Llama 2-7B2026.06 | 28 | — | — | — | |
| RLCRLoss type=Single Loss, Prompting strategy=Single Prompt, k=32026.03 | 27 | 0.89 | 2.13 | 518.07 | |
| LLaDAModel=LLaDA 8B-Instruct, Decoding Method=Top Probability + Suffix Anchor2026.05 | 25.8 | — | — | — | |
| LLaDAModel=LLaDA 8B-Instruct, Decoding Method=Top Margin2026.05 | 24.4 | — | — | — | |
| LLaDAModel=LLaDA 8B-Instruct, Decoding Method=Random2026.05 | 22.4 | — | — | — | |
| LLaDAModel=LLaDA 8B-Instruct, Decoding Method=Top Probability2026.05 | 19.2 | — | — | — |