Multiple Choice Question Answering on ARC-c (Accuracy)
54.1AccuracyDeLask
Evaluation results
| Method | Links | |
|---|---|---|
| DeLaskModel=LLaMA3-8B, Decoding=DeLask2026.05 | 54.1 | |
| UADModel=LLaMA3-8B, Decoding=UAD2026.05 | 53.72 | |
| RegularModel=LLaMA3-8B, Decoding=Regular2026.05 | 53.33 | |
| ITIModel=LLaMA3-8B, Decoding=ITI2026.05 | 52.49 | |
| DOLAModel=LLaMA3-8B, Decoding=DOLA2026.05 | 50.34 | |
| ADModel=LLaMA3-8B, Decoding=AD2026.05 | 48.93 | |
| DERNBackbone=DeepSeek-V2 Lite, Evaluation setting=15-shot, Average Accuracy (Av)=57.06, Parameter Reduction (P)=22.56%2026.04 | 48.14 | |
| DeLaskModel=LLaMA2-7B, Decoding=DeLask2026.05 | 46.24 | |
| ADModel=LLaMA2-7B, Decoding=AD2026.05 | 46.07 | |
| RegularModel=LLaMA2-7B, Decoding=Regular2026.05 | 45.98 | |
| DOLAModel=LLaMA2-7B, Decoding=DOLA2026.05 | 45.63 | |
| Bidirectional dLLMShots=25-sh, Candidate Type=long-target2026.06 | 45.3 | |
| ITIModel=LLaMA2-7B, Decoding=ITI2026.05 | 44.13 | |
| MoEITSBackbone=DeepSeek-V2 Lite, Evaluation setting=15-shot, Average Accuracy (Av)=67.99, Parameter Reduction (P)=55.42%2026.04 | 43.77 | |
| UADModel=LLaMA2-7B, Decoding=UAD2026.05 | 43.17 | |
| MoE-I^2Backbone=DeepSeek-V2 Lite, Evaluation setting=15-shot, Average Accuracy (Av)=62.79, Parameter Reduction (P)=53.98%2026.04 | 42.58 | |
| R2LM-PI (plug-in)Shots=25-sh, Candidate Type=long-target2026.06 | 40.9 | |
| MoE-PrunerBackbone=DeepSeek-V2 Lite, Evaluation setting=15-shot, Average Accuracy (Av)=61.422026.04 | 40.02 | |
| R2LMShots=25-sh, Candidate Type=long-target2026.06 | 39.4 | |
| Causal dLLMShots=25-sh, Candidate Type=long-target2026.06 | 38.7 | |
| MDLMevaluation protocol=likelihood-based answer scoring2026.05 | 24.66 | |
| CoFReevaluation protocol=likelihood-based answer scoring2026.05 | 22.87 |