Multiple-choice Question Answering on ARC-E
87.8AccuracyUQ4CT
Evaluation results
| Method | Links | ||
|---|---|---|---|
| UQ4CTBackbone=Llama3.1-8B, Fine-tuned Dataset=OBQA, Number of forward passes=12024.10 | 87.8 | 3.61 | |
| BLoB(N=10)Backbone=Llama3.1-8B, Fine-tuned Dataset=OBQA, Number of forward passes=102024.10 | 87.2 | 3 | |
| MC DropBackbone=Llama3.1-8B, Fine-tuned Dataset=OBQA2024.10 | 86.92 | 10.21 | |
| LoRABackbone=Llama3.1-8B, Fine-tuned Dataset=OBQA2024.10 | 86.7 | 9.6 | |
| MixLoRABackbone=Llama3.1-8B, Fine-tuned Dataset=OBQA2024.10 | 86.7 | 9.9 | |
| BLoB(Mean)Backbone=Llama3.1-8B, Fine-tuned Dataset=OBQA2024.10 | 86.6 | 6.61 | |
| LABackbone=Llama3.1-8B, Fine-tuned Dataset=OBQA2024.10 | 86.4 | 6.4 | |
| EnsembleBackbone=Llama3.1-8B, Fine-tuned Dataset=OBQA2024.10 | 85.4 | 10.7 | |
| AR (autoregressive, ours)Zero-shot=true, Likelihood-based evaluation=true, Number of Parameters=1.7B2026.05 | 73.1 | — | |
| AR (autoregressive)Zero-shot=true, Likelihood-based evaluation=true, Number of Parameters=1.7B2026.05 | 72.7 | — | |
| Duo (uniform diffusion)Zero-shot=true, Likelihood-based evaluation=true, Number of Parameters=1.7B2026.05 | 53.4 | — | |
| MDLM (masked diffusion)Zero-shot=true, Likelihood-based evaluation=true, Number of Parameters=1.7B2026.05 | 50.5 | — | |
| CFM (ours)Zero-shot=true, Likelihood-based evaluation=true, Number of Parameters=1.7B2026.05 | 47.1 | — | |
| Eso-LM (interpolating)Zero-shot=true, Likelihood-based evaluation=true, Number of Parameters=1.7B2026.05 | 46 | — | |
| SMDM-1BZero-shot=true, Likelihood-based evaluation=true, Number of Parameters=1B2026.05 | 37.4 | — | |
| ChanceZero-shot=true, Likelihood-based evaluation=true2026.05 | 24.7 | — |