Multiple Choice Question Answering on ARC Easy (Accuracy, ECE, NLL)
86AccuracyLoRA-Ensemble
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| LoRA-EnsembleBackbone=LLaMA-2-7B, Ensemble size (M)=52026.01 | 86 | 9 | 0.92 | |
| Deep EnsembleBackbone=LLaMA-2-7B, Ensemble size (M)=32026.01 | 85.8 | 9.9 | 0.83 | |
| Ckpt EnsembleBackbone=LLaMA-2-7B, Ensemble size (M)=32026.01 | 85.8 | 9.8 | 0.8 | |
| SV-EnsembleBackbone=LLaMA-2-7B, Ensemble size (M)=162026.01 | 85.8 | 3.8 | 0.43 | |
| BlobBackbone=LLaMA-2-7B, Ensemble size (N)=102026.01 | 85.5 | 3.6 | 0.4 | |
| Bayes-LoRA (LA)Backbone=LLaMA-2-7B2026.01 | 85.1 | 5.4 | 0.49 | |
| MC DropoutBackbone=LLaMA-2-7B, Ensemble size (N)=102026.01 | 85 | 12.4 | 1.11 | |
| SingleBackbone=LLaMA-2-7B2026.01 | 84.7 | 13.4 | 1.26 | |
| Bayes-LoRA (LLLA)Backbone=LLaMA-2-7B2026.01 | 84.7 | 11.6 | 0.87 | |
| BlobBackbone=LLaMA-2-7B, Ensemble size (N)=52026.01 | 84.7 | 6.2 | 0.46 | |
| Single w/ SVFBackbone=LLaMA-2-7B2026.01 | 83.2 | 12.1 | 0.77 | |
| Last-Layer EnsembleBackbone=LLaMA-2-7B, Ensemble size (M)=52026.01 | 72 | 6 | 0.79 |