Commonsense Reasoning on WG-S
70.9AccuracyBayesian-LoRA (S=4)
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Bayesian-LoRA (S=4)S (Samples)=4, Evaluation Protocol=End-to-End, rind=9, cind=92026.01 | 70.9 | 4.9 | 0.79 | |
| BLoBBackbone=Llama2-7B, Method=LoRA, Steps=5,000, N=02024.06 | 70.89 | 20.62 | 0.91 | |
| ENSBackbone=Llama2-7B, Method=LoRA, Steps=5,0002024.06 | 69.57 | 28.52 | 2.71 | |
| MCDBackbone=Llama2-7B, Method=LoRA, Steps=5,0002024.06 | 69.46 | 27.98 | 2.79 | |
| MAPBackbone=Llama2-7B, Method applied to=LoRA, Gradient steps=50002026.07 | 69.37 | 29.76 | 2.86 | |
| LAPBackbone=Llama2-7B, Method=LoRA, Steps=5,0002024.06 | 69.2 | 4.15 | 0.6 | |
| BLoBBackbone=Llama2-7B, Method=LoRA, Steps=5,000, N=102024.06 | 69.07 | 9.35 | 0.63 | |
| BLoB (N=10)N (Ensemble Size)=10, rind=9, cind=92026.01 | 69.07 | 9.35 | 0.63 | |
| MCDBackbone=Llama2-7B, Method applied to=LoRA, Samples (M)=10, Gradient steps=50002026.07 | 69.06 | 28.49 | 2.5 | |
| MLEBackbone=Llama2-7B, Method=LoRA, Steps=5,0002024.06 | 68.99 | 29.83 | 3.17 | |
| Deep EnsembleBackbone=Llama2-7B, Method applied to=LoRA, Samples (M)=10, Gradient steps=50002026.07 | 68.98 | 28.72 | 2.44 | |
| MAPBackbone=Llama2-7B, Method=LoRA, Steps=5,0002024.06 | 68.62 | 29.76 | 2.46 | |
| LABackbone=Llama2-7B, Method applied to=LoRA, Gradient steps=50002026.07 | 68.18 | 11.41 | 0.62 | |
| MAPrind=9, cind=92026.01 | 68 | 30.8 | 2.75 | |
| Temprind=9, cind=92026.01 | 67 | 12.8 | 0.68 | |
| LLLA (post-hoc)Evaluation Protocol=post-hoc, rind=9, cind=92026.01 | 66.9 | 11.6 | 0.68 | |
| LA (post-hoc)Evaluation Protocol=post-hoc, rind=9, cind=92026.01 | 66.9 | 7.8 | 0.66 | |
| TFBBackbone=Llama2-7B, Method applied to=LoRA, Samples (M)=10, Gradient steps=50002026.07 | 66.84 | 9.36 | 0.62 | |
| Dropoutrind=9, cind=92026.01 | 66.7 | 29.5 | 2.54 | |
| Ckpt Ensrind=9, cind=92026.01 | 66.7 | 25.2 | 1.31 | |
| DALorRABackbone=Llama2-7B, Method applied to=LoRA, Samples (M)=10, Gradient steps=50002026.07 | 66.61 | 7.84 | 0.62 | |
| BLoBBackbone=Llama2-7B, Method applied to=LoRA, Samples (M)=10, Gradient steps=50002026.07 | 66.55 | 11.23 | 0.66 | |
| BLoBBackbone=Llama2-7B, Method=LoRA, Steps=5,000, N=52024.06 | 66.3 | 10.89 | 0.68 | |
| C-LoRABackbone=Llama2-7B, Method applied to=LoRA, Samples (M)=10, Gradient steps=50002026.07 | 66.21 | 7.86 | 0.63 | |
| BBBBackbone=Llama2-7B, Method=LoRA, Steps=5,0002024.06 | 56.54 | 21.81 | 1.4 | |
| BBBrind=9, cind=92026.01 | 56.54 | 21.81 | 1.4 |