Commonsense Reasoning on BoolQ (test)
89.45AccuracyBaRA
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| BaRAParams (M)=5.6082026.06 | 89.45 | — | — | 1.74 | 0.22 | |
| EnsembleParams (M)=11.3052026.06 | 89.13 | — | — | 2.61 | 0.24 | |
| MLEParams (M)=3.7682026.06 | 89.06 | — | — | 3.79 | 0.25 | |
| BLoBParams (M)=5.4032026.06 | 89.05 | — | — | 1.4 | 0.23 | |
| MAPParams (M)=3.7682026.06 | 89.04 | — | — | 3.81 | 0.25 | |
| MC-DropoutParams (M)=3.7682026.06 | 88.99 | — | — | 3.88 | 0.25 | |
| FVAE-LoRAParams (M)=33.532026.06 | 88.99 | — | — | 4.92 | 0.29 | |
| C-LoRAParams (M)=4.2752026.06 | 88.73 | — | — | 2.5 | 0.27 | |
| ScalaBLParams (M)=3.7692026.06 | 88.48 | — | — | 1.96 | 0.24 | |
| AUSteer-FFNModel=Gemma2-27B-it2026.02 | 88.41 | — | — | — | — | |
| AUSteer-HeadModel=Gemma2-27B-it2026.02 | 88.1 | — | — | — | — | |
| LaplaceParams (M)=3.7682026.06 | 87.77 | — | — | 1.18 | 0.29 | |
| VanillaModel=Gemma2-27B-it2026.02 | 86.88 | — | — | — | — | |
| AUSteer-HeadModel=LLaMA2-13B-Chat2026.02 | 85.25 | — | — | — | — | |
| AUSteer-FFNModel=LLaMA2-13B-Chat2026.02 | 85.02 | — | — | — | — | |
| VanillaModel=LLaMA2-13B-Chat2026.02 | 84.01 | — | — | — | — | |
| LoRABackbone=Llama-2-13b, Rank=32, Param × 10⁵=262.1, Protocol=Fine-tuning2025.06 | 84 | — | — | — | — | |
| AUSteer-FFNModel=LLaMA3.1-8B-Instruct2026.02 | 83.79 | — | — | — | — | |
| LoTRBackbone=Llama-2-13b, Rank=64, Param × 10⁵=9.83, Protocol=Fine-tuning2025.06 | 83.6 | — | — | — | — | |
| LoRABackbone=Llama-2-13b, Rank=8, Param × 10⁵=65.5, Protocol=Fine-tuning2025.06 | 83.4 | — | — | — | — | |
| AUSteer-HeadModel=LLaMA3.1-8B-Instruct2026.02 | 83.18 | — | — | — | — | |
| LoTRBackbone=Llama-2-13b, Rank=16, Param × 10⁵=1.84, Protocol=Fine-tuning2025.06 | 83.1 | — | — | — | — | |
| MetaTT-4DBackbone=Llama-2-13b, Rank=64, Param × 10⁵=8.3, Protocol=Fine-tuning2025.06 | 83 | — | — | — | — | |
| VanillaModel=LLaMA3.1-8B-Instruct2026.02 | 82.57 | — | — | — | — | |
| LoRABackbone=Llama-2-7b, Rank=16, Param × 10⁵=83.9, Protocol=Fine-tuning2025.06 | 81 | — | — | — | — | |
| MetaTT-4DBackbone=Llama-2-13b, Rank=16, Param × 10⁵=1.75, Protocol=Fine-tuning2025.06 | 81 | — | — | — | — | |
| LoRABackbone=Llama-2-7b, Rank=8, Param × 10⁵=41.9, Protocol=Fine-tuning2025.06 | 80.8 | — | — | — | — | |
| VeRABackbone=Llama-2-13b, Rank=256, Param × 10⁵=4.3, Protocol=Fine-tuning2025.06 | 80.4 | — | — | — | — | |
| MetaTT-4DBackbone=Llama-2-7b, Rank=64, Param × 10⁵=6.6, Protocol=Fine-tuning2025.06 | 80 | — | — | — | — | |
| LoTRBackbone=Llama-2-7b, Rank=16, Param × 10⁵=1.47, Protocol=Fine-tuning2025.06 | 78 | — | — | — | — | |
| LoTRBackbone=Llama-2-7b, Rank=64, Param × 10⁵=7.86, Protocol=Fine-tuning2025.06 | 78 | — | — | — | — | |
| MetaTT-4DBackbone=Llama-2-7b, Rank=16, Param × 10⁵=1.40, Protocol=Fine-tuning2025.06 | 77 | — | — | — | — | |
| VeRABackbone=Llama-2-7b, Rank=1024, Param × 10⁵=3.27, Protocol=Fine-tuning2025.06 | 75.8 | — | — | — | — | |
| Zero ShotBackbone=Llama-2-7b, Protocol=Zero-shot2025.06 | 74.7 | — | — | — | — | |
| Zero ShotBackbone=Llama-2-13b, Protocol=Zero-shot2025.06 | 71 | — | — | — | — | |
| MAXIMUM ENTROPYBase Model=LLaMA2-7B2024.03 | — | 60.65 | 0.48 | — | — | |
| MAXIMUM ENTROPY w/ STARBase Model=LLaMA2-7B2024.03 | — | 61.91 | 3.67 | — | — | |
| PREDICTIVE ENTROPYBase Model=LLaMA2-7B2024.03 | — | 58.39 | -5.24 | — | — | |
| PREDICTIVE ENTROPY w/ STARBase Model=LLaMA2-7B2024.03 | — | 61.84 | 3.49 | — | — | |
| RANDOMBase Model=LLaMA2-7B2024.03 | — | 60.46 | — | — | — |