Commonsense Reasoning on OBQA
89.2AccuracyHydraLoRA
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| HydraLoRABackbone=Qwen2.5-3B2026.02 | 89.2 | — | — | |
| MoSLoRABackbone=Qwen2.5-3B2026.02 | 89 | — | — | |
| UQ4CTBackbone=Llama3.1-8B2024.10 | 88.4 | 3.341 | — | |
| MixLoRABackbone=Llama3.1-8B2024.10 | 88.27 | 6.58 | — | |
| LoRABackbone=Llama3.1-8B2024.10 | 88.001 | 7.3 | — | |
| HypLoRABackbone=Qwen2.5-3B2026.02 | 87.8 | — | — | |
| BLoB(Mean)Backbone=Llama3.1-8B, Variant=Mean2024.10 | 87.601 | 6.83 | — | |
| BLoB(N=10)Backbone=Llama3.1-8B, N=102024.10 | 87.13 | 3.841 | — | |
| MC DropBackbone=Llama3.1-8B2024.10 | 87.121 | 7.24 | — | |
| LoRABackbone=Qwen2.5-3B2026.02 | 87 | — | — | |
| EnsembleBackbone=Llama3.1-8B2024.10 | 86.47 | 8.63 | — | |
| LABackbone=Llama3.1-8B2024.10 | 86.006 | 11.975 | — | |
| BoHAModel=Llama-3.2-3B, #Params=48.63M2025.09 | 85.73 | — | — | |
| FFTModel=Llama-3.2-3B, #Params=3.21B2025.09 | 85 | — | — | |
| ABBAModel=Llama-3.2-3B, #Params=48.63M2025.09 | 84.27 | — | — | |
| GraLoRAModel=Llama-3.2-3B, #Params=48.63M2025.09 | 83.73 | — | — | |
| HiRAModel=Llama-3.2-3B, #Params=48.63M2025.09 | 83.32 | — | — | |
| BLoBBackbone=Llama2-7B, Method=LoRA, Steps=5,000, N=02024.06 | 82.73 | 8.36 | 0.56 | |
| LAPBackbone=Llama2-7B, Method=LoRA, Steps=5,0002024.06 | 82.12 | 8.7 | 0.52 | |
| BBBBackbone=Llama2-7B, Method=LoRA, Steps=5,0002024.06 | 82.06 | 11.38 | 0.66 | |
| BLoBBackbone=Llama2-7B, Method=LoRA, Steps=5,000, N=52024.06 | 81.79 | 3.4 | 0.53 | |
| MCDBackbone=Llama2-7B, Method=LoRA, Steps=5,0002024.06 | 81.72 | 13.1 | 0.77 | |
| rsLoRAModel=Llama-3.2-3B, #Params=48.63M2025.09 | 81.72 | — | — | |
| MLEBackbone=Llama2-7B, Method=LoRA, Steps=5,0002024.06 | 81.52 | 12.55 | 0.73 | |
| BLoBBackbone=Llama2-7B, Method=LoRA, Steps=5,000, N=102024.06 | 81.52 | 3.77 | 0.5 | |
| MAPBackbone=Llama2-7B, Method=LoRA, Steps=5,0002024.06 | 81.38 | 13.26 | 0.75 | |
| ENSBackbone=Llama2-7B, Method=LoRA, Steps=5,0002024.06 | 81.38 | 15.34 | 1.06 | |
| PiSSAModel=Llama-3.2-3B, #Params=48.63M2025.09 | 80.8 | — | — | |
| L-LoRA-SParameters=10M, Phase=MAP2026.05 | 80.8 | 16.4 | 1.12 | |
| L-LoRA-XSRank=100, Phase=MAP2026.05 | 80.8 | 15.6 | 1.12 | |
| L-LoRA-SParameters=10M, Phase=Bayesian2026.05 | 80.6 | 6.5 | 0.6 | |
| MoREparams/task=4.5M, Backbone=Llama2-7B2026.03 | 80.5 | — | — | |
| L-LoRA-XSRank=100, Phase=Bayesian2026.05 | 80.2 | 10.7 | 0.63 | |
| DoRAModel=Llama-3.2-3B, #Params=49.40M2025.09 | 79.8 | — | — | |
| LoRAModel=Llama-3.2-3B, #Params=48.63M2025.09 | 79.6 | — | — | |
| Laplace (LA)Phase=Bayesian2026.05 | 78.9 | 6.4 | 0.65 | |
| L-LoRA-SParameters=4M, Phase=MAP2026.05 | 78.8 | 16.5 | 1.09 | |
| L-LoRA-SParameters=4M, Phase=Bayesian2026.05 | 78.8 | 10.1 | 0.77 | |
| Laplace (LA)Phase=MAP2026.05 | 78.7 | 16.1 | 0.99 | |
| B-LoRA-XSRank=64, Phase=Bayesian2026.05 | 78.6 | 10.7 | 0.74 | |
| EPTparams/task=3.3M, Backbone=Llama2-7B2026.03 | 78.4 | — | — | |
| B-LoRA-XSRank=64, Phase=MAP2026.05 | 78.2 | 14.4 | 0.91 | |
| FullBackbone=Deepseek-R1-Distill-Qwen-7B2025.12 | 78 | — | — | |
| L-LoRA-XSRank=64, Phase=Bayesian2026.05 | 78 | 9.2 | 0.66 | |
| L-LoRA-XSRank=64, Phase=MAP2026.05 | 76.8 | 18 | 1.16 | |
| B-LoRA-XSRank=32, Phase=MAP2026.05 | 76.8 | 15.7 | 0.96 | |
| B-LoRA-XSRank=32, Phase=Bayesian2026.05 | 76.4 | 10.2 | 0.71 | |
| SnapKVBackbone=Deepseek-R1-Distill-Qwen-7B, Budget=5122025.12 | 76 | — | — | |
| L-LoRA-XSRank=32, Phase=MAP2026.05 | 76 | 18.5 | 1.05 | |
| L-LoRA-XSRank=32, Phase=Bayesian2026.05 | 75.4 | 6 | 0.67 | |
| SnapKVBackbone=Deepseek-R1-Distill-Qwen-7B, Budget=2562025.12 | 75 | — | — | |
| LoRAparams/task=2.1M, Backbone=Llama2-7B2026.03 | 74 | — | — | |
| FFTModel=Llama-3.2-1B, #Params=1.24B2025.09 | 74 | — | — | |
| BoHAModel=Llama-3.2-1B, #Params=22.54M2025.09 | 73.33 | — | — | |
| ABBAModel=Llama-3.2-1B, #Params=22.54M2025.09 | 72.93 | — | — | |
| Base ModelBackbone=Llama3.1-8B2024.10 | 72.8 | 11.39 | — | |
| DoRAModel=Llama-3.2-1B, #Params=22.92M2025.09 | 72.2 | — | — | |
| PiSSAModel=Llama-3.2-1B, #Params=22.54M2025.09 | 72.2 | — | — | |
| DRAGON2022.10 | 72 | — | — | |
| LoRAModel=Llama-3.2-1B, #Params=22.54M2025.09 | 71.4 | — | — | |
| GraLoRAModel=Llama-3.2-1B, #Params=22.54M2025.09 | 71.4 | — | — | |
| rsLoRAModel=Llama-3.2-1B, #Params=22.54M2025.09 | 71.11 | — | — | |
| SnapKVBackbone=Deepseek-R1-Distill-Qwen-7B, Budget=1282025.12 | 71 | — | — | |
| HiRAModel=Llama-3.2-1B, #Params=22.54M2025.09 | 70.2 | — | — | |
| H2OBackbone=Deepseek-R1-Distill-Qwen-7B, Budget=3842025.12 | 69 | — | — | |
| MultiLoRAparams/task=10M, Backbone=Llama2-7B2026.03 | 68.2 | — | — | |
| SnapKVBackbone=Deepseek-R1-Distill-Qwen-7B, Budget=3842025.12 | 68 | — | — | |
| QAGNN2022.10 | 67.8 | — | — | |
| MOELoRAparams/task=4.5M, Backbone=Llama2-7B2026.03 | 67.8 | — | — | |
| H2OBackbone=Deepseek-R1-Distill-Qwen-7B, Budget=5122025.12 | 67 | — | — | |
| GreaseLM2022.10 | 66.9 | — | — | |
| RoBERTa2022.10 | 64.9 | — | — | |
| H2OBackbone=Deepseek-R1-Distill-Qwen-7B, Budget=2562025.12 | 64 | — | — | |
| CKT-baseBackbone=T5-base, Method=CKT2023.06 | 61.58 | — | — | |
| CALMBackbone=T5-base2023.06 | 60.9 | — | — | |
| LLaMAParameters=65B, Zero-shot=true, Likelihood Normalization=Answer-conditional2023.02 | 60.2 | — | — | |
| ERNIE-baseBackbone=ERNIE-base2023.06 | 58.9 | — | — | |
| LLaMAParameters=33B, Zero-shot=true, Likelihood Normalization=Answer-conditional2023.02 | 58.6 | — | — | |
| T5-base + SSMBackbone=T5-base, Augmentation=SSM2023.06 | 58.6 | — | — | |
| KnowBERTBackbone=KnowBERT2023.06 | 58.5 | — | — | |
| T5-base + TIBackbone=T5-base, Augmentation=TI2023.06 | 58.43 | — | — | |
| T5-baseBackbone=T5-base2023.06 | 58.2 | — | — | |
| T5-base + CSKG (Rule)Backbone=T5-base, Augmentation=CSKG (Rule)2023.06 | 57.97 | — | — | |
| GPT-3Parameters=175B, Zero-shot=true, Likelihood Normalization=Answer-conditional2023.02 | 57.6 | — | — | |
| BERT-baseBackbone=BERT-base2023.06 | 57.6 | — | — | |
| LLaMAParameters=7B, Zero-shot=true, Likelihood Normalization=Answer-conditional2023.02 | 57.2 | — | — | |
| CKT w/ GPT-2Backbone=GPT-2, Method=CKT2023.06 | 56.95 | — | — | |
| T5-base + KDBackbone=T5-base, Augmentation=KD2023.06 | 56.54 | — | — | |
| LLaMAParameters=13B, Zero-shot=true, Likelihood Normalization=Answer-conditional2023.02 | 56.4 | — | — | |
| T5-base + CSKG (TI)Backbone=T5-base, Augmentation=CSKG (TI)2023.06 | 56.17 | — | — | |
| PaLMParameters=540B, Zero-shot=true, Likelihood Normalization=Answer-conditional2023.02 | 53.4 | — | — | |
| COMETBackbone=BART2023.06 | 51.2 | — | — | |
| PaLMParameters=62B, Zero-shot=true, Likelihood Normalization=Answer-conditional2023.02 | 50.4 | — | — | |
| PASERQuantization=GPTQ 4 bits, Backbone=LLaMA2-13B2025.02 | 44.2 | — | — | |
| NuggetsQuantization=GPTQ 4 bits, Backbone=LLaMA2-13B2025.02 | 42.7 | — | — | |
| H2OBackbone=Deepseek-R1-Distill-Qwen-7B, Budget=1282025.12 | 42 | — | — | |
| w/o trainingQuantization=w/o Quant, Backbone=LLaMA2-13B2025.02 | 42 | — | — | |
| PASERQuantization=RTN 4 bits, Backbone=LLaMA2-13B2025.02 | 41.7 | — | — | |
| IFDQuantization=GPTQ 4 bits, Backbone=LLaMA2-13B2025.02 | 41.4 | — | — | |
| Instruction MiningQuantization=GPTQ 4 bits, Backbone=LLaMA2-13B2025.02 | 41.3 | — | — |