Commonsense Reasoning on Commonsense Reasoning Suite (test)
0.9594HellaSwag AccuracyPERA (r = 16)
Evaluation results
| Method | Links | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| PERA (r = 16)Model=Llama-3-8B, Rank (r)=16, #Params (%)=0.35152026.04 | 0.9594 | 0.8738 | — | 0.8857 | 0.9373 | 0.8362 | — | — | 0.886 | — | 0.8927 | 0.7615 | 0.8316 | — | |
| PERA (r = 32)Model=Llama-3-8B, Rank (r)=32, #Params (%)=0.70122026.04 | 0.9561 | 0.8712 | — | 0.8983 | 0.9356 | 0.8353 | — | — | 0.884 | — | 0.8895 | 0.7554 | 0.8158 | — | |
| DoRA (r = 32)Model=Llama-3-8B, Rank (r)=32, #Params (%)=0.70022026.04 | 0.955 | 0.852 | — | 0.893 | 0.905 | 0.804 | — | — | 0.858 | — | 0.856 | 0.746 | 0.799 | — | |
| PERA (r = 16)Model=Qwen2.5-7B, Rank (r)=16, #Params (%)=0.35432026.04 | 0.9538 | 0.8829 | — | 0.8961 | 0.9617 | 0.8891 | — | — | 0.918 | — | 0.8934 | 0.7306 | 0.8204 | — | |
| HiRA (r = 32)Model=Llama-3-8B, Rank (r)=32, #Params (%)=0.70022026.04 | 0.9536 | 0.8672 | — | 0.897 | 0.9327 | 0.829 | — | — | 0.8832 | — | 0.877 | 0.754 | 0.8115 | — | |
| HiRA (r = 16)Model=Llama-3-8B, Rank (r)=16, #Params (%)=0.35132026.04 | 0.9485 | 0.8608 | — | 0.8912 | 0.9306 | 0.8259 | — | — | 0.874 | — | 0.8674 | 0.7385 | 0.8106 | — | |
| LoRA (r = 16)Model=Llama-3-8B, Rank (r)=16, #Params (%)=0.35132026.04 | 0.935 | 0.828 | — | 0.867 | 0.877 | 0.757 | — | — | 0.828 | — | 0.848 | 0.723 | 0.793 | — | |
| HiRA (r = 16)Model=Qwen2.5-7B, Rank (r)=16, #Params (%)=0.35412026.04 | 0.923 | 0.854 | — | 0.883 | 0.954 | 0.887 | — | — | 0.88 | — | 0.81 | 0.69 | 0.808 | — | |
| LoRA (r = 32)Model=Llama-3-8B, Rank (r)=32, #Params (%)=0.70022026.04 | 0.917 | 0.8079 | — | 0.852 | 0.842 | 0.712 | — | — | 0.79 | — | 0.843 | 0.708 | 0.799 | — | |
| DoRA (r = 32)Model=Llama-2-7B, Rank (r)=32, #Params (%)=0.82562026.04 | 0.891 | 0.7969 | — | 0.837 | 0.837 | 0.682 | — | — | 0.824 | — | 0.826 | 0.718 | 0.76 | — | |
| PERA (r = 32)Model=Llama-2-7B, Rank (r)=32, #Params (%)=0.82682026.04 | 0.8898 | 0.8214 | — | 0.8439 | 0.8779 | 0.7329 | — | — | 0.832 | — | 0.8627 | 0.737 | 0.7953 | — | |
| PERA (r = 16)Model=Llama-2-7B, Rank (r)=16, #Params (%)=0.41482026.04 | 0.8866 | 0.8261 | — | 0.8531 | 0.8855 | 0.7355 | — | — | 0.854 | — | 0.8713 | 0.7183 | 0.8035 | — | |
| HiRA (r = 32)Model=Llama-2-7B, Rank (r)=32, #Params (%)=0.82562026.04 | 0.8812 | 0.8142 | — | 0.8335 | 0.8674 | 0.7381 | — | — | 0.846 | — | 0.8398 | 0.7122 | 0.7953 | — | |
| LoRA (r = 32)Model=Llama-2-7B, Rank (r)=32, #Params (%)=0.82562026.04 | 0.836 | 0.7761 | — | 0.799 | 0.798 | 0.647 | — | — | 0.81 | — | 0.826 | 0.698 | 0.795 | — | |
| DensePrune%=0%, Evaluation=Zero-shot, Model=Mistral-7B2025.12 | 0.8105 | 0.714 | — | 0.8226 | 0.8093 | 0.5392 | — | — | 0.442 | — | 0.738 | 0.8364 | — | — | |
| Qwen2.5 7B-NLBackbone=Qwen2.5 7B, Strategy=NL-tuned, Zero-shot=true, Greedy decoding=true2025.05 | 0.81 | — | — | 0.84 | — | — | — | — | — | — | 0.62 | — | — | 0.95 | |
| LoRA (r = 16)Model=Qwen2.5-7B, Rank (r)=16, #Params (%)=0.35412026.04 | 0.786 | 0.738 | — | 0.736 | 0.859 | 0.717 | — | — | 0.744 | — | 0.758 | 0.6 | 0.7 | — | |
| ChatGPTModel=ChatGPT2026.04 | 0.785 | 0.7701 | — | 0.854 | 0.898 | 0.799 | — | — | 0.748 | — | 0.661 | 0.731 | 0.685 | — | |
| Token FilteringPrune%=20%, Evaluation=Zero-shot, Model=Mistral-7B2025.12 | 0.7664 | 0.6582 | — | 0.7661 | 0.7614 | 0.4573 | — | — | 0.414 | — | 0.693 | 0.7498 | — | — | |
| Dense w/oRM Blocks=0(0.0%)2026.06 | 0.7602 | 0.6436 | — | 0.7911 | 0.763 | 0.4633 | — | — | 0.442 | — | 0.6914 | 0.7771 | 0.4606 | — | |
| ShortGPTPruning=static, RM Blocks=4(12.5%)2026.06 | 0.7369 | 0.6316 | — | 0.7715 | 0.7256 | 0.4445 | — | — | 0.42 | — | 0.6914 | 0.7869 | 0.4759 | — | |
| BUDDYPruning=dynamic, RM Blocks=4(12.5%)2026.06 | 0.7364 | 0.6263 | — | 0.7709 | 0.7302 | 0.448 | — | — | 0.43 | — | 0.6882 | 0.733 | 0.4734 | — | |
| SLEBPruning=static, RM Blocks=4(12.5%)2026.06 | 0.726 | 0.6088 | — | 0.7748 | 0.734 | 0.4104 | — | — | 0.414 | — | 0.6401 | 0.7196 | 0.4514 | — | |
| Shortened LlamaPruning=static, RM Blocks=4(12.5%)2026.06 | 0.7253 | 0.6084 | — | 0.7802 | 0.7327 | 0.4283 | — | — | 0.406 | — | 0.629 | 0.7125 | 0.4529 | — | |
| POTARAModel=Llama-3.2-3B-Instruct2026.02 | 0.7173 | 0.7418 | — | 0.788 | 0.878 | 0.752 | — | — | 0.7847 | — | 0.6453 | 0.6787 | 0.69 | — | |
| Llama 3.1 8B-NLBackbone=Llama 3.1 8B, Strategy=NL-tuned, Zero-shot=true, Greedy decoding=true2025.05 | 0.71 | — | — | 0.77 | — | — | — | — | — | — | 0.66 | — | — | 0.82 | |
| Llama 3.1 8B-PCBackbone=Llama 3.1 8B, Strategy=Pseudo-code tuned, Zero-shot=true, Greedy decoding=true2025.05 | 0.71 | — | — | 0.82 | — | — | — | — | — | — | 0.61 | — | — | 0.89 | |
| PuDDingPruning=dynamic, RM Blocks=4(12.5%)2026.06 | 0.6973 | 0.5948 | — | 0.7595 | 0.6932 | 0.3951 | — | — | 0.4 | — | 0.6448 | 0.7355 | 0.433 | — | |
| LocalModel=Llama-3.2-3B-Instruct2026.02 | 0.6893 | 0.7205 | — | 0.7727 | 0.862 | 0.7213 | — | — | 0.754 | — | 0.6407 | 0.6627 | 0.6613 | — | |
| FedALTModel=Llama-3.2-3B-Instruct2026.02 | 0.6833 | 0.7148 | — | 0.79 | 0.85 | 0.7273 | — | — | 0.7667 | — | 0.6033 | 0.6427 | 0.6553 | — | |
| Granite 3.1 8B-PCBackbone=Granite 3.1 8B, Strategy=Pseudo-code tuned, Zero-shot=true, Greedy decoding=true2025.05 | 0.68 | — | — | 0.78 | — | — | — | — | — | — | 0.7 | — | — | 0.92 | |
| Olmo 3 7B-NLBackbone=Olmo 3 7B, Strategy=NL-tuned, Zero-shot=true, Greedy decoding=true2025.05 | 0.68 | — | — | 0.72 | — | — | — | — | — | — | 0.57 | — | — | 0.92 | |
| ShortGPTPruning=static, RM Blocks=8(25%)2026.06 | 0.6766 | 0.572 | — | 0.7242 | 0.6528 | 0.3891 | — | — | 0.396 | — | 0.6709 | 0.6294 | 0.4371 | — | |
| BUDDYPruning=dynamic, RM Blocks=8(25%)2026.06 | 0.6664 | 0.5844 | — | 0.7318 | 0.6528 | 0.3805 | — | — | 0.392 | — | 0.6685 | 0.7263 | 0.4565 | — | |
| Olmo 3 7B-PCBackbone=Olmo 3 7B, Strategy=Pseudo-code tuned, Zero-shot=true, Greedy decoding=true2025.05 | 0.66 | — | — | 0.74 | — | — | — | — | — | — | 0.55 | — | — | 0.92 | |
| FedDPAModel=Llama-3.2-3B-Instruct2026.02 | 0.6573 | 0.7233 | — | 0.804 | 0.876 | 0.7473 | — | — | 0.7707 | — | 0.6147 | 0.638 | 0.6787 | — | |
| SLEBPruning=static, RM Blocks=8(25%)2026.06 | 0.6572 | 0.5678 | — | 0.7378 | 0.6658 | 0.3549 | — | — | 0.398 | — | 0.6046 | 0.6924 | 0.4319 | — | |
| FedSAModel=Llama-3.2-3B-Instruct2026.02 | 0.6493 | 0.7115 | — | 0.7773 | 0.8467 | 0.7153 | — | — | 0.7487 | — | 0.6467 | 0.6427 | 0.6653 | — | |
| FedITModel=Llama-3.2-3B-Instruct2026.02 | 0.6493 | 0.7194 | — | 0.788 | 0.8593 | 0.7427 | — | — | 0.7433 | — | 0.6053 | 0.6687 | 0.6987 | — | |
| SlimGPT w/oPrune%=20%, Evaluation=Zero-shot, Model=Mistral-7B2025.12 | 0.6467 | 0.5551 | — | 0.7176 | 0.5295 | 0.3439 | — | — | 0.346 | — | 0.644 | 0.6581 | — | — | |
| Shortened LlamaPruning=static, RM Blocks=8(25%)2026.06 | 0.6315 | 0.5377 | — | 0.7427 | 0.6326 | 0.349 | — | — | 0.37 | — | 0.5462 | 0.6187 | 0.411 | — | |
| Mistral 7B v0.3-PCBackbone=Mistral 7B v0.3, Strategy=Pseudo-code tuned, Zero-shot=true, Greedy decoding=true2025.05 | 0.62 | — | — | 0.77 | — | — | — | — | — | — | 0.6 | — | — | 0.8 | |
| Token FilteringPrune%=50%, Evaluation=Zero-shot, Model=Mistral-7B2025.12 | 0.6119 | 0.5037 | — | 0.6899 | 0.4928 | 0.343 | — | — | 0.32 | — | 0.6369 | 0.4315 | — | — | |
| PuDDingPruning=dynamic, RM Blocks=8(25%)2026.06 | 0.603 | 0.5307 | — | 0.7182 | 0.6141 | 0.3558 | — | — | 0.366 | — | 0.5667 | 0.6287 | 0.393 | — | |
| ShortGPTPruning=static, RM Blocks=12(37.5%)2026.06 | 0.5876 | 0.5331 | — | 0.6692 | 0.5446 | 0.3413 | — | — | 0.332 | — | 0.6409 | 0.7122 | 0.4365 | — | |
| FLAPPrune%=20%, Evaluation=Zero-shot, Model=Mistral-7B2025.12 | 0.5806 | 0.5772 | — | 0.6899 | 0.67 | 0.3763 | — | — | 0.386 | — | 0.6519 | 0.6859 | — | — | |
| Qwen2.5 7B-CPBackbone=Qwen2.5 7B, Strategy=Code-Prompted, Zero-shot=true, Greedy decoding=true2025.05 | 0.58 | — | — | 0.83 | — | — | — | — | — | — | 0.62 | — | — | 0.96 | |
| BUDDYPruning=dynamic, RM Blocks=12(37.5%)2026.06 | 0.5787 | 0.5354 | — | 0.7078 | 0.5808 | 0.3114 | — | — | 0.356 | — | 0.6022 | 0.7281 | 0.4181 | — | |
| FFA-LoRAModel=Llama-3.2-3B-Instruct2026.02 | 0.5773 | 0.688 | — | 0.7907 | 0.8553 | 0.7327 | — | — | 0.7387 | — | 0.5413 | 0.648 | 0.62 | — | |
| SLEBPruning=static, RM Blocks=12(37.5%)2026.06 | 0.5483 | 0.5093 | — | 0.7138 | 0.5976 | 0.3106 | — | — | 0.36 | — | 0.5296 | 0.6028 | 0.4115 | — | |
| Mistral 7B v0.3-NLBackbone=Mistral 7B v0.3, Strategy=NL-tuned, Zero-shot=true, Greedy decoding=true2025.05 | 0.53 | — | — | 0.65 | — | — | — | — | — | — | 0.4 | — | — | 0.74 | |
| Shortened LlamaPruning=static, RM Blocks=12(37.5%)2026.06 | 0.5281 | 0.4992 | — | 0.7089 | 0.5728 | 0.2986 | — | — | 0.342 | — | 0.5233 | 0.6214 | 0.3987 | — | |
| Qwen2.5 7B-PCBackbone=Qwen2.5 7B, Strategy=Pseudo-code tuned, Zero-shot=true, Greedy decoding=true2025.05 | 0.52 | — | — | 0.84 | — | — | — | — | — | — | 0.56 | — | — | 0.95 | |
| BUDDYPruning=dynamic, RM Blocks=16(50%)2026.06 | 0.4866 | 0.469 | — | 0.6474 | 0.4891 | 0.2841 | — | — | 0.328 | — | 0.5351 | 0.6021 | 0.3792 | — | |
| FiRSTPruning=dynamic, RM Blocks=4(12.5%)2026.06 | 0.4766 | 0.4691 | — | 0.562 | 0.4827 | 0.3473 | — | — | 0.36 | — | 0.5422 | 0.5774 | 0.4048 | — | |
| PuDDingPruning=dynamic, RM Blocks=12(37.5%)2026.06 | 0.4736 | 0.4494 | — | 0.6469 | 0.4916 | 0.2935 | — | — | 0.316 | — | 0.5359 | 0.4642 | 0.3736 | — | |
| PuDDingPruning=dynamic, RM Blocks=16(50%)2026.06 | 0.4736 | 0.4491 | — | 0.6447 | 0.4916 | 0.2935 | — | — | 0.316 | — | 0.5359 | 0.4642 | 0.3736 | — | |
| ShortGPTPruning=static, RM Blocks=16(50%)2026.06 | 0.4713 | 0.4641 | — | 0.6181 | 0.4491 | 0.2918 | — | — | 0.298 | — | 0.5722 | 0.622 | 0.39 | — | |
| Granite 8B Code-NLBackbone=Granite 8B Code, Strategy=NL-tuned, Zero-shot=true, Greedy decoding=true2025.05 | 0.47 | — | — | 0.63 | — | — | — | — | — | — | 0.54 | — | — | 0.71 | |
| FiRSTPruning=dynamic, RM Blocks=8(25%)2026.06 | 0.4452 | 0.4397 | — | 0.5522 | 0.452 | 0.3294 | — | — | 0.354 | — | 0.4038 | 0.5774 | 0.4038 | — | |
| SlimGPT w/oPrune%=50%, Evaluation=Zero-shot, Model=Mistral-7B2025.12 | 0.4445 | 0.432 | — | 0.6413 | 0.3096 | 0.2364 | — | — | 0.312 | — | 0.5221 | 0.5587 | — | — | |
| FiRSTPruning=dynamic, RM Blocks=12(37.5%)2026.06 | 0.4399 | 0.4512 | — | 0.5517 | 0.4533 | 0.3089 | — | — | 0.362 | — | 0.5367 | 0.5587 | 0.3982 | — | |
| MoRA (r = 32)Model=Llama-3-8B, Rank (r)=32, #Params (%)=0.69972026.04 | 0.4353 | 0.7863 | — | 0.8743 | 0.9116 | 0.7961 | — | — | 0.856 | — | 0.8674 | 0.7428 | 0.8071 | — | |
| SLEBPruning=static, RM Blocks=16(50%)2026.06 | 0.4293 | 0.46 | — | 0.6507 | 0.4949 | 0.2637 | — | — | 0.31 | — | 0.5257 | 0.6171 | 0.3884 | — | |
| FLAPPrune%=50%, Evaluation=Zero-shot, Model=Mistral-7B2025.12 | 0.4052 | 0.4457 | — | 0.5911 | 0.4481 | 0.2477 | — | — | 0.314 | — | 0.5616 | 0.5524 | — | — | |
| Granite 8B Code-PCBackbone=Granite 8B Code, Strategy=Pseudo-code tuned, Zero-shot=true, Greedy decoding=true2025.05 | 0.4 | — | — | 0.54 | — | — | — | — | — | — | 0.51 | — | — | 0.54 | |
| FiRSTPruning=dynamic, RM Blocks=16(50%)2026.06 | 0.3973 | 0.4215 | — | 0.5321 | 0.4112 | 0.2927 | — | — | 0.332 | — | 0.5328 | 0.5453 | 0.3291 | — | |
| ResidualD=12, n=42026.01 | 0.366 | 0.462 | 0.354 | 0.654 | 0.584 | 0.264 | 0.63 | 0.366 | 0.326 | 0.575 | 0.52 | 0.44 | — | — | |
| KromHCD=12, n=42026.01 | 0.364 | 0.477 | 0.384 | 0.65 | 0.578 | 0.276 | 0.66 | 0.3 | 0.312 | 0.586 | 0.548 | 0.584 | — | — | |
| mHCD=12, n=42026.01 | 0.362 | 0.445 | 0.376 | 0.642 | 0.586 | 0.274 | 0.64 | 0.324 | 0.336 | 0.604 | 0.532 | 0.546 | — | — | |
| Shortened LlamaPruning=static, RM Blocks=16(50%)2026.06 | 0.3546 | 0.4517 | — | 0.6551 | 0.4971 | 0.2662 | — | — | 0.308 | — | 0.513 | 0.6217 | 0.3976 | — | |
| mHC-liteD=12, n=42026.01 | 0.352 | 0.444 | 0.362 | 0.646 | 0.586 | 0.272 | 0.63 | 0.236 | 0.304 | 0.586 | 0.492 | 0.416 | — | — | |
| MoRA (r = 32)Model=Llama-2-7B, Rank (r)=32, #Params (%)=0.82412026.04 | 0.2909 | 0.7246 | — | 0.8079 | 0.8531 | 0.7142 | — | — | 0.812 | — | 0.8019 | 0.7217 | 0.7953 | — | |
| KromHCD=6, n=42026.01 | 0.28 | 0.424 | 0.278 | 0.59 | 0.414 | 0.23 | 0.54 | 0.288 | 0.254 | 0.535 | 0.5 | 0.608 | — | — | |
| mHC-liteD=6, n=42026.01 | 0.274 | 0.423 | 0.282 | 0.564 | 0.428 | 0.244 | 0.53 | 0.35 | 0.256 | 0.534 | 0.5 | 0.544 | — | — | |
| ResidualD=6, n=42026.01 | 0.27 | 0.411 | 0.29 | 0.604 | 0.428 | 0.218 | 0.55 | 0.318 | 0.238 | 0.593 | 0.492 | 0.374 | — | — | |
| mHCD=6, n=42026.01 | 0.264 | 0.422 | 0.284 | 0.608 | 0.428 | 0.224 | 0.6 | 0.23 | 0.248 | 0.553 | 0.504 | 0.544 | — | — | |
| Mistral 7B v0.3-CPBackbone=Mistral 7B v0.3, Strategy=Code-Prompted, Zero-shot=true, Greedy decoding=true2025.05 | 0.25 | — | — | 0.76 | — | — | — | — | — | — | 0.45 | — | — | 0.87 | |
| Llama 3.1 8B-CPBackbone=Llama 3.1 8B, Strategy=Code-Prompted, Zero-shot=true, Greedy decoding=true2025.05 | 0.24 | — | — | 0.76 | — | — | — | — | — | — | 0.44 | — | — | 0.9 | |
| Granite 8B Code-CPBackbone=Granite 8B Code, Strategy=Code-Prompted, Zero-shot=true, Greedy decoding=true2025.05 | 0.24 | — | — | 0.58 | — | — | — | — | — | — | 0.47 | — | — | 0.6 | |
| Granite 3.1 8B-CPBackbone=Granite 3.1 8B, Strategy=Code-Prompted, Zero-shot=true, Greedy decoding=true2025.05 | 0.23 | — | — | 0.73 | — | — | — | — | — | — | 0.43 | — | — | 0.9 | |
| Prompt TuningModel=Llama-3-8B, #Params (%)=0.00102026.04 | 0.1401 | 0.3696 | — | 0.4505 | 0.3274 | 0.3157 | — | — | 0.292 | — | 0.5012 | 0.5685 | 0.3613 | — | |
| Granite 3.1 8B-NLBackbone=Granite 3.1 8B, Strategy=NL-tuned, Zero-shot=true, Greedy decoding=true2025.05 | 0.14 | — | — | 0.4 | — | — | — | — | — | — | 0.38 | — | — | 0.16 | |
| Olmo 3 7B-CPBackbone=Olmo 3 7B, Strategy=Code-Prompted, Zero-shot=true, Greedy decoding=true2025.05 | 0.07 | — | — | 0.72 | — | — | — | — | — | — | 0.49 | — | — | 0.91 | |
| Prompt TuningModel=Llama-2-7B, #Params (%)=0.00122026.04 | 0.0691 | 0.2129 | — | 0.1235 | 0.0863 | 0.0606 | — | — | 0.094 | — | 0.4057 | 0.5593 | 0.305 | — | |
| P-TuningModel=Llama-3-8B, #Params (%)=0.62402026.04 | 0.0177 | 0.1811 | — | 0.1164 | 0.0863 | 0.0742 | — | — | 0.096 | — | 0.3765 | 0.5997 | 0.0819 | — | |
| P-TuningModel=Llama-2-7B, #Params (%)=0.74282026.04 | 0.0001 | 0.1224 | — | 0.3602 | 0.0198 | 0.0017 | — | — | 0.008 | — | 0 | 0.5875 | 0.002 | — |