Question Answering on Winogrande (WG)
74.1AccuracyFP16
Evaluation results
| Method | Links | |
|---|---|---|
| FP16Model=LLaMA3.1-8B-Instruct, Precision=MXFP42026.04 | 74.1 | |
| LLaMA-3 8BPruning Ratio=0%, Base Model=LLaMA-3 8B, Evaluation Protocol=Zero-shot2026.02 | 72.77 | |
| DuQuant++Model=LLaMA3-8B-Instruct, Precision=MXFP42026.04 | 72.1 | |
| DuQuant++*Model=LLaMA3.1-8B-Instruct, Precision=MXFP42026.04 | 71.7 | |
| DuQuant++*Model=LLaMA3-8B-Instruct, Precision=MXFP42026.04 | 71.6 | |
| FP16Model=LLaMA3-8B-Instruct, Precision=MXFP42026.04 | 71.4 | |
| FlatQuantModel=LLaMA3-8B-Instruct, Precision=MXFP42026.04 | 71.1 | |
| FlatQuantModel=LLaMA3.1-8B-Instruct, Precision=MXFP42026.04 | 71.1 | |
| CoMePruning Ratio=20%, Base Model=LLaMA-3 8B, Evaluation Protocol=Zero-shot2026.02 | 70.96 | |
| QuaRot*Model=LLaMA3.1-8B-Instruct, Precision=MXFP42026.04 | 70.7 | |
| MR-GPTQModel=LLaMA3.1-8B-Instruct, Precision=MXFP42026.04 | 70.6 | |
| DuQuant++Model=LLaMA3.1-8B-Instruct, Precision=MXFP42026.04 | 70.6 | |
| LINEARPATCHPruning Ratio=20%, Base Model=LLaMA-3 8B, Evaluation Protocol=Zero-shot2026.02 | 70.17 | |
| HPTQAvg. Bitwidth=4.125, Backbone=Llama-2-7B, Zero-shot=true2025.07 | 69.61 | |
| FP16 BaselineAvg. Bitwidth=16.000, Backbone=Llama-2-7B, Zero-shot=true2025.07 | 69.46 | |
| LLaMA-2 7BPruning Ratio=0%, Base Model=LLaMA-2 7B, Evaluation Protocol=Zero-shot2026.02 | 69.14 | |
| MR-GPTQModel=LLaMA3-8B-Instruct, Precision=MXFP42026.04 | 69 | |
| SSQR-1%Avg. Bitwidth=4.445, Backbone=Llama-2-7B, Zero-shot=true2025.07 | 68.82 | |
| GPTQAvg. Bitwidth=4.125, Backbone=Llama-2-7B, Zero-shot=true2025.07 | 68.82 | |
| QuaRot*Model=LLaMA3-8B-Instruct, Precision=MXFP42026.04 | 68.8 | |
| QuaRotModel=LLaMA3.1-8B-Instruct, Precision=MXFP42026.04 | 68.4 | |
| HPTQAvg. Bitwidth=3.125, Backbone=Llama-2-7B, Zero-shot=true2025.07 | 67.72 | |
| QuIP#Avg. Bitwidth=4.000, Backbone=Llama-2-7B, Zero-shot=true2025.07 | 67.6 | |
| LINEARPATCHPruning Ratio=20%, Base Model=LLaMA-2 7B, Evaluation Protocol=Zero-shot2026.02 | 67.4 | |
| AQLMAvg. Bitwidth=5.020, Backbone=Llama-2-7B, Zero-shot=true2025.07 | 67.4 | |
| AQLMAvg. Bitwidth=4.040, Backbone=Llama-2-7B, Zero-shot=true2025.07 | 67.32 | |
| QuaRotModel=LLaMA3-8B-Instruct, Precision=MXFP42026.04 | 67.3 | |
| CoMePruning Ratio=20%, Base Model=LLaMA-2 7B, Evaluation Protocol=Zero-shot2026.02 | 67.25 | |
| QTIPAvg. Bitwidth=4.000, Backbone=Llama-2-7B, Zero-shot=true2025.07 | 67.1 | |
| AQLMAvg. Bitwidth=3.040, Backbone=Llama-2-7B, Zero-shot=true2025.07 | 66.93 | |
| AmoebaLLMPruning Ratio=20%, Base Model=LLaMA-2 7B, Evaluation Protocol=Zero-shot2026.02 | 66.9 | |
| QTIPAvg. Bitwidth=3.000, Backbone=Llama-2-7B, Zero-shot=true2025.07 | 66.9 | |
| MoPPruning Ratio=20%, Base Model=LLaMA-3 8B, Evaluation Protocol=Zero-shot2026.02 | 66.8 | |
| LINEARPATCHPruning Ratio=30%, Base Model=LLaMA-2 7B, Evaluation Protocol=Zero-shot2026.02 | 66.69 | |
| QuIP#Avg. Bitwidth=3.000, Backbone=Llama-2-7B, Zero-shot=true2025.07 | 66.5 | |
| PruneNetPruning Ratio=20%, Base Model=LLaMA-2 7B, Evaluation Protocol=Zero-shot2026.02 | 66.22 | |
| HPTQAvg. Bitwidth=2.125, Backbone=Llama-2-7B, Zero-shot=true2025.07 | 65.82 | |
| AQLMAvg. Bitwidth=2.290, Backbone=Llama-2-7B, Zero-shot=true2025.07 | 65.67 | |
| AQLMAvg. Bitwidth=2.020, Backbone=Llama-2-7B, Zero-shot=true2025.07 | 65.67 | |
| SSQR-1%Avg. Bitwidth=3.445, Backbone=Llama-2-7B, Zero-shot=true2025.07 | 65.43 | |
| MoPPruning Ratio=20%, Base Model=LLaMA-2 7B, Evaluation Protocol=Zero-shot2026.02 | 65.19 | |
| Pretrained ModelModel Size=7B, Training Strategy=Standard Pretraining2026.03 | 65.19 | |
| SliceGPTPruning Ratio=20%, Base Model=LLaMA-2 7B, Evaluation Protocol=Zero-shot2026.02 | 65.11 | |
| GPTQAvg. Bitwidth=3.125, Backbone=Llama-2-7B, Zero-shot=true2025.07 | 64.96 | |
| QuIP#Avg. Bitwidth=2.000, Backbone=Llama-2-7B, Zero-shot=true2025.07 | 64.9 | |
| SlimLLMPruning Ratio=20%, Base Model=LLaMA-2 7B, Evaluation Protocol=Zero-shot2026.02 | 64.88 | |
| QTIPAvg. Bitwidth=2.000, Backbone=Llama-2-7B, Zero-shot=true2025.07 | 64.7 | |
| ModHiFi-PPruning Ratio=20%, Base Model=LLaMA-2 7B, Evaluation Protocol=Zero-shot2026.02 | 64.64 | |
| DISP-LLMPruning Ratio=30%, Base Model=LLaMA-2 7B, Evaluation Protocol=Zero-shot2026.02 | 63.93 | |
| Pretrained ModelModel Size=2.7B, Training Strategy=Standard Pretraining2026.03 | 63.85 | |
| MoPPruning Ratio=30%, Base Model=LLaMA-3 8B, Evaluation Protocol=Zero-shot2026.02 | 63.75 | |
| CoMePruning Ratio=30%, Base Model=LLaMA-2 7B, Evaluation Protocol=Zero-shot2026.02 | 63.38 | |
| Split Model TrainingModel Size=2.7B, Training Strategy=Split Training2026.03 | 63.22 | |
| PruneNetPruning Ratio=30%, Base Model=LLaMA-2 7B, Evaluation Protocol=Zero-shot2026.02 | 62.9 | |
| MoPPruning Ratio=30%, Base Model=LLaMA-2 7B, Evaluation Protocol=Zero-shot2026.02 | 62.38 | |
| Split Model TrainingModel Size=1.3B, Training Strategy=Split Training2026.03 | 62.19 | |
| ProDiaLModel Size=Mamba-1.4B, Parameters=0.05B, Projection Type=Both-Proj2024.11 | 61.96 | |
| Pretrained ModelModel Size=1.3B, Training Strategy=Standard Pretraining2026.03 | 61.88 | |
| StrongModel Size=Mamba-1.4B, Parameters=0.06B, Projection Type=Base2024.11 | 61.8 | |
| ProDiaLModel Size=Mamba-1.4B, Parameters=0.02B, Projection Type=Out-Proj2024.11 | 61.8 | |
| FTModel Size=Mamba-1.4B, Parameters=1.21B, Projection Type=Both-Proj2024.11 | 61.72 | |
| LoRAModel Size=Mamba-1.4B, Parameters=0.05B, Projection Type=Both-Proj2024.11 | 61.72 | |
| DoRAModel Size=Mamba-1.4B, Parameters=0.05B, Projection Type=Both-Proj2024.11 | 61.72 | |
| FTModel Size=Mamba-1.4B, Parameters=0.40B, Projection Type=Out-Proj2024.11 | 61.72 | |
| FTModel Size=Mamba-1.4B, Parameters=0.81B, Projection Type=In-Proj2024.11 | 61.56 | |
| LoRAModel Size=Mamba-1.4B, Parameters=0.03B, Projection Type=In-Proj2024.11 | 61.56 | |
| ProDiaLModel Size=Mamba-1.4B, Parameters=0.03B, Projection Type=In-Proj2024.11 | 61.56 | |
| LoRAModel Size=Mamba-1.4B, Parameters=0.02B, Projection Type=Out-Proj2024.11 | 61.56 | |
| AMPPruning Ratio=20%, Base Model=LLaMA-2 7B, Evaluation Protocol=Zero-shot2026.02 | 61.56 | |
| DoRAModel Size=Mamba-1.4B, Parameters=0.03B, Projection Type=In-Proj2024.11 | 61.48 | |
| DoRAModel Size=Mamba-1.4B, Parameters=0.02B, Projection Type=Out-Proj2024.11 | 61.48 | |
| MoPPruning Ratio=40%, Base Model=LLaMA-3 8B, Evaluation Protocol=Zero-shot2026.02 | 61.38 | |
| SliceGPTPruning Ratio=30%, Base Model=LLaMA-2 7B, Evaluation Protocol=Zero-shot2026.02 | 61.33 | |
| BitFitModel Size=Mamba-1.4B, Parameters=0.39M, Projection Type=Base2024.11 | 61.25 | |
| AMPPruning Ratio=30%, Base Model=LLaMA-2 7B, Evaluation Protocol=Zero-shot2026.02 | 61.25 | |
| Full-FTModel Size=Mamba-1.4B, Parameters=1.40B, Projection Type=Base2024.11 | 61.17 | |
| Shortened LLaMAPruning Ratio=30%, Base Model=LLaMA-2 7B, Evaluation Protocol=Zero-shot2026.02 | 61.09 | |
| Yang et al.Pruning Ratio=30%, Base Model=LLaMA-2 7B, Evaluation Protocol=Zero-shot2026.02 | 60.77 | |
| AMPPruning Ratio=30%, Base Model=LLaMA-3 8B, Evaluation Protocol=Zero-shot2026.02 | 60.77 | |
| CoMePruning Ratio=30%, Base Model=LLaMA-3 8B, Evaluation Protocol=Zero-shot2026.02 | 60.62 | |
| Yang et al.Pruning Ratio=30%, Base Model=LLaMA-3 8B, Evaluation Protocol=Zero-shot2026.02 | 60.3 | |
| AMPPruning Ratio=20%, Base Model=LLaMA-3 8B, Evaluation Protocol=Zero-shot2026.02 | 59.91 | |
| ModHiFi-PPruning Ratio=30%, Base Model=LLaMA-2 7B, Evaluation Protocol=Zero-shot2026.02 | 59.35 | |
| MoPPruning Ratio=40%, Base Model=LLaMA-2 7B, Evaluation Protocol=Zero-shot2026.02 | 59.35 | |
| ProDiaLModel Size=Mamba-370M, Parameters=2.02M, Projection Type=Out-Proj2024.11 | 57.3 | |
| FTModel Size=Mamba-370M, Parameters=302M, Projection Type=Both-Proj2024.11 | 57.22 | |
| DoRAModel Size=Mamba-370M, Parameters=4.13M, Projection Type=In-Proj2024.11 | 57.22 | |
| ProDiaLModel Size=Mamba-370M, Parameters=3.74M, Projection Type=In-Proj2024.11 | 57.22 | |
| FTModel Size=Mamba-370M, Parameters=101M, Projection Type=Out-Proj2024.11 | 57.22 | |
| DoRAModel Size=Mamba-370M, Parameters=2.41M, Projection Type=Out-Proj2024.11 | 57.22 | |
| StrongModel Size=Mamba-370M, Parameters=8.19M, Projection Type=Base2024.11 | 57.14 | |
| ProDiaLModel Size=Mamba-370M, Parameters=5.75M, Projection Type=Both-Proj2024.11 | 57.06 | |
| FTModel Size=Mamba-370M, Parameters=201M, Projection Type=In-Proj2024.11 | 57.06 | |
| Full-FTModel Size=Mamba-370M, Parameters=370M, Projection Type=Base2024.11 | 56.99 | |
| BitFitModel Size=Mamba-370M, Parameters=0.20M, Projection Type=Base2024.11 | 56.99 | |
| DoRAModel Size=Mamba-370M, Parameters=6.54M, Projection Type=Both-Proj2024.11 | 56.99 | |
| LoRAModel Size=Mamba-370M, Parameters=2.36M, Projection Type=Out-Proj2024.11 | 56.91 | |
| AMPPruning Ratio=40%, Base Model=LLaMA-3 8B, Evaluation Protocol=Zero-shot2026.02 | 56.91 | |
| LoRAModel Size=Mamba-370M, Parameters=6.29M, Projection Type=Both-Proj2024.11 | 56.75 | |
| LoRAModel Size=Mamba-370M, Parameters=3.93M, Projection Type=In-Proj2024.11 | 56.75 |