Reasoning on WinoGrande
85.2AccuracyInternLM2-20B
Evaluation results
| Method | Links | ||
|---|---|---|---|
| InternLM2-20BEvaluation Protocol=0-shot, Model Category=Base Model, Model Scale=~20B2024.03 | 85.2 | — | |
| InternLM2-7BEvaluation Protocol=0-shot, Model Category=Base Model, Model Scale=~7B2024.03 | 84.7 | — | |
| Qwen3.5-9BShots=52026.04 | 82.64 | — | |
| Llama-3.3-70B-InstructShots=52026.04 | 81.06 | — | |
| Mixtral-8x7B-v0.1Evaluation Protocol=0-shot, Model Category=Base Model, Model Scale=~20B2024.03 | 77.3 | — | |
| InternLM2-20B-BaseEvaluation Protocol=0-shot, Model Category=Base Model, Model Scale=~20B2024.03 | 76.2 | — | |
| Qwen3-14BShots=52026.04 | 76.16 | — | |
| Mistral-7B-v0.1Evaluation Protocol=0-shot, Model Category=Base Model, Model Scale=~7B2024.03 | 75.3 | — | |
| InternLM2-Chat-20B-SFTEvaluation Protocol=0-shot, Model Category=Chat Model, Model Scale=~20B, Fine-tuning Strategy=SFT2024.03 | 75 | — | |
| Phi-mini (teacher)Setting=Teachers (reference), Few-shot protocol=3-shot2026.05 | 74.9 | 63.72 | |
| InternLM2-Chat-20BEvaluation Protocol=0-shot, Model Category=Chat Model, Model Scale=~20B2024.03 | 74.8 | — | |
| InternLM2-7B-BaseEvaluation Protocol=0-shot, Model Category=Base Model, Model Scale=~7B2024.03 | 74.6 | — | |
| AWQBackbone=Llama-3-8B, #Bit=4.13, Zero-shot=true2025.03 | 74 | — | |
| Hybrid-PrefixModel=Vicuna-7b-v1.52026.04 | 74 | — | |
| XekRung-8BShots=52026.04 | 73.24 | — | |
| LLAMA-3-8BModel=LLAMA-3-8B, Bits=FP16, Zero-shot=true2026.04 | 73.16 | — | |
| ClusCompBackbone=Llama-3-8B, #Bit=4.13, Zero-shot=true2025.03 | 72.9 | — | |
| Llama-3-8BBackbone=Llama-3-8B, #Bit=16.00, Zero-shot=true2025.03 | 72.8 | — | |
| SliM-LLMBackbone=Llama-3-8B, #Bit=3.13, Zero-shot=true2025.03 | 72.8 | — | |
| GPTQBackbone=Llama-3-8B, #Bit=4.13, Zero-shot=true2025.03 | 72.6 | — | |
| SliM-LLMBackbone=Llama-3-8B, #Bit=4.13, Zero-shot=true2025.03 | 72.6 | — | |
| QuIPBackbone=Llama-3-8B, #Bit=3.00, Zero-shot=true2025.03 | 72.5 | — | |
| SecGPT-14BShots=52026.04 | 72.45 | — | |
| Llama2-13BEvaluation Protocol=0-shot, Model Category=Base Model, Model Scale=~20B2024.03 | 72.3 | — | |
| Qwen-4B (teacher)Setting=Teachers (reference), Few-shot protocol=3-shot2026.05 | 72.3 | 66.42 | |
| AWQBackbone=Llama-3-8B, #Bit=3.13, Zero-shot=true2025.03 | 72.1 | — | |
| GPTQBackbone=Llama-3-8B, #Bit=3.13, Zero-shot=true2025.03 | 71.1 | — | |
| RTNBackbone=Llama-3-8B, #Bit=4.13, Zero-shot=true2025.03 | 71 | — | |
| ClusCompBackbone=Llama-3-8B, #Bit=2.87, Zero-shot=true2025.03 | 71 | — | |
| Llama-3B (teacher)Setting=Teachers (reference), Few-shot protocol=3-shot2026.05 | 70.01 | 46.93 | |
| Baichuan2-13B-BaseEvaluation Protocol=0-shot, Model Category=Base Model, Model Scale=~20B2024.03 | 70 | — | |
| OriginalCompression Ratio=0%, Memory (GB)=26.95 GB2026.02 | 70 | — | |
| GPTQCompression Ratio=73%, Memory (GB)=7.16 GB2026.02 | 70 | — | |
| SVD-LLM V2Compression Ratio=0.22026.05 | 70 | — | |
| BaselineModel=Qwen3-MoE, Weight Bits=16, Zero-shot=true2026.04 | 69.53 | — | |
| Llama2-7BEvaluation Protocol=0-shot, Model Category=Base Model, Model Scale=~7B2024.03 | 69.5 | — | |
| BaseBackbone=LLaMA-2-7B, Compression Ratio=0%2025.10 | 69.06 | — | |
| Hybrid-PromptModel=Llama-2-7b2026.04 | 68.9 | — | |
| BaselineModel=OLMoE, Weight Bits=16, Zero-shot=true2026.04 | 68.82 | — | |
| Qwen3-8BShots=52026.04 | 68.75 | — | |
| Baichuan2-7B-BaseEvaluation Protocol=0-shot, Model Category=Base Model, Model Scale=~7B2024.03 | 68.7 | — | |
| GPT-3.5Evaluation Protocol=0-shot, Model Category=Chat Model, Model Scale=API Models2024.03 | 68.7 | — | |
| Qwen-7BEvaluation Protocol=0-shot, Model Category=Base Model, Model Scale=~7B2024.03 | 68.6 | — | |
| FO-LoRAModel=Llama-2-7b2026.04 | 68.5 | — | |
| SPQCompression Ratio=75%, Memory (GB)=6.86 GB2026.02 | 68 | — | |
| BaselineModel=DeepSeekV2-Lite, Weight Bits=16, Zero-shot=true2026.04 | 67.64 | — | |
| Qwen-14BEvaluation Protocol=0-shot, Model Category=Base Model, Model Scale=~20B2024.03 | 67.5 | — | |
| FO-PromptModel=Llama-2-7b2026.04 | 67.2 | — | |
| LLAMA-2-7BModel=LLAMA-2-7B, Bits=FP16, Zero-shot=true2026.04 | 67.17 | — | |
| ClusCompBackbone=Llama-3-8B, #Bit=2.27, Zero-shot=true2025.03 | 67.1 | — | |
| LLAMA-1-7BModel=LLAMA-1-7B, Bits=FP16, Zero-shot=true2026.04 | 67.01 | — | |
| BaselineCompression Ratio=0.02026.05 | 67 | — | |
| SAES-SVD + PARSECompression Ratio=0.22026.05 | 67 | — | |
| FO-LoRAModel=Vicuna-7b-v1.52026.04 | 66.7 | — | |
| Hybrid-LoRAModel=Vicuna-7b-v1.52026.04 | 66.7 | — | |
| Hybrid-LoRAModel=Llama-2-7b2026.04 | 66.3 | — | |
| Hybrid-PromptModel=Vicuna-7b-v1.52026.04 | 66.3 | — | |
| FO-PrefixModel=Llama-2-7b2026.04 | 66.2 | — | |
| ASVDCompression Ratio=21%, Memory (GB)=21.41 GB2026.02 | 66 | — | |
| SAES-SVD + PARSECompression Ratio=0.42026.05 | 66 | — | |
| Phi-3Zero-shot=true, Base Model=Phi-32025.05 | 65.98 | — | |
| ArrowZero-shot=true, Base Model=Phi-32025.05 | 65.98 | — | |
| MoBiEModel=Qwen3-MoE, Weight Bits=1.34, Zero-shot=true2026.04 | 65.9 | — | |
| InternLM2-Chat-7B-SFTEvaluation Protocol=0-shot, Model Category=Chat Model, Model Scale=~7B, Fine-tuning Strategy=SFT2024.03 | 65.8 | — | |
| InternLM2-Chat-7BEvaluation Protocol=0-shot, Model Category=Chat Model, Model Scale=~7B2024.03 | 65.8 | — | |
| FO-PromptModel=Vicuna-7b-v1.52026.04 | 65.8 | — | |
| GPTQModel=Qwen3-MoE, Weight Bits=3, Zero-shot=true2026.04 | 65.48 | — | |
| SparseGPTCompression Ratio=50%, Memory (GB)=13.40 GB2026.02 | 65 | — | |
| NoWagModel=Qwen3-MoE, Weight Bits=2.11, Zero-shot=true2026.04 | 64.96 | — | |
| GenKnowSubSetting=En, Zero-shot=true, Base Model=Phi-32025.05 | 64.72 | — | |
| AWQBackbone=Llama-3-8B, #Bit=3.00, Zero-shot=true2025.03 | 64.7 | — | |
| GenKnowSubSetting=Avg, Zero-shot=true, Base Model=Phi-32025.05 | 64.64 | — | |
| PGSVDBackbone=LLaMA-2-7B, Compression Ratio=20%2025.10 | 64.56 | — | |
| GenKnowSubSetting=Fr, Zero-shot=true, Base Model=Phi-32025.05 | 64.4 | — | |
| Hybrid-PrefixModel=Llama-2-7b2026.04 | 64.3 | — | |
| FO-PrefixModel=Vicuna-7b-v1.52026.04 | 64.1 | — | |
| X-TokenSetting=Multi-teacher, Teachers=Phi-mini + Qwen-4B + Llama-3B, Few-shot protocol=3-shot2026.05 | 63.61 | 40.15 | |
| X-Token (P-KL)Setting=Cross tokenizer (single teacher), Teacher=Qwen-4B, Few-shot protocol=3-shot2026.05 | 63.46 | 38.85 | |
| LLM-PrunerBackbone=LLaMA-2-7B, Compression Ratio=20%2025.10 | 63.38 | — | |
| GenKnowSubSetting=De, Zero-shot=true, Base Model=Phi-32025.05 | 63.3 | — | |
| X-TokenSetting=Multi-teacher, Teachers=Phi-mini + Llama-3B, Few-shot protocol=3-shot2026.05 | 63.3 | 40.48 | |
| SAES-SVD + PARSECompression Ratio=0.62026.05 | 63 | — | |
| GOLDSetting=Cross tokenizer (single teacher), Teacher=Qwen-4B, Few-shot protocol=3-shot2026.05 | 62.95 | 35.03 | |
| X-TokenSetting=Multi-teacher, Teachers=Phi-mini + Qwen-4B, Few-shot protocol=3-shot2026.05 | 62.74 | 38.49 | |
| Llama-3B → 1BSetting=Same tokenizer, Teacher=Llama-3B, Few-shot protocol=3-shot2026.05 | 62.7 | 38.4 | |
| GOLDSetting=Cross tokenizer (single teacher), Teacher=Phi-mini, Few-shot protocol=3-shot2026.05 | 62.6 | 38.66 | |
| ULDSetting=Cross tokenizer (single teacher), Teacher=Phi-mini, Few-shot protocol=3-shot2026.05 | 62.59 | 38.31 | |
| ClusCompBackbone=Llama-3-8B, #Bit=2.14, Zero-shot=true2025.03 | 62.4 | — | |
| ULDSetting=Cross tokenizer (single teacher), Teacher=Qwen-4B, Few-shot protocol=3-shot2026.05 | 61.96 | 36.77 | |
| X-Token (H-KL)Setting=Cross tokenizer (single teacher), Teacher=Phi-mini, Few-shot protocol=3-shot2026.05 | 61.87 | 39.18 | |
| LBLLMModel=LLAMA-1-7B, Bits=W(1+1)A4, Zero-shot=true2026.04 | 61.72 | — | |
| ChatGLM3-6BEvaluation Protocol=0-shot, Model Category=Chat Model, Model Scale=~7B2024.03 | 61.7 | — | |
| Continued pre-trainingSetting=No distillation, Few-shot protocol=3-shot2026.05 | 61.6 | 36.63 | |
| Llama-1B (base)Setting=No distillation, Few-shot protocol=3-shot2026.05 | 61.48 | 33.96 | |
| Mixtral-8x7B-Instruct-v0.1Evaluation Protocol=0-shot, Model Category=Chat Model, Model Scale=~20B2024.03 | 60.9 | — | |
| GPTQBackbone=Llama-3-8B, #Bit=3.00, Zero-shot=true2025.03 | 60.9 | — | |
| SmolLM 2Params.=1.7B, Tokens=11T2025.06 | 60.1 | — | |
| Hybrid-PrefixModel=OPT-1.3b2026.04 | 60 | — | |
| Hybrid-PromptModel=OPT-1.3b2026.04 | 59.9 | — | |
| SliceGPTBackbone=LLaMA-2-7B, Compression Ratio=20%2025.10 | 59.83 | — |