Commonsense Reasoning on WinoGrande
7,364AccuracyGPTQ
Evaluation results
| Method | Links | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| GPTQFormat=MXFP2025.09 | 7,364 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Human Performance2024.01 | 94.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| VERA-T5 (Multitask Supervised)Evaluation Protocol=Supervised2024.01 | 92.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| PaLM 2Number of exemplars (k-shot)=5, Model variant=Instruction-tuned2023.05 | 90.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BF16Backbone=DeepSeek-V3.1 671B2026.02 | 89.34 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| HiF4Backbone=DeepSeek-V3.1 671B2026.02 | 89.11 | — | — | — | -0.23 | — | — | — | — | — | — | — | — | — | — | — | |
| BaLoRAModel=Llama-3-8B, Params (%)=0.71452026.04 | 88.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| NVFP4+PTSBackbone=DeepSeek-V3.1 671B2026.02 | 88 | — | — | — | -1.34 | — | — | — | — | — | — | — | — | — | — | — | |
| NVFP4Backbone=DeepSeek-V3.1 671B2026.02 | 87.92 | — | — | — | -1.42 | — | — | — | — | — | — | — | — | — | — | — | |
| Dense ModelBase Model=Mistral-7B, Fine-tuning Protocol=FFT, Sparsity=0%2026.01 | 87.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| HiRAModel=Llama-3-8B, Params (%)=0.70022026.04 | 87.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GPT-4Number of exemplars (k-shot)=52023.05 | 87.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| AGRPOBase Model=LLaDA 8B, Training Algorithm=AGRPO2026.03 | 87.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LFPO (All Loss)Base Model=LLaDA 8B, Training Algorithm=LFPO (All Loss)2026.03 | 86.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Dense ModelBase Model=Llama3.1-8B, Fine-tuning Protocol=FFT, Sparsity=0%2026.01 | 86.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MoRAModel=Llama-3-8B, Params (%)=0.69972026.04 | 86.74 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Ours (1-vec)Model=Llama-3.1-8B, Params (%)=0.001%2026.02 | 86.6 | — | — | — | — | — | — | 0.2 | — | — | — | — | — | — | — | — | |
| GradPrunerBase Model=Llama3.1-8B, Fine-tuning Protocol=FFT, Sparsity=40%2026.01 | 86.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GradPrunerBase Model=Mistral-7B, Fine-tuning Protocol=FFT, Sparsity=40%2026.01 | 86 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DeepSeek V3.1Model Variant=Base, # Shots=5-shot, # Activated Params=37B, # Total Params=671B2026.02 | 85.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DeepSeek-V3.1 Base# Shots=5-shot, # Activated Params=37B, # Total Params=671B2026.01 | 85.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Ours r=1Model=Llama-3.1-8B, Params (%)=0.003%2026.02 | 85.9 | — | — | — | — | — | — | -0.5 | — | — | — | — | — | — | — | — | |
| DeepSeek V3.2Model Variant=Exp Base, # Shots=5-shot, # Activated Params=37B, # Total Params=671B2026.02 | 85.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DeepSeek-V3.2 Exp Base# Shots=5-shot, # Activated Params=37B, # Total Params=671B2026.01 | 85.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| coupled-GRPOBase Model=LLaDA 8B, Training Algorithm=coupled-GRPO2026.03 | 85.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DoRAModel=Llama-3-8B, Params (%)=0.70022026.04 | 85.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Kimi-K2Model Variant=Base, # Shots=5-shot, # Activated Params=32B, # Total Params=1043B2026.02 | 85.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Kimi-K2 Base# Shots=5-shot, # Activated Params=32B, # Total Params=1043B2026.01 | 85.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GLM-4.5 Baseshots=5-shot2026.06 | 85.24 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| PaLMNumber of exemplars (k-shot)=52023.05 | 85.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SE-GPTModel=GPT-42024.07 | 84.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| UniGRPOBase Model=LLaDA 8B, Training Algorithm=UniGRPO2026.03 | 84.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Dense ModelBase Model=Mistral-7B, Fine-tuning Protocol=LoRA, Sparsity=0%2026.01 | 84.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SPGBase Model=LLaDA 8B, Training Algorithm=SPG2026.03 | 84.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LoRAModel=Llama-3-8B, Params (%)=0.70022026.04 | 84.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Kimi-K2 Baseshots=5-shot2026.06 | 84.21 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DeBERTa-v3-L (Supervised)Evaluation Protocol=Supervised2024.01 | 84.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| HiRAModel=Llama-2-7B, Params (%)=0.82562026.04 | 83.98 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MiMo-V2 FlashModel Variant=Base, # Shots=5-shot, # Activated Params=15B, # Total Params=309B2026.02 | 83.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MiMo-V2-Flash Base# Shots=5-shot, # Activated Params=15B, # Total Params=309B2026.01 | 83.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| PaLMprompting=1-shot2023.05 | 83.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LFPO (Neg. Only)Base Model=LLaDA 8B, Training Algorithm=LFPO (Neg. Only)2026.03 | 83.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BaLoRAModel=Llama-2-7B, Params (%)=0.84272026.04 | 83.66 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| FT(Llama3.2)Base Model=Llama3.2-3B, Fine-tuning Protocol=FFT, Sparsity=0%2026.01 | 83.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SATBase Model=Mistral-7B, Fine-tuning Protocol=FFT, Sparsity=40%2026.01 | 83.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SATBase Model=Llama3.1-8B, Fine-tuning Protocol=FFT, Sparsity=40%2026.01 | 83.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DeepSeek-V3.2 Exp-Baseshots=5-shot2026.06 | 83.43 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Zero-shot-CoTModel=GPT-42024.07 | 83.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LLaMA 2 70B5-shot=true2024.01 | 83.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GradPrunerBase Model=Mistral-7B, Fine-tuning Protocol=LoRA, Sparsity=40%2026.01 | 83.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| diffu-GRPOBase Model=LLaDA 8B, Training Algorithm=diffu-GRPO2026.03 | 83.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LFPO (Pos. Only)Base Model=LLaDA 8B, Training Algorithm=LFPO (Pos. Only)2026.03 | 83.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| PaLM 2-Lprompting=1-shot2023.05 | 83 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MINITRONBase Model=Mistral-7B, Fine-tuning Protocol=FFT, Sparsity=40%2026.01 | 83 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Self-EXPModel=GPT-42024.07 | 82.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LacoBase Model=Mistral-7B, Fine-tuning Protocol=FFT, Sparsity=40%2026.01 | 82.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Self-ICL-CoTModel=GPT-42024.07 | 82.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LoRAModel=Llama-2-7B, Params (%)=0.82562026.04 | 82.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DoRAModel=Llama-2-7B, Params (%)=0.82562026.04 | 82.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Mashup LearningBackbone=Mistral-7B-Instruct-v0.2, Setup=Lots-of-LoRAs2026.03 | 82.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Dense ModelBase Model=Llama3.1-8B, Fine-tuning Protocol=LoRA, Sparsity=0%2026.01 | 82.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Mistral-Large-3 675B-Base-2512shots=5-shot2026.06 | 82.08 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LacoBase Model=Llama3.1-8B, Fine-tuning Protocol=FFT, Sparsity=40%2026.01 | 81.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Mistral Nemo Inst 2407Classifier=Self-labeled2026.01 | 81.7 | — | — | — | — | 7.2 | — | — | — | — | — | — | — | — | — | — | |
| Llama 3.1-70BPrecision=BF162026.02 | 81.61 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GPT-3.55-shot=true2024.01 | 81.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LLMPrunerBase Model=Mistral-7B, Fine-tuning Protocol=FFT, Sparsity=40%2026.01 | 81.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SATBase Model=Mistral-7B, Fine-tuning Protocol=LoRA, Sparsity=40%2026.01 | 81.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| APTBase Model=Mistral-7B, Fine-tuning Protocol=LoRA, Sparsity=40%2026.01 | 81.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MINITRONBase Model=Llama3.1-8B, Fine-tuning Protocol=FFT, Sparsity=40%2026.01 | 81.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| From scratchBackbone=Mistral-7B-Instruct-v0.2, Setup=Lots-of-LoRAs2026.03 | 81.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Mixtral 8x7B5-shot=true2024.01 | 81.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Zero-shotModel=GPT-42024.07 | 81.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GradPrunerBase Model=Llama3.1-8B, Fine-tuning Protocol=LoRA, Sparsity=40%2026.01 | 81.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LacoBase Model=Mistral-7B, Fine-tuning Protocol=LoRA, Sparsity=40%2026.01 | 81.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Qwen3 0.6BClassifier=Self-labeled2026.01 | 81.2 | — | — | — | — | 28.3 | — | — | — | — | — | — | — | — | — | — | |
| Qwen3 0.6BClassifier=Self-labeled2026.01 | 81.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| PaLM-540BModel Category=LLM, Evaluation Mode=Zero-shot2023.09 | 81.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LLMPrunerBase Model=Llama3.1-8B, Fine-tuning Protocol=FFT, Sparsity=40%2026.01 | 81 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Llama-3-70B#Bit=16.00, Base Model=Llama-3-70B, Evaluation Protocol=Zero-shot2025.03 | 80.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| FT(Llama3.2)Base Model=Llama3.2-3B, Fine-tuning Protocol=LoRA, Sparsity=0%2026.01 | 80.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MINITRONBase Model=Mistral-7B, Fine-tuning Protocol=LoRA, Sparsity=40%2026.01 | 80.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LO-BCQ (g128)Backbone=Llama2-70B, Bitwidth=4.56, #codebooks=162025.02 | 80.43 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LLaMA 2 70BActive Params=70B2024.01 | 80.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BF16Backbone=LongCat 560B2026.02 | 80.27 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Ours r=1Model=Qwen 3 4B, Params (%)=0.005%2026.02 | 80.2 | — | — | — | — | — | — | -2.8 | — | — | — | — | — | — | — | — | |
| MoRAModel=Llama-2-7B, Params (%)=0.82412026.04 | 80.19 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LLMPrunerBase Model=Mistral-7B, Fine-tuning Protocol=LoRA, Sparsity=40%2026.01 | 80.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BF16Models=Mixtral-8x7B, Quantization Strategy=Embedding-wise, Quantization Bit-width=BF162026.04 | 79.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LO-BCQ (g128)Backbone=Llama2-70B, Bitwidth=4.44, #codebooks=82025.02 | 79.79 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| F-L-SModel Zoo=Mistral2024.10 | 79.72 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Mistral Small 24B Inst 2501Classifier=Self-labeled2026.01 | 79.7 | — | — | — | — | 3.5 | — | — | — | — | — | — | — | — | — | — | |
| HiF4Backbone=LongCat 560B2026.02 | 79.56 | — | — | — | -0.71 | — | — | — | — | — | — | — | — | — | — | — | |
| Self-ICLModel=GPT-42024.07 | 79.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| NVFP4+PTSBackbone=LongCat 560B2026.02 | 79.4 | — | — | — | -0.87 | — | — | — | — | — | — | — | — | — | — | — | |
| Attn-QATModel=Llama 3.1-70B, Precision=Attn-QAT2026.02 | 79.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Nemotron-3-Ultra 550B-A55B-Baseshots=5-shot2026.06 | 79.32 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ROBERTa-L (Supervised)Evaluation Protocol=Supervised2024.01 | 79.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| NVFP4Backbone=LongCat 560B2026.02 | 79.24 | — | — | — | -1.03 | — | — | — | — | — | — | — | — | — | — | — | |
| LO-BCQ (g128)Backbone=Llama2-70B, Bitwidth=4.19, #codebooks=22025.02 | 79.24 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — |