Pronoun Resolution on WinoGrande
89.4AccuracyUnifiedQA-T5-11B
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| UnifiedQA-T5-11BSupervision Type=Unsupervised, Zero-shot Transfer=True2021.05 | 89.4 | 0.8571 | — | |
| TTTTTSupervision Type=Unsupervised, Zero-shot Transfer=True2021.05 | 84.6 | 0.7673 | — | |
| CorefBERT LARGESupervision Type=Supervised, Zero-shot Transfer=True2021.05 | 80.8 | — | — | |
| RoBERTa-largeSupervision Type=Unsupervised, Zero-shot Transfer=True2021.05 | 79.1 | 0.6641 | — | |
| CorefRoBERTa LARGESupervision Type=Supervised, Zero-shot Transfer=True2021.05 | 77.9 | — | — | |
| QERAZero-shot=true, Model=Vicuna-13B2026.03 | 75.33 | — | — | |
| HiF4+HiGPTQModel=Qwen2.5-14B, A-W Quant Type=HiF4+HiGPTQ2026.02 | 75.06 | — | 0.32 | |
| ZeroQuant-V2Zero-shot=true, Model=Vicuna-13B2026.03 | 75 | — | — | |
| BF16Model=Qwen2.5-14B, A-W Quant Type=BF162026.02 | 74.74 | — | — | |
| GlowQZero-shot=true, Model=Vicuna-13B2026.03 | 74.67 | — | — | |
| NVFP4Model=Qwen2.5-14B, A-W Quant Type=NVFP42026.02 | 74.43 | — | -0.31 | |
| FP16Rank=-, Zero-shot=true, Model=Vicuna-13B2026.03 | 74.33 | — | — | |
| GlowQ-SZero-shot=true, Model=Vicuna-13B2026.03 | 74.33 | — | — | |
| BF16Model=Mistral-7B, A-W Quant Type=BF162026.02 | 74.11 | — | — | |
| L2QERRank=64, Zero-shot=true, Model=Vicuna-13B2026.03 | 74 | — | — | |
| HiF4Model=Qwen2.5-14B, A-W Quant Type=HiF42026.02 | 73.72 | — | -1.02 | |
| HiF4+HiGPTQModel=Mistral-7B, A-W Quant Type=HiF4+HiGPTQ2026.02 | 73.4 | — | -0.71 | |
| UnifiedQA-Bart-largeSupervision Type=Unsupervised, Zero-shot Transfer=True2021.05 | 73.3 | 0.6358 | — | |
| NVFP4+PTSModel=Qwen2.5-14B, A-W Quant Type=NVFP4+PTS2026.02 | 73.16 | — | -1.58 | |
| BF16Model=Llama3-8B, A-W Quant Type=BF162026.02 | 72.93 | — | — | |
| NVFP4+PTSModel=Mistral-7B, A-W Quant Type=NVFP4+PTS2026.02 | 72.45 | — | -1.66 | |
| HiF4Model=Mistral-7B, A-W Quant Type=HiF42026.02 | 72.06 | — | -2.05 | |
| HiF4+HiGPTQModel=Llama3-8B, A-W Quant Type=HiF4+HiGPTQ2026.02 | 71.87 | — | -1.06 | |
| HiF4Model=Llama3-8B, A-W Quant Type=HiF42026.02 | 71.79 | — | -1.14 | |
| NVFP4Model=Llama3-8B, A-W Quant Type=NVFP42026.02 | 71.43 | — | -1.5 | |
| NVFP4+PTSModel=Llama3-8B, A-W Quant Type=NVFP4+PTS2026.02 | 70.8 | — | -2.13 | |
| NVFP4Model=Llama2-7B, A-W Quant Type=NVFP42026.02 | 70.48 | — | 1.42 | |
| BF16Model=Llama2-7B, A-W Quant Type=BF162026.02 | 69.06 | — | — | |
| HiF4+HiGPTQModel=Llama2-7B, A-W Quant Type=HiF4+HiGPTQ2026.02 | 68.99 | — | -0.07 | |
| HiF4Model=Llama2-7B, A-W Quant Type=HiF42026.02 | 68.39 | — | -0.67 | |
| NVFP4+PTSModel=Llama2-7B, A-W Quant Type=NVFP4+PTS2026.02 | 66.69 | — | -2.37 | |
| CSSSupervision Type=Unsupervised, Zero-shot Transfer=True2021.05 | 65 | 0.6046 | — | |
| BERTSupervision Type=Unsupervised, Zero-shot Transfer=True2021.05 | 64.9 | 0.5289 | — | |
| FIXED_E=128Experts=1282026.05 | 63.06 | — | — | |
| EMO (Stage 5)E=64→1282026.05 | 61.88 | — | — | |
| BLADETrain Toks.=3B, Backbone=TinyLlama-1.1B2026.06 | 61.25 | — | — | |
| RHO-1Train Toks.=3B, Backbone=TinyLlama-1.1B2026.06 | 60.62 | — | — | |
| FIXED_E=32Experts=322026.05 | 60.46 | — | — | |
| Base-CTTrain Toks.=5B, Backbone=TinyLlama-1.1B2026.06 | 60.46 | — | — | |
| MATES*Train Toks.=3B, Backbone=TinyLlama-1.1B2026.06 | 60.14 | — | — | |
| RandomTrain Toks.=3B, Backbone=TinyLlama-1.1B2026.06 | 59.83 | — | — | |
| MNPPSupervision Type=Unsupervised, Zero-shot Transfer=True (Accuracy), Finetuned Mode=Full WinoGrande data splits (for AUC)2021.05 | 59.2 | 0.6706 | — | |
| BaseTrain Toks.=-, Backbone=TinyLlama-1.1B2026.06 | 59.12 | — | — | |
| RoBERTa-DPRSupervision Type=Supervised, Finetuning Dataset=DPR, Zero-shot Transfer=True2021.05 | 58.9 | — | — | |
| EMO (Stage 4)E=32→642026.05 | 58.56 | — | — | |
| FIXED_E=16Experts=162026.05 | 58.48 | — | — | |
| EMO (Stage 3)E=16→322026.05 | 57.85 | — | — | |
| EMO (Stage 1)E=82026.05 | 56.99 | — | — | |
| EMO (Stage 2)E=8→162026.05 | 56.2 | — | — | |
| Mamba-3model size=152M, mb=42026.06 | 52.7 | — | — | |
| SISAmodel size=152M, ds=16, mb=42026.06 | 52.5 | — | — | |
| BERT (local context)Supervision Type=Supervised, Zero-shot Transfer=True2021.05 | 51.9 | — | — | |
| SISAmodel size=152M, ds=128, mb=42026.06 | 51.9 | — | — | |
| SISAmodel size=152M, ds=32, mb=22026.06 | 51.8 | — | — | |
| GMT v7shot=0-shot, eval-framework=lm-evaluation-harness2026.04 | 51.46 | — | — | |
| Mamba-2model size=152M, mb=42026.06 | 51.4 | — | — | |
| Transformermodel size=152M, mb=42026.06 | 51.2 | — | — | |
| BERT-DPRSupervision Type=Supervised, Finetuning Dataset=DPR, Zero-shot Transfer=True2021.05 | 51 | — | — | |
| Ensemble LMsSupervision Type=Unsupervised, Zero-shot Transfer=True2021.05 | 50.9 | — | — | |
| SISAmodel size=152M, ds=64, mb=42026.06 | 50.8 | — | — | |
| Baselineshot=0-shot, eval-framework=lm-evaluation-harness2026.04 | 50.51 | — | — | |
| RoBERTa (local context)Supervision Type=Supervised, Zero-shot Transfer=True2021.05 | 50 | — | — | |
| Wino Know. HuntingSupervision Type=Unsupervised, Zero-shot Transfer=True2021.05 | 49.6 | — | — | |
| NVFP4Model=Mistral-7B, A-W Quant Type=NVFP42026.02 | 48.3 | — | -25.81 |