Coreference Resolution on Winograd
90.5AccuracyPaLM 2-M
Evaluation Results
| Method | Links | |
|---|---|---|
| PaLM 2-Mprompting=1-shot2023.05 | 90.5 | |
| PaLM 2-Lprompting=1-shot2023.05 | 89.5 | |
| PaLMprompting=1-shot2023.05 | 87.5 | |
| PaLM 2-Sprompting=1-shot2023.05 | 87.5 | |
| Ettin-Dec-1BModel Size Category=XL, Parameters=908M-1.2B, Zero-shot=true, Closed-book=true2025.07 | 79.1 | |
| OLMo-1B-0724Model Size Category=XL, Parameters=908M-1.2B, Zero-shot=true, Closed-book=true2025.07 | 76.9 | |
| Llama-3.2-1BModel Size Category=XL, Parameters=908M-1.2B, Zero-shot=true, Closed-book=true2025.07 | 74.7 | |
| Ettin-Dec-400mModel Size Category=Large, Parameters=360-410M, Zero-shot=true, Closed-book=true2025.07 | 71.8 | |
| SmolLM2-360mModel Size Category=Large, Parameters=360-410M, Zero-shot=true, Closed-book=true2025.07 | 70.3 | |
| Pythia-410mModel Size Category=Large, Parameters=360-410M, Zero-shot=true, Closed-book=true2025.07 | 65.2 | |
| SmolLM2-135mModel Size Category=Base, Parameters=135-160M, Zero-shot=true, Closed-book=true2025.07 | 59.7 | |
| Ettin-Dec-150mModel Size Category=Base, Parameters=135-160M, Zero-shot=true, Closed-book=true2025.07 | 59 | |
| Pythia-160mModel Size Category=Base, Parameters=135-160M, Zero-shot=true, Closed-book=true2025.07 | 58.2 | |
| Ettin-Dec-68mModel Size Category=Small, Parameters=68-82M, Zero-shot=true, Closed-book=true2025.07 | 55.3 | |
| DistilGPTModel Size Category=Small, Parameters=68-82M, Zero-shot=true, Closed-book=true2025.07 | 53.8 | |
| Pythia-14mModel Size Category=XXS, Parameters=14-17M, Zero-shot=true, Closed-book=true2025.07 | 51.6 | |
| Ettin-Dec-32mModel Size Category=XS, Parameters=32M, Zero-shot=true, Closed-book=true2025.07 | 50.2 | |
| Ettin-Dec-17mModel Size Category=XXS, Parameters=14-17M, Zero-shot=true, Closed-book=true2025.07 | 48 |