List your company
ResearchFormerly WizWand

Coreference Resolution on Winograd

90.5AccuracyPaLM 2-M
46.357.77569.2580.725May 17, 2023Sep 25, 2023Feb 4, 2024Jun 15, 2024Oct 24, 2024Mar 5, 2025Jul 15, 2025
Updated 8mo ago

Evaluation results

MethodLinks
PaLM 2-Mprompting=1-shot2023.0590.5
PaLM 2-Lprompting=1-shot2023.0589.5
PaLMprompting=1-shot2023.0587.5
PaLM 2-Sprompting=1-shot2023.0587.5
Ettin-Dec-1BModel Size Category=XL, Parameters=908M-1.2B, Zero-shot=true, Closed-book=true2025.0779.1
OLMo-1B-0724Model Size Category=XL, Parameters=908M-1.2B, Zero-shot=true, Closed-book=true2025.0776.9
Llama-3.2-1BModel Size Category=XL, Parameters=908M-1.2B, Zero-shot=true, Closed-book=true2025.0774.7
Ettin-Dec-400mModel Size Category=Large, Parameters=360-410M, Zero-shot=true, Closed-book=true2025.0771.8
SmolLM2-360mModel Size Category=Large, Parameters=360-410M, Zero-shot=true, Closed-book=true2025.0770.3
Pythia-410mModel Size Category=Large, Parameters=360-410M, Zero-shot=true, Closed-book=true2025.0765.2
SmolLM2-135mModel Size Category=Base, Parameters=135-160M, Zero-shot=true, Closed-book=true2025.0759.7
Ettin-Dec-150mModel Size Category=Base, Parameters=135-160M, Zero-shot=true, Closed-book=true2025.0759
Pythia-160mModel Size Category=Base, Parameters=135-160M, Zero-shot=true, Closed-book=true2025.0758.2
Ettin-Dec-68mModel Size Category=Small, Parameters=68-82M, Zero-shot=true, Closed-book=true2025.0755.3
DistilGPTModel Size Category=Small, Parameters=68-82M, Zero-shot=true, Closed-book=true2025.0753.8
Pythia-14mModel Size Category=XXS, Parameters=14-17M, Zero-shot=true, Closed-book=true2025.0751.6
Ettin-Dec-32mModel Size Category=XS, Parameters=32M, Zero-shot=true, Closed-book=true2025.0750.2
Ettin-Dec-17mModel Size Category=XXS, Parameters=14-17M, Zero-shot=true, Closed-book=true2025.0748