# Coreference Resolution on Winograd Source: [Coreference Resolution on Winograd]() Collection index: [llms.txt]() Data updated: 2026-03-13T08:42:44.198Z Evaluation of Coreference Resolution performance on the Winograd dataset, typically measured using Accuracy. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Coreference Resolution | | Dataset (including any stated split) | Winograd | | Metric | Accuracy | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 18 of 18 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Accuracy](): Higher is better; unit: PERCENT. ## Recorded results | Method | Accuracy | Result date | Report paper | | --- | --- | --- | --- | | PaLM 2-M | 90.5 | Not recorded | 68fa708e1e209131cea4bc0a | | PaLM 2-L | 89.5 | Not recorded | 68fa708e1e209131cea4bc0a | | PaLM | 87.5 | Not recorded | 68fa708e1e209131cea4bc0a | | PaLM 2-S | 87.5 | Not recorded | 68fa708e1e209131cea4bc0a | | Ettin-Dec-1B | 79.1 | Not recorded | 68fa7a111e209131cea80ee7 | | OLMo-1B-0724 | 76.9 | Not recorded | 68fa7a111e209131cea80ee7 | | Llama-3.2-1B | 74.7 | Not recorded | 68fa7a111e209131cea80ee7 | | Ettin-Dec-400m | 71.8 | Not recorded | 68fa7a111e209131cea80ee7 | | SmolLM2-360m | 70.3 | Not recorded | 68fa7a111e209131cea80ee7 | | Pythia-410m | 65.2 | Not recorded | 68fa7a111e209131cea80ee7 | | SmolLM2-135m | 59.7 | Not recorded | 68fa7a111e209131cea80ee7 | | Ettin-Dec-150m | 59 | Not recorded | 68fa7a111e209131cea80ee7 | | Pythia-160m | 58.2 | Not recorded | 68fa7a111e209131cea80ee7 | | Ettin-Dec-68m | 55.3 | Not recorded | 68fa7a111e209131cea80ee7 | | DistilGPT | 53.8 | Not recorded | 68fa7a111e209131cea80ee7 | | Pythia-14m | 51.6 | Not recorded | 68fa7a111e209131cea80ee7 | | Ettin-Dec-32m | 50.2 | Not recorded | 68fa7a111e209131cea80ee7 | | Ettin-Dec-17m | 48 | Not recorded | 68fa7a111e209131cea80ee7 | ### PaLM 2-M Accuracy: 90.5. Result ID: 699148263a6c314849070087. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"prompting":"1-shot"}. This record does not assert independent reproduction. ### PaLM 2-L Accuracy: 89.5. Result ID: 699148263a6c314849070088. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"prompting":"1-shot"}. This record does not assert independent reproduction. ### PaLM Accuracy: 87.5. Result ID: 699148263a6c314849070085. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"prompting":"1-shot"}. This record does not assert independent reproduction. ### PaLM 2-S Accuracy: 87.5. Result ID: 699148263a6c314849070086. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"prompting":"1-shot"}. This record does not assert independent reproduction. ### Ettin-Dec-1B Accuracy: 79.1. Result ID: 69b3cdf447f34ce9249df248. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size Category":"XL","Parameters":"908M-1.2B","Zero-shot":"true","Closed-book":"true"}. This record does not assert independent reproduction. ### OLMo-1B-0724 Accuracy: 76.9. Result ID: 69b3cdf447f34ce9249df246. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model Size Category":"XL","Parameters":"908M-1.2B","Zero-shot":"true","Closed-book":"true"}. This record does not assert independent reproduction. ### Llama-3.2-1B Accuracy: 74.7. Result ID: 69b3cdf447f34ce9249df247. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model Size Category":"XL","Parameters":"908M-1.2B","Zero-shot":"true","Closed-book":"true"}. This record does not assert independent reproduction. ### Ettin-Dec-400m Accuracy: 71.8. Result ID: 69b3cdf447f34ce9249df245. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size Category":"Large","Parameters":"360-410M","Zero-shot":"true","Closed-book":"true"}. This record does not assert independent reproduction. ### SmolLM2-360m Accuracy: 70.3. Result ID: 69b3cdf447f34ce9249df244. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size Category":"Large","Parameters":"360-410M","Zero-shot":"true","Closed-book":"true"}. This record does not assert independent reproduction. ### Pythia-410m Accuracy: 65.2. Result ID: 69b3cdf447f34ce9249df243. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model Size Category":"Large","Parameters":"360-410M","Zero-shot":"true","Closed-book":"true"}. This record does not assert independent reproduction. ### SmolLM2-135m Accuracy: 59.7. Result ID: 69b3cdf447f34ce9249df240. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size Category":"Base","Parameters":"135-160M","Zero-shot":"true","Closed-book":"true"}. This record does not assert independent reproduction. ### Ettin-Dec-150m Accuracy: 59. Result ID: 69b3cdf447f34ce9249df242. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size Category":"Base","Parameters":"135-160M","Zero-shot":"true","Closed-book":"true"}. This record does not assert independent reproduction. ### Pythia-160m Accuracy: 58.2. Result ID: 69b3cdf447f34ce9249df241. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model Size Category":"Base","Parameters":"135-160M","Zero-shot":"true","Closed-book":"true"}. This record does not assert independent reproduction. ### Ettin-Dec-68m Accuracy: 55.3. Result ID: 69b3cdf447f34ce9249df23f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size Category":"Small","Parameters":"68-82M","Zero-shot":"true","Closed-book":"true"}. This record does not assert independent reproduction. ### DistilGPT Accuracy: 53.8. Result ID: 69b3cdf447f34ce9249df23e. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model Size Category":"Small","Parameters":"68-82M","Zero-shot":"true","Closed-book":"true"}. This record does not assert independent reproduction. ### Pythia-14m Accuracy: 51.6. Result ID: 69b3cdf447f34ce9249df23b. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model Size Category":"XXS","Parameters":"14-17M","Zero-shot":"true","Closed-book":"true"}. This record does not assert independent reproduction. ### Ettin-Dec-32m Accuracy: 50.2. Result ID: 69b3cdf447f34ce9249df23d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size Category":"XS","Parameters":"32M","Zero-shot":"true","Closed-book":"true"}. This record does not assert independent reproduction. ### Ettin-Dec-17m Accuracy: 48. Result ID: 69b3cdf447f34ce9249df23c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size Category":"XXS","Parameters":"14-17M","Zero-shot":"true","Closed-book":"true"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.