Coreference Resolution on OntoNotes
93.7MUCGPT-4
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| GPT-4Base Encoders=API, Gold Mentions=true, Additional Resources=false2025.12 | 93.7 | 88.8 | 82.8 | 88.4 | |
| GPT-4Base Encoders=API, Gold Mentions=true2025.10 | 93.7 | 88.8 | 82.8 | 88.4 | |
| ImCoref-CeCogpt4oBase Encoders=DeBERTalarge+API, Params=531M, Training Time=13h, Training Hardware=1×RTX4090-24G, Gold Mentions=false, Additional Resources=false2025.12 | 91.2 | 84.9 | 81.8 | 86 | |
| ImCoref-CeSgpt4Base Encoders=DeBERTalarge+API, Params=531M, Training Time=13h, Training Hardware=1 × RTX4090-24G, Inference Time=2m, Inference Mem.=4.052025.10 | 91.2 | 84.9 | 81.8 | 86 | |
| ImCoref-CeSdpskBase Encoders=DeBERTalarge+API, Params=531M, Training Time=13h, Training Hardware=1 × RTX4090-24G, Inference Time=5m, Inference Mem.=4.052025.10 | 90.8 | 84.6 | 81.6 | 85.7 | |
| MEIC-DTBase Encoders=DeBERTalarge, Params=479M, Training Time=14h, Training Hardware=1×RTX4090-24G, Gold Mentions=false, Additional Resources=false2025.12 | 90.5 | 84.4 | 80.2 | 85.1 | |
| ImCoref-CeSqwenBase Encoders=DeBERTalarge+API, Params=531M, Training Time=13h, Training Hardware=1 × RTX4090-24G, Inference Time=3m, Inference Mem.=4.052025.10 | 89.6 | 84.1 | 81 | 84.9 | |
| InstructGPTBase Encoders=API, Gold Mentions=true, Additional Resources=false2025.12 | 89.2 | 79.4 | 73.7 | 80.8 | |
| InstructGPTBase Encoders=API, Gold Mentions=true2025.10 | 89.2 | 79.4 | 73.7 | 80.8 | |
| ImCorefmesBase Encoders=DeBERTalarge, Params=531M, Training Time=13h, Training Hardware=1×RTX4090-24G, Gold Mentions=false, Additional Resources=false2025.12 | 88.2 | 83.6 | 81 | 84.3 | |
| ImCorefBase Encoders=DeBERTalarge, Params=531M, Training Time=13h, Training Hardware=1×RTX4090-24G, Inference Time=15s, Inference Mem.=4.052025.10 | 88.2 | 83.6 | 81 | 84.3 | |
| CorefQABase Encoders=SpanBERTlarge, Params=740M, Training Hardware=1×TPUv3-128G, Gold Mentions=false, Additional Resources=true2025.12 | 88 | 82.2 | 79.1 | 83.1 | |
| MaverickmesBase Encoders=DeBERTalarge, Params=504M, Training Time=14h, Training Hardware=1×RTX4090-24G, Gold Mentions=false, Additional Resources=false2025.12 | 88 | 82.8 | 79.9 | 83.6 | |
| CorefQABase Encoders=SpanBERTlarge, Params=740M, Training Hardware=1×TPUv3-128G, Additional Resources=true2025.10 | 88 | 82.2 | 79.1 | 83.1 | |
| MaverickmesBase Encoders=DeBERTalarge, Params=504M, Training Time=14h, Training Hardware=1×RTX4090-24G, Inference Time=14s, Inference Mem.=4.002025.10 | 88 | 82.8 | 79.9 | 83.6 | |
| MaverickincrBase Encoders=DeBERTalarge, Params=452M, Training Time=29h, Training Hardware=1×RTX4090-24G, Gold Mentions=false, Additional Resources=false2025.12 | 87.9 | 82.7 | 79.8 | 83.5 | |
| Link-AppendBase Encoders=mT5XXL, Params=13B, Training Time=48h, Training Hardware=128×TPUv4-32G, Gold Mentions=false, Additional Resources=false2025.12 | 87.8 | 82.6 | 79.5 | 83.3 | |
| Link-AppendBase Encoders=mT5XXL, Params=13B, Training Time=48h, Training Hardware=128×TPUv4-32G, Inference Time=30m2025.10 | 87.8 | 82.6 | 79.5 | 83.3 | |
| seq2seqBase Encoders=T0XXL, Params=11B, Training Hardware=8×A100-80G, Gold Mentions=false, Additional Resources=false2025.12 | 87.6 | 82.4 | 79.5 | 83.2 | |
| seq2seqBase Encoders=T0XXL, Params=11B, Training Hardware=8×A100-80G, Inference Time=40m2025.10 | 87.6 | 82.4 | 79.5 | 83.2 | |
| ASPBase Encoders=FLAN-T5XXL, Params=11B, Training Time=45h, Training Hardware=6×A100-80G, Gold Mentions=false, Additional Resources=false2025.12 | 87.2 | 81.7 | 78.6 | 82.5 | |
| ASPBase Encoders=FLAN-T5XXL, Params=11B, Training Time=45h, Training Hardware=6×A100-80G, Inference Time=20m2025.10 | 87.2 | 81.7 | 78.6 | 82.5 | |
| LingMessBase Encoders=LongFormerlarge, Params=590M, Training Time=23h, Training Hardware=1×V100-32G, Gold Mentions=false, Additional Resources=false2025.12 | 86.6 | 80.5 | 77.3 | 81.4 | |
| LingMessBase Encoders=LongFormerlarge, Params=590M, Training Time=23h, Training Hardware=1×V100-32G, Inference Time=20s, Inference Mem.=4.802025.10 | 86.6 | 80.5 | 77.3 | 81.4 | |
| wl-corefBase Encoders=RoBERTalarge, Params=360M, Training Time=5h, Training Hardware=1×RTX8000-48G, Gold Mentions=false, Additional Resources=false2025.12 | 86.3 | 79.9 | 76.6 | 81 | |
| Dual-cacheBase Encoders=LongFormerlarge, Params=471M, Training Hardware=1×T4-16G, Gold Mentions=false, Additional Resources=false2025.12 | 86.3 | 80.3 | 76.8 | 81.1 | |
| wl-corefBase Encoders=RoBERTalarge, Params=360M, Training Time=5h, Training Hardware=1×RTX8000-48G, Inference Time=11s, Inference Mem.=2.302025.10 | 86.3 | 79.9 | 76.6 | 81 | |
| Dual cacheBase Encoders=LongFormerlarge, Params=471M, Training Hardware=1×T4-16G2025.10 | 86.3 | 80.3 | 76.8 | 81.1 | |
| GPT-3.5-turboBase Encoders=API, Gold Mentions=true, Additional Resources=false2025.12 | 86.2 | 79.3 | 68.3 | 77.9 | |
| GPT-3.5-turboBase Encoders=API, Gold Mentions=true2025.10 | 86.2 | 79.3 | 68.3 | 77.9 | |
| s2e-corefBase Encoders=LongFormerlarge, Params=494M, Training Hardware=1× 32G, Gold Mentions=false, Additional Resources=false2025.12 | 85.8 | 79.1 | 76.1 | 80.3 | |
| s2e-corefBase Encoders=LongFormerlarge, Params=494M, Training Hardware=1× 32G, Inference Time=17s, Inference Mem.=3.902025.10 | 85.8 | 79.1 | 76.1 | 80.3 | |
| c2f-corefBase Encoders=SpanBERTlarge, Params=370M, Training Hardware=1× 32G, Gold Mentions=false, Additional Resources=false2025.12 | 85.3 | 78.1 | 75.3 | 79.6 | |
| IcorefBase Encoders=SpanBERTlarge, Params=377M, Training Time=40h, Training Hardware=1×1080TI-12G, Gold Mentions=false, Additional Resources=false2025.12 | 85.3 | 77.8 | 75.2 | 79.4 | |
| longdocBase Encoders=LongFormerlarge, Params=471M, Training Time=16h, Training Hardware=1× A6000-48G, Gold Mentions=false, Additional Resources=false2025.12 | 85.3 | 78 | 75.3 | 79.6 | |
| c2f-corefBase Encoders=SpanBERTlarge, Params=370M, Training Hardware=1× 32G, Inference Time=50s, Inference Mem.=11.902025.10 | 85.3 | 78.1 | 75.3 | 79.6 | |
| IcorefBase Encoders=SpanBERTlarge, Params=377M, Training Time=40h, Training Hardware=1×1080TI-12G, Inference Time=38s, Inference Mem.=2.902025.10 | 85.3 | 77.8 | 75.2 | 79.4 | |
| longdocBase Encoders=LongFormerlarge, Params=471M, Training Time=16h, Training Hardware=1× A6000-48G, Inference Time=25s, Inference Mem.=2.102025.10 | 85.3 | 78 | 75.3 | 79.6 | |
| f-corefBase Encoders=DistilRoBERTa, Params=91M, Training Hardware=1×V100-32G, Gold Mentions=false, Additional Resources=true2025.12 | 84.4 | 76.6 | 74.5 | 78.5 | |
| f-corefBase Encoders=DistilRoBERTa, Params=91M, Training Hardware=1×V100-32G, Inference Time=3s, Inference Mem.=1.00, Additional Resources=true2025.10 | 84.4 | 76.6 | 74.5 | 78.5 | |
| GPT-4Base Encoders=API, Gold Mentions=false, Additional Resources=false2025.12 | 73.7 | 62.7 | 52.3 | 62.9 | |
| GPT-4Base Encoders=API, Gold Mentions=false2025.10 | 73.7 | 62.7 | 52.3 | 62.9 | |
| InstructGPTBase Encoders=API, Gold Mentions=false, Additional Resources=false2025.12 | 70.4 | 58.4 | 51.7 | 60.1 | |
| InstructGPTBase Encoders=API, Gold Mentions=false2025.10 | 70.4 | 58.4 | 51.7 | 60.1 | |
| GPT-3.5-turboBase Encoders=API, Gold Mentions=false, Additional Resources=false2025.12 | 66.9 | 55.5 | 46.5 | 56.3 | |
| GPT-3.5-turboBase Encoders=API, Gold Mentions=false2025.10 | 66.9 | 55.5 | 46.5 | 56.3 |