Entity Linking on AIDA (testa)
92.9Micro F1SPEL-large
Evaluation results
| Method | Links | |
|---|---|---|
| SPEL-largeCandidate Set=PPRforNED candidate set, context-mode=context-aware, #params on GPU=361.1M, speed sec/doc=0.2672023.10 | 92.9 | |
| SPEL-baseCandidate Set=PPRforNED candidate set, context-mode=context-aware, #params on GPU=128.9M, speed sec/doc=0.1562023.10 | 92.7 | |
| SPEL-largeCandidate Set=PPRforNED candidate set, context-mode=context-agnostic, #params on GPU=361.1M, speed sec/doc=0.2682023.10 | 92 | |
| SPEL-baseCandidate Set=PPRforNED candidate set, context-mode=context-agnostic, #params on GPU=128.9M, speed sec/doc=0.1562023.10 | 91.7 | |
| SPEL-largeCandidate set setting=no mention-specific, US$ for 1000 docs=2.642023.10 | 91.6 | |
| SPEL-largeCandidate Set=no mention-specific candidate set, #params on GPU=361.1M, speed sec/doc=0.2732023.10 | 91.6 | |
| SPEL-baseCandidate set setting=no mention-specific, US$ for 1000 docs=2.282023.10 | 91.3 | |
| SPEL-baseCandidate Set=no mention-specific candidate set, #params on GPU=128.9M, speed sec/doc=0.0842023.10 | 91.3 | |
| Poerner et al. (2020)Architecture=BERT, #params on GPU=131.1M2023.10 | 90.8 | |
| SPEL-largeCandidate Set=KB+Yago candidate set, #params on GPU=361.1M, speed sec/doc=0.2672023.10 | 90.8 | |
| SPEL-baseCandidate Set=KB+Yago candidate set, #params on GPU=128.9M, speed sec/doc=0.1582023.10 | 90.6 | |
| De Cao et al. (2021a)Architecture=RoBERTa+LSTM, Candidate Set=using PPRforNED candidate set, #params on GPU=124.8M, speed sec/doc=0.1942023.10 | 90.1 | |
| Kolitsas et al. (2018)Architecture=LSTM, #params on GPU=330.7M, speed sec/doc=0.0972023.10 | 89.4 | |
| Broscheit (2019)Architecture=BERT, #params on GPU=495.1M, speed sec/doc=0.6132023.10 | 86 | |
| Martins et al. (2019)Architecture=Stack-LSTM2023.10 | 85.2 | |
| van Hulst et al. (2020)Architecture=LSTM, #params on GPU=19.0M, speed sec/doc=0.3372023.10 | 83.3 | |
| Peters et al. (2019)Architecture=BERT2023.10 | 82.1 | |
| Févry et al. (2020)Architecture=Transformer2023.10 | 79.7 | |
| Hoffart et al. (2011)Architecture=Linear2023.10 | 72.4 | |
| GPT-4.0Prompting strategy=few-shot w/ CoT, Candidate set setting=no mention-specific, US$ for 1000 docs=59.372023.10 | 62.4 | |
| De Cao et al. (2021a)Architecture=RoBERTa+LSTM, Candidate Set=no mention-specific candidate set, #params on GPU=124.8M, speed sec/doc=0.2682023.10 | 61.9 | |
| GPT-3.5Prompting strategy=zero-shot, Candidate set setting=no mention-specific, US$ for 1000 docs=4.222023.10 | 47.3 | |
| GPT-4.0Prompting strategy=zero-shot, Candidate set setting=no mention-specific, US$ for 1000 docs=42.172023.10 | 40.4 |