Entity Linking on AIDA (testb)
88.6Micro F1SPEL-large
Evaluation results
| Method | Links | |
|---|---|---|
| SPEL-largeCandidate Set=PPRforNED candidate set, context-mode=context-aware, #params on GPU=361.1M, speed sec/doc=0.2672023.10 | 88.6 | |
| SPEL-baseCandidate Set=PPRforNED candidate set, context-mode=context-aware, #params on GPU=128.9M, speed sec/doc=0.1562023.10 | 88.1 | |
| SPEL-largeCandidate Set=PPRforNED candidate set, context-mode=context-agnostic, #params on GPU=361.1M, speed sec/doc=0.2682023.10 | 87.3 | |
| SPEL-baseCandidate Set=PPRforNED candidate set, context-mode=context-agnostic, #params on GPU=128.9M, speed sec/doc=0.1562023.10 | 86.8 | |
| Feng et al. (2022)Architecture=BERT, #params on GPU=157.3M2023.10 | 86.3 | |
| SPEL-largeCandidate set setting=no mention-specific, US$ for 1000 docs=2.642023.10 | 85.8 | |
| Zhang et al. (2022)Architecture=BLINK+ELECTRA, #params on GPU=1004.3M2023.10 | 85.8 | |
| SPEL-largeCandidate Set=no mention-specific candidate set, #params on GPU=361.1M, speed sec/doc=0.2732023.10 | 85.8 | |
| Mrini et al. (2022)Architecture=BART, #params on GPU=(train) 811.5M, (test) 406.2M2023.10 | 85.7 | |
| SPEL-baseCandidate Set=KB+Yago candidate set, #params on GPU=128.9M, speed sec/doc=0.1582023.10 | 85.7 | |
| SPEL-largeCandidate Set=KB+Yago candidate set, #params on GPU=361.1M, speed sec/doc=0.2672023.10 | 85.7 | |
| SPEL-baseCandidate set setting=no mention-specific, US$ for 1000 docs=2.282023.10 | 85.5 | |
| De Cao et al. (2021a)Architecture=RoBERTa+LSTM, Candidate Set=using PPRforNED candidate set, #params on GPU=124.8M, speed sec/doc=0.1942023.10 | 85.5 | |
| SPEL-baseCandidate Set=no mention-specific candidate set, #params on GPU=128.9M, speed sec/doc=0.0842023.10 | 85.5 | |
| Poerner et al. (2020)Architecture=BERT, #params on GPU=131.1M2023.10 | 85 | |
| De Cao et al. (2021b)Architecture=BART, #params on GPU=406.3M, speed sec/doc=40.9692023.10 | 83.7 | |
| Kannan Ravi et al. (2021)Architecture=BERT2023.10 | 83.1 | |
| Kolitsas et al. (2018)Architecture=LSTM, #params on GPU=330.7M, speed sec/doc=0.0972023.10 | 82.4 | |
| van Hulst et al. (2020)Architecture=LSTM, #params on GPU=19.0M, speed sec/doc=0.3372023.10 | 82.4 | |
| Martins et al. (2019)Architecture=Stack-LSTM2023.10 | 81.9 | |
| Broscheit (2019)Architecture=BERT, #params on GPU=495.1M, speed sec/doc=0.6132023.10 | 79.3 | |
| Févry et al. (2020)Architecture=Transformer2023.10 | 76.7 | |
| Peters et al. (2019)Architecture=BERT2023.10 | 73.7 | |
| Hoffart et al. (2011)Architecture=Linear2023.10 | 72.8 | |
| GPT-4.0Prompting strategy=few-shot w/ CoT, Candidate set setting=no mention-specific, US$ for 1000 docs=59.372023.10 | 66.2 | |
| GPT-4.0Prompting strategy=zero-shot, Candidate set setting=no mention-specific, US$ for 1000 docs=42.172023.10 | 54.1 | |
| GPT-3.5Prompting strategy=zero-shot, Candidate set setting=no mention-specific, US$ for 1000 docs=4.222023.10 | 52.9 | |
| De Cao et al. (2021a)Architecture=RoBERTa+LSTM, Candidate Set=no mention-specific candidate set, #params on GPU=124.8M, speed sec/doc=0.2682023.10 | 49.4 |