Entity Disambiguation on ZELDA Benchmark (test)
91.1AIDA-BCL-RECALL
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| CL-RECALLType=Baseline (Recall of candidate list)2024.04 | 91.1 | 94 | 98.4 | 98.3 | 92.4 | 98.8 | 98.8 | 56.7 | 73.1 | 89.1 | |
| Upper boundEvaluation protocol=Baseline2026.01 | 91.1 | 94 | 98.4 | 98.3 | 92.4 | 98.8 | 98.8 | 56.7 | 73.1 | 89.1 | |
| VERBALIZED + iter. trainingTrain data=Z2025.05 | 88.2 | 78.9 | 92.2 | 88.4 | 71.5 | 90.8 | 98.2 | 66.3 | 65.9 | 82.3 | |
| VERBALIZEDTrain data=Z, iterative training=true2025.05 | 88.2 | 78.9 | 92.2 | 88.4 | 71.5 | 90.8 | 98.2 | 66.3 | 65.9 | 82.3 | |
| VERBALIZED ITEvaluation protocol=Classical (with training data)2026.01 | 88.2 | 78.9 | 92.2 | 88.4 | 71.5 | 90.8 | 98.2 | 66.3 | 65.9 | 82.3 | |
| VERBALIZED + iter. predictionTrain data=Z2025.05 | 84.4 | 78.4 | 89.5 | 87.1 | 70.4 | 91.2 | 98.7 | 65.3 | 67.2 | 81.4 | |
| BLINK-OG crossTrain data=B2025.05 | 84.2 | 82.4 | 92.8 | 91.2 | 77.3 | 82.3 | 99.2 | 64.8 | 74.2 | 83.2 | |
| LELAEvaluation protocol=True Zero-Shot, LLM selection model=Qwen3-30B-A3B2026.01 | 84.2 | 79.1 | 83.9 | 84.6 | 64.8 | 87.5 | 97.3 | 48.8 | 67 | 77.5 | |
| LELAEvaluation protocol=True Zero-Shot, LLM selection model=Qwen3-4B2026.01 | 84 | 78 | 84.1 | 82.6 | 66.3 | 85.1 | 97.1 | 48.2 | 65 | 76.7 | |
| LELAEvaluation protocol=True Zero-Shot, LLM selection model=Magistral-Small-25092026.01 | 83.3 | 79.1 | 80.7 | 79.5 | 66.7 | 87.5 | 97.2 | 48.9 | 66.7 | 76.6 | |
| VERBALIZEDTrain data=Z2025.05 | 82.6 | 78.9 | 89.2 | 86.2 | 69.8 | 91.4 | 98.6 | 65.3 | 67 | 81 | |
| VERBALIZEDTrain data=Z, iterative training=false2025.05 | 82.6 | 78.9 | 89.2 | 86.2 | 69.8 | 91.4 | 98.6 | 65.3 | 67 | 81 | |
| LUKEPCategory=Classification, Mode=Fine-tuned2024.04 | 81.2 | 77.9 | 81.5 | 78.5 | 70.3 | 76.5 | 98 | 22.5 | 51.8 | 71 | |
| LUKE_FTTrain data=Z2025.05 | 81.2 | 77.9 | 81.5 | 78.5 | 70.3 | 76.5 | 98 | 22.5 | 51.8 | 71 | |
| LUKE FTEvaluation protocol=Classical (with training data)2026.01 | 81.2 | 77.9 | 81.5 | 78.5 | 70.3 | 76.5 | 98 | 22.5 | 51.8 | 71 | |
| BLINK-OG biTrain data=B2025.05 | 80.6 | 77.3 | 90.8 | 87.8 | 68.2 | 79.8 | 97.9 | 50.1 | 57.3 | 76.6 | |
| FUSIONEDCategory=Generative2024.04 | 80.1 | 81.4 | 93.9 | 92.3 | 73.6 | 89 | 98.3 | 41.5 | 57.9 | 78.7 | |
| FUSIONEDTrain data=Z2025.05 | 80.1 | 81.4 | 93.9 | 92.3 | 73.6 | 89 | 98.3 | 41.5 | 57.9 | 78.7 | |
| FusionEDEvaluation protocol=Classical (with training data)2026.01 | 80.1 | 81.4 | 93.9 | 92.3 | 73.6 | 89 | 98.3 | 41.5 | 57.9 | 78.7 | |
| FEVRYCategory=Classification, Configuration=CL (Using candidate lists)2024.04 | 79.5 | 76.9 | 89 | 86.5 | 70.3 | 84.5 | 87.6 | 31.9 | 47.7 | 72.7 | |
| FEVRY_CLTrain data=Z2025.05 | 79.5 | 76.9 | 89 | 86.5 | 70.3 | 84.5 | 87.6 | 31.9 | 47.7 | 72.7 | |
| FEVRY CLEvaluation protocol=Classical (with training data)2026.01 | 79.5 | 76.9 | 89 | 86.5 | 70.3 | 84.5 | 87.6 | 31.9 | 47.7 | 72.7 | |
| LUKEPCategory=Classification, Mode=Pre-trained2024.04 | 79.3 | 73.8 | 76.1 | 69.9 | 66.8 | 68.4 | 97.7 | 20.4 | 50.8 | 67 | |
| LUKE_PRETrain data=Z2025.05 | 79.3 | 73.8 | 76.1 | 69.9 | 66.8 | 68.4 | 97.7 | 20.4 | 50.8 | 67 | |
| FEVRYCategory=Classification, Configuration=ALL (Restricted generation process)2024.04 | 79.2 | 71.8 | 88.5 | 84.1 | 68 | 84.3 | 63.8 | 43.4 | 53.1 | 70.7 | |
| FEVRY_ALLTrain data=Z2025.05 | 79.2 | 71.8 | 88.5 | 84.1 | 68 | 84.3 | 63.8 | 43.4 | 53.1 | 70.7 | |
| GENRECategory=Generative, Configuration=CL (Using candidate lists)2024.04 | 78.6 | 80.1 | 92.8 | 91.5 | 73.6 | 88.4 | 99.6 | 37.3 | 52.8 | 77.2 | |
| GENRE_CLTrain data=Z2025.05 | 78.6 | 80.1 | 92.8 | 91.5 | 73.6 | 88.4 | 99.6 | 37.3 | 52.8 | 77.2 | |
| GENRE CLEvaluation protocol=Classical (with training data)2026.01 | 78.6 | 80.1 | 92.8 | 91.5 | 73.6 | 88.4 | 99.6 | 37.3 | 52.8 | 77.2 | |
| LELAEvaluation protocol=True Zero-Shot, LLM selection strategy=None (w/o LLM selection)2026.01 | 76.4 | 75.1 | 79 | 79.9 | 65 | 76.2 | 97 | 43.6 | 60.6 | 72.5 | |
| GENRECategory=Generative, Configuration=ALL (Without candidate lists)2024.04 | 72.4 | 75.9 | 88.8 | 83.9 | 66.5 | 85.2 | 95.3 | 38.7 | 43.5 | 72.2 | |
| GENRE_ALLTrain data=Z2025.05 | 72.4 | 75.9 | 88.8 | 83.9 | 66.5 | 85.2 | 95.3 | 38.7 | 43.5 | 72.2 | |
| BLINK-ZTrain data=Z2025.05 | 65.5 | 72.1 | 83.1 | 79.1 | 58.1 | 73.1 | 96.3 | 41.8 | 42.6 | 68 | |
| BLINK biEvaluation protocol=Classical (with training data)2026.01 | 65.5 | 72.1 | 83.1 | 79.1 | 58.1 | 73.1 | 96.3 | 41.8 | 42.6 | 68 | |
| MFS baselineEvaluation protocol=Baseline2026.01 | 63.5 | 72.3 | 83.4 | 81 | 61.2 | 65.1 | 99.4 | 14.9 | 41.3 | 64.7 |