Fine-grained Entity Recognition on OVEN Entity 1.0 (test)
29.6HMGiT-Large
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| GiT-Large#par (B)=0.4, Training Dataset=REW-47M (Ours), Evaluation Protocol=+ seen finetune2024.10 | 29.6 | 36 | 25.1 | |
| GiT-Large#par (B)=0.4, Training Dataset=REW-47M (Ours), Evaluation Protocol=Zero-shot2024.10 | 23.6 | 25.7 | 21.7 | |
| GER-ALD#par (B)=0.4, Training Dataset=Entity-WebLI [7], Evaluation Protocol=+ seen finetune2024.10 | 22.7 | 31.5 | 17.7 | |
| GiT-Large#par (B)=0.4, Training Dataset=Entity-WebLI [7], Evaluation Protocol=+ seen finetune2024.10 | 20.1 | 25.9 | 16.4 | |
| GiT-Large#par (B)=0.4, Training Dataset=Entity-WebLI [7], Evaluation Protocol=Zero-shot2024.10 | 19.1 | 19.8 | 18.5 | |
| GER-ALD#par (B)=0.4, Training Dataset=Entity-WebLI [7], Evaluation Protocol=Zero-shot2024.10 | 17.7 | 18.3 | 17.2 | |
| PaLI-17B#par (B)=17, Training Dataset=WebLI-1B [9], Evaluation Protocol=+ seen finetune2024.10 | 16 | 28.3 | 11.2 | |
| CLIP2CLIP#par (B)=0.9, Training Dataset=OpenAI [41], Evaluation Protocol=+ seen finetune2024.10 | 11.5 | 12.6 | 10.5 | |
| PaLI-3B#par (B)=3, Training Dataset=WebLI-1B [9], Evaluation Protocol=+ seen finetune2024.10 | 9.1 | 19.1 | 6 | |
| CLIPfusion#par (B)=0.9, Training Dataset=OpenAI [41], Evaluation Protocol=+ seen finetune2024.10 | 8.4 | 33.6 | 4.8 | |
| GiT-Large#par (B)=0.4, Training Dataset=WebLI-100M [9], Evaluation Protocol=+ seen finetune2024.10 | 6.5 | 13.7 | 4.2 | |
| CLIPfusion#par (B)=0.9, Training Dataset=OpenAI [41], Evaluation Protocol=Zero-shot2024.10 | 5.2 | 5.6 | 4.9 | |
| CLIP2CLIP#par (B)=0.9, Training Dataset=OpenAI [41], Evaluation Protocol=Zero-shot2024.10 | 5.2 | 5.6 | 4.9 | |
| GiT-Large#par (B)=0.4, Training Dataset=WebLI-100M [9], Evaluation Protocol=Zero-shot2024.10 | 2.1 | 4.7 | 1.4 | |
| PaLI-17B#par (B)=17, Training Dataset=WebLI-1B [9], Evaluation Protocol=Zero-shot2024.10 | 1.8 | 3.3 | 1.2 |