Binary Classification on VALSE zero-shot
66.9ExistenceUNITER
Evaluation results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| UNITERArchitecture=Single-Stream, Training Source=HNCfull2026.05 | 66.9 | 50.7 | 57.5 | 53.6 | 52.6 | 50.7 | 55.9 | 52.8 | 50.1 | 50.4 | 54.1 | 53.9 | |
| UNITERArchitecture=Single-Stream, Training Source=HNCsubset2026.05 | 63.5 | 51.3 | 53.2 | 54.2 | 51 | 50.3 | 54.4 | 52 | 49.8 | 50.1 | 51.2 | 52.8 | |
| VisualBERTArchitecture=Single-Stream, Training Source=HNCfull2026.05 | 63.1 | 52.8 | 58.8 | 50.2 | 53.2 | 51.4 | 57.6 | 54.3 | 49.9 | 48.9 | 54.8 | 54.4 | |
| LXMERTArchitecture=Dual-Stream, Training Source=HNCfull2026.05 | 60.3 | 50.3 | 55.9 | 50.2 | 52.6 | 51.6 | 57.4 | 55.1 | 48.9 | 48.6 | 52.4 | 53 | |
| ViLBERTArchitecture=Dual-Stream, Training Source=HNCfull2026.05 | 59.8 | 51.4 | 58.6 | 53.2 | 52.4 | 50.9 | 58 | 52.9 | 47.2 | 48.2 | 51.8 | 53 | |
| VL-BERTArchitecture=Single-Stream, Training Source=HNCfull2026.05 | 59.2 | 51.3 | 58.1 | 52.7 | 53.3 | 50.1 | 56.3 | 52.3 | 50.2 | 51.1 | 53.6 | 53.5 | |
| VisualBERTArchitecture=Single-Stream, Training Source=HNCsubset2026.05 | 58.9 | 51.8 | 53.2 | 50.4 | 50.3 | 50.4 | 58.3 | 51 | 49.7 | 49.8 | 51.5 | 52.3 | |
| VL-BERTArchitecture=Single-Stream, Training Source=HNCsubset2026.05 | 58.6 | 51.6 | 52.5 | 53.3 | 51.5 | 50.7 | 53.8 | 49.7 | 49.9 | 49.4 | 50 | 51.9 | |
| BLIPArchitecture=Contrastive Dual-Encoder, Training Source=HNCfull2026.05 | 57.77 | 55.6 | 63.05 | 57.76 | 58.02 | 54.07 | 57.41 | 58.41 | 50.11 | 53.55 | 59.2 | 57.13 | |
| LXMERTArchitecture=Dual-Stream, Training Source=HNCsubset2026.05 | 52.2 | 50.2 | 50.4 | 54 | 50.5 | 50.5 | 55.4 | 53.2 | 49.8 | 49.5 | 50.1 | 51.4 | |
| ViLBERTArchitecture=Dual-Stream, Training Source=HNCsubset2026.05 | 52.1 | 51.4 | 51.1 | 54.6 | 49.9 | 50.9 | 55.8 | 51.6 | 50 | 50 | 50.4 | 51.6 | |
| LXMERTArchitecture=Dual-Stream, Training Source=FOIL2026.05 | 52.1 | 50.1 | 49.1 | 50.7 | 50.4 | 49.9 | 53.2 | 52.1 | 50.1 | 50 | 77.5 | 50.8 | |
| BLIPArchitecture=Contrastive Dual-Encoder, Training Source=ITM2026.05 | 52.06 | 57.95 | 53.3 | 55.96 | 53.46 | 54.97 | 52.47 | 61.94 | 50.66 | 50.35 | 72.1 | 57.16 | |
| ViLBERTArchitecture=Dual-Stream, Training Source=FOIL2026.05 | 49.8 | 50.4 | 49.3 | 52.5 | 49.6 | 49.8 | 54.6 | 53 | 50.1 | 50 | 77 | 50.9 | |
| UNITERArchitecture=Single-Stream, Training Source=FOIL2026.05 | 49.8 | 50.1 | 49.7 | 50.7 | 50 | 50 | 51 | 51 | 50.1 | 50 | 77.1 | 50.2 | |
| VL-BERTArchitecture=Single-Stream, Training Source=FOIL2026.05 | 49.7 | 50.4 | 49.4 | 50 | 49.8 | 50.2 | 50.4 | 50.5 | 49.9 | 50 | 78.1 | 50 | |
| VisualBERTArchitecture=Single-Stream, Training Source=FOIL2026.05 | 49.3 | 50.3 | 50 | 50.7 | 49.7 | 50 | 51.5 | 50.3 | 49.9 | 50 | 79 | 50.2 | |
| ViLBERTArchitecture=Dual-Stream, Training Source=VOLTA2026.05 | 47.8 | 50 | 49.4 | 49.5 | 49.8 | 49.8 | 48.1 | 47 | 49.9 | 50 | 46 | 48.8 | |
| LXMERTArchitecture=Dual-Stream, Training Source=VOLTA2026.05 | 47.3 | 49.9 | 49.5 | 49.6 | 49.9 | 50 | 49 | 48.1 | 50 | 50 | 45.8 | 48.9 | |
| VL-BERTArchitecture=Single-Stream, Training Source=VOLTA2026.05 | 47.1 | 49.6 | 49.5 | 49.5 | 49.7 | 49.9 | 48.3 | 47.4 | 50 | 50 | 44.2 | 48.5 | |
| VisualBERTArchitecture=Single-Stream, Training Source=VOLTA2026.05 | 46.9 | 49.5 | 49.6 | 48.9 | 49.6 | 49.8 | 47.9 | 47.8 | 50 | 50 | 43.7 | 48.4 | |
| UNITERArchitecture=Single-Stream, Training Source=VOLTA2026.05 | 46.2 | 50 | 49.4 | 49.1 | 49.9 | 50 | 47.9 | 47.5 | 49.9 | 50 | 43.8 | 48.4 |