Classification on CPT zero-shot
69.01Part-Whole ScoreUNITER
Evaluation Results
| Method | Links | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| UNITERStream Architecture=Single-Stream, Evaluation Split=HNCfull, Protocol=zero-shot2026.05 | 69.01 | 70.65 | 55.75 | 46.27 | 58.5 | 62.17 | 57.97 | 68.71 | 59.63 | 77 | 62.32 | |
| VisualBERTStream Architecture=Single-Stream, Evaluation Split=HNCfull, Protocol=zero-shot2026.05 | 68.84 | 68.84 | 52.02 | 52.3 | 63.27 | 66.9 | 57.54 | 70.33 | 57.22 | 70 | 62.16 | |
| VisualBERTStream Architecture=Single-Stream, Evaluation Split=HNCsubset, Protocol=zero-shot2026.05 | 66.47 | 70.89 | 53.26 | 50.19 | 61.34 | 68.17 | 57.01 | 67.79 | 58.34 | 68.4 | 61.5 | |
| VisualBERTStream Architecture=Single-Stream, Evaluation Split=FOIL, Protocol=zero-shot2026.05 | 64.46 | 69.69 | 50.16 | 51.51 | 61.75 | 65.6 | 58.54 | 68.66 | 57.78 | 66 | 61.2 | |
| UNITERStream Architecture=Single-Stream, Evaluation Split=HNCsubset, Protocol=zero-shot2026.05 | 64.1 | 66.42 | 50.99 | 46.53 | 57.29 | 62.8 | 53.32 | 65.95 | 58.75 | 63.2 | 59.27 | |
| UNITERStream Architecture=Single-Stream, Evaluation Split=FOIL, Protocol=zero-shot2026.05 | 63.95 | 67.27 | 49.53 | 46.59 | 56.69 | 54.57 | 57.47 | 64.93 | 57.72 | 64 | 59.29 | |
| VL-BERTStream Architecture=Single-Stream, Evaluation Split=HNCfull, Protocol=zero-shot2026.05 | 63.43 | 68.6 | 49.69 | 53.31 | 62.74 | 68.04 | 57.47 | 65.74 | 51.43 | 67 | 60.28 | |
| VisualBERTStream Architecture=Single-Stream, Evaluation Split=VOLTA, Protocol=zero-shot2026.05 | 63.35 | 65.7 | 48.45 | 50.98 | 56.84 | 61.2 | 58.1 | 67.22 | 57.27 | 65 | 59.24 | |
| UNITERStream Architecture=Single-Stream, Evaluation Split=VOLTA, Protocol=zero-shot2026.05 | 63 | 65.94 | 46.43 | 47.86 | 54.04 | 61.58 | 57.41 | 65.36 | 58.51 | 62 | 58.53 | |
| VL-BERTStream Architecture=Single-Stream, Evaluation Split=FOIL, Protocol=zero-shot2026.05 | 61.37 | 71.38 | 49.22 | 51.67 | 62.21 | 63.48 | 56.66 | 65.74 | 52.22 | 66 | 59.33 | |
| VL-BERTStream Architecture=Single-Stream, Evaluation Split=HNCsubset, Protocol=zero-shot2026.05 | 59.43 | 67.08 | 48.88 | 50.5 | 57.93 | 65.52 | 53.85 | 65.15 | 50.48 | 64.4 | 57.88 | |
| VL-BERTStream Architecture=Single-Stream, Evaluation Split=VOLTA, Protocol=zero-shot2026.05 | 58.37 | 66.06 | 47.2 | 51.83 | 55.4 | 62.23 | 56.47 | 65.12 | 50.42 | 63 | 57.42 | |
| LXMERTStream Architecture=Dual-Stream, Evaluation Split=HNCfull, Protocol=zero-shot2026.05 | 56.48 | 60.51 | 46.43 | 50.24 | 54.8 | 70.11 | 56.79 | 60.62 | 52.05 | 54 | 55.49 | |
| ViLBERTStream Architecture=Dual-Stream, Evaluation Split=HNCfull, Protocol=zero-shot2026.05 | 55.97 | 54.83 | 41.3 | 39.28 | 52.08 | 56.85 | 50.47 | 57.32 | 47.05 | 53 | 50.87 | |
| ViLBERTStream Architecture=Dual-Stream, Evaluation Split=VOLTA, Protocol=zero-shot2026.05 | 55.02 | 55.92 | 42.24 | 47.17 | 49.89 | 57.39 | 52.91 | 60.48 | 56.6 | 52 | 53.95 | |
| ViLBERTStream Architecture=Dual-Stream, Evaluation Split=FOIL, Protocol=zero-shot2026.05 | 54.59 | 56.76 | 40.06 | 45.79 | 52.38 | 56.63 | 51.97 | 57.8 | 54.41 | 54 | 52.99 | |
| LXMERTStream Architecture=Dual-Stream, Evaluation Split=VOLTA, Protocol=zero-shot2026.05 | 53.73 | 59.42 | 43.01 | 49.23 | 51.55 | 62.45 | 55.6 | 62.87 | 54.8 | 56 | 55.43 | |
| LXMERTStream Architecture=Dual-Stream, Evaluation Split=HNCsubset, Protocol=zero-shot2026.05 | 52.93 | 61.98 | 44.38 | 47.56 | 54.78 | 67.2 | 55.04 | 63.91 | 54.6 | 53.4 | 55.94 | |
| ViLBERTStream Architecture=Dual-Stream, Evaluation Split=HNCsubset, Protocol=zero-shot2026.05 | 52.65 | 54.56 | 43.11 | 41.5 | 52.53 | 55.64 | 45.41 | 57.57 | 53.23 | 58.6 | 51.15 | |
| LXMERTStream Architecture=Dual-Stream, Evaluation Split=FOIL, Protocol=zero-shot2026.05 | 50.13 | 55.68 | 43.17 | 49.81 | 52.46 | 64.18 | 52.72 | 57.51 | 51.88 | 54 | 52.22 |