POPE
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
POPE Adversarial
121.4CIDEr
50
POPE Aggregated random, popular, and adversarial
86.53Accuracy (POPE Aggregated)
47
POPE Adversarial v1.0
89.26Accuracy
45
POPE MSCOCO Adversarial
87.33Accuracy
43
POPE
91.22Accuracy
43
POPE Adversarial
83.9Accuracy
41
POPE Random
90.57Accuracy
41
POPE MS-COCO Adversarial sampling (val)
85.48Accuracy
39
POPE (average across random and popular)
91.56Accuracy (POPE)
38
POPE average over three sampling strategies
83.94AUROC
35
POPE GQA (Adversarial)
84.27Accuracy
34
POPE official (val)
99.13A-PR
34
POPE
90.81Accuracy
32
POPE Adversarial v1.0 (test)
88.96Accuracy
31
POPE Popular v1.0 (test)
90.34Accuracy
31
POPE Random v1.0 (test)
91.17Accuracy
31
POPE
27.48CAE
30
POPE MSCOCO Popular
89.07Accuracy
29
POPE MSCOCO (Random)
90.77Accuracy
29
POPE GQA (test)
84.72Average Accuracy
29
POPE popular
86.59F1 Score
27
POPE averaged across MS-COCO, A-OKVQA, and GQA (test)
88.7Random F1-score
26
POPE
91.5F1 Score (Random)
25
POPE GQA
86.62Accuracy
24
POPE A-OKVQA
87.61Accuracy
24