SugarCrepe
Benchmarks
TaskDatasetSOTA resultTrendResults
SugarCrepe
95.2Overall Accuracy
95
SugarCrepe
98.44R@1 (Add)
30
SugarCrepe swap att (test)
82.1Accuracy
27
SugarCrepe++
75.2Average Performance
25
SugarCrepe++ TOT
72.9Accuracy
21
SugarCrepe++ ITT
73.1Accuracy
21
SugarCrepe
94.2Add Score
21
SugarCrepe ++ (test)
79.7Replace ITT
21
SugarCrepe (test)
100Replace: Object (R@1)
21
SugarCrepe
88.06Accuracy
20
SugarCrepe++
66.24Accuracy
20
SugarCrepe (test)
95.52Replace (Object)
20
SugarCrepe 1.0 (test)
98.86Avg-M Score
18
SugarCrepe
16.7AURC
17
SugarCrepe++ T2I
61.05Accuracy
15
SugarCrepe
89.23Accuracy
15
SugarCrepe++
79.8Replace-I2T
11
SugarCrepe
89.5Replace Accuracy
11
SugarCrepe clean
36.84Transfer R@1
9
SugarCrepe clean
88.41Attr. Neg. Selectivity
9
SugarCrepe
82.14Simple Accuracy
9
SugarCrepe
88.7Replacement Score
9
SUGARCREPE (test)
86.3Accuracy
8
SugarCrepe 1.0 (test)
100Replace Acc (Object)
8
SugarCrepe 1.0 (test)
88.1Recall@1 (Replace, Object)
8