Listener Task on PhotoBook (val)
95AccuracyHuman
Evaluation results
| Method | Links | |
|---|---|---|
| Human2023.06 | 95 | |
| DeBERTa-based listener model (Ours)seed runs=3, CLIPScore=true, Dense learning signals=true, VisAttn=false2023.06 | 84.8 | |
| DeBERTa-based listener model (Ours + VisAttn)seed runs=3, CLIPScore=true, Dense learning signals=true, VisAttn=true2023.06 | 75 | |
| DeBERTa-based listener model (Ours - CLIPScore)seed runs=3, CLIPScore=false, Dense learning signals=true, VisAttn=false2023.06 | 70.7 | |
| DeBERTa-based listener model (Ours - CLIPScore + VisAttn)seed runs=3, CLIPScore=false, Dense learning signals=true, VisAttn=true2023.06 | 69.8 | |
| Modified (Takmaz et al., 2020) w/ CLIPScore ref chainsseed runs=3, CLIPScore ref chains=true2023.06 | 65 | |
| Modified (Takmaz et al., 2020)seed runs=32023.06 | 64.2 | |
| DeBERTa-based listener model (Ours - Dense learning signals)seed runs=3, CLIPScore=true, Dense learning signals=false, VisAttn=false2023.06 | 59.4 | |
| Random guess2023.06 | 50 |