Listener Task on PhotoBook (test)
94.5AccuracyHuman
Evaluation results
| Method | Links | |
|---|---|---|
| Human2023.06 | 94.5 | |
| DeBERTa-based listener model (Ours)seed runs=3, CLIPScore=true, Dense learning signals=true, VisAttn=false2023.06 | 77.3 | |
| DeBERTa-based listener model (Ours + VisAttn)seed runs=3, CLIPScore=true, Dense learning signals=true, VisAttn=true2023.06 | 69.8 | |
| DeBERTa-based listener model (Ours - CLIPScore + VisAttn)seed runs=3, CLIPScore=false, Dense learning signals=true, VisAttn=true2023.06 | 64.9 | |
| DeBERTa-based listener model (Ours - CLIPScore)seed runs=3, CLIPScore=false, Dense learning signals=true, VisAttn=false2023.06 | 64.8 | |
| Modified (Takmaz et al., 2020) w/ CLIPScore ref chainsseed runs=3, CLIPScore ref chains=true2023.06 | 59.7 | |
| Modified (Takmaz et al., 2020)seed runs=32023.06 | 59 | |
| DeBERTa-based listener model (Ours - Dense learning signals)seed runs=3, CLIPScore=true, Dense learning signals=false, VisAttn=false2023.06 | 55.9 | |
| Random guess2023.06 | 50 |