Retrieval on VisDial (test)
49.29Avg RDesc - Guessing
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Desc - Guessingshot=zero-shot, backbone=LLaMA-2-13B-Chat, decoding=greedy decoding2024.07 | 49.29 | 29.46 | 53.73 | 64.68 | — | |
| Desc - Guessingzero-shot=true, Backbone (LLM)=LLaMA-2-13B-Chat2024.07 | 49.29 | — | — | — | — | |
| Desc - Queriesshot=zero-shot, backbone=LLaMA-2-13B-Chat, decoding=greedy decoding2024.07 | 44.45 | 24.52 | 49.13 | 59.69 | — | |
| Desc - Querieszero-shot=true, Backbone (LLM)=LLaMA-2-13B-Chat2024.07 | 44.45 | — | — | — | — | |
| Desc - Queries (all)shot=zero-shot, backbone=LLaMA-2-13B-Chat, decoding=greedy decoding, queries=original five plus actions, atmosphere, and lighting2024.07 | 43.8 | 24.3 | 47.34 | 59.74 | — | |
| Desc - Queries (all)zero-shot=true, Backbone (LLM)=LLaMA-2-13B-Chat2024.07 | 43.8 | — | — | — | — | |
| FROMAGeshot=zero-shot2024.07 | 40.57 | 20.8 | 44.9 | 56 | — | |
| Koh et al.zero-shot=true2024.07 | 40.57 | — | — | — | — | |
| Desc - Summaryshot=zero-shot, backbone=LLaMA-2-13B-Chat, decoding=greedy decoding2024.07 | 35.77 | 19.43 | 39.05 | 48.84 | — | |
| Desc - Summaryzero-shot=true, Backbone (LLM)=LLaMA-2-13B-Chat2024.07 | 35.77 | — | — | — | — | |
| Desc - Diagshot=zero-shot, backbone=LLaMA-2-13B-Chat, decoding=greedy decoding2024.07 | 22.53 | 11 | 24.03 | 32.56 | — | |
| Desc - Diagzero-shot=true, Backbone (LLM)=LLaMA-2-13B-Chat2024.07 | 22.53 | — | — | — | — | |
| FTretrieval_mode=fine-tuned, LLM_questioner_aligned=false2024.06 | — | — | — | — | 1.0106 | |
| PlugIRretrieval_mode=zero-shot, LLM_questioner_aligned=true, context_reformulation=true2024.06 | — | — | — | — | 0.7674 | |
| ZSretrieval_mode=zero-shot, LLM_questioner_aligned=false2024.06 | — | — | — | — | 1.0006 |