Multi-modal dialogue retrieval on PhotoChat (test)
28.3R@1DRIBER GPT-4 1106
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| DRIBER GPT-4 1106Evaluation Protocol=Zero-shot, LLM Backbone=GPT-4 11062023.10 | 28.3 | 47.4 | 55.2 | — | 37.6 | |
| DRIBER ChatGPT 0613Evaluation Protocol=Zero-shot, LLM Backbone=ChatGPT 06132023.10 | 26.6 | 46.1 | 54.2 | — | 36 | |
| DRIBER ChatGPT 1106Evaluation Protocol=Zero-shot, LLM Backbone=ChatGPT 11062023.10 | 26.3 | 45.6 | 54.3 | — | 35.4 | |
| DRIBER Vicuna-13BEvaluation Protocol=Zero-shot, LLM Backbone=Vicuna-13B2023.10 | 25.8 | 45 | 53.1 | — | 35 | |
| DRIBER LLAMa2-Chat-70BEvaluation Protocol=Zero-shot, LLM Backbone=LLAMa2-Chat-70B2023.10 | 24.5 | 43.5 | 52.6 | — | 34 | |
| Human2023.10 | 19.6 | 37.5 | 44.7 | — | — | |
| DialCLIPEvaluation Protocol=Fine-tuned2023.10 | 19.5 | 44 | 55.8 | — | — | |
| PaCE2023.05 | 15.2 | 36.7 | 49.6 | 101.5 | — | |
| PaCEEvaluation Protocol=Fine-tuned2023.10 | 15.2 | 36.7 | 49.6 | — | — | |
| CLIP-largeEvaluation Protocol=Zero-shot, Model Type=VLM2023.10 | 14.1 | 28.7 | 35.3 | — | 21.5 | |
| VLMo2023.05 | 13.8 | 30 | 39.4 | 83.2 | — | |
| VLMOEvaluation Protocol=Fine-tuned2023.10 | 13.8 | 30 | 39.4 | — | — | |
| GPT4-VEvaluation Protocol=Zero-shot, Model Type=Large Multi-Modal Model2023.10 | 13.8 | 27.9 | 35.9 | — | 21.3 | |
| CLIP-baseEvaluation Protocol=Zero-shot, Model Type=VLM2023.10 | 13.7 | 28 | 35.2 | — | 20.8 | |
| LLaVA v1.5 13BEvaluation Protocol=Zero-shot, Model Type=Large Multi-Modal Model2023.10 | 12.1 | 25.6 | 32.3 | — | 19.3 | |
| Qwen-VL-Chat 7BEvaluation Protocol=Zero-shot, Model Type=Large Multi-Modal Model2023.10 | 12.1 | 27.4 | 36.1 | — | 20.2 | |
| MiniGPT-4 Vicuna 13BEvaluation Protocol=Zero-shot, Model Type=Large Multi-Modal Model2023.10 | 11.7 | 27.7 | 35.5 | — | 19.8 | |
| MiniGPT-4 Vicuna 7BEvaluation Protocol=Zero-shot, Model Type=Large Multi-Modal Model2023.10 | 11.6 | 26.5 | 34 | — | 19.1 | |
| ViLT2023.05 | 11.5 | 25.6 | 33.8 | 71 | — | |
| ViLTEvaluation Protocol=Fine-tuned2023.10 | 11.5 | 25.6 | 33.8 | — | — | |
| LLaVA v1.5 7BEvaluation Protocol=Zero-shot, Model Type=Large Multi-Modal Model2023.10 | 11.1 | 26.5 | 33.3 | — | 18.8 | |
| SCAN2023.05 | 10.4 | 27 | 37.1 | 74.5 | — | |
| SCANEvaluation Protocol=Fine-tuned2023.10 | 10.4 | 27 | 37.1 | — | — | |
| VSE++2023.05 | 10.2 | 25.4 | 34.2 | 69.8 | — | |
| VSE++Evaluation Protocol=Fine-tuned2023.10 | 10.2 | 25.4 | 34.2 | — | — | |
| DE*2023.05 | 9 | 26.4 | 35.7 | 71.1 | — | |
| DEEvaluation Protocol=Fine-tuned2023.10 | 9 | 26.4 | 35.7 | — | — | |
| BM252023.05 | 6.6 | 15.4 | 23 | 45 | — | |
| BM25Evaluation Protocol=Fine-tuned2023.10 | 6.6 | 15.4 | 23 | — | — |