Text-to-Image Retrieval on PhotoChat (test)
44.17R@1Desc - Summary
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Desc - SummaryLLM=LLaMA-2-13B-Chat, Evaluation Setting=Fully-Trained2024.07 | 44.17 | 64.23 | 72.66 | 60.35 | |
| Desc - QueriesLLM=LLaMA-2-13B-Chat, Evaluation Setting=Fully-Trained2024.07 | 44 | 64.78 | 73.95 | 60.91 | |
| Desc - GuessingLLM=LLaMA-2-13B-Chat, Evaluation Setting=Fully-Trained2024.07 | 43.18 | 65.45 | 73.43 | 60.69 | |
| Desc - SummaryLLM=LLaMA-2-7B-Chat, Evaluation Setting=Fully-Trained2024.07 | 42.81 | 62.42 | 71.35 | 58.86 | |
| Desc - GuessingLLM=LLaMA-2-7B-Chat, Evaluation Setting=Fully-Trained2024.07 | 42.55 | 64.22 | 72.29 | 59.69 | |
| Desc - DiagEvaluation Setting=Fully-Trained2024.07 | 40.35 | 58.77 | 66.88 | 55.33 | |
| Desc - QueriesLLM=LLaMA-2-7B-Chat, Evaluation Setting=Fully-Trained2024.07 | 37.34 | 57.52 | 66.62 | 53.83 | |
| Desc - GuessingLLM=LLaMA-2-13B-Chat, Evaluation Setting=Zero-Shot2024.07 | 29.3 | 51.3 | 59.8 | 46.8 | |
| Desc - GuessingLLM=LLaMA-2-7B-Chat, Evaluation Setting=Zero-Shot2024.07 | 27.6 | 47.8 | 58.1 | 44.5 | |
| Desc - QueriesLLM=LLaMA-2-13B-Chat, Evaluation Setting=Zero-Shot2024.07 | 26.4 | 45.8 | 55.1 | 42.43 | |
| Desc - SummaryLLM=LLaMA-2-13B-Chat, Evaluation Setting=Zero-Shot2024.07 | 24.4 | 40.5 | 48.3 | 37.73 | |
| Desc - SummaryLLM=LLaMA-2-7B-Chat, Evaluation Setting=Zero-Shot2024.07 | 22.9 | 40.1 | 47.6 | 36.87 | |
| Desc - QueriesLLM=LLaMA-2-7B-Chat, Evaluation Setting=Zero-Shot2024.07 | 22.6 | 42.2 | 50.4 | 38.4 | |
| Desc - CaptionLLM=BLIP-2, Evaluation Setting=Zero-Shot2024.07 | 16.68 | 35.34 | 45.17 | 32.4 | |
| Desc - DiagEvaluation Setting=Zero-Shot2024.07 | 16 | 30.9 | 37.7 | 28.2 | |
| DE - Diag (CLIP)Evaluation Setting=Fully-Trained2024.07 | 14.76 | 35.78 | 47.12 | 32.55 | |
| DE - Diag (BERT)Evaluation Setting=Fully-Trained2024.07 | 12.88 | 35.13 | 47.75 | 31.92 | |
| SCANEvaluation Setting=Fully-Trained2024.07 | 10.4 | 27 | 37.1 | 24.83 | |
| VSE++Evaluation Setting=Fully-Trained2024.07 | 10.2 | 25.4 | 34.2 | 23.27 |