Chat-based Image Retrieval on VisDial (val)
29.53Hits@1 (R1)PlugIR
Evaluation results
| Method | Links | ||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| PlugIRVLM=CLIP-ViT-L/14, LLM=LLaMA3-8B2025.03 | 29.53 | 33.62 | 35.9 | 39.24 | 41.28 | 43.07 | 44.33 | 45.16 | 45.98 | 47.24 | 56.23 | 60.69 | 62.94 | 65.79 | 67.64 | 69.53 | 70.78 | 71.85 | 72.67 | 73.84 | |
| Zero-shot CLIPVLM=CLIP-ViT-L/14, Protocol=Zero-shot2025.03 | 29.51 | 33.14 | 35.32 | 36.87 | 38.13 | 39.24 | 40.16 | 40.36 | 40.6 | 40.6 | 55.91 | 60.51 | 62.98 | 64.78 | 66.18 | 67.01 | 67.73 | 68.12 | 68.22 | 68.27 | |
| ImageScopeVLM=CLIP-ViT-L/142025.03 | 26.74 | 35.8 | 42.1 | 47.04 | 49.66 | 52.71 | 55.14 | 56.49 | 58.28 | 59.4 | 57.32 | 63.23 | 65.21 | 69.04 | 71.9 | 73.69 | 75.87 | 77.03 | 78.05 | 79.89 | |
| PlugIRVLM=CLIP-ViT-B/32, LLM=LLaMA3-8B2025.03 | 22.75 | 25.55 | 27.7 | 30.72 | 32.8 | 34.54 | 36.05 | 37.55 | 38.37 | 39.49 | 50.64 | 55.57 | 58.72 | 61.29 | 63.32 | 65.07 | 66.42 | 67.64 | 68.6 | 69.82 | |
| ImageScopeVLM=CLIP-ViT-B/322025.03 | 22.67 | 31.54 | 36.72 | 40.5 | 44.04 | 47.53 | 49.76 | 51.45 | 53.54 | 55.18 | 53.44 | 60.9 | 64.73 | 67.78 | 69.91 | 72.48 | 73.74 | 74.95 | 75.97 | 77.03 | |
| Zero-shot CLIPVLM=CLIP-ViT-B/32, Protocol=Zero-shot2025.03 | 22.53 | 26.55 | 28.73 | 29.84 | 31.49 | 32.41 | 33.33 | 33.62 | 33.72 | 33.77 | 50.53 | 55.04 | 57.61 | 59.11 | 60.85 | 62.11 | 62.79 | 63.18 | 63.32 | 63.37 |