Retrieval on HotelMatch Vision-driven (test)
0.247MRR@10HotelMatch-LLM
Evaluation results
| Method | Links | ||
|---|---|---|---|
| HotelMatch-LLMModality Setting=Multimodal2025.06 | 0.247 | 0.362 | |
| MARVELModality Setting=Multimodal, Evaluation Protocol=Fine-tuned2025.06 | 0.219 | 0.326 | |
| VISTAModality Setting=Multimodal, Evaluation Protocol=Fine-tuned2025.06 | 0.216 | 0.321 | |
| CLIPModality Setting=Multimodal, Evaluation Protocol=Zero-Shot2025.06 | 0.172 | 0.254 | |
| HotelMatch-LLM w/o visionModality Setting=Text-only ablation2025.06 | 0.154 | 0.239 | |
| GTR-largeModality Setting=Text-only, Evaluation Protocol=Zero-Shot2025.06 | 0.148 | 0.224 | |
| GTR-baseModality Setting=Text-only, Evaluation Protocol=Zero-Shot2025.06 | 0.142 | 0.219 | |
| CLIP-TextModality Setting=Text-only, Evaluation Protocol=Zero-Shot2025.06 | 0.14 | 0.197 | |
| BM25Modality Setting=Text-only2025.06 | 0.138 | 0.195 |