Retrieval on HotelMatch Out-of-distribution (test)
70.4MRR@10HotelMatch-LLM
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| HotelMatch-LLMModality Setting=Multimodal2025.06 | 70.4 | 55.8 | |
| VISTAModality Setting=Multimodal, Evaluation Protocol=Fine-tuned2025.06 | 66.2 | 51.3 | |
| MARVELModality Setting=Multimodal, Evaluation Protocol=Fine-tuned2025.06 | 66 | 51.5 | |
| HotelMatch-LLM w/o visionModality Setting=Text-only ablation2025.06 | 65.8 | 53.5 | |
| GTR-largeModality Setting=Text-only, Evaluation Protocol=Zero-Shot2025.06 | 65.6 | 53.4 | |
| GTR-baseModality Setting=Text-only, Evaluation Protocol=Zero-Shot2025.06 | 65 | 52.1 | |
| BM25Modality Setting=Text-only2025.06 | 58.8 | 48.9 | |
| CLIPModality Setting=Multimodal, Evaluation Protocol=Zero-Shot2025.06 | 56.1 | 43.9 | |
| CLIP-TextModality Setting=Text-only, Evaluation Protocol=Zero-Shot2025.06 | 55.9 | 42.8 |