Retrieval on HotelMatch Real-world (test)
0.681MRR@10HotelMatch-LLM
Evaluation results
| Method | Links | ||
|---|---|---|---|
| HotelMatch-LLMModality Setting=Multimodal2025.06 | 0.681 | 0.6 | |
| MARVELModality Setting=Multimodal, Evaluation Protocol=Fine-tuned2025.06 | 0.603 | 0.503 | |
| HotelMatch-LLM w/o visionModality Setting=Text-only ablation2025.06 | 0.595 | 0.482 | |
| VISTAModality Setting=Multimodal, Evaluation Protocol=Fine-tuned2025.06 | 0.582 | 0.465 | |
| GTR-baseModality Setting=Text-only, Evaluation Protocol=Zero-Shot2025.06 | 0.547 | 0.426 | |
| GTR-largeModality Setting=Text-only, Evaluation Protocol=Zero-Shot2025.06 | 0.545 | 0.429 | |
| BM25Modality Setting=Text-only2025.06 | 0.506 | 0.401 | |
| CLIPModality Setting=Multimodal, Evaluation Protocol=Zero-Shot2025.06 | 0.46 | 0.402 | |
| CLIP-TextModality Setting=Text-only, Evaluation Protocol=Zero-Shot2025.06 | 0.452 | 0.381 |