Multi-modal knowledge base retrieval on ReMuQ (test)
96.3R@5VISTA
Evaluation results
| Method | Links | ||
|---|---|---|---|
| VISTAEvaluation Protocol=Supervised fine-tuning, learning rate=2e-5, batch size=1920, training steps=2002024.06 | 96.3 | 97.3 | |
| VISTAMode=Fine-tuning2024.11 | 96.3 | 97.3 | |
| MIReMode=Fine-tuning2024.11 | 94.4 | 96.2 | |
| MIReMode=Fine-tuning, Backbone=ViT-large2024.11 | 94.38 | 96.18 | |
| MIReMode=Fine-tuning, Pre-training=None2024.11 | 92.44 | 94.38 | |
| GeMKREvaluation Protocol=Supervised fine-tuning2024.06 | 90.3 | 92.7 | |
| GeMKRMode=Fine-tuning2024.11 | 90.3 | 92.7 | |
| ReViz-ICTEvaluation Protocol=Supervised fine-tuning2024.06 | 76.2 | 83.3 | |
| SEALEvaluation Protocol=Supervised fine-tuning2024.06 | 66.4 | 74.1 | |
| FLMRMode=Fine-tuning2024.11 | 62.76 | 74.67 | |
| ReVizEvaluation Protocol=Supervised fine-tuning2024.06 | 62.4 | 71.6 | |
| DPREvaluation Protocol=Supervised fine-tuning2024.06 | 43.4 | 48.8 | |
| ReVizMode=Fine-tuning2024.11 | 23.61 | 39.43 | |
| BM25Evaluation Protocol=Supervised fine-tuning2024.06 | 8.8 | 10.8 |