Visual Document Question Answering on REAL-MM-RAG
55.45Accuracy (Financial Report)Oracle
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| OracleTop-k retrieval=3, Input modality=I + T, Judge model=Qwen2.5-7B-Instruct, Backbone Vision Language Model=Qwen2.5-VL-7B-Instruct2025.09 | 55.45 | 70.72 | 73.42 | 76.14 | |
| OracleTop-k retrieval=3, Input modality=I, Judge model=Qwen2.5-7B-Instruct, Backbone Vision Language Model=Qwen2.5-VL-7B-Instruct2025.09 | 53.11 | 74.52 | 71.72 | 75.48 | |
| CMRAGTop-k retrieval=3, Judge model=Qwen2.5-7B-Instruct, Backbone Vision Language Model=Qwen2.5-VL-7B-Instruct2025.09 | 32.13 | 60.46 | 45.6 | 64.18 | |
| SigLIPTop-k retrieval=3, Judge model=Qwen2.5-7B-Instruct, Backbone Vision Language Model=Qwen2.5-VL-7B-Instruct2025.09 | 29.54 | 55.99 | 41.65 | 62.26 | |
| CLIP-L/14-336Top-k retrieval=3, Judge model=Qwen2.5-7B-Instruct, Backbone Vision Language Model=Qwen2.5-VL-7B-Instruct2025.09 | 27.9 | 54.94 | 35.47 | 57.09 |