Document Visual Question Answering on DUDE
60ANLSGPT-4o+Ours (Spot-IT)
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| GPT-4o+Ours (Spot-IT)zero-shot=true2025.08 | 60 | 45 | 60 | |
| Claude-3.7-SonnetSize=-, Paradigm=End-to-End2025.12 | 58.1 | — | — | |
| GPT-4o+CoTzero-shot=true2025.08 | 58 | 43 | 57 | |
| GPT-4ozero-shot=true2025.08 | 55 | 42 | 56 | |
| GPT-4oSize=-, Paradigm=End-to-End2025.12 | 52.7 | — | — | |
| GPT-4o-mini+Ours (Spot-IT)zero-shot=true2025.08 | 52 | 41 | 55 | |
| GPT-4.1Size=-, Paradigm=End-to-End2025.12 | 50.2 | — | — | |
| GPT-4o-minizero-shot=true2025.08 | 47 | 34 | 50 | |
| GPT-4o+OCRzero-shot=true2025.08 | 47 | 34 | 47 | |
| GPT-4o miniSize=-, Paradigm=End-to-End2025.12 | 46.5 | — | — | |
| CogDocSize=7B, Paradigm=Unified Thinking2025.12 | 46.2 | — | — | |
| mPLUG-DocOwl2Size=8B, Paradigm=End-to-End2025.12 | 45.6 | — | — | |
| Gem-1.5-Flash+Ours (Spot-IT)zero-shot=true2025.08 | 45 | 34 | 47 | |
| SV-RAGSize=8B, Paradigm=RAG2025.12 | 45 | — | — | |
| Qwen2.5-VL-72B-InstructSize=72B, Paradigm=End-to-End2025.12 | 44.5 | — | — | |
| VDocRAGSize=8B, Paradigm=RAG2025.12 | 44 | — | — | |
| InternVL3-8BSize=8B, Paradigm=End-to-End2025.12 | 43.4 | — | — | |
| MiMo-VL-7B-SFTSize=7B, Paradigm=End-to-End2025.12 | 43.1 | — | — | |
| VisRAGSize=8B, Paradigm=RAG2025.12 | 43.1 | — | — | |
| FRAGSize=8B, Paradigm=RAG2025.12 | 43.1 | — | — | |
| Gem-1.5-Flashzero-shot=true2025.08 | 42 | 30 | 42 | |
| Qwen2.5-VL-7B-InstructSize=7B, Paradigm=End-to-End2025.12 | 41.8 | — | — | |
| DocopilotSize=8B, Paradigm=End-to-End2025.12 | 40.7 | — | — | |
| M3DocRAGSize=10B, Paradigm=RAG2025.12 | 39.5 | — | — | |
| Qwen2-7B+Ours (Spot-IT)zero-shot=true2025.08 | 32 | 27 | 37 | |
| Qwen2-7Bzero-shot=true2025.08 | 28 | 21 | 32 | |
| Llama-3.2+Ours (Spot-IT)zero-shot=true2025.08 | 24 | 19 | 29 | |
| Llama-3.2-VL-11Bzero-shot=true2025.08 | 18 | 13 | 23 | |
| Llama-3.2+CoTzero-shot=true2025.08 | 17 | 11 | 23 | |
| Llama-3.2+OCRzero-shot=true2025.08 | 8 | 5 | 19 |