OCR-related Understanding Tasks on TextVQA (val)
86.62AccuracyMIRROR (ours)
Evaluation results
| Method | Links | |
|---|---|---|
| MIRROR (ours)Param Size=7B2026.02 | 86.62 | |
| Qwen2.5-VL-7B AIFmethod=Adaptive Information Flow2026.04 | 86 | |
| MIRROR (w/o tool)Param Size=7B2026.02 | 85.37 | |
| Qwen2.5-VL-7BModel Size=7B2025.02 | 84.9 | |
| Qwen2.5-VL-7BParam Size=7B2026.02 | 84.9 | |
| Qwen2.5-VL-7B2026.04 | 84.9 | |
| Qwen2-VLModel Size=8B2025.01 | 83.8 | |
| Qwen2.5-VL-72BModel Size=72B2025.02 | 83.5 | |
| Qwen2.5-VL-72B2026.04 | 83.5 | |
| InternVL2.5-78BModel Size=78B2025.02 | 83.4 | |
| Iso-CMethod Class=Merging2025.05 | 83.33 | |
| DeepSeekVL-2Model Size=8B2025.01 | 83.3 | |
| TSV MergingMethod Class=Merging2025.05 | 83.15 | |
| OptMergeMethod Class=Merging2025.05 | 82.97 | |
| TA w/ DAREMethod Class=Merging2025.05 | 82.93 | |
| TIES w/ DAREMethod Class=Merging2025.05 | 82.61 | |
| TIES MergingMethod Class=Merging2025.05 | 82.4 | |
| Task ArithmeticMethod Class=Merging2025.05 | 81.95 | |
| WUDI MergingMethod Class=Merging2025.05 | 80.78 | |
| InternVL3Param Size=8B2026.02 | 80.51 | |
| DeepSeekVL-2Model Size=2B2025.01 | 80.5 | |
| Qwen2.5-VL-3BModel Size=3B2025.02 | 79.3 | |
| InternVL2.5-MPOModel Size=8B2025.01 | 79.2 | |
| Qwen2.5-VL-3BParam Size=3B2026.02 | 79.12 | |
| Gemini 1.5 Pro2025.02 | 78.8 | |
| Qwen2-VLModel Size=2B2025.01 | 78.8 | |
| Qwen2-VL-7B-GRPO-8kCheckpoint Type=Fine-tuned2025.05 | 78.58 | |
| Qwen2-VL-InstructMethod Class=Baseline2025.05 | 78.38 | |
| SAIL-VLModel Size=8B2025.01 | 77.7 | |
| GPT-4o2025.02 | 77.4 | |
| GPT-4o2026.04 | 77.4 | |
| olmOCR-7B-0225-previewCheckpoint Type=Fine-tuned2025.05 | 77.24 | |
| InternVL2.5-MPOModel Size=2B2025.01 | 77.2 | |
| InternVL3Param Size=2B2026.02 | 77 | |
| Qwen2-VL-7B-PokemonCheckpoint Type=Fine-tuned2025.05 | 76.75 | |
| Claude-3.5 Sonnet2025.02 | 76.5 | |
| Claude-3.5 Sonnet2026.04 | 76.5 | |
| TA w/ DAREProtocol=full fine-tuning2025.05 | 76.3 | |
| TIES MergingProtocol=full fine-tuning2025.05 | 76.29 | |
| Task ArithmeticProtocol=full fine-tuning2025.05 | 76.26 | |
| TIES w/ DAREProtocol=full fine-tuning2025.05 | 76.19 | |
| LLaVA-OneVisionParam Size=7B2026.02 | 76.1 | |
| OptMergeProtocol=full fine-tuning2025.05 | 76.01 | |
| WUDI MergingProtocol=full fine-tuning2025.05 | 75.95 | |
| SAIL-VLModel Size=2B2025.01 | 75.7 | |
| TSV MergingProtocol=full fine-tuning2025.05 | 75.66 | |
| Weight AverageProtocol=full fine-tuning2025.05 | 74.54 | |
| Individual OCRProtocol=full fine-tuning2025.05 | 73 | |
| Mixture TrainingProtocol=full fine-tuning2025.05 | 72.96 | |
| InternVL2.5-InstructProtocol=full fine-tuning2025.05 | 72.51 | |
| EraX-VL-7B-V1.0Checkpoint Type=Fine-tuned2025.05 | 70.7 | |
| Iso-CProtocol=full fine-tuning2025.05 | 69.34 | |
| Individual VQAProtocol=full fine-tuning2025.05 | 63.68 | |
| Individual GroundingProtocol=full fine-tuning2025.05 | 58.39 | |
| Individual GeometryProtocol=full fine-tuning2025.05 | 56.91 | |
| Individual ChartProtocol=full fine-tuning2025.05 | 54.36 | |
| MRoPE-IPositional Encoding Method=MRoPE-I, DIPE Enhancement=+DIPE2026.03 | 52.34 | |
| Vanilla RoPEPositional Encoding Method=Vanilla RoPE, DIPE Enhancement=+DIPE2026.03 | 51.6 | |
| MRoPEPositional Encoding Method=MRoPE, DIPE Enhancement=+DIPE2026.03 | 50.03 | |
| LLaVA-1.5-7B AIFmethod=Adaptive Information Flow2026.04 | 49.9 | |
| MRoPE-IPositional Encoding Method=MRoPE-I, DIPE Enhancement=Base2026.03 | 49.75 | |
| Vanilla RoPEPositional Encoding Method=Vanilla RoPE, DIPE Enhancement=Base2026.03 | 48.17 | |
| LLaVA-1.5-7B2026.04 | 47.8 | |
| MRoPEPositional Encoding Method=MRoPE, DIPE Enhancement=Base2026.03 | 46.61 |