Multimodal Optical Character Recognition on OCRBench
274Recognition ScoreH2OVL-Mississippi-0.8B
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| H2OVL-Mississippi-0.8BParams (B)=0.8, Language Model=H2O-Danube3 0.5B, Vision Model=InternViT-300M2024.10 | 274 | 162 | 112 | 152 | — | 751 | 51 | |
| Qwen2-VL-2B-InstructParams (B)=2.1, Language Model=Qwen2-1.5B, Vision Model=ViT-600M2024.10 | 265 | 172 | 146 | 174 | — | 812 | 55 | |
| MiniCPM-V2.6Params (B)=8, Language Model=Qwen2-7B, Vision Model=SigLIP-400M2024.10 | 259 | 184 | 169 | 182 | — | 836 | 42 | |
| H2OVL-Mississippi-2BParams (B)=2, Language Model=H2O-Danube2 1.8B, Vision Model=InternViT-300M2024.10 | 252 | 171 | 140 | 166 | — | 782 | 53 | |
| InternVL2-26BParams (B)=26, Language Model=internlm2-chat-20b, Vision Model=InternViT-6B2024.10 | 251 | 184 | 153 | 168 | — | 823 | 67 | |
| MiniMonkeyParams (B)=2.2, Language Model=InternLM2-1.8B, Vision Model=InternViT-300M2024.10 | 250 | 178 | 126 | 171 | — | 792 | 67 | |
| InternVL2-2BParams (B)=2, Language Model=InternLM2-1.8B, Vision Model=InternViT-300M2024.10 | 246 | 170 | 133 | 167 | — | 785 | 69 | |
| GOT-OCR2.0Params (B)=0.6, Language Model=Qwen2-0.5B, Vision Model=VitDet-80M2024.10 | 245 | 99 | 83 | 164 | — | 622 | 31 | |
| MiniCPM-V-2Params (B)=2.8, Language Model=MiniCPM-2.4B, Vision Model=SigLIP-400M2024.10 | 243 | 168 | 100 | 85 | — | 596 | 0 | |
| InternVL2-1BParams (B)=0.8, Language Model=Qwen2-0.5B, Vision Model=InternViT-300M2024.10 | 242 | 164 | 127 | 150 | — | 755 | 72 | |
| PaliGemma-3B-mix-448Params (B)=2.9, Language Model=Gemma-2B, Vision Model=SigLIP-400M2024.10 | 242 | 165 | 88 | 118 | — | 613 | 0 | |
| InternVL2-4BParams (B)=4, Language Model=Phi-3, Vision Model=InternViT-300M2024.10 | 236 | 170 | 154 | 158 | — | 785 | 67 | |
| InternVL-Chat-V1.5Params (B)=26, Language Model=InternLM2-20B, Vision Model=InternViT-6B2024.10 | 236 | 181 | 149 | 153 | — | 722 | 3 | |
| Mini-InternVL-Chat-2B-V1.5Params (B)=2, Language Model=InternLM2-1.8B, Vision Model=InternViT-300M2024.10 | 222 | 161 | 126 | 139 | — | 652 | 4 | |
| MiniCPM-Llama3-V2.5Params (B)=8, Language Model=Llama-3-8B-Instruct, Vision Model=SigLIP-400M2024.10 | 221 | 171 | 125 | 155 | — | 725 | 53 | |
| Geminimode=zero-shot2024.06 | 215 | 174 | 128 | 134 | 651 | — | — | |
| LaRAmode=zero-shot2024.06 | 211 | 147 | 85 | 105 | 548 | — | — | |
| Phi-3-VisionParams (B)=4.2, Language Model=Phi-3, Vision Model=CLIP VIT-L/142024.10 | 196 | 159 | 137 | 148 | — | 640 | 0 | |
| Mini-InternVL-Chat-4B-V1.5Params (B)=4, Language Model=Phi-3, Vision Model=InternViT-300M2024.10 | 193 | 160 | 146 | 135 | — | 640 | 6 | |
| LLaVARmode=zero-shot2024.06 | 186 | 122 | 25 | 13 | 346 | — | — | |
| LLaVA-1.5Model Scale=13B2024.10 | 176 | 129 | 19 | 7 | 331 | — | 0 | |
| LLaVA1.5-13Bmode=zero-shot2024.06 | 176 | 129 | 19 | 7 | 331 | — | — | |
| Monkeymode=zero-shot2024.06 | 174 | 161 | 91 | 88 | 514 | — | — | |
| mPLUG-Owlmode=zero-shot2024.06 | 172 | 104 | 18 | 3 | 297 | — | — | |
| MMFuserBase Model=LLaVA-1.5, Model Scale=13B2024.10 | 171 | 136 | 25 | 11 | 343 | — | 0 | |
| InstructBLIP2024.10 | 168 | 93 | 14 | 1 | 276 | — | 0 | |
| GPT-4vmode=zero-shot2024.06 | 167 | 163 | 146 | 160 | 636 | — | — | |
| BLIVA2024.10 | 165 | 103 | 22 | 1 | 291 | — | 0 | |
| LLaVA-1.5Model Scale=7B2024.10 | 160 | 117 | 15 | 5 | 297 | — | 0 | |
| MMFuserBase Model=LLaVA-1.5, Model Scale=7B2024.10 | 159 | 128 | 20 | 8 | 315 | — | 0 | |
| BLIP22024.10 | 154 | 71 | 10 | 0 | 235 | — | 0 | |
| mPLUG-Owl2mode=zero-shot2024.06 | 153 | 153 | 41 | 19 | 366 | — | — | |
| MiniGPT4V22024.10 | 124 | 29 | 4 | 0 | 157 | — | 0 | |
| MiniGPT-V2mode=zero-shot2024.06 | 124 | 29 | 4 | 0 | 157 | — | — | |
| MASQuantModel=Qwen2.5-VL-7B, Bits=W4A162026.03 | 84.6 | — | — | — | — | — | — | |
| MASQuantModel=Qwen2.5-VL-7B, Bits=W8A82026.03 | 84.2 | — | — | — | — | — | — | |
| DenseModel=Qwen2.5-VL-7B, Bits=FP162026.03 | 83.8 | — | — | — | — | — | — | |
| RTNModel=Qwen2.5-VL-7B, Bits=W8A82026.03 | 83.8 | — | — | — | — | — | — | |
| SQModel=Qwen2.5-VL-7B, Bits=W8A82026.03 | 83.8 | — | — | — | — | — | — | |
| RTNModel=Qwen2.5-VL-7B, Bits=W4A162026.03 | 83.7 | — | — | — | — | — | — | |
| AWQModel=Qwen2.5-VL-7B, Bits=W4A162026.03 | 83.7 | — | — | — | — | — | — | |
| MBQModel=Qwen2.5-VL-7B, Bits=W8A82026.03 | 83.5 | — | — | — | — | — | — | |
| MBQModel=Qwen2.5-VL-7B, Bits=W4A162026.03 | 82.8 | — | — | — | — | — | — | |
| MASQuantModel=Qwen2.5-VL-3B, Bits=W8A82026.03 | 79.5 | — | — | — | — | — | — | |
| DenseModel=Qwen2.5-VL-3B, Bits=FP162026.03 | 79.3 | — | — | — | — | — | — | |
| SQModel=Qwen2.5-VL-3B, Bits=W8A82026.03 | 79 | — | — | — | — | — | — | |
| MBQModel=Qwen2.5-VL-3B, Bits=W8A82026.03 | 78.9 | — | — | — | — | — | — | |
| RTNModel=Qwen2.5-VL-3B, Bits=W8A82026.03 | 78.7 | — | — | — | — | — | — | |
| AWQModel=Qwen2.5-VL-3B, Bits=W4A162026.03 | 78.6 | — | — | — | — | — | — | |
| MASQuantModel=Qwen2.5-VL-3B, Bits=W4A162026.03 | 78.6 | — | — | — | — | — | — | |
| MBQModel=Qwen2.5-VL-3B, Bits=W4A162026.03 | 78.5 | — | — | — | — | — | — | |
| RTNModel=Qwen2.5-VL-3B, Bits=W4A162026.03 | 77.9 | — | — | — | — | — | — | |
| MBQModel=Qwen2.5-VL-7B, Bits=W4A82026.03 | 74.1 | — | — | — | — | — | — | |
| MASQuantModel=Qwen2.5-VL-7B, Bits=W4A82026.03 | 72.8 | — | — | — | — | — | — | |
| MBQModel=Qwen2.5-VL-7B, Bits=W4A62026.03 | 71.7 | — | — | — | — | — | — | |
| MASQuantModel=Qwen2.5-VL-7B, Bits=W4A62026.03 | 70.3 | — | — | — | — | — | — | |
| SQModel=Qwen2.5-VL-7B, Bits=W4A82026.03 | 70.2 | — | — | — | — | — | — | |
| RTNModel=Qwen2.5-VL-7B, Bits=W4A82026.03 | 68.3 | — | — | — | — | — | — | |
| SQModel=Qwen2.5-VL-7B, Bits=W4A62026.03 | 67.7 | — | — | — | — | — | — | |
| MASQuantModel=Qwen2.5-VL-3B, Bits=W4A82026.03 | 67.2 | — | — | — | — | — | — | |
| SQModel=Qwen2.5-VL-3B, Bits=W4A82026.03 | 66.9 | — | — | — | — | — | — | |
| MBQModel=Qwen2.5-VL-3B, Bits=W4A82026.03 | 66.9 | — | — | — | — | — | — | |
| SQModel=Qwen2.5-VL-3B, Bits=W4A62026.03 | 66.7 | — | — | — | — | — | — | |
| MASQuantModel=Qwen2.5-VL-3B, Bits=W4A62026.03 | 66.7 | — | — | — | — | — | — | |
| MBQModel=Qwen2.5-VL-3B, Bits=W4A62026.03 | 65.6 | — | — | — | — | — | — | |
| RTNModel=Qwen2.5-VL-3B, Bits=W4A82026.03 | 0 | — | — | — | — | — | — | |
| DocTR-defaultParams (B)=0.052024.10 | — | — | — | — | — | 177 | — | |
| DocTR-V2M(H2O.AI)Params (B)=0.052024.10 | — | — | — | — | — | 256 | — |