Visual Machine Reading Comprehension on VisualMRC (test)
364.2CIDErLayoutT5
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| LayoutT5parameters=770M, uses OCR engine=true2024.11 | 364.2 | — | — | — | — | |
| LayoutT5Evaluation Setting=end-to-end, ROI Detection=with2021.01 | 317.9 | 41 | 33.2 | 52.2 | 90.3 | |
| T5-TextEvaluation Setting=end-to-end, ROI Detection=with2021.01 | 297.6 | 38.6 | 29.8 | 50.2 | 90 | |
| LayoutBARTEvaluation Setting=end-to-end, ROI Detection=with2021.01 | 293.9 | 36.4 | 30.5 | 50.5 | 90.4 | |
| LayoutT5Evaluation Setting=end-to-end, ROI Detection=without2021.01 | 292.8 | 39.1 | 31 | 49.3 | 89.8 | |
| T5-TextEvaluation Setting=end-to-end, ROI Detection=without2021.01 | 284.3 | 37.5 | 28.8 | 48.6 | 89.5 | |
| LayoutBARTEvaluation Setting=end-to-end, ROI Detection=without2021.01 | 277.3 | 33.8 | 29.6 | 48.6 | 90 | |
| BART-TextEvaluation Setting=end-to-end, ROI Detection=with2021.01 | 265.6 | 34.6 | 27.5 | 47.3 | 90 | |
| BART-TextEvaluation Setting=end-to-end, ROI Detection=without2021.01 | 258.6 | 33.2 | 27.2 | 46.7 | 89.7 | |
| BLIP-2-OPT-2.7B + OursModel Params=3.8B, Trainable Params=14M, Pre-training Data=129M, Fine-tuning Data=650K2024.11 | 228.7 | — | — | — | — | |
| HVFAbackbone=mPLUG-Owl, fine-tuning dataset=650K2024.11 | 226.4 | — | — | — | — | |
| mPLUG-Owl-7B + OursModel Params=7.2B, Trainable Params=96M, Pre-training Data=1.1B, Fine-tuning Data=650K2024.11 | 226.4 | — | — | — | — | |
| mPLUG-Owl-7B + UReaderModel Params=7.2B, Trainable Params=86M, Pre-training Data=1.1B, Fine-tuning Data=650K2024.11 | 221.7 | — | — | — | — | |
| BLIP-2-OPT-2.7B + UReaderModel Params=3.8B, Trainable Params=8M, Pre-training Data=129M, Fine-tuning Data=650K2024.11 | 214.3 | — | — | — | — | |
| M4CEvaluation Setting=end-to-end, ROI Detection=with2021.01 | 97.6 | 10.2 | 12.7 | 28 | 86.1 | |
| DonutModel Params=143M, Trainable Params=143M, Pre-training Data=13M, Fine-tuning Data=Task-specific2024.11 | 93.91 | — | — | — | — | |
| Qwen-VLModel Params=9.6B, Trainable Params=207M, Pre-training Data=1.4B+76.8M, Fine-tuning Data=Task-specific2024.11 | 65.7 | — | — | — | — | |
| MonkeyModel Params=9.8B, Trainable Params=207M, Pre-training Data=1.4B+76.8M, Fine-tuning Data=1.44M2024.11 | 65.1 | — | — | — | — |