PDF Multimodal Understanding on PaperPDF Single-Evidence Subset
41.5ANLSPDF-WuKong
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| PDF-WuKong# param=8.5B, input format=parsed content as input2024.10 | 41.5 | 42.8 | 39.8 | |
| PDF-WuKong# param=8.5B, input format=page image input2024.10 | 36.6 | 35.2 | 31.7 | |
| MiniCPM-2.5# param=8.5B2024.10 | 31.8 | 28.2 | 24.8 | |
| Monkey# param=9.8B2024.10 | 30 | 24.4 | 22.3 | |
| Qwen-VL# param=9.6B2024.10 | 26.4 | 19.6 | 18.3 | |
| Emu2-Chat# param=37B2024.10 | 26 | 24.4 | 23.4 | |
| CogVLM2# param=17B2024.10 | 24.8 | 27.4 | 26.3 | |
| IXC2-4KHD# param=8B2024.10 | 24.5 | 20 | 18 | |
| IXC2-VL# param=8B2024.10 | 23.4 | 20.8 | 21.3 | |
| mPLUG-Owl2# param=8.2B2024.10 | 19.5 | 20.3 | 22.7 |