Document Information Extraction on VRDU Registration Form (Unseen)
90.22Micro-F1LMDXPaLM 2-S
Evaluation results
| Method | Links | |
|---|---|---|
| LMDXPaLM 2-S|D| (Training Data Size)=200, Modality=T+L, Localized?=true2023.09 | 90.22 | |
| LMDXPaLM 2-S|D| (Training Data Size)=100, Modality=T+L, Localized?=true2023.09 | 89.7 | |
| LMDXPaLM 2-S|D| (Training Data Size)=50, Modality=T+L, Localized?=true2023.09 | 88.43 | |
| LMDXPaLM 2-S|D| (Training Data Size)=10, Modality=T+L, Localized?=true2023.09 | 86.87 | |
| FormNet|D| (Training Data Size)=200, Modality=T+L, Localized?=true2023.09 | 77.29 | |
| LMDXGemini Pro|D| (Training Data Size)=0, Modality=T+L, Localized?=true2023.09 | 77.18 | |
| LMDXPaLM 2-S|D| (Training Data Size)=0, Modality=T+L, Localized?=true2023.09 | 74.94 | |
| Gemini Pro+OCR|D| (Training Data Size)=0, Modality=T, Localized?=false2023.09 | 73.66 | |
| FormNet|D| (Training Data Size)=100, Modality=T+L, Localized?=true2023.09 | 72.58 | |
| LayoutLMv2|D| (Training Data Size)=200, Modality=T+L+I, Localized?=true2023.09 | 72.03 | |
| LayoutLM|D| (Training Data Size)=200, Modality=T+L+I, Localized?=true2023.09 | 70.47 | |
| GPT-4V+Image|D| (Training Data Size)=0, Modality=I, Localized?=false2023.09 | 69.44 | |
| FormNet|D| (Training Data Size)=50, Modality=T+L, Localized?=true2023.09 | 68.29 | |
| GPT-3.5+OCR|D| (Training Data Size)=0, Modality=T, Localized?=false2023.09 | 67.49 | |
| LayoutLMv2|D| (Training Data Size)=100, Modality=T+L+I, Localized?=true2023.09 | 65.96 | |
| LayoutLM|D| (Training Data Size)=100, Modality=T+L+I, Localized?=true2023.09 | 63.68 | |
| PaLM 2-S+OCR|D| (Training Data Size)=0, Modality=T, Localized?=false2023.09 | 63.51 | |
| LayoutLMv3|D| (Training Data Size)=200, Modality=T+L+I, Localized?=true2023.09 | 62.58 | |
| LayoutLMv2|D| (Training Data Size)=50, Modality=T+L+I, Localized?=true2023.09 | 61.36 | |
| LayoutLMv3|D| (Training Data Size)=100, Modality=T+L+I, Localized?=true2023.09 | 57.69 | |
| LayoutLM|D| (Training Data Size)=50, Modality=T+L+I, Localized?=true2023.09 | 55.86 | |
| LayoutLMv2|D| (Training Data Size)=10, Modality=T+L+I, Localized?=true2023.09 | 54.21 | |
| Gemini Pro+Image|D| (Training Data Size)=0, Modality=I, Localized?=false2023.09 | 53.72 | |
| FormNet|D| (Training Data Size)=10, Modality=T+L, Localized?=true2023.09 | 50.53 | |
| LayoutLMv3|D| (Training Data Size)=50, Modality=T+L+I, Localized?=true2023.09 | 47.85 | |
| LayoutLM|D| (Training Data Size)=10, Modality=T+L+I, Localized?=true2023.09 | 25.54 | |
| LayoutLMv3|D| (Training Data Size)=10, Modality=T+L+I, Localized?=true2023.09 | 21.17 | |
| LLaVA-v1.5-13B+Image|D| (Training Data Size)=0, Modality=I, Localized?=false2023.09 | 5.05 |