Document Information Extraction on VRDU Ad-buy Form Mixed Template
79.82Micro-F1LMDXPaLM 2-S
Evaluation results
| Method | Links | ||
|---|---|---|---|
| LMDXPaLM 2-S|D| (Training Data Size)=200, Modality=T+L, Localized?=true2023.09 | 79.82 | 72.09 | |
| LMDXPaLM 2-S|D| (Training Data Size)=100, Modality=T+L, Localized?=true2023.09 | 78.05 | 69.77 | |
| LMDXPaLM 2-S|D| (Training Data Size)=50, Modality=T+L, Localized?=true2023.09 | 75.08 | 65.42 | |
| LMDXPaLM 2-S|D| (Training Data Size)=10, Modality=T+L, Localized?=true2023.09 | 54.35 | 39.35 | |
| LayoutLMv2|D| (Training Data Size)=200, Modality=T+L+I, Localized?=true2023.09 | 46.54 | 25.46 | |
| LayoutLMv3|D| (Training Data Size)=200, Modality=T+L+I, Localized?=true2023.09 | 45.16 | 24.51 | |
| LayoutLMv2|D| (Training Data Size)=100, Modality=T+L+I, Localized?=true2023.09 | 44.97 | 23.52 | |
| LayoutLM|D| (Training Data Size)=200, Modality=T+L+I, Localized?=true2023.09 | 44.66 | 23.9 | |
| FormNet|D| (Training Data Size)=200, Modality=T+L, Localized?=true2023.09 | 43.23 | 21.86 | |
| LayoutLMv3|D| (Training Data Size)=100, Modality=T+L+I, Localized?=true2023.09 | 42.63 | 22.08 | |
| LayoutLM|D| (Training Data Size)=100, Modality=T+L+I, Localized?=true2023.09 | 42.38 | 21.26 | |
| LayoutLMv2|D| (Training Data Size)=50, Modality=T+L+I, Localized?=true2023.09 | 42.23 | 20.98 | |
| FormNet|D| (Training Data Size)=50, Modality=T+L, Localized?=true2023.09 | 40.68 | 19.06 | |
| FormNet|D| (Training Data Size)=100, Modality=T+L, Localized?=true2023.09 | 40.38 | 18.8 | |
| LayoutLM|D| (Training Data Size)=50, Modality=T+L+I, Localized?=true2023.09 | 39.76 | 19.5 | |
| LMDXPaLM 2-S|D| (Training Data Size)=0, Modality=T+L, Localized?=true2023.09 | 39.74 | 21.21 | |
| LayoutLMv3|D| (Training Data Size)=50, Modality=T+L+I, Localized?=true2023.09 | 39.49 | 19.53 | |
| LMDXGemini Pro|D| (Training Data Size)=0, Modality=T+L, Localized?=true2023.09 | 38.02 | 23.29 | |
| Gemini Pro+OCR|D| (Training Data Size)=0, Modality=T, Localized?=false2023.09 | 34.46 | 19.25 | |
| GPT-4V+Image|D| (Training Data Size)=0, Modality=I, Localized?=false2023.09 | 31.95 | 4.45 | |
| PaLM 2-S+OCR|D| (Training Data Size)=0, Modality=T, Localized?=false2023.09 | 30.24 | 9.86 | |
| GPT-3.5+OCR|D| (Training Data Size)=0, Modality=T, Localized?=false2023.09 | 30.05 | 7.65 | |
| LayoutLMv2|D| (Training Data Size)=10, Modality=T+L+I, Localized?=true2023.09 | 25.36 | 9.96 | |
| FormNet|D| (Training Data Size)=10, Modality=T+L, Localized?=true2023.09 | 20.47 | 5.72 | |
| LayoutLM|D| (Training Data Size)=10, Modality=T+L+I, Localized?=true2023.09 | 20.2 | 6.95 | |
| Gemini Pro+Image|D| (Training Data Size)=0, Modality=I, Localized?=false2023.09 | 15.38 | 0.91 | |
| LayoutLMv3|D| (Training Data Size)=10, Modality=T+L+I, Localized?=true2023.09 | 10.16 | 5.92 | |
| LLaVA-v1.5-13B+Image|D| (Training Data Size)=0, Modality=I, Localized?=false2023.09 | 0.34 | 0 |