Document Information Extraction on VRDU Ad-buy Form Unseen
0.7842Micro-F1LMDXPaLM 2-S
Evaluation results
| Method | Links | |
|---|---|---|
| LMDXPaLM 2-S|D| (Training Data Size)=200, Modality=T+L, Localized?=true2023.09 | 0.7842 | |
| LMDXPaLM 2-S|D| (Training Data Size)=100, Modality=T+L, Localized?=true2023.09 | 0.7599 | |
| LMDXPaLM 2-S|D| (Training Data Size)=50, Modality=T+L, Localized?=true2023.09 | 0.757 | |
| LMDXPaLM 2-S|D| (Training Data Size)=10, Modality=T+L, Localized?=true2023.09 | 0.5482 | |
| LayoutLMv2|D| (Training Data Size)=200, Modality=T+L+I, Localized?=true2023.09 | 0.4631 | |
| LayoutLMv3|D| (Training Data Size)=200, Modality=T+L+I, Localized?=true2023.09 | 0.4443 | |
| LayoutLMv2|D| (Training Data Size)=100, Modality=T+L+I, Localized?=true2023.09 | 0.4435 | |
| LayoutLM|D| (Training Data Size)=200, Modality=T+L+I, Localized?=true2023.09 | 0.4418 | |
| FormNet|D| (Training Data Size)=200, Modality=T+L, Localized?=true2023.09 | 0.4287 | |
| LayoutLMv2|D| (Training Data Size)=50, Modality=T+L+I, Localized?=true2023.09 | 0.4159 | |
| LayoutLMv3|D| (Training Data Size)=100, Modality=T+L+I, Localized?=true2023.09 | 0.4154 | |
| LayoutLM|D| (Training Data Size)=100, Modality=T+L+I, Localized?=true2023.09 | 0.4146 | |
| FormNet|D| (Training Data Size)=100, Modality=T+L, Localized?=true2023.09 | 0.3988 | |
| FormNet|D| (Training Data Size)=50, Modality=T+L, Localized?=true2023.09 | 0.3952 | |
| LMDXPaLM 2-S|D| (Training Data Size)=0, Modality=T+L, Localized?=true2023.09 | 0.3933 | |
| LayoutLMv3|D| (Training Data Size)=50, Modality=T+L+I, Localized?=true2023.09 | 0.3843 | |
| LayoutLM|D| (Training Data Size)=50, Modality=T+L+I, Localized?=true2023.09 | 0.3842 | |
| LMDXGemini Pro|D| (Training Data Size)=0, Modality=T+L, Localized?=true2023.09 | 0.3794 | |
| Gemini Pro+OCR|D| (Training Data Size)=0, Modality=T, Localized?=false2023.09 | 0.329 | |
| GPT-4V+Image|D| (Training Data Size)=0, Modality=I, Localized?=false2023.09 | 0.3184 | |
| GPT-3.5+OCR|D| (Training Data Size)=0, Modality=T, Localized?=false2023.09 | 0.2984 | |
| PaLM 2-S+OCR|D| (Training Data Size)=0, Modality=T, Localized?=false2023.09 | 0.297 | |
| LayoutLMv2|D| (Training Data Size)=10, Modality=T+L+I, Localized?=true2023.09 | 0.2517 | |
| FormNet|D| (Training Data Size)=10, Modality=T+L, Localized?=true2023.09 | 0.2028 | |
| LayoutLM|D| (Training Data Size)=10, Modality=T+L+I, Localized?=true2023.09 | 0.1992 | |
| Gemini Pro+Image|D| (Training Data Size)=0, Modality=I, Localized?=false2023.09 | 0.1524 | |
| LayoutLMv3|D| (Training Data Size)=10, Modality=T+L+I, Localized?=true2023.09 | 0.1001 | |
| LLaVA-v1.5-13B+Image|D| (Training Data Size)=0, Modality=I, Localized?=false2023.09 | 0.0038 |