Document Understanding on DUE Benchmark
86.7DocVQALayoutLMv2LARGE + QG
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| LayoutLMv2LARGE + QGModality=T+L+V2024.02 | 86.7 | — | — | — | — | — | — | — | |
| UDOPModality=V+T+L2022.12 | 84.7 | 47.4 | 82.8 | 28.9 | 85.5 | 47.2 | 78.9 | 65.1 | |
| UDOPModality=T+L+V2024.02 | 84.7 | 47.4 | — | — | — | 47.2 | 78.9 | 64.6 | |
| UDOP-DualModality=V+T+L2022.12 | 84.4 | 47.1 | 81.9 | 28.7 | 85.2 | 46.7 | 79.5 | 64.7 | |
| StructuralLMlargeModality=T+L2022.12 | 83.9 | — | — | — | — | — | — | — | |
| LayoutLMv3largeModality=V+T+L2022.12 | 83.4 | 45.1 | 77.1 | 26.9 | 84 | 45.7 | 78.1 | 62.9 | |
| LayoutLMV3LARGEModality=T+L+V2024.02 | 83.4 | 45.1 | — | — | — | 45.7 | 78.1 | 63.1 | |
| LATIN-Prompt (Claude)Modality=T+L2024.02 | 82.6 | 54.5 | — | — | — | — | — | — | |
| T5large+2D+UModality=T+L2022.12 | 81 | 46.1 | 75.9 | 26.8 | 83.3 | 43.3 | 78.6 | 59.8 | |
| T5LARGE+2D+UModality=T+L2024.02 | 81 | 46.1 | — | — | — | 43.3 | 78.6 | 62.3 | |
| LAPDocModality=T+L, Verbalization Strategy=SPATIALFORMAT2024.02 | 79.8 | 54.9 | — | — | — | 47.7 | 70.1 | 63.1 | |
| LayoutLMv2largeModality=V+T+L2022.12 | 78.8 | — | — | — | — | — | — | — | |
| T5large+UModality=T2022.12 | 76.3 | 37.1 | 76 | 27.6 | 82.9 | 38.1 | 76 | 56.5 | |
| LAPDocModality=T, Verbalization Strategy=PLAINTEXT2024.02 | 76.3 | 47.7 | — | — | — | 45.1 | 68.4 | 59.4 | |
| LAPDocModality=T+L, Verbalization Strategy=SPATIALFORMATY2024.02 | 76.3 | 49.6 | — | — | — | 45.5 | 70.3 | 60.4 | |
| LAPDocModality=T+L, Verbalization Strategy=CENTERPOINT2024.02 | 75.1 | 47.4 | — | — | — | 38.2 | 67.8 | 57.1 | |
| LAPDocModality=T+L, Verbalization Strategy=BOUNDINGBOX2024.02 | 74.8 | 46.4 | — | — | — | 35 | 68.5 | 56.2 | |
| LAPDocModality=T+L, Verbalization Strategy=BOUNDINGBOXMARKUP2024.02 | 74.6 | 45.8 | — | — | — | 36.2 | 68.6 | 56.3 | |
| DonutModality=V2022.12 | 72.1 | — | — | — | — | — | — | — | |
| DonutModality=V2024.02 | 72.1 | — | — | — | — | — | — | — | |
| T5largeModality=T2022.12 | 70.4 | 36.7 | 74.3 | 25.3 | 74.4 | 33.3 | 58.9 | 50.7 | |
| T5large+2DModality=T+L2022.12 | 69.8 | 39.2 | 72.6 | 25.7 | 74 | 30.8 | 58 | 50.4 | |
| BERTlargeModality=T2022.12 | 67.5 | — | — | — | — | — | — | — | |
| BERTLARGEModality=T2024.02 | 67.5 | — | — | — | — | — | — | — | |
| LAMBERTModality=T+L2022.12 | — | — | 81.3 | — | — | — | — | — |