# GlobalDoc: A Cross-Modal Vision-Language Framework for Real-World Document Image Retrieval and Classification

Source: [GlobalDoc: A Cross-Modal Vision-Language Framework for Real-World Document Image Retrieval and Classification](<https://www.sota2.com/research/paper/68fa75871e209131cea6864a>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-01-04T03:50:03.643Z

Authors: Souhail Bakkali, Sanket Biswas, Zuheng Ming, Micka\\"el Coustaty, Mar\\c{c}al Rusi\\~nol, Oriol Ramos Terrades, Josep Llad\\'os

Published: 2023-09-11T18:35:14.000Z

## Abstract

Visual document understanding (VDU) has rapidly advanced with the development of powerful multi-modal language models. However, these models typically require extensive document pre-training data to learn intermediate representations and often suffer a significant performance drop in real-world online industrial settings. A primary issue is their heavy reliance on OCR engines to extract local positional information within document pages, which limits the models' ability to capture global information and hinders their generalizability, flexibility, and robustness. In this paper, we introduce GlobalDoc, a cross-modal transformer-based architecture pre-trained in a self-supervised manner using three novel pretext objective tasks. GlobalDoc improves the learning of richer semantic concepts by unifying language and visual representations, resulting in more transferable models. For proper evaluation, we also propose two novel document-level downstream VDU tasks, Few-Shot Document Image Classification (DIC) and Content-based Document Image Retrieval (DIR), designed to simulate industrial scenarios more closely. Extensive experimentation has been conducted to demonstrate GlobalDoc's effectiveness in practical settings.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Information Extraction on FUNSD (test)](<https://www.sota2.com/research/sota/information-extraction-on-funsd-test.md>): Information Extraction; FUNSD (test); F1 Score: 79.4.

- [Document Classification on RVL-CDIP (test)](<https://www.sota2.com/research/sota/document-classification-on-rvl-cdip-test.md>): Document Classification; RVL-CDIP (test); Accuracy: 94.04.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2309.05756>)

[PDF](<https://arxiv.org/pdf/2309.05756.pdf>)
