# Self-Supervised Representation Learning on Document Images

Source: [Self-Supervised Representation Learning on Document Images](<https://www.sota2.com/research/paper/68fa67941e209131cea17267>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-27T06:02:46.151Z

Authors: Adrian Cosma, Mihai Ghidoveanu, Michael Panaitescu-Liess, Marius Popescu

Published: 2020-04-18T10:14:06.000Z

## Abstract

This work analyses the impact of self-supervised pre-training on document images in the context of document image classification. While previous approaches explore the effect of self-supervision on natural images, we show that patch-based pre-training performs poorly on document images because of their different structural properties and poor intra-sample semantic information. We propose two context-aware alternatives to improve performance on the Tobacco-3482 image classification task. We also propose a novel method for self-supervision, which makes use of the inherent multi-modality of documents (image and text), which performs better than other popular self-supervised methods, including supervised ImageNet pre-training, on document image classification scenarios with a limited amount of data.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2004.10605>)

[PDF](<https://arxiv.org/pdf/2004.10605.pdf>)