# VSR: A Unified Framework for Document Layout Analysis combining Vision, Semantics and Relations Source: [VSR: A Unified Framework for Document Layout Analysis combining Vision, Semantics and Relations]() Collection index: [llms.txt]() Data updated: 2026-01-04T03:31:17.508Z Authors: Peng Zhang, Can Li, Liang Qiao, Zhanzhan Cheng, Shiliang Pu, Yi Niu, Fei Wu Published: 2021-05-13T12:20:30.000Z ## Abstract Document layout analysis is crucial for understanding document structures. On this task, vision and semantics of documents, and relations between layout components contribute to the understanding process. Though many works have been proposed to exploit the above information, they show unsatisfactory results. NLP-based methods model layout analysis as a sequence labeling task and show insufficient capabilities in layout modeling. CV-based methods model layout analysis as a detection or segmentation task, but bear limitations of inefficient modality fusion and lack of relation modeling between layout components. To address the above limitations, we propose a unified framework VSR for document layout analysis, combining vision, semantics and relations. VSR supports both NLP-based and CV-based methods. Specifically, we first introduce vision through document image and semantics through text embedding maps. Then, modality-specific visual and semantic features are extracted using a two-stream network, which are adaptively fused to make full use of complementary information. Finally, given component candidates, a relation module based on graph neural network is incorported to model relations between components and output final results. On three popular benchmarks, VSR outperforms previous models by large margins. Code will be released soon. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Layout Analysis on DocBank 1.0 (test)](): Layout Analysis; DocBank 1.0 (test); Abstract: 98.29. - [Document Layout Analysis on PubLayNet (test)](): Document Layout Analysis; PubLayNet (test); Text Score: 96.7. - [Document Layout Analysis on Article Regions (test)](): Document Layout Analysis; Article Regions (test); mAP: 0.945. - [Object Detection on SCUT-CAB Logical (test)](): Object Detection; SCUT-CAB Logical (test); mAP: 0.557. - [Document Layout Analysis on PubLayNet (val)](): Document Layout Analysis; PubLayNet (val); mAP: 0.957. - [Instance Segmentation on SCUT-CAB Physical (test)](): Instance Segmentation; SCUT-CAB Physical (test); mAP: 78.7. - [Object Detection on SCUT-CAB Physical (test)](): Object Detection; SCUT-CAB Physical (test); mAP: 78.7. - [Instance Segmentation on SCUT-CAB Logical (test)](): Instance Segmentation; SCUT-CAB Logical (test); mAP: 55.1. ## Original sources and implementation [Original paper]() [PDF]()