# MolNexTR: A Generalized Deep Learning Model for Molecular Image Recognition

Source: [MolNexTR: A Generalized Deep Learning Model for Molecular Image Recognition](<https://www.sota2.com/research/paper/68fa77581e209131cea72899>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T04:15:51.701Z

Authors: Yufan Chen, Ching Ting Leung, Yong Huang, Jianwei Sun, Hao Chen, Hanyu Gao

Published: 2024-03-06T13:17:41.000Z

## Abstract

In the field of chemical structure recognition, the task of converting molecular images into machine-readable data formats such as SMILES string stands as a significant challenge, primarily due to the varied drawing styles and conventions prevalent in chemical literature. To bridge this gap, we proposed MolNexTR, a novel image-to-graph deep learning model that collaborates to fuse the strengths of ConvNext, a powerful Convolutional Neural Network variant, and Vision-TRansformer. This integration facilitates a more detailed extraction of both local and global features from molecular images. MolNexTR can predict atoms and bonds simultaneously and understand their layout rules. It also excels at flexibly integrating symbolic chemistry principles to discern chirality and decipher abbreviated structures. We further incorporate a series of advanced algorithms, including an improved data augmentation module, an image contamination module, and a post-processing module for getting the final SMILES output. These modules cooperate to enhance the model's robustness to diverse styles of molecular images found in real literature. In our test sets, MolNexTR has demonstrated superior performance, achieving an accuracy rate of 81-97%, marking a significant advancement in the domain of molecular structure recognition.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Optical Chemical Structure Recognition (OCSR) on BioVista Benchmark (test)](<https://www.sota2.com/research/sota/optical-chemical-structure-recognition-ocsr-on-biovista-benchmark-test.md>): Optical Chemical Structure Recognition (OCSR); BioVista Benchmark (test); Full Score: 69.5.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2403.03691>)

[PDF](<https://arxiv.org/pdf/2403.03691.pdf>)
