# FoldToken: Learning Protein Language via Vector Quantization and Beyond

Source: [FoldToken: Learning Protein Language via Vector Quantization and Beyond](<https://www.sota2.com/research/paper/68fa5f561e209131ce9e90a7>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-25T19:13:45.935Z

Authors: Zhangyang Gao, Cheng Tan, Jue Wang, Yufei Huang, Lirong Wu, Stan Z. Li

Published: 2024-02-04T12:18:51.000Z

## Abstract

Is there a foreign language describing protein sequences and structures simultaneously? Protein structures, represented by continuous 3D points, have long posed a challenge due to the contrasting modeling paradigms of discrete sequences. We introduce \\textbf{FoldTokenizer} to represent protein sequence-structure as discrete symbols. This innovative approach involves projecting residue types and structures into a discrete space, guided by a reconstruction loss for information preservation. We refer to the learned discrete symbols as \\textbf{FoldToken}, and the sequence of FoldTokens serves as a new protein language, transforming the protein sequence-structure into a unified modality. We apply the created protein language on general backbone inpainting and antibody design tasks, building the first GPT-style model (\\textbf{FoldGPT}) for sequence-structure co-generation with promising results. Key to our success is the substantial enhancement of the vector quantization module, Soft Conditional Vector Quantization (\\textbf{SoftCVQ}).

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Protein Reconstruction on CATH 512 samples](<https://www.sota2.com/research/sota/protein-reconstruction-on-cath-512-samples.md>): Protein Reconstruction; CATH 512 samples; RMSD: 1.3.

- [Protein Reconstruction on AFDB 512 samples](<https://www.sota2.com/research/sota/protein-reconstruction-on-afdb-512-samples.md>): Protein Reconstruction; AFDB 512 samples; RMSD: 2.16.

- [Protein Reconstruction on CAMEO (512 samples)](<https://www.sota2.com/research/sota/protein-reconstruction-on-cameo-512-samples.md>): Protein Reconstruction; CAMEO (512 samples); RMSD: 2.54.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2403.09673>)

[PDF](<https://arxiv.org/pdf/2403.09673.pdf>)
