# Aligning LLMs with Biomedical Knowledge using Balanced Fine-Tuning

Source: [Aligning LLMs with Biomedical Knowledge using Balanced Fine-Tuning](<https://www.sota2.com/research/paper/6927dafbd6b0dd3342930c8f>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-05-06T05:00:11.062Z

Authors: Zhenchao Tang, Fang Wang, Haohuai He, Jiale Zhou, Tianxu Lv, Jun Zhu, Shouzhi Chen, Minghao Yang, Yu Wang, Jiayang Wu, Yidong Song, Yaokun Li, Jiehui Huang, Bing He, Jianhua Yao

Published: 2025-11-26T05:34:26.000Z

## Abstract

Engineering LLMs to accelerate life sciences research requires a robust alignment with biomedical knowledge. We observe that biomedical text exhibits a fundamentally different uncertainty structure from general text: dense low-confidence runs encode epistemic knowledge gaps (dense causal chains, rare entities) rather than the sparse aleatoric stylistic variation typical of general text. Based on this discovery, we propose Balanced Fine-Tuning (BFT), a dual-scale post-training method that combines group-normalized token reweighting with sequence-level reallocation toward knowledge-dense samples exhibiting dense epistemic uncertainty. Across medical evaluation, biological reasoning, sparse-reward RL, and biological representation tasks, BFT provides more consistent gains than SFT and DFT under a shared training setup. When replacing the default closed-source backbones in GeneAgent (GPT-4o) and VCWorld (Gemini-2.5-Flash), the BFT-aligned 70B model delivers stronger performance across biological process reasoning and chemical perturbation prediction. Critically, all BFT variants further improve after subsequent GRPO with sparse rewards, while SFT and DFT degrade, suggesting that epistemic-aware post-training provides a more robust policy initialization. Beyond text generation, BFT-aligned LLMs produce more accurate and professional biomedical profile texts; after encoding these profiles with a text embedding model, the resulting representations support gene-level, cell-level, and perturbation-response tasks, suggesting that BFT-enhanced generation can facilitate biological representation and, in turn, broader biomedical downstream tasks.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Cell-level clustering on Phenotype labels](<https://www.sota2.com/research/sota/cell-level-clustering-on-phenotype-labels.md>): Cell-level clustering; Phenotype labels; ARI: 16.

- [Cell-level clustering on Cell Type labels](<https://www.sota2.com/research/sota/cell-level-clustering-on-cell-type-labels.md>): Cell-level clustering; Cell Type labels; ARI: 33.

- [Gene-Gene Interaction prediction on GGI](<https://www.sota2.com/research/sota/gene-gene-interaction-prediction-on-ggi.md>): Gene-Gene Interaction prediction; GGI; AUC: 82.7.

- [Gene-level biological representation evaluation on TF Range](<https://www.sota2.com/research/sota/gene-level-biological-representation-evaluation-on-tf-range.md>): Gene-level biological representation evaluation; TF Range; AUC: 62.6.

- [Gene-level biological representation evaluation on Dosage](<https://www.sota2.com/research/sota/gene-level-biological-representation-evaluation-on-dosage.md>): Gene-level biological representation evaluation; Dosage; AUC: 91.1.

- [Gene-level biological representation evaluation on No-methy](<https://www.sota2.com/research/sota/gene-level-biological-representation-evaluation-on-no-methy.md>): Gene-level biological representation evaluation; No-methy; AUC: 92.9.

- [Protein-Protein Interaction prediction on PPI](<https://www.sota2.com/research/sota/protein-protein-interaction-prediction-on-ppi.md>): Protein-Protein Interaction prediction; PPI; AUC: 76.6.

- [Gene-level biological representation evaluation on Lys4](<https://www.sota2.com/research/sota/gene-level-biological-representation-evaluation-on-lys4.md>): Gene-level biological representation evaluation; Lys4; AUC: 90.7.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2511.21075>)

[PDF](<https://arxiv.org/pdf/2511.21075.pdf>)
