# Uncertainty-Aware Gradient Signal-to-Noise Data Selection for Instruction Tuning Source: [Uncertainty-Aware Gradient Signal-to-Noise Data Selection for Instruction Tuning]() Collection index: [llms.txt]() Data updated: 2026-01-24T06:00:02.060Z Authors: Zhihang Yuan, Chengyu Yue, Long Huang, Litu Ou, Lei Shi Published: 2026-01-20T07:51:32.000Z ## Abstract Instruction tuning is a standard paradigm for adapting large language models (LLMs), but modern instruction datasets are large, noisy, and redundant, making full-data fine-tuning costly and often unnecessary. Existing data selection methods either build expensive gradient datastores or assign static scores from a weak proxy, largely ignoring evolving uncertainty, and thus missing a key source of LLM interpretability. We propose GRADFILTERING, an objective-agnostic, uncertainty-aware data selection framework that utilizes a small GPT-2 proxy with a LoRA ensemble and aggregates per-example gradients into a Gradient Signal-to-Noise Ratio (G-SNR) utility. Our method matches or surpasses random subsets and strong baselines in most LLM-as-a-judge evaluations as well as in human assessment. Moreover, GRADFILTERING-selected subsets converge faster than competitive filters under the same compute budget, reflecting the benefit of uncertainty-aware scoring. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Instruction Tuning on Alpaca instruction-tuning 52k](): Instruction Tuning; Alpaca instruction-tuning 52k; Pairwise Winning Score: 116. ## Original sources and implementation [Original paper]() [PDF]()