# MIRA: Mid-training Rubric Anchoring for Source-Aware Data Selection Source: [MIRA: Mid-training Rubric Anchoring for Source-Aware Data Selection]() Collection index: [llms.txt]() Data updated: 2026-06-06T06:00:02.017Z Authors: Haowen Wang, Yaxin Du, Jian Yang, Jiajun Wu, Shukai Liu, Yuxuan Zhang, Pingjie Wang, Siheng Chen, Tuney Zheng, Ming Zhou, Xianglong Liu, Bryan Dai Published: 2026-05-28T17:40:40.000Z ## Abstract Mid-training has become an important stage in modern LLM development, using large-scale curated mixtures to strengthen capabilities before final post-training. Its data selection problem is distinct: the data are optimized under a pretraining-style objective at near-pretraining scale, but are curated toward downstream capabilities and drawn from heterogeneous sources with different formats and training roles. As a result, effective selection requires both scalability and source-adaptive semantic criteria. Existing model-based methods scale well, but provide only implicit quality signals. Semantic selection methods offer stronger judgments, but usually assume fixed rubrics or standardized data formats. To address this mismatch, we propose MIRA, a source-aware filtering framework based on self-anchored rubric discovery. The key idea is to make rubric construction part of data selection: MIRA first discovers what should be evaluated for each source group, then distills those judgments into scalable student scorers for full-corpus filtering. On code-oriented mid-training with 21 sources and 5 source groups, MIRA outperforms selection baselines across nine code benchmarks and matches the full-corpus run while using only half the tokens. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Aggregated Programming Capability Evaluation on Overall Evaluation Suite](): Aggregated Programming Capability Evaluation; Overall Evaluation Suite; Macro Average Score: 64.2. - [Code Generation on MBPP, MBPP+, BCB-f, BCB-h, and LCB](): Code Generation; Code Generation Benchmarks MBPP, MBPP+, BCB-f, BCB-h, LCB; MBPP Score: 88.9. - [Multilingual Code Generation on Multipl-E](): Multilingual Code Generation; MultiPL-E; MultiplE: 72.84. - [Software Engineering Repair on SWE Multi](): Software Engineering Repair; SWE Multi; SWE Average Score: 36.33. - [Text-to-SQL Execution on SQL Execution Benchmarks Spider, BIRD](): Text-to-SQL Execution; SQL Execution Benchmarks Spider, BIRD; Execution Accuracy (Spider): 98.26. ## Original sources and implementation [Original paper]() [PDF]()