# Efficient Dataset Distillation for Pre-Trained Self-Supervised Models via Statistical Flow Matching Source: [Efficient Dataset Distillation for Pre-Trained Self-Supervised Models via Statistical Flow Matching]() Collection index: [llms.txt]() Data updated: 2026-05-13T05:00:27.562Z Authors: Qianxin Xia, Jiawei Du, Xin Zhang, Yuhan Zhang, Jielei Wang, Guoming Lu Published: 2026-02-05T07:18:48.000Z ## Abstract Dataset distillation seeks to synthesize a highly compact dataset that achieves performance comparable to the original dataset on downstream tasks. For the classification task that use pre-trained self-supervised models as backbones, previous linear gradient matching optimizes synthetic images by encouraging them to mimic the gradient updates induced by real images on the linear classifier. However, this batch-level formulation requires loading thousands of real images and applying multiple rounds of differentiable augmentations to synthetic images at each distillation step, leading to substantial computational and memory overhead. In this paper, we introduce statistical flow matching , a stable and efficient supervised learning framework that optimizes synthetic images by aligning constant statistical flows from target class centers to non-target class centers in the original data. Our approach loads raw statistics only once and performs a single augmentation pass on the synthetic data, achieving performance comparable to or better than the state-of-the-art methods with 10x lower GPU memory usage and 4x shorter runtime. Furthermore, we propose a classifier inheritance strategy that reuses the classifier trained on the original dataset for inference, requiring only an extremely lightweight linear projector and marginal storage while achieving substantial performance gains. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Image Classification on CUB-200 1.0 (test)](): Image Classification; CUB-200 1.0 (test); Average Accuracy: 69.9. - [Image Classification on ImageNet-100 1.0 (test)](): Image Classification; ImageNet-100 1.0 (test); CLIP Score: 92.2. - [Image Classification on Stanford Dogs 1.0 (test)](): Image Classification; Stanford Dogs 1.0 (test); CLIP Score: 62.4. ## Original sources and implementation [Original paper]() [PDF]()