# Hierarchical Features Matter: A Deep Exploration of Progressive Parameterization Method for Dataset Distillation Source: [Hierarchical Features Matter: A Deep Exploration of Progressive Parameterization Method for Dataset Distillation]() Collection index: [llms.txt]() Data updated: 2025-10-26T04:20:40.032Z Authors: Xinhao Zhong, Hao Fang, Bin Chen, Xulin Gu, Meikang Qiu, Shuhan Qi, Shu-Tao Xia Published: 2024-06-09T09:15:54.000Z ## Abstract Dataset distillation is an emerging dataset reduction method, which condenses large-scale datasets while maintaining task accuracy. Current parameterization methods achieve enhanced performance under extremely high compression ratio by optimizing determined synthetic dataset in informative feature domain. However, they limit themselves to a fixed optimization space for distillation, neglecting the diverse guidance across different informative latent spaces. To overcome this limitation, we propose a novel parameterization method dubbed Hierarchical Parameterization Distillation (H-PD), to systematically explore hierarchical feature within provided feature space (e.g., layers within pre-trained generative adversarial networks). We verify the correctness of our insights by applying the hierarchical optimization strategy on GAN-based parameterization method. In addition, we introduce a novel class-relevant feature distance metric to alleviate the computational burden associated with synthetic dataset evaluation, bridging the gap between synthetic and original datasets. Experimental results demonstrate that the proposed H-PD achieves a significant performance improvement under various settings with equivalent time consumption, and even surpasses current generative distillation using diffusion models under extreme compression ratios IPC=1 and IPC=10. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Image Classification on ImageNet subsets 1.0 (test)](): Image Classification; ImageNet subsets 1.0 (test); Nette Score: 30.8. - [Image Classification on ImageNet 128x128 (test)](): Image Classification; ImageNet 128x128 (test); Nette Accuracy: 45.4. ## Original sources and implementation [Original paper]() [PDF]()