# Channel-wise Dynamic Knowledge Distillation via Adaptive Sample Generation for Action Recognition

Source: [Channel-wise Dynamic Knowledge Distillation via Adaptive Sample Generation for Action Recognition](<https://www.sota2.com/research/paper/6a72c37fbc6f0f0705846e46>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-08-06T05:00:05.119Z

Authors: Ping Li, Chenhao Ping, Jie Song, Mingli Song

Published: 2026-08-04T04:18:13.000Z

## Abstract

Knowledge Distillation (KD) offers a promising yet underexplored path for compressing large action recognition models. However, existing KD methods suffer from two key limitations: 1) reliance on fixed input samples leads to suboptimal feature alignment between the frozen teacher (larger model) and the learnable student (smaller model), and 2) applying a uniform distillation strength for all channels fails to account for their varying importance in capturing distinct knowledge (e.g., motion tempo or magnitude) across training epochs. This motivates us to develop an Adaptive Sample-aware Channel-wise Dynamic (ASCD) KD approach, which operates in two stages. First, we use an adaptive sample generation module to create updated samples by incorporating semantics from sample gradients, which are derived by minimizing a feature loss weighted by channel centroid frequency differences at each layer. Meanwhile, crucial motion-related details are preserved by applying a Gaussian mask to frequency features. Second, we employ a channel-wise dynamic distillation module to train student on these generated samples, guided by sample gradients and feature frequencies. For efficiency, samples are updated periodically rather than per epoch. Extensive experiments on three video benchmarks (UCF101, Kinetics-400, Something-Something-v2) and two image datasets (CIFAR-100, ImageNet) demonstrate the state-of-the-art performance of our method. Code is available at https://github.com/mlvccn/ASCD\_KD\_Action.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2608.03100>)

[PDF](<https://arxiv.org/pdf/2608.03100.pdf>)
