# Training Noise Token Pruning

Source: [Training Noise Token Pruning](<https://www.sota2.com/research/paper/68fa77741e209131cea7333c>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T04:18:49.615Z

Authors: Mingxing Rao, Bohan Jiang, Daniel Moyer

Published: 2024-11-27T07:04:00.000Z

## Abstract

In the present work we present Training Noise Token (TNT) Pruning for vision transformers. Our method relaxes the discrete token dropping condition to continuous additive noise, providing smooth optimization in training, while retaining discrete dropping computational gains in deployment settings. We provide theoretical connections to Rate-Distortion literature, and empirical evaluations on the ImageNet dataset using ViT and DeiT architectures demonstrating TNT's advantages over previous pruning methods.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2411.18092>)

[PDF](<https://arxiv.org/pdf/2411.18092.pdf>)