# Tiny Models are the Computational Saver for Large Models

Source: [Tiny Models are the Computational Saver for Large Models](<https://www.sota2.com/research/paper/68fa76891e209131cea6d808>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-01-04T03:55:46.773Z

Authors: Qingyuan Wang, Barry Cardiff, Antoine Frapp\\'e, Benoit Larras, Deepu John

Published: 2024-03-26T14:14:30.000Z

## Abstract

This paper introduces TinySaver, an early-exit-like dynamic model compression approach which employs tiny models to substitute large models adaptively. Distinct from traditional compression techniques, dynamic methods like TinySaver can leverage the difficulty differences to allow certain inputs to complete their inference processes early, thereby conserving computational resources. Most existing early exit designs are implemented by attaching additional network branches to the model's backbone. Our study, however, reveals that completely independent tiny models can replace a substantial portion of the larger models' job with minimal impact on performance. Employing them as the first exit can remarkably enhance computational efficiency. By searching and employing the most appropriate tiny model as the computational saver for a given large model, the proposed approaches work as a novel and generic method to model compression. This finding will help the research community in exploring new compression methods to address the escalating computational demands posed by rapidly evolving AI models. Our evaluation of this approach in ImageNet-1k classification demonstrates its potential to reduce the number of compute operations by up to 90\\%, with only negligible losses in performance, across various modern vision models.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Object Detection on COCO 2017 (val)](<https://www.sota2.com/research/sota/object-detection-on-coco-2017-val.md>): Object Detection; COCO 2017 (val); AP: not recorded.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2403.17726>)

[PDF](<https://arxiv.org/pdf/2403.17726.pdf>)

[Implementation](<https://github.com/QingyuanWang/tinysaver>)
