# AutoLR: Layer-wise Pruning and Auto-tuning of Learning Rates in Fine-tuning of Deep Networks

Source: [AutoLR: Layer-wise Pruning and Auto-tuning of Learning Rates in Fine-tuning of Deep Networks](<https://www.sota2.com/research/paper/68fa69121e209131cea20061>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-25T22:52:51.647Z

Authors: Youngmin Ro, Jin Young Choi

Published: 2020-02-14T14:24:40.000Z

## Abstract

Existing fine-tuning methods use a single learning rate over all layers. In this paper, first, we discuss that trends of layer-wise weight variations by fine-tuning using a single learning rate do not match the well-known notion that lower-level layers extract general features and higher-level layers extract specific features. Based on our discussion, we propose an algorithm that improves fine-tuning performance and reduces network complexity through layer-wise pruning and auto-tuning of layer-wise learning rates. The proposed algorithm has verified the effectiveness by achieving state-of-the-art performance on the image retrieval benchmark datasets (CUB-200, Cars-196, Stanford online product, and Inshop). Code is available at https://github.com/youngminPIL/AutoLR.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Lesion Detection on BUSI](<https://www.sota2.com/research/sota/lesion-detection-on-busi.md>): Lesion Detection; BUSI; Training Time (m): 4.9.

- [Lesion Detection on POCUS](<https://www.sota2.com/research/sota/lesion-detection-on-pocus.md>): Lesion Detection; POCUS; Training Time (m): 17.5.

- [Lesion Detection on Chest X-ray](<https://www.sota2.com/research/sota/lesion-detection-on-chest-x-ray.md>): Lesion Detection; Chest X-ray; Training Time (m): 13.3.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2002.06048>)

[PDF](<https://arxiv.org/pdf/2002.06048.pdf>)
