# KernelWarehouse: Towards Parameter-Efficient Dynamic Convolution

Source: [KernelWarehouse: Towards Parameter-Efficient Dynamic Convolution](<https://www.sota2.com/research/paper/68fa705d1e209131cea4aa38>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T01:40:08.455Z

Authors: Chao Li, Anbang Yao

Published: 2023-08-16T13:35:09.000Z

## Abstract

Dynamic convolution learns a linear mixture of $n$ static kernels weighted with their sample-dependent attentions, demonstrating superior performance compared to normal convolution. However, existing designs are parameter-inefficient: they increase the number of convolutional parameters by $n$ times. This and the optimization difficulty lead to no research progress in dynamic convolution that can allow us to use a significant large value of $n$ (e.g., $n\>100$ instead of typical setting $n\<10$) to push forward the performance boundary. In this paper, we propose $KernelWarehouse$, a more general form of dynamic convolution, which can strike a favorable trade-off between parameter efficiency and representation power. Its key idea is to redefine the basic concepts of "$kernels$" and "$assembling$ $kernels$" in dynamic convolution from the perspective of reducing kernel dimension and increasing kernel number significantly. In principle, KernelWarehouse enhances convolutional parameter dependencies within the same layer and across successive layers via tactful kernel partition and warehouse sharing, yielding a high degree of freedom to fit a desired parameter budget. We validate our method on ImageNet and MS-COCO datasets with different ConvNet architectures, and show that it attains state-of-the-art results. For instance, the ResNet18\|ResNet50\|MobileNetV2\|ConvNeXt-Tiny model trained with KernelWarehouse on ImageNet reaches 76.05%\|81.05%\|75.52%\|82.51% top-1 accuracy. Thanks to its flexible design, KernelWarehouse can even reduce the model size of a ConvNet while improving the accuracy, e.g., our ResNet18 model with 36.45%\|65.10% parameter reduction to the baseline shows 2.89%\|2.29% absolute improvement to top-1 accuracy.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2308.08361>)

[PDF](<https://arxiv.org/pdf/2308.08361.pdf>)