# Attentive Normalization Source: [Attentive Normalization]() Collection index: [llms.txt]() Data updated: 2026-01-04T03:07:13.759Z Authors: Xilai Li, Wei Sun, Tianfu Wu Published: 2019-08-04T02:17:34.000Z ## Abstract In state-of-the-art deep neural networks, both feature normalization and feature attention have become ubiquitous. % with significant performance improvement shown in a vast amount of tasks. They are usually studied as separate modules, however. In this paper, we propose a light-weight integration between the two schema and present Attentive Normalization (AN). Instead of learning a single affine transformation, AN learns a mixture of affine transformations and utilizes their weighted-sum as the final affine transformation applied to re-calibrate features in an instance-specific way. The weights are learned by leveraging channel-wise feature attention. In experiments, we test the proposed AN using four representative neural architectures in the ImageNet-1000 classification benchmark and the MS-COCO 2017 object detection and instance segmentation benchmark. AN obtains consistent performance improvement for different neural architectures in both benchmarks with absolute increase of top-1 accuracy in ImageNet-1000 between 0.5\\% and 2.7\\%, and absolute increase up to 1.8\\% and 2.2\\% for bounding box and mask AP in MS-COCO respectively. We observe that the proposed AN provides a strong alternative to the widely used Squeeze-and-Excitation (SE) module. The source codes are publicly available at https://github.com/iVMCL/AOGNet-v2 (the ImageNet Classification Repo) and https://github.com/iVMCL/AttentiveNorm\\\_Detection (the MS-COCO Detection and Segmentation Repo). ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Image Classification on ImageNet 1000 (val)](): Image Classification; ImageNet 1000 (val); Top-1 Error: 18.13. - [Instance Segmentation on MS COCO (val)](): Instance Segmentation; MS-COCO (val); AP: 40. - [Object Detection on MS COCO (val)](): Object Detection; MS-COCO (val); mAP: 0.465. - [Object Detection on COCO 2017 (val)](): Object Detection; COCO 2017 (val); AP: not recorded. - [Instance Segmentation on COCO 2017 (val)](): Instance Segmentation; COCO 2017 (val); APm: not recorded. ## Original sources and implementation [Original paper]() [PDF]()