# NomMer: Nominate Synergistic Context in Vision Transformer for Visual Recognition

Source: [NomMer: Nominate Synergistic Context in Vision Transformer for Visual Recognition](<https://www.sota2.com/research/paper/68fa6bfe1e209131cea3186a>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-01-04T05:12:38.746Z

Authors: Hao Liu, Xinghua Jiang, Xin Li, Zhimin Bao, Deqiang Jiang, Bo Ren

Published: 2021-11-25T10:07:54.000Z

## Abstract

Recently, Vision Transformers (ViT), with the self-attention (SA) as the de facto ingredients, have demonstrated great potential in the computer vision community. For the sake of trade-off between efficiency and performance, a group of works merely perform SA operation within local patches, whereas the global contextual information is abandoned, which would be indispensable for visual recognition tasks. To solve the issue, the subsequent global-local ViTs take a stab at marrying local SA with global one in parallel or alternative way in the model. Nevertheless, the exhaustively combined local and global context may exist redundancy for various visual data, and the receptive field within each layer is fixed. Alternatively, a more graceful way is that global and local context can adaptively contribute per se to accommodate different visual data. To achieve this goal, we in this paper propose a novel ViT architecture, termed NomMer, which can dynamically Nominate the synergistic global-local context in vision transforMer. By investigating the working pattern of our proposed NomMer, we further explore what context information is focused. Beneficial from this "dynamic nomination" mechanism, without bells and whistles, the NomMer can not only achieve 84.5% Top-1 classification accuracy on ImageNet with only 73M parameters, but also show promising performance on dense prediction tasks, i.e., object detection and semantic segmentation. The code and models will be made publicly available at https://github.com/TencentYoutuResearch/VisualRecognition-NomMer

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Image Classification on ImageNet-21K](<https://www.sota2.com/research/sota/image-classification-on-imagenet-21k.md>): Image Classification; ImageNet-21K; Top-1 Accuracy: 86.6.

- [Classification on ImageNet-1K 1.0 (val)](<https://www.sota2.com/research/sota/classification-on-imagenet-1k-1-0-val.md>): Classification; ImageNet-1K 1.0 (val); Top-1 Accuracy (%): 84.9.

- [Semantic Segmentation on ADE20K (val)](<https://www.sota2.com/research/sota/semantic-segmentation-on-ade20k-val.md>): Semantic segmentation; ADE20K (val); mIoU: 50.

- [Object Detection on COCO 2017 (val)](<https://www.sota2.com/research/sota/object-detection-on-coco-2017-val.md>): Object Detection; COCO 2017 (val); AP: not recorded.

- [Instance Segmentation on COCO 2017 (val)](<https://www.sota2.com/research/sota/instance-segmentation-on-coco-2017-val.md>): Instance Segmentation; COCO 2017 (val); APm: not recorded.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2111.12994>)

[PDF](<https://arxiv.org/pdf/2111.12994.pdf>)

[Implementation](<https://github.com/TencentYoutuResearch/VisualRecognition-NomMer>)