# TRT-ViT: TensorRT-oriented Vision Transformer

Source: [TRT-ViT: TensorRT-oriented Vision Transformer](<https://www.sota2.com/research/paper/68fa6d031e209131cea374c3>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T00:24:36.116Z

Authors: Xin Xia, Jiashi Li, Jie Wu, Xing Wang, Xuefeng Xiao, Min Zheng, Rui Wang

Published: 2022-05-19T14:20:25.000Z

## Abstract

We revisit the existing excellent Transformers from the perspective of practical application. Most of them are not even as efficient as the basic ResNets series and deviate from the realistic deployment scenario. It may be due to the current criterion to measure computation efficiency, such as FLOPs or parameters is one-sided, sub-optimal, and hardware-insensitive. Thus, this paper directly treats the TensorRT latency on the specific hardware as an efficiency metric, which provides more comprehensive feedback involving computational capacity, memory cost, and bandwidth. Based on a series of controlled experiments, this work derives four practical guidelines for TensorRT-oriented and deployment-friendly network design, e.g., early CNN and late Transformer at stage-level, early Transformer and late CNN at block-level. Accordingly, a family of TensortRT-oriented Transformers is presented, abbreviated as TRT-ViT. Extensive experiments demonstrate that TRT-ViT significantly outperforms existing ConvNets and vision Transformers with respect to the latency/accuracy trade-off across diverse visual tasks, e.g., image classification, object detection and semantic segmentation. For example, at 82.7% ImageNet-1k top-1 accuracy, TRT-ViT is 2.7$\\times$ faster than CSWin and 2.0$\\times$ faster than Twins. On the MS-COCO object detection task, TRT-ViT achieves comparable performance with Twins, while the inference speed is increased by 2.8$\\times$.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Object Detection on COCO (AP^b, AP^b\_50, AP^b\_75)](<https://www.sota2.com/research/sota/object-detection-on-coco-ap-b-ap-b_50-ap-b_75.md>): Object Detection; COCO; AP50 (Box): 67.7.

- [Instance Segmentation on COCO](<https://www.sota2.com/research/sota/instance-segmentation-on-coco.md>): Instance Segmentation; COCO; APmask: 41.

- [Image Classification on ImageNet-1K 1.0 (val)](<https://www.sota2.com/research/sota/image-classification-on-imagenet-1k-1-0-val.md>): Image Classification; ImageNet-1K 1.0 (val); Top-1 Accuracy: 83.4.

- [Semantic Segmentation on ADE20K (val)](<https://www.sota2.com/research/sota/semantic-segmentation-on-ade20k-val.md>): Semantic segmentation; ADE20K (val); mIoU: 48.8.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2205.09579>)

[PDF](<https://arxiv.org/pdf/2205.09579.pdf>)
