# Technical Report for ICCV 2021 Challenge SSLAD-Track3B: Transformers Are Better Continual Learners

Source: [Technical Report for ICCV 2021 Challenge SSLAD-Track3B: Transformers Are Better Continual Learners](<https://www.sota2.com/research/paper/68fa6b941e209131cea2effb>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-25T23:52:06.013Z

Authors: Duo Li, Guimei Cao, Yunlu Xu, Zhanzhan Cheng, Yi Niu

Published: 2022-01-13T12:41:36.000Z

## Abstract

In the SSLAD-Track 3B challenge on continual learning, we propose the method of COntinual Learning with Transformer (COLT). We find that transformers suffer less from catastrophic forgetting compared to convolutional neural network. The major principle of our method is to equip the transformer based feature extractor with old knowledge distillation and head expanding strategies to compete catastrophic forgetting. In this report, we first introduce the overall framework of continual learning for object detection. Then, we analyse the key elements' effect on withstanding catastrophic forgetting in our solution. Our method achieves 70.78 mAP on the SSLAD-Track 3B challenge test set.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2201.04924>)

[PDF](<https://arxiv.org/pdf/2201.04924.pdf>)
