# DATA: Domain-Aware and Task-Aware Self-supervised Learning Source: [DATA: Domain-Aware and Task-Aware Self-supervised Learning]() Collection index: [llms.txt]() Data updated: 2026-01-04T05:10:32.656Z Authors: Qing Chang, Junran Peng, Lingxie Xie, Jiajun Sun, Haoran Yin, Qi Tian, Zhaoxiang Zhang Published: 2022-03-17T02:38:49.000Z ## Abstract The paradigm of training models on massive data without label through self-supervised learning (SSL) and finetuning on many downstream tasks has become a trend recently. However, due to the high training costs and the unconsciousness of downstream usages, most self-supervised learning methods lack the capability to correspond to the diversities of downstream scenarios, as there are various data domains, different vision tasks and latency constraints on models. Neural architecture search (NAS) is one universally acknowledged fashion to conquer the issues above, but applying NAS on SSL seems impossible as there is no label or metric provided for judging model selection. In this paper, we present DATA, a simple yet effective NAS approach specialized for SSL that provides Domain-Aware and Task-Aware pre-training. Specifically, we (i) train a supernet which could be deemed as a set of millions of networks covering a wide range of model scales without any label, (ii) propose a flexible searching mechanism compatible with SSL that enables finding networks of different computation costs, for various downstream vision tasks and data domains without explicit metric provided. Instantiated With MoCo v2, our method achieves promising results across a wide range of computation costs on downstream tasks, including image classification, object detection and semantic segmentation. DATA is orthogonal to most existing SSL methods and endows them the ability of customization on downstream needs. Extensive experiments on other SSL methods demonstrate the generalizability of the proposed method. Code is released at https://github.com/GAIA-vision/GAIA-ssl ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Semi-supervised Image Classification on ImageNet 1% (val)](): Semi-supervised Image Classification; ImageNet 1% (val); Top-1 Acc: 49.5. - [Semi-supervised Image Classification on ImageNet (10% labeled)](): Semi-supervised Image Classification; ImageNet (10% labeled); Top-1 Accuracy: 65.6. - [Semantic Segmentation on Cityscapes (val) (mIoU only)](): Semantic segmentation; Cityscapes (val); mIoU: 78.2. - [Object Detection on PASCAL VOC 2007 (test)](): Object Detection; PASCAL VOC 2007 (test); mAP: 60.5. - [Object Detection on COCO 2017 (val)](): Object Detection; COCO 2017 (val); AP: not recorded. - [Instance Segmentation on COCO 2017 (val)](): Instance Segmentation; COCO 2017 (val); APm: not recorded. ## Original sources and implementation [Original paper]() [PDF]() [Implementation]()