# Colorization as a Proxy Task for Visual Understanding

Source: [Colorization as a Proxy Task for Visual Understanding](<https://www.sota2.com/research/paper/68fa63881e209131ce9ff514>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-25T20:43:17.087Z

Authors: Gustav Larsson, Michael Maire, Gregory Shakhnarovich

Published: 2017-03-11T23:32:59.000Z

## Abstract

We investigate and improve self-supervision as a drop-in replacement for ImageNet pretraining, focusing on automatic colorization as the proxy task. Self-supervised training has been shown to be more promising for utilizing unlabeled data than other, traditional unsupervised learning methods. We build on this success and evaluate the ability of our self-supervised network in several contexts. On VOC segmentation and classification tasks, we present results that are state-of-the-art among methods not using ImageNet labels for pretraining representations.   Moreover, we present the first in-depth analysis of self-supervision via colorization, concluding that formulation of the loss, training details and network architecture play important roles in its effectiveness. This investigation is further expanded by revisiting the ImageNet pretraining paradigm, asking questions such as: How much training data is needed? How many labels are needed? How much do features change when fine-tuned? We relate these questions back to self-supervision by showing that colorization provides a similarly powerful supervisory signal as various flavors of ImageNet pretraining.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Classification on PASCAL VOC 2007 (test)](<https://www.sota2.com/research/sota/classification-on-pascal-voc-2007-test.md>): Classification; PASCAL VOC 2007 (test); mAP (%): 65.9.

- [Semantic Segmentation on PASCAL VOC 2012](<https://www.sota2.com/research/sota/semantic-segmentation-on-pascal-voc-2012.md>): Semantic segmentation; PASCAL VOC 2012; mIoU: 38.4.

- [Semantic Segmentation on PASCAL VOC 2012 (test)](<https://www.sota2.com/research/sota/semantic-segmentation-on-pascal-voc-2012-test.md>): Semantic segmentation; PASCAL VOC 2012 (test); mIoU: 38.4.

- [Semantic Segmentation on PASCAL VOC 2012 (val)](<https://www.sota2.com/research/sota/semantic-segmentation-on-pascal-voc-2012-val.md>): Semantic segmentation; PASCAL VOC 2012 (val); Mean IoU: 38.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/1703.04044>)

[PDF](<https://arxiv.org/pdf/1703.04044.pdf>)