# Colorization Transformer

Source: [Colorization Transformer](<https://www.sota2.com/research/paper/68fa696f1e209131cea22069>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-01-04T03:21:17.894Z

Authors: Manoj Kumar, Dirk Weissenborn, Nal Kalchbrenner

Published: 2021-02-08T18:45:06.000Z

## Abstract

We present the Colorization Transformer, a novel approach for diverse high fidelity image colorization based on self-attention. Given a grayscale image, the colorization proceeds in three steps. We first use a conditional autoregressive transformer to produce a low resolution coarse coloring of the grayscale image. Our architecture adopts conditional transformer layers to effectively condition grayscale input. Two subsequent fully parallel networks upsample the coarse colored low resolution image into a finely colored high resolution image. Sampling from the Colorization Transformer produces diverse colorings whose fidelity outperforms the previous state-of-the-art on colorising ImageNet based on FID results and based on a human evaluation in a Mechanical Turk test. Remarkably, in more than 60% of cases human evaluators prefer the highest rated among three generated colorings over the ground truth. The code and pre-trained checkpoints for Colorization Transformer are publicly available at https://github.com/google-research/google-research/tree/master/coltran

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Image Colorization on ImageNet 500-image human evaluation (test)](<https://www.sota2.com/research/sota/image-colorization-on-imagenet-500-image-human-evaluation-test.md>): Image Colorization; ImageNet 500-image human evaluation (test); AMT Fooling Rate: 62.

- [Image Colorization on ImageNet (val)](<https://www.sota2.com/research/sota/image-colorization-on-imagenet-val.md>): Image Colorization; ImageNet (val); FID: 19.37.

- [Colorization on ImageNet (test)](<https://www.sota2.com/research/sota/colorization-on-imagenet-test.md>): Colorization; ImageNet (test); FID: 6.44.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2102.04432>)

[PDF](<https://arxiv.org/pdf/2102.04432.pdf>)

[Implementation](<https://github.com/google-research/google-research/tree/master/coltran>)