# The Animation Transformer: Visual Correspondence via Segment Matching

Source: [The Animation Transformer: Visual Correspondence via Segment Matching](<https://www.sota2.com/research/paper/68fa6ab71e209131cea29fd0>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-27T06:10:35.496Z

Authors: Evan Casey, V\\'ictor P\\'erez, Zhuoru Li, Harry Teitelman, Nick Boyajian, Tim Pulver, Mike Manh, William Grisaitis

Published: 2021-09-06T17:23:40.000Z

## Abstract

Visual correspondence is a fundamental building block on the way to building assistive tools for hand-drawn animation. However, while a large body of work has focused on learning visual correspondences at the pixel-level, few approaches have emerged to learn correspondence at the level of line enclosures (segments) that naturally occur in hand-drawn animation. Exploiting this structure in animation has numerous benefits: it avoids the intractable memory complexity of attending to individual pixels in high resolution images and enables the use of real-world animation datasets that contain correspondence information at the level of per-segment colors. To that end, we propose the Animation Transformer (AnT) which uses a transformer-based architecture to learn the spatial and visual relationships between segments across a sequence of images. AnT enables practical ML-assisted colorization for professional animation workflows and is publicly accessible as a creative tool in Cadmium.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Paint Bucket Colorization on 3D rendered (test)](<https://www.sota2.com/research/sota/paint-bucket-colorization-on-3d-rendered-test.md>): Paint Bucket Colorization; 3D rendered (test); Acc: 66.34.

- [Paint Bucket Colorization on Hand-drawn (test)](<https://www.sota2.com/research/sota/paint-bucket-colorization-on-hand-drawn-test.md>): Paint Bucket Colorization; Hand-drawn (test); Accuracy: 72.24.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2109.02614>)

[PDF](<https://arxiv.org/pdf/2109.02614.pdf>)
