# Fingerspelling Detection in American Sign Language

Source: [Fingerspelling Detection in American Sign Language](<https://www.sota2.com/research/paper/68fa69b41e209131cea239aa>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-01-04T05:01:58.494Z

Authors: Bowen Shi, Diane Brentari, Greg Shakhnarovich, Karen Livescu

Published: 2021-04-03T02:11:09.000Z

## Abstract

Fingerspelling, in which words are signed letter by letter, is an important component of American Sign Language. Most previous work on automatic fingerspelling recognition has assumed that the boundaries of fingerspelling regions in signing videos are known beforehand. In this paper, we consider the task of fingerspelling detection in raw, untrimmed sign language videos. This is an important step towards building real-world fingerspelling recognition systems. We propose a benchmark and a suite of evaluation metrics, some of which reflect the effect of detection on the downstream fingerspelling recognition task. In addition, we propose a new model that learns to detect fingerspelling via multi-task training, incorporating pose estimation and fingerspelling recognition (transcription) along with detection, and compare this model to several alternatives. The model outperforms all alternative approaches across all metrics, establishing a state of the art on the benchmark.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Fingerspelling Video Search (Text to Video) on ChicagoFSWild (test)](<https://www.sota2.com/research/sota/fingerspelling-video-search-text-to-video-on-chicagofswild-test.md>): Fingerspelling Video Search (Text to Video); ChicagoFSWild (test); mAP: 33.2.

- [Fingerspelling Video Search (Text to Video) on ChicagoFSWild+ (test)](<https://www.sota2.com/research/sota/fingerspelling-video-search-text-to-video-on-chicagofswild-plus-test.md>): Fingerspelling Video Search (Text to Video); ChicagoFSWild+ (test); mAP: 59.3.

- [Fingerspelling Word Search (Video to Text) on ChicagoFSWild (test)](<https://www.sota2.com/research/sota/fingerspelling-word-search-video-to-text-on-chicagofswild-test.md>): Fingerspelling Word Search (Video to Text); ChicagoFSWild (test); mAP: 38.3.

- [Fingerspelling Word Search (Video to Text) on ChicagoFSWild+ (test)](<https://www.sota2.com/research/sota/fingerspelling-word-search-video-to-text-on-chicagofswild-plus-test.md>): Fingerspelling Word Search (Video to Text); ChicagoFSWild+ (test); mAP: 63.3.

- [Fingerspelling Detection on ChicagoFSWild (test)](<https://www.sota2.com/research/sota/fingerspelling-detection-on-chicagofswild-test.md>): Fingerspelling detection; ChicagoFSWild (test); AP@0.1: 49.5.

- [Fingerspelling detection on ChicagoFSWild+ (test)](<https://www.sota2.com/research/sota/fingerspelling-detection-on-chicagofswild-plus-test.md>): Fingerspelling detection; ChicagoFSWild+ (test); AP (IoU=0.1): 59.

- [Fingerspelling detection on ChicagoFSWild+ 1.0 (test)](<https://www.sota2.com/research/sota/fingerspelling-detection-on-chicagofswild-plus-1-0-test.md>): Fingerspelling detection; ChicagoFSWild+ 1.0 (test); AP (IoU=0.1): not recorded.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2104.01291>)

[PDF](<https://arxiv.org/pdf/2104.01291.pdf>)