# Searching for fingerspelled content in American Sign Language

Source: [Searching for fingerspelled content in American Sign Language](<https://www.sota2.com/research/paper/68fa6c1c1e209131cea324d5>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-01-04T05:12:06.926Z

Authors: Bowen Shi, Diane Brentari, Greg Shakhnarovich, Karen Livescu

Published: 2022-03-24T18:36:22.000Z

## Abstract

Natural language processing for sign language video - including tasks like recognition, translation, and search - is crucial for making artificial intelligence technologies accessible to deaf individuals, and is gaining research interest in recent years. In this paper, we address the problem of searching for fingerspelled key-words or key phrases in raw sign language videos. This is an important task since significant content in sign language is often conveyed via fingerspelling, and to our knowledge the task has not been studied before. We propose an end-to-end model for this task, FSS-Net, that jointly detects fingerspelling and matches it to a text sequence. Our experiments, done on a large public dataset of ASL fingerspelling in the wild, show the importance of fingerspelling detection as a component of a search and retrieval model. Our model significantly outperforms baseline methods adapted from prior work on related tasks

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Fingerspelling Video Search (Text to Video) on ChicagoFSWild (test)](<https://www.sota2.com/research/sota/fingerspelling-video-search-text-to-video-on-chicagofswild-test.md>): Fingerspelling Video Search (Text to Video); ChicagoFSWild (test); mAP: 39.4.

- [Fingerspelling Video Search (Text to Video) on ChicagoFSWild+ (test)](<https://www.sota2.com/research/sota/fingerspelling-video-search-text-to-video-on-chicagofswild-plus-test.md>): Fingerspelling Video Search (Text to Video); ChicagoFSWild+ (test); mAP: 63.8.

- [Fingerspelling Word Search (Video to Text) on ChicagoFSWild (test)](<https://www.sota2.com/research/sota/fingerspelling-word-search-video-to-text-on-chicagofswild-test.md>): Fingerspelling Word Search (Video to Text); ChicagoFSWild (test); mAP: 43.4.

- [Fingerspelling Word Search (Video to Text) on ChicagoFSWild+ (test)](<https://www.sota2.com/research/sota/fingerspelling-word-search-video-to-text-on-chicagofswild-plus-test.md>): Fingerspelling Word Search (Video to Text); ChicagoFSWild+ (test); mAP: 67.4.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2203.13291>)

[PDF](<https://arxiv.org/pdf/2203.13291.pdf>)