# Few-shot Keypoint Detection with Uncertainty Learning for Unseen Species

Source: [Few-shot Keypoint Detection with Uncertainty Learning for Unseen Species](<https://www.sota2.com/research/paper/68fa6c301e209131cea32b44>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-01-04T05:12:47.002Z

Authors: Changsheng Lu, Piotr Koniusz

Published: 2021-12-12T08:39:47.000Z

## Abstract

Current non-rigid object keypoint detectors perform well on a chosen kind of species and body parts, and require a large amount of labelled keypoints for training. Moreover, their heatmaps, tailored to specific body parts, cannot recognize novel keypoints (keypoints not labelled for training) on unseen species. We raise an interesting yet challenging question: how to detect both base (annotated for training) and novel keypoints for unseen species given a few annotated samples? Thus, we propose a versatile Few-shot Keypoint Detection (FSKD) pipeline, which can detect a varying number of keypoints of different kinds. Our FSKD provides the uncertainty estimation of predicted keypoints. Specifically, FSKD involves main and auxiliary keypoint representation learning, similarity learning, and keypoint localization with uncertainty modeling to tackle the localization noise. Moreover, we model the uncertainty across groups of keypoints by multivariate Gaussian distribution to exploit implicit correlations between neighboring keypoints. We show the effectiveness of our FSKD on (i) novel keypoint detection for unseen species, and (ii) few-shot Fine-Grained Visual Recognition (FGVR) and (iii) Semantic Alignment (SA) downstream tasks. For FGVR, detected keypoints improve the classification accuracy. For SA, we showcase a novel thin-plate-spline warping that uses estimated keypoint uncertainty under imperfect keypoint corespondences.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Keypoint Detection on Animal Pose Dataset novel keypoints](<https://www.sota2.com/research/sota/keypoint-detection-on-animal-pose-dataset-novel-keypoints.md>): Keypoint Detection; Animal Pose Dataset novel keypoints; Cat Score: 52.36.

- [Keypoint Detection on CUB novel keypoints](<https://www.sota2.com/research/sota/keypoint-detection-on-cub-novel-keypoints.md>): Keypoint Detection; CUB novel keypoints; Keypoint Detection Score: 77.9.

- [Keypoint Detection on NABird novel keypoints](<https://www.sota2.com/research/sota/keypoint-detection-on-nabird-novel-keypoints.md>): Keypoint Detection; NABird novel keypoints; Score: 56.04.

- [Keypoint Detection on CUB Seen Species, Base Setting](<https://www.sota2.com/research/sota/keypoint-detection-on-cub-seen-species-base-setting.md>): Keypoint Detection; CUB Seen Species, Base Setting; PCK: 90.8.

- [Keypoint Detection on NABird Seen Species, Novel Setting](<https://www.sota2.com/research/sota/keypoint-detection-on-nabird-seen-species-novel-setting.md>): Keypoint Detection; NABird Seen Species, Novel Setting; PCK: 58.35.

- [Keypoint Detection on Animal Pose Dataset Unseen Species Base Setting](<https://www.sota2.com/research/sota/keypoint-detection-on-animal-pose-dataset-unseen-species-base-setting.md>): Keypoint Detection; Animal Pose Dataset Unseen Species Base Setting; Cat Score: 57.12.

- [Keypoint Detection on Animal Pose Dataset Seen Species, Novel Setting](<https://www.sota2.com/research/sota/keypoint-detection-on-animal-pose-dataset-seen-species-novel-setting.md>): Keypoint Detection; Animal Pose Dataset Seen Species, Novel Setting; Cat AP: 60.84.

- [Keypoint Detection on Animal Pose Dataset Seen Species Base Setting](<https://www.sota2.com/research/sota/keypoint-detection-on-animal-pose-dataset-seen-species-base-setting.md>): Keypoint Detection; Animal Pose Dataset Seen Species Base Setting; Cat Accuracy: 68.66.

- [Keypoint Detection on CUB Unseen Species, Base Setting](<https://www.sota2.com/research/sota/keypoint-detection-on-cub-unseen-species-base-setting.md>): Keypoint Detection; CUB Unseen Species, Base Setting; PCK: 87.94.

- [Keypoint Detection on CUB Seen Species, Novel Setting](<https://www.sota2.com/research/sota/keypoint-detection-on-cub-seen-species-novel-setting.md>): Keypoint Detection; CUB Seen Species, Novel Setting; PCK: 78.17.

- [Keypoint Detection on NABird Unseen Species Base Setting](<https://www.sota2.com/research/sota/keypoint-detection-on-nabird-unseen-species-base-setting.md>): Keypoint Detection; NABird Unseen Species Base Setting; PCK: 87.84.

- [Keypoint Detection on NABird Seen Species, Base Setting](<https://www.sota2.com/research/sota/keypoint-detection-on-nabird-seen-species-base-setting.md>): Keypoint Detection; NABird Seen Species, Base Setting; PCK: 88.16.

- [Keypoint Detection on DeepFashion 2](<https://www.sota2.com/research/sota/keypoint-detection-on-deepfashion-2.md>): Keypoint Detection; DeepFashion 2; PCK@0.1: 34.29.

- [Keypoint Detection on CUB (Novel species)](<https://www.sota2.com/research/sota/keypoint-detection-on-cub-novel-species.md>): Keypoint Detection; CUB (Novel species); PCK: 76.99.

- [Keypoint Detection on NABird Base species](<https://www.sota2.com/research/sota/keypoint-detection-on-nabird-base-species.md>): Keypoint Detection; NABird Base species; PCK: 84.74.

- [Keypoint Detection on Animal](<https://www.sota2.com/research/sota/keypoint-detection-on-animal.md>): Keypoint Detection; Animal; PCK@0.1: 48.81.

- [Keypoint Detection on CUB](<https://www.sota2.com/research/sota/keypoint-detection-on-cub.md>): Keypoint Detection; CUB; PCK@0.1: 79.45.

- [Keypoint Detection on NABird](<https://www.sota2.com/research/sota/keypoint-detection-on-nabird.md>): Keypoint Detection; NABird; PCK@0.1: 57.11.

- [Keypoint Detection on AwA Pose](<https://www.sota2.com/research/sota/keypoint-detection-on-awa-pose.md>): Keypoint Detection; AwA Pose; PCK@0.1: 72.2.

- [Fine-grained Visual Recognition on CUB (test)](<https://www.sota2.com/research/sota/fine-grained-visual-recognition-on-cub-test.md>): Fine-grained Visual Recognition; CUB (test); All-Shot Accuracy: 66.37.

- [Fine-grained visual classification on NABirds (test)](<https://www.sota2.com/research/sota/fine-grained-visual-classification-on-nabirds-test.md>): Fine-grained visual classification; NABirds (test); Top-1 Accuracy: not recorded.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2112.06183>)

[PDF](<https://arxiv.org/pdf/2112.06183.pdf>)

[Implementation](<https://github.com/AlanLuSun/Few-shot-keypoint-detection>)
