# Segment Anything Meets Point Tracking

Source: [Segment Anything Meets Point Tracking](<https://www.sota2.com/research/paper/68fa71551e209131cea50952>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T02:03:39.970Z

Authors: Frano Raji\\v{c}, Lei Ke, Yu-Wing Tai, Chi-Keung Tang, Martin Danelljan, Fisher Yu

Published: 2023-07-03T17:58:01.000Z

## Abstract

The Segment Anything Model (SAM) has established itself as a powerful zero-shot image segmentation model, enabled by efficient point-centric annotation and prompt-based models. While click and brush interactions are both well explored in interactive image segmentation, the existing methods on videos focus on mask annotation and propagation. This paper presents SAM-PT, a novel method for point-centric interactive video segmentation, empowered by SAM and long-term point tracking. SAM-PT leverages robust and sparse point selection and propagation techniques for mask generation. Compared to traditional object-centric mask propagation strategies, we uniquely use point propagation to exploit local structure information agnostic to object semantics. We highlight the merits of point-based tracking through direct evaluation on the zero-shot open-world Unidentified Video Objects (UVO) benchmark. Our experiments on popular video object segmentation and multi-object segmentation tracking benchmarks, including DAVIS, YouTube-VOS, and BDD100K, suggest that a point-based segmentation tracker yields better zero-shot performance and efficient interactions. We release our code that integrates different point trackers and video segmentation benchmarks at https://github.com/SysCV/sam-pt.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Video Object Segmentation on PanoVOS (test)](<https://www.sota2.com/research/sota/video-object-segmentation-on-panovos-test.md>): Video Object Segmentation; PanoVOS (test); J&F Score: 41.

- [Video Object Segmentation on PanoVOS (val)](<https://www.sota2.com/research/sota/video-object-segmentation-on-panovos-val.md>): Video Object Segmentation; PanoVOS (val); J&F Score: 47.5.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2307.01197>)

[PDF](<https://arxiv.org/pdf/2307.01197.pdf>)