# Object segmentation in the wild with foundation models: application to vision assisted neuro-prostheses for upper limbs

Source: [Object segmentation in the wild with foundation models: application to vision assisted neuro-prostheses for upper limbs](<https://www.sota2.com/research/paper/68fa7a351e209131cea81ca0>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T05:15:13.470Z

Authors: Bolutife Atoki, Jenny Benois-Pineau, Renaud P\\'eteri, Fabien Baldacci, Aymar de Rugy

Published: 2025-07-24T15:40:44.000Z

## Abstract

In this work, we address the problem of semantic object segmentation using foundation models. We investigate whether foundation models, trained on a large number and variety of objects, can perform object segmentation without fine-tuning on specific images containing everyday objects, but in highly cluttered visual scenes. The ''in the wild'' context is driven by the target application of vision guided upper limb neuroprostheses. We propose a method for generating prompts based on gaze fixations to guide the Segment Anything Model (SAM) in our segmentation scenario, and fine-tune it on egocentric visual data. Evaluation results of our approach show an improvement of the IoU segmentation quality metric by up to 0.51 points on real-world challenging data of Grasping-in-the-Wild corpus which is made available on the RoboFlow Platform (https://universe.roboflow.com/iwrist/grasping-in-the-wild)

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2507.18517>)

[PDF](<https://arxiv.org/pdf/2507.18517.pdf>)
