# Learning Discrete Abstractions for Visual Rearrangement Tasks Using Vision-Guided Graph Coloring

Source: [Learning Discrete Abstractions for Visual Rearrangement Tasks Using Vision-Guided Graph Coloring](<https://www.sota2.com/research/paper/68fa7b611e209131cea88090>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-03-24T05:00:03.763Z

Authors: Abhiroop Ajith, Constantinos Chamzas

Published: 2025-09-17T22:25:06.000Z

## Abstract

Learning abstractions directly from data is a core challenge in robotics. Humans naturally operate at an abstract level, reasoning over high-level subgoals while delegating execution to low-level motor skills -- an ability that enables efficient problem solving in complex environments. In robotics, abstractions and hierarchical reasoning have long been central to planning, yet they are typically hand-engineered, demanding significant human effort and limiting scalability. Automating the discovery of useful abstractions directly from visual data would make planning frameworks more scalable and more applicable to real-world robotic domains. In this work, we focus on rearrangement tasks where the state is represented with raw images, and propose a method to induce discrete, graph-structured abstractions by combining structural constraints with an attention-guided visual distance. Our approach leverages the inherent bipartite structure of rearrangement problems, integrating structural constraints and visual embeddings into a unified framework. This enables the autonomous discovery of abstractions from vision alone, which can subsequently support high-level planning. We evaluate our method on two rearrangement tasks in simulation and show that it consistently identifies meaningful abstractions that facilitate effective planning and outperform existing approaches.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Visual Task Planning and Graph Learning on Multi Fruit 2x3](<https://www.sota2.com/research/sota/visual-task-planning-and-graph-learning-on-multi-fruit-2x3.md>): Visual Task Planning and Graph Learning; Multi Fruit 2x3; Opt: 66.

- [Visual Task Planning and Graph Learning on Blocks 2](<https://www.sota2.com/research/sota/visual-task-planning-and-graph-learning-on-blocks-2.md>): Visual Task Planning and Graph Learning; Blocks 2; Opt: 93.7.

- [Visual Task Planning and Graph Learning on Blocks 3](<https://www.sota2.com/research/sota/visual-task-planning-and-graph-learning-on-blocks-3.md>): Visual Task Planning and Graph Learning; Blocks 3; Opt: 72.

- [Visual Task Planning and Graph Learning on Fruit-4x6](<https://www.sota2.com/research/sota/visual-task-planning-and-graph-learning-on-fruit-4x6.md>): Visual Task Planning and Graph Learning; Fruit-4x6; Opt Rate: 78.1.

- [Visual Task Planning and Graph Learning on Multi Fruit 4x6](<https://www.sota2.com/research/sota/visual-task-planning-and-graph-learning-on-multi-fruit-4x6.md>): Visual Task Planning and Graph Learning; Multi Fruit 4x6; Opt: 71.8.

- [Visual Task Planning and Graph Learning on Fruit 6x8](<https://www.sota2.com/research/sota/visual-task-planning-and-graph-learning-on-fruit-6x8.md>): Visual Task Planning and Graph Learning; Fruit 6x8; Opt: 64.8.

- [Visual Task Planning and Graph Learning on Fruit-2x3](<https://www.sota2.com/research/sota/visual-task-planning-and-graph-learning-on-fruit-2x3.md>): Visual Task Planning and Graph Learning; Fruit-2x3; Optimality Score: 100.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2509.14460>)

[PDF](<https://arxiv.org/pdf/2509.14460.pdf>)