# Hierarchical Prompting with Dual LLM Modules for Robotic Task and Motion Planning

Source: [Hierarchical Prompting with Dual LLM Modules for Robotic Task and Motion Planning](<https://www.sota2.com/research/paper/6a02b3f8dbc836b87d8245a0>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-07-01T05:00:04.992Z

Authors: Karolina \\'Zr\\'obek, Tessa Pulli, Pawe{\\l} Gajewski, Antonio Galiza Cerdeira Gonzalez, Bipin Indurkhya

Published: 2026-05-08T17:36:13.000Z

## Abstract

We present a hierarchical language-driven framework for robotic task and motion planning to improve natural, intuitive human-robot interaction in service and assistance scenarios. The proposed system employs two large language model (LLM) modules: a high-level planning agent and a low-level spatial reasoning sub-module. The primary agent processes natural language commands and generates action sequences using a ReAct-style prompt, interacting with tools for object perception and manipulation (e.g., pick, place, release). For precise spatial placement, such as interpreting "place the mug next to the plate", a separate sub-prompting module handles 3D reasoning based on object geometry and scene layout. The system integrates YOLOX-GDRNet for object detection and pose estimation, along with a motion execution stub. We evaluated the system in 24 test scenarios, ranging from simple spatial commands to high-level instructions and infeasible requests. The system achieved an overall task success rate of 86%.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2605.08330>)

[PDF](<https://arxiv.org/pdf/2605.08330.pdf>)
