# Traj2Action: A Co-Denoising Framework for Trajectory-Guided Human-to-Robot Skill Transfer

Source: [Traj2Action: A Co-Denoising Framework for Trajectory-Guided Human-to-Robot Skill Transfer](<https://www.sota2.com/research/paper/68fa7bc31e209131cea8a464>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-04-14T05:00:04.234Z

Authors: Han Zhou, Jinjin Cao, Liyuan Ma, Xueji Fang, Guo-jun Qi

Published: 2025-10-01T04:21:12.000Z

## Abstract

Learning diverse manipulation skills for real-world robots is severely bottlenecked by the reliance on costly and hard-to-scale teleoperated demonstrations. While human videos offer a scalable alternative, effectively transferring manipulation knowledge is fundamentally hindered by the significant morphological gap between human and robotic embodiments. To address this challenge and facilitate skill transfer from human to robot, we introduce Traj2Action, a novel framework that bridges this embodiment gap by using the 3D trajectory of the operational endpoint as a unified intermediate representation, and then transfers the manipulation knowledge embedded in this trajectory to the robot's actions. Our policy first learns to generate a coarse trajectory, which forms a high-level motion plan by leveraging both human and robot data. This plan then conditions the synthesis of precise, robot-specific actions (e.g., orientation and gripper state) within a co-denoising framework. Our work centers on two core objectives: first, the systematic verification of the Traj2Action framework's effectiveness-spanning architectural design, cross-task generalization, and data efficiency and second, the revelation of key laws that govern robot policy learning during the integration of human hand demonstration data. This research focus enables us to provide a scalable paradigm tailored to address human-to-robot skill transfer across morphological gaps. Extensive real-world experiments on a Franka robot demonstrate that Traj2Action boosts the performance by up to 27% and 22.25% over $\\pi\_0$ baseline on short- and long-horizon real-world tasks, and achieves significant gains as human data scales in robot policy learning.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Robotic Manipulation on PWB Short-Horizon](<https://www.sota2.com/research/sota/robotic-manipulation-on-pwb-short-horizon.md>): Robotic Manipulation; PWB Short-Horizon; Success Rate: 76.

- [Robotic Manipulation on PTT (Pick up Tomato and put it in the Tray) Short-Horizon](<https://www.sota2.com/research/sota/robotic-manipulation-on-ptt-pick-up-tomato-and-put-it-in-the-tray-short-horizon.md>): Robotic Manipulation; PTT (Pick up Tomato and put it in the Tray) Short-Horizon; Success Rate (SR): 76.

- [Robotic Manipulation on SRP (Stack the Rings on the Pillar) Long-Horizon](<https://www.sota2.com/research/sota/robotic-manipulation-on-srp-stack-the-rings-on-the-pillar-long-horizon.md>): Robotic Manipulation; SRP (Stack the Rings on the Pillar) Long-Horizon; TP Rate: 44.75.

- [Robotic Manipulation on SPC (Stack the Paper Cups) Long-Horizon](<https://www.sota2.com/research/sota/robotic-manipulation-on-spc-stack-the-paper-cups-long-horizon.md>): Robotic Manipulation; SPC (Stack the Paper Cups) Long-Horizon; TP Rate (%): 61.25.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2510.00491>)

[PDF](<https://arxiv.org/pdf/2510.00491.pdf>)
