# Manual2Skill++: Connector-Aware General Robotic Assembly from Instruction Manuals via Vision-Language Models

Source: [Manual2Skill++: Connector-Aware General Robotic Assembly from Instruction Manuals via Vision-Language Models](<https://www.sota2.com/research/paper/68fa7c531e209131cea8d907>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-03-21T05:00:04.296Z

Authors: Chenrui Tie, Shengxiang Sun, Yudi Lin, Yanbo Wang, Zhongrui Li, Zhouhan Zhong, Jinxuan Zhu, Yiman Pang, Haonan Chen, Junting Chen, Ruihai Wu, Lin Shao

Published: 2025-10-18T04:13:26.000Z

## Abstract

Assembly hinges on reliably forming connections between parts; yet most robotic approaches plan assembly sequences and part poses while treating connectors as an afterthought. Connections represent the foundational physical constraints of assembly execution; while task planning sequences operations, the precise establishment of these constraints ultimately determines assembly success. In this paper, we treat connections as explicit, primary entities in assembly representation, directly encoding connector types, specifications, and locations for every assembly step. Drawing inspiration from how humans learn assembly tasks through step-by-step instruction manuals, we present Manual2Skill++, a vision-language framework that automatically extracts structured connection information from assembly manuals. We encode assembly tasks as hierarchical graphs where nodes represent parts and sub-assemblies, and edges explicitly model connection relationships between components. A large-scale vision-language model parses symbolic diagrams and annotations in manuals to instantiate these graphs, leveraging the rich connection knowledge embedded in human-designed instructions. We curate a dataset containing over 20 assembly tasks with diverse connector types to validate our representation extraction approach, and evaluate the complete task understanding-to-execution pipeline across four complex assembly scenarios in simulation, spanning furniture, toys, and manufacturing components with real-world correspondence. More detailed information can be found at https://nus-lins-lab.github.io/Manual2SkillPP/

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Assembly Graph Extraction (Pair Matching) on Assembly Manual Dataset 85 steps (test)](<https://www.sota2.com/research/sota/assembly-graph-extraction-pair-matching-on-assembly-manual-dataset-85-steps-test.md>): Assembly Graph Extraction (Pair Matching); Assembly Manual Dataset 85 steps (test); F1 Score: 63.42.

- [Assembly Graph Extraction (Set Matching) on Assembly Manual Dataset (85 steps) (test)](<https://www.sota2.com/research/sota/assembly-graph-extraction-set-matching-on-assembly-manual-dataset-85-steps-test.md>): Assembly Graph Extraction (Set Matching); Assembly Manual Dataset 85 steps (test); F1 Score: 86.06.

- [Pose Alignment on Manual2Skill dataset](<https://www.sota2.com/research/sota/pose-alignment-on-manual2skill-dataset.md>): Pose Alignment; Manual2Skill dataset; Geometric Distance (GD): 0.03.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2510.16344>)

[PDF](<https://arxiv.org/pdf/2510.16344.pdf>)
