# GraspNeRF: Multiview-based 6-DoF Grasp Detection for Transparent and Specular Objects Using Generalizable NeRF Source: [GraspNeRF: Multiview-based 6-DoF Grasp Detection for Transparent and Specular Objects Using Generalizable NeRF]() Collection index: [llms.txt]() Data updated: 2025-10-26T01:05:49.615Z Authors: Qiyu Dai, Yan Zhu, Yiran Geng, Ciyu Ruan, Jiazhao Zhang, He Wang Published: 2022-10-12T20:31:23.000Z ## Abstract In this work, we tackle 6-DoF grasp detection for transparent and specular objects, which is an important yet challenging problem in vision-based robotic systems, due to the failure of depth cameras in sensing their geometry. We, for the first time, propose a multiview RGB-based 6-DoF grasp detection network, GraspNeRF, that leverages the generalizable neural radiance field (NeRF) to achieve material-agnostic object grasping in clutter. Compared to the existing NeRF-based 3-DoF grasp detection methods that rely on densely captured input images and time-consuming per-scene optimization, our system can perform zero-shot NeRF construction with sparse RGB inputs and reliably detect 6-DoF grasps, both in real-time. The proposed framework jointly learns generalizable NeRF and grasp detection in an end-to-end manner, optimizing the scene representation construction for the grasping. For training data, we generate a large-scale photorealistic domain-randomized synthetic dataset of grasping in cluttered tabletop scenes that enables direct transfer to the real world. Our extensive experiments in synthetic and real-world environments demonstrate that our method significantly outperforms all the baselines in all the experiments while remaining in real-time. Project page can be found at https://pku-epic.github.io/GraspNeRF ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [6-DoF Grasping on GraspNet-1Billion Novel Kinect](): 6-DoF Grasping; GraspNet-1Billion Novel Kinect; AP: 12.86. - [6-DoF Grasping on GraspNet-1Billion Average RealSense](): 6-DoF Grasping; GraspNet-1Billion Average RealSense; AP: 15.91. - [6-DoF Grasping on GraspNet-1Billion Seen Kinect](): 6-DoF Grasping; GraspNet-1Billion Seen Kinect; AP: 24.61. - [6-DoF Grasping on GraspNet-1Billion Similar Kinect](): 6-DoF Grasping; GraspNet-1Billion Similar Kinect; AP: 17.67. - [6-DoF Grasping on GraspNet-1Billion Average Kinect](): 6-DoF Grasping; GraspNet-1Billion Average Kinect; AP: 18.38. - [Grasp Detection on GraspNet-1Billion RealSense Novel](): Grasp Detection; GraspNet-1Billion RealSense Novel; AP: 11.08. - [Grasp Detection on GraspNet-1Billion RealSense (Seen)](): Grasp Detection; GraspNet-1Billion RealSense (Seen); AP: 22.49. - [Grasp Detection on GraspNet-1Billion RealSense Similar](): Grasp Detection; GraspNet-1Billion RealSense Similar; AP: 0.1415. ## Original sources and implementation [Original paper]() [PDF]()