Unsupervised Monocular Depth and Ego-motion Learning with Structure and Semantics

About

We present an approach which takes advantage of both structure and semantics for unsupervised monocular learning of depth and ego-motion. More specifically, we model the motion of individual objects and learn their 3D motion vector jointly with depth and ego-motion. We obtain more accurate results, especially for challenging dynamic scenes not addressed by previous approaches. This is an extended version of Casser et al. [AAAI'19]. Code and models have been open sourced at https://sites.google.com/corp/view/struct2depth.

Vincent Casser, Soeren Pirk, Reza Mahjourian, Anelia Angelova · 2019

Related benchmarks

Task	Dataset	Result
Depth Estimation	KITTI (Eigen split)	RMSE4.7503	291
Monocular Depth Estimation	KITTI (test)	Abs Rel Error0.1529	114
Monocular Depth Estimation	Cityscapes	Accuracy (delta < 1.25)82.6	74
Depth Prediction	Cityscapes (test)	RMSE7.0237	52
Depth Estimation	Cityscapes (test)	Abs Err—	40
Monocular Depth Estimation	Cityscapes (test)	RMSE7.024	31
Depth Estimation	KITTI 50m cap (test)	Abs Rel0.103	24
Monocular Depth Estimation	Cityscapes 12 (test)	Abs Rel0.145	11
Depth Estimation	KITTI 80m cap revised evaluation code (test)	Abs Rel0.1108	9