3D Object Detection from a Single Fisheye Image Without a Single Fisheye Training Image

Existing monocular 3D object detection methods have been demonstrated on rectilinear perspective images and fail in images with alternative projections such as those acquired by fisheye cameras. Previous works on object detection in fisheye images have focused on 2D object detection, partly due to the lack of 3D datasets of such images. In this work, we show how to use existing monocular 3D object detection models, trained only on rectilinear images, to detect 3D objects in images from fisheye cameras, without using any fisheye training data. We outperform the only existing method for monocular 3D object detection in panoramas on a benchmark of synthetic data, despite the fact that the existing method trains on the target non-rectilinear projection whereas we train only on rectilinear images. We also experiment with an internal dataset of real fisheye images.

Elad Plaut, Erez Ben Yaacov, Bat El Shlomo · 2020