LIDAR Constrained NeRF on Outdoor Scenes

Last updated on Oct 20, 2022 Computer Vision, Computer Graphics, Deep Learning
PDF
Depth Map comparisons Depth Map comparisons

The goal of this project was to extend the NeRF architecture to make it work on outdoor scenes. In our case, we focused on autonomous driving scenarios with the KITTI-360 dataset. As baseline we built upon the DS-NeRF architecture and adapted it to work on LIDAR data and NDC space to deal with unbounded scenes. The main methods we have tried were:

  1. Depth Extrapolation + Inverse Depth Smoothness Loss as a regularizer
  2. Semantic Segmentation Loss to guide the optimization process
  3. Feature Loss (using a pre-trained VGG19 to create feature vectors)

Depth extrapolation steps

Computer Vision Computer Graphics Deep Learning
Yiğit Aras Tunalı
Yiğit Aras Tunalı
MSc. Student at Technische Universität München

My research interests include Computer Vision, Computer Graphics, Machine Learning and Simulations.

Related