Authors:
Augusto R. Castro
1
;
Valdir Grassi Jr.
1
and
Moacir A. Ponti
2
Affiliations:
1
Department of Electrical and Computer Engineering, São Carlos School of Engineering, University of São Paulo, São Carlos, Brazil
;
2
Instituto de Ciências Matemáticas e de Computação, Universidade de São Paulo, São Carlos, Brazil
Keyword(s):
Deep Learning, Depth Completion, RGB+Depth, Depth Sensing, Distance Transforms.
Abstract:
Low-cost depth-sensing devices can provide real-time depth maps to many applications, such as robotics and augmented reality. However, due to physical limitations in the acquisition process, the depth map obtained can present missing areas corresponding to irregular, transparent, or reflective surfaces. Therefore, when there is more computing power than just the embedded processor in low-cost depth sensors, models developed to complete depth maps can boost the system's performance. To exploit the generalization capability of deep learning models, we propose a method composed of a U-Net followed by a refinement module to complete depth maps provided by Microsoft Kinect. We applied the Euclidean distance transform in the loss function to increase the influence of missing pixels when adjusting our network filters and reduce blur in predictions. We outperform state-of-the-art methods for completed depth maps in a benchmark dataset. Our novel loss function combining the distance transform
, gradient and structural similarity measure presents promising results in guiding the model to reduce unnecessary blurring of final depth maps predicted by a convolutional network.
(More)