ATTENTION: This is a web archive! The IMS Group was split up in 2018 and does not exist anymore. Recent work of former members can be found at the VR/AR Group and the Computer Vision Group.

Interactive Media Systems, TU Wien

Depth Super Resolution by Rigid Body Self-Similarity in 3D

By Michael Hornacek, Christoph Rhemann, Margrit Gelautz, and Carsten Rother

Abstract

We tackle the problem of jointly increasing the spatial resolution and apparent measurement accuracy of an input low-resolution, noisy, and perhaps heavily quantized depth map. In stark contrast to earlier work, we make no use of ancillary data like a color image at the target resolution, multiple aligned depth maps, or a database of high-resolution depth exemplars. Instead, we proceed by identifying and merging patch correspondences within the input depth map itself, exploiting patchwise scene self-similarity across depth such as repetition of geometric primitives or object symmetry. While the notion of 'single-image' super resolution has successfully been applied in the context of color and intensity images, we are to our knowledge the first to present a tailored analogue for depth images. Rather than reason in terms of patches of 2D pixels as others have before us, our key contribution is to proceed by reasoning in terms of patches of 3D points, with matched patch pairs related by a respective 6 DoF rigid body motion in 3D. In support of obtaining a dense correspondence field in reasonable time, we introduce a new 3D variant of PatchMatch. A third contribution is a simple, yet effective patch upscaling and merging technique, which predicts sharp object boundaries at the target resolution. We show that our results are highly competitive with those of alternative techniques leveraging even a color image at the target resolution or a database of high-resolution depth exemplars.

Reference

M. Hornacek, C. Rhemann, M. Gelautz, C. Rother: "Depth Super Resolution by Rigid Body Self-Similarity in 3D"; Poster: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2013, Portland, Oregon, USA; 06-23-2013 - 06-28-2013; in: "IEEE Computer Vision and Pattern Recognition", (2013), 1123 - 1130.

Additional Information:

Supplementary material:

Poster (CVPR)

Talk (UCL/McGill)

Downloads

Poster 7.26 MB PDF document Download
Talk 9.48 MB PDF document Download

BibTeX

Click into the text area and press Ctrl+A/Ctrl+C or ⌘+A/⌘+C to copy the BibTeX into your clipboard… or download the BibTeX.