Interactive Post-Processing of 2D-Plus-Depth Videos: Algorithms, Implementation and Evaluation
Thesis by Braulio Sespede
Supervision by Margrit Gelautz and Florian Seitner
Abstract
One of the approaches to reconstruct the depth of real-world objects or scenes consists of using stereo cameras to record them in image or video format, and estimate the disparity between views of the captured scene with the aid of stereo correspondence algorithms. Due to disparity mismatches during the correspondence estimation, surface reconstruction algorithms can fail to generate accurate geometric representations. Additionally, when reconstructing 3D models from 2D-plus-depth video, flickering between frames might occur. Post-processing techniques can be used to remove inconsistencies, suppress noise, and correct erroneously labeled disparities. This thesis addresses the development and evaluation of automatic and user-supported post-processing techniques for correction of 2D-plus-depth videos. Multiple stereo cameras capture the videos from different viewing directions. An interactive RGBD segmentation algorithm based on efficient cost-volume filtering is used to guide local corrections and extract objects of interest for 3D animated reconstructions. Additionally, state-of-the-art filters are implemented to smooth surfaces and reduce temporal flicker. Finally, the techniques are integrated into a tool with interactive visualizations of 2D-plus-depth content. Results show that user-assisted segmentation enables the extraction of precise masks, allowing the correction of wrongly labeled disparities. Additionally, spatio-temporal edge-preserving filters such as the weighted median filter and guided filter prove to be key components of post-processing. Spatio-temporal filters not only smooth surfaces but also improve the consistency of segmentation masks as well. Finally, the addition of an interactive visualization tool enables a better understanding of the post-processing techniques and other elements of the pipeline such as the calibration and the underlying stereo correspondence algorithm.
Reference
B. Sespede: "Interactive Post-Processing of 2D-Plus-Depth Videos: Algorithms, Implementation and Evaluation"; Supervisor: M. Gelautz, F. Seitner; University of Applied Sciences Technikum Wien, 2018; final examination: 06-12-2018.
BibTeX
Click into the text area and press Ctrl+A/Ctrl+C or ⌘+A/⌘+C to copy the BibTeX into your clipboard… or download the BibTeX.