Streaming and Exploration of Dynamically Changing Dense Surface Reconstructions in Immersive Virtual Reality
Thesis by Manuel Kröter
Supervision by Hannes Kaufmann and Annette Mossel
Abstract
Low cost commodity depth cameras like the Microsoft Kinect allow to sense the structure of the environment. With the aid of dense surface reconstruction methods, a detailed 3D model can be computed in real-time from the acquired camera data. Autonomous robots can apply this techniques in order to build a map of the scene while they are exploring it. This allows the robot to locate itself and to navigate in unknown environments. Besides that, the reconstructed model can be interesting for different parties, who want to explore these environments as well. Especially, distant or dangerous areas can be scanned by robots while remote observers are able to safely get an overview of the scene. In order to support remote exploration while the scene is still scanned, the reconstructed information has to be streamed incrementally over wireless network. Since currently no solution exists with this feature, the existing reconstruction framework InfiniTAM is extended to support the transmission of a large scale, dynamically changing model. The visualization and exploration of the model is performed with the aid of Unreal Engine 4, a state-of-the-art 3d engine. For this purpose, a triangular mesh representation is favored, while dense reconstruction methods mostly operate on a volumetric representation. In current approaches, the mesh is extracted in a post-processing step, which is not applicable when the scene should be explored while being scanned and updated. The used reconstruction framework is therefore adapted to maintain an up-to-date mesh in real-time. The reconstructed mesh finally is explored in a virtual reality setup using a head-mounted display and an omnidirectional treadmill. The usage of virtual reality hardware enhances the ease of use and makes it possible to navigate in a natural way. The developed system is evaluated in terms of memory requirements and data rates as well as within a user study, that analyzes the effect of the incremental streaming and the virtual reality exploration on spatial knowledge acquisition.
Reference
M. Kröter: "Streaming and Exploration of Dynamically Changing Dense Surface Reconstructions in Immersive Virtual Reality"; Supervisor: H. Kaufmann, A. Mossel; Institut für Softwaretechnik und Interaktive Systeme, 2016; final examination: 04-29-2016.
BibTeX
Click into the text area and press Ctrl+A/Ctrl+C or ⌘+A/⌘+C to copy the BibTeX into your clipboard… or download the BibTeX.