3D Object Detection from RGB-D and Material Data in AR/VR Applications

Master Thesis, Bachelor Thesis, or Practicum

About This Topic

Experiment with the newest depth sensing technologies and develop tools to test their suitability for certain object detection and reconstruction tasks in AR/VR applications!

Additional Information

A novel type of RGB-D sensor (Trinamix XperYenZ http://www.trinamix.de/) allows not only to capture color and depth images, but also provides information about the object material. This is especially interesting with a view towards AR/VR applications, where the property can e.g. aid hand tracking by identifying skin. The task of this topic is to evaluate and extend existing object detection algorithms and assess how/if the performance can be improved with the additional information. The test tools should be based on an existing software framework, which already provides methods for object detection such as OpenCV or Point Cloud Library (PCL), to avoid integration/implementation overhead. Based on the chosen framework simple applications should be implemented to show the performance in standardized scenes. The effort of this topic can be adapted to different courses or theses, by choosing implementation of one or more test cases.

Hardware

Trinamix XperYenZ

Software

OpenCV, C++