ATTENTION: This is a web archive! The IMS Group was split up in 2018 and does not exist anymore. Recent work of former members can be found at the VR/AR Group and the Computer Vision Group.

Interactive Media Systems, TU Wien

Techniques for Improving Mobile Video Creation

Thesis by Dominik Schörkhuber

Supervision by Margrit Gelautz

Abstract

In this thesis, we explore methods to assist non-professional users with video creation on mobile devices. The developed algorithms are embedded into a video creation application featuring a storyboard based workflow. We present three kinds of assistance systems which help the user avoiding mistakes commonly made by amateur users and follow cinematographic guidelines during recording. In order to improve the resulting video quality, we address the problems of (a) video stabilization, (b) shot-type classification, and (c) lens occlusion. In the context of video stabilization, the camera path is first reconstructed and then different optimization strategies are employed to improve the camera path. We use a Linear Programming approach to create a piece-wise linear path and compare it with a local smoothing method. Next, we present an approach to automatically infer the shot-type for a scene observed by a camera. Person keypoint detectors are used to extract joint information for all actors. We compute the skeletal representation of the main actor and classify it into a cinematographic description of the scene. Among the compared approaches for classification, support vector machines showed the best performance. For training and evaluation, we produce datasets based on image recordings at a set distance and manually annotated movie scenes. The result can be compared to a given storyboard in order to give feedback to the user accordingly. Finally, we address the problem of accidentally occluding the camera lens, which is a common mistake during recording with a smart phone. We formulate this task as a semantic segmentation problem and solve it with classical image processing as well as a deep learning method. The classical image processing approach is clearly outperformed by a combination of Mobilenets and Fully Convolutional Neural Networks.

Reference

D. Schörkhuber: "Techniques for Improving Mobile Video Creation"; Supervisor: M. Gelautz; Fakultät für Informatik der Technischen Universität Wien, 2018.

BibTeX

Click into the text area and press Ctrl+A/Ctrl+C or ⌘+A/⌘+C to copy the BibTeX into your clipboard… or download the BibTeX.