ATTENTION: This is a web archive! The IMS Group was split up in 2018 and does not exist anymore. Recent work of former members can be found at the VR/AR Group and the Computer Vision Group.

Interactive Media Systems, TU Wien

Highly Overparameterized Optical Flow Using PatchMatch Belief Propagation

By Michael Hornacek, Frederic Besse, Jan Kautz, Andrew Fitzgibbon, and Carsten Rother

Abstract

Motion in the image plane is ultimately a function of 3D motion in space. We propose to compute optical flow using what is ostensibly an extreme overparameterization: depth, surface normal, and frame-to-frame 3D rigid body motion at every pixel, giving a total of 9 DoF. The advantages of such an overparameterization are twofold: first, geometrically meaningful reasoning can be called upon in the optimization, reflecting possible 3D motion in the underlying scene; second, the `fronto-parallel´ assumption implicit in the use of traditional matching pixel windows is ameliorated because the parameterization determines a plane-induced homography at every pixel. We show that optimization over this high-dimensional, continuous state space can be carried out using an adaptation of the recently introduced PatchMatch Belief Propagation (PMBP) energy minimization algorithm, and that the resulting flow fields compare favorably to the state of the art on a number of small and large-displacement datasets.

Reference

M. Hornacek, F. Besse, J. Kautz, A. Fitzgibbon, C. Rother: "Highly Overparameterized Optical Flow Using PatchMatch Belief Propagation"; Poster: European Conference on Computer Vision (ECCV) 2014, Zürich, Schweiz; 09-06-2014 - 09-12-2014; in: "European Conference on Computer Vision", (2014), 220 - 234.

BibTeX

Click into the text area and press Ctrl+A/Ctrl+C or ⌘+A/⌘+C to copy the BibTeX into your clipboard… or download the BibTeX.