Object oriented structure from motion : can a scribble help? -

Abstract

The concept of anywhere anytime scanning of 3D objects is very appealing. One promising solution to extract structure is to rely on a monocular camera to perform, what is well-known as Structure from Motion (SfM). In spite of the significant progress achieved in SfM in the past decades, the structures that are obtained are still well below par the quality of reconstruction obtained through laser scanning. This thesis attempts to utilize a minimal user interaction, in the form of a scribble, to guide the reconstruction process and improve the structure estimation part of SfM. The first form of guidance we will consider is the incorporation of scene priors on surfaces with pre-defined models such as planar scenes; we will then generalize the concept to objects of any kind by treating points in the scene non-uniformly, whereby the major focus would be on pertinent objects in the scene, leading to what we call object oriented SfM (OOSfM). To address this semantic variant of SfM, we formulate the bundle adjustment step in a novel manner, which emphasizes more the objects of interest. Furthermore, the proposed system should allow the user to select, at a post-acquisition stage, the object of focus and accordingly obtain different outputs. Testing is done on both real and synthetic datasets, and results of the different variants of the bundle adjustment formulation are reported and compared to the conventional (or vanilla) SfM pipeline results.