Pose estimation for augmented reality
Pose estimation for augmented reality: a hands-on survey

Table of Contents


Augmented Reality (AR) has now progressed to the point where real-time applications are being considered and needed. At the same time it is important that synthetic elements are rendered and aligned in the scene in an accurate and visually acceptable way. In order to address these issues, real-time, robust and efficient tracking algorithms have to be considered. The tracking of objects in the scene amounts to calculating the location (or pose) between the camera and the scene.

In the paper [pdf]:

E. Marchand, H. Uchiyama and F. Spindler. Pose estimation for augmented reality:
a hands-on survey. IEEE Trans. on Visualization and Computer Graphics, 22(12):2633-2651, December 2016.

a brief but almost self contented introduction to the most important approaches dedicated to camera localization along with a survey of the extension that have been proposed in the recent years. We also try to link these methodological concepts to the main libraries and SDK available on the market.

The aim of this paper is then to provide researchers and practitioners with an almost comprehensive and consolidate introduction to effective tools to facilitate research in augmented reality. It is also dedicated to academics involved in teaching augmented reality at the undergraduate and graduate level.

For most of the presented approaches, we also provide links to code of short examples. This should allow readers to easily bridge the gap between theoretical aspects and practice. These examples have been written using OpenCV but also ViSP developed at Inria. This page contains the documentation of these documented source code proposed as a supplementary material of the paper.

We hope this article and source code will be accessible and interesting to experts and students alike.



The source code available from http://github.com/lagadic/camera_localization was designed to work with OpenCV or with ViSP. During CMake configuration the user can choose which of these two 3rd parties are used. Prior to build this project, the user has to download and install OpenCV and/or ViSP. For example under Ubuntu OpenCV installation is done using:

sudo apt-get libopencv-dev

while ViSP installation is performed using:

sudo apt-get libvisp-dev

How to build

Once ViSP is installed, download the lastest source code release from github https://github.com/lagadic/camera_localization/releases.

Unzip the archive:

$ unzip camera_localization-2.0.1.zip

or extract the code from tarball:

$ tar xvzf camera_localization-2.0.1.tar.gz

Using cmake run:

$ cd camera_localization
$ cmake .
$ make

To generate the documentation you can run:

$ make doc

Pose estimation relying on a 3D model

Pose estimation from a known model

In this section we give base algorithm for camera localization.

Extension to markerless model-based tracking

Pose estimation relying on an image model

Homography estimation

Direct motion estimation through template matching