Visual Servoing Platform  version 3.2.0 under development (2019-01-22)
Tutorial: Planar image projection

Table of Contents

Introduction

The aim of this tutorial is to explain how to use vpImageSimulator class to project an image of a planar scene at a given camera position. For example, this capability can then be used during the simulation of a visual-servo as described in Tutorial: Image-based visual servo to introduce an image processing.

All the material (source code and images) described in this tutorial is part of ViSP source code and could be downloaded using the following command:

$ svn export https://github.com/lagadic/visp.git/trunk/tutorial/simulator/image

Image projection

Given the image of a planar 20cm by 20cm square target as the one presented in the next image, we show here after how to project this image at a given camera position, and how to get the resulting image.

img-target-square.png
Image of a planar 20cm by 20cm square target.

This is done by the following code also available in tutorial-image-simulator.cpp:

#include <visp3/gui/vpDisplayGDI.h>
#include <visp3/gui/vpDisplayOpenCV.h>
#include <visp3/gui/vpDisplayX.h>
#include <visp3/io/vpImageIo.h>
#include <visp3/robot/vpImageSimulator.h>
int main()
{
try {
vpImageIo::read(target, "./target_square.pgm");
for (int i = 0; i < 4; i++)
X[i].resize(3);
// Top left Top right Bottom right Bottom left
X[0][0] = -0.1;
X[1][0] = 0.1;
X[2][0] = 0.1;
X[3][0] = -0.1;
X[0][1] = -0.1;
X[1][1] = -0.1;
X[2][1] = 0.1;
X[3][1] = 0.1;
X[0][2] = 0;
X[1][2] = 0;
X[2][2] = 0;
X[3][2] = 0;
vpCameraParameters cam(840, 840, I.getWidth() / 2, I.getHeight() / 2);
vpHomogeneousMatrix cMo(0, 0, 0.35, 0, vpMath::rad(30), vpMath::rad(15));
sim.init(target, X);
// Get the new image of the projected planar image target
sim.getImage(I, cam);
try {
vpImageIo::write(I, "./rendered_image.jpg");
} catch (...) {
std::cout << "Unsupported image format" << std::endl;
}
#if defined(VISP_HAVE_X11)
vpDisplayX d(I);
#elif defined(VISP_HAVE_GDI)
#elif defined(VISP_HAVE_OPENCV)
#else
std::cout << "No image viewer is available..." << std::endl;
#endif
vpDisplay::setTitle(I, "Planar image projection");
std::cout << "A click to quit..." << std::endl;
} catch (const vpException &e) {
std::cout << "Catch an exception: " << e << std::endl;
}
}

The result of this program is shown in the next image.

img-simu-image-target-square.png
Resulting projection of the planar image at a given camera position.

The provide hereafter the explanation of the new lines that were introduced.

#include <visp3/robot/vpImageSimulator.h>

Include the header of the vpImageSimulator class that allows to project an image to a given camera position.

Then in the main() function we create an instance of a gray level image that corresponds to the image of the planar target, and then we read the image from the disk.

vpImageIo::read(target, "./target_square.pgm");

Since the previous image corresponds to a 20cm by 20cm target, we initialize the 3D coordinates of each corner in the plane Z=0. Each

for (int i = 0; i < 4; i++)
X[i].resize(3);
// Top left Top right Bottom right Bottom left
X[0][0] = -0.1;
X[1][0] = 0.1;
X[2][0] = 0.1;
X[3][0] = -0.1;
X[0][1] = -0.1;
X[1][1] = -0.1;
X[2][1] = 0.1;
X[3][1] = 0.1;
X[0][2] = 0;
X[1][2] = 0;
X[2][2] = 0;
X[3][2] = 0;

Then we create an instance of the image I that will contain the rendered image from a given camera position.

Since the projection depends on the camera, we set its intrinsic parameters.

vpCameraParameters cam(840, 840, I.getWidth() / 2, I.getHeight() / 2);

We also set the render position of the camera as an homogeneous transformation between the camera frame and the target frame.

vpHomogeneousMatrix cMo(0, 0, 0.35, 0, vpMath::rad(30), vpMath::rad(15));

We create here an instance of the planar image projector, set the interpolation to bilinear and initialize the projector with the image of the target and the coordinates of its corners.

Now to retrieve the rendered image we first clean the content of the image to render, set the camera position, and finally get the image using the camera parameters.

sim.getImage(I, cam);

Then, if libjpeg is available, the rendered image is saved in the same directory then the executable.

try {
vpImageIo::write(I, "./rendered_image.jpg");
} catch (...) {
std::cout << "Unsupported image format" << std::endl;
}

Finally, as in Tutorial: How to create and build a CMake project that uses ViSP on Unix or Windows we open a window to display the rendered image.

Note that this planar image projection capability has been also introduced in vpVirtualGrabber class exploited in tutorial-ibvs-4pts-image-tracking.cpp. Thus the next Tutorial: Image-based visual servo shows how to use it in order to introduce an image processing that does the tracking of the target during a visual-servo simulation.