Visual Servoing Platform  version 3.5.1 under development (2023-02-04)
Tutorial: Planar image projection

Table of Contents

Introduction

The aim of this tutorial is to explain how to use vpImageSimulator class to project an image of a planar scene at a given camera position. For example, this capability can then be used during the simulation of a visual-servo as described in Tutorial: Image-based visual servo to introduce an image processing.

All the material (source code and images) described in this tutorial is part of ViSP source code and could be downloaded using the following command:

$ svn export https://github.com/lagadic/visp.git/trunk/tutorial/simulator/image

Image projection

Given the image of a planar 20cm by 20cm square target as the one presented in the next image, we show here after how to project this image at a given camera position, and how to get the resulting image.

Image of a planar 20cm by 20cm square target.

This is done by the following code also available in tutorial-image-simulator.cpp:

#include <visp3/gui/vpDisplayGDI.h>
#include <visp3/gui/vpDisplayOpenCV.h>
#include <visp3/gui/vpDisplayX.h>
#include <visp3/io/vpImageIo.h>
#include <visp3/robot/vpImageSimulator.h>
int main()
{
try {
vpImageIo::read(target, "./target_square.pgm");
for (int i = 0; i < 4; i++)
X[i].resize(3);
// Top left Top right Bottom right Bottom left
X[0][0] = -0.1;
X[1][0] = 0.1;
X[2][0] = 0.1;
X[3][0] = -0.1;
X[0][1] = -0.1;
X[1][1] = -0.1;
X[2][1] = 0.1;
X[3][1] = 0.1;
X[0][2] = 0;
X[1][2] = 0;
X[2][2] = 0;
X[3][2] = 0;
vpCameraParameters cam(840, 840, I.getWidth() / 2, I.getHeight() / 2);
vpHomogeneousMatrix cMo(0, 0, 0.35, 0, vpMath::rad(30), vpMath::rad(15));
sim.init(target, X);
// Get the new image of the projected planar image target
sim.getImage(I, cam);
try {
vpImageIo::write(I, "./rendered_image.jpg");
} catch (...) {
std::cout << "Unsupported image format" << std::endl;
}
#if defined(VISP_HAVE_X11)
vpDisplayX d(I);
#elif defined(VISP_HAVE_GDI)
#elif defined(VISP_HAVE_OPENCV)
#else
std::cout << "No image viewer is available..." << std::endl;
#endif
vpDisplay::setTitle(I, "Planar image projection");
std::cout << "A click to quit..." << std::endl;
} catch (const vpException &e) {
std::cout << "Catch an exception: " << e << std::endl;
}
}
Generic class defining intrinsic camera parameters.
Implementation of column vector and the associated operations.
Definition: vpColVector.h:131
Display for windows using GDI (available on any windows 32 platform).
Definition: vpDisplayGDI.h:129
The vpDisplayOpenCV allows to display image using the OpenCV library. Thus to enable this class OpenC...
Use the X11 console to display images on unix-like OS. Thus to enable this class X11 should be instal...
Definition: vpDisplayX.h:135
static bool getClick(const vpImage< unsigned char > &I, bool blocking=true)
static void display(const vpImage< unsigned char > &I)
static void setTitle(const vpImage< unsigned char > &I, const std::string &windowtitle)
static void flush(const vpImage< unsigned char > &I)
error that can be emited by ViSP classes.
Definition: vpException.h:72
Implementation of an homogeneous matrix and operations on such kind of matrices.
static void read(vpImage< unsigned char > &I, const std::string &filename, int backend=IO_DEFAULT_BACKEND)
Definition: vpImageIo.cpp:148
static void write(const vpImage< unsigned char > &I, const std::string &filename, int backend=IO_DEFAULT_BACKEND)
Definition: vpImageIo.cpp:292
Class which enables to project an image in the 3D space and get the view of a virtual camera.
void getImage(vpImage< unsigned char > &I, const vpCameraParameters &cam)
void init(const vpImage< unsigned char > &I, vpColVector *X)
void setCleanPreviousImage(const bool &clean, const vpColor &color=vpColor::white)
void setInterpolationType(const vpInterpolationType interplt)
void setCameraPosition(const vpHomogeneousMatrix &cMt)
unsigned int getWidth() const
Definition: vpImage.h:247
unsigned int getHeight() const
Definition: vpImage.h:189
static double rad(double deg)
Definition: vpMath.h:121

The result of this program is shown in the next image.

Resulting projection of the planar image at a given camera position.

The provide hereafter the explanation of the new lines that were introduced.

#include <visp3/robot/vpImageSimulator.h>

Include the header of the vpImageSimulator class that allows to project an image to a given camera position.

Then in the main() function we create an instance of a gray level image that corresponds to the image of the planar target, and then we read the image from the disk.

vpImageIo::read(target, "./target_square.pgm");

Since the previous image corresponds to a 20cm by 20cm target, we initialize the 3D coordinates of each corner in the plane Z=0. Each

for (int i = 0; i < 4; i++)
X[i].resize(3);
// Top left Top right Bottom right Bottom left
X[0][0] = -0.1;
X[1][0] = 0.1;
X[2][0] = 0.1;
X[3][0] = -0.1;
X[0][1] = -0.1;
X[1][1] = -0.1;
X[2][1] = 0.1;
X[3][1] = 0.1;
X[0][2] = 0;
X[1][2] = 0;
X[2][2] = 0;
X[3][2] = 0;

Then we create an instance of the image I that will contain the rendered image from a given camera position.

Since the projection depends on the camera, we set its intrinsic parameters.

vpCameraParameters cam(840, 840, I.getWidth() / 2, I.getHeight() / 2);

We also set the render position of the camera as an homogeneous transformation between the camera frame and the target frame.

vpHomogeneousMatrix cMo(0, 0, 0.35, 0, vpMath::rad(30), vpMath::rad(15));

We create here an instance of the planar image projector, set the interpolation to bilinear and initialize the projector with the image of the target and the coordinates of its corners.

Now to retrieve the rendered image we first clean the content of the image to render, set the camera position, and finally get the image using the camera parameters.

sim.getImage(I, cam);

Then, if libjpeg is available, the rendered image is saved in the same directory then the executable.

try {
vpImageIo::write(I, "./rendered_image.jpg");
} catch (...) {
std::cout << "Unsupported image format" << std::endl;
}

Finally, as in Tutorial: How to create and build a project that uses ViSP and CMake on Unix or Windows we open a window to display the rendered image.

Note that this planar image projection capability has been also introduced in vpVirtualGrabber class exploited in tutorial-ibvs-4pts-image-tracking.cpp. Thus the next Tutorial: Image-based visual servo shows how to use it in order to introduce an image processing that does the tracking of the target during a visual-servo simulation.