Tutorial: Planar image projection

Table of Contents

• Introduction
• Image projection

Introduction

The aim of this tutorial is to explain how to use vpImageSimulator class to project an image of a planar scene at a given camera position. For example, this capability can then be used during the simulation of a visual-servo as described in Tutorial: Image-based visual servo to introduce an image processing.

Note that all the material (source code and images) described in this tutorial is part of ViSP source code (in simulator/image folder) and could be found in https://github.com/lagadic/visp/tree/master/simulator/image.

Image projection

Given the image of a planar 20cm by 20cm square target as the one presented in the next image, we show here after how to project this image at a given camera position, and how to get the resulting image.

Image of a planar 20cm by 20cm square target.

This is done by the following code also available in tutorial-image-simulator.cpp:

#include <visp3/core/vpConfig.h>
#include <visp3/gui/vpDisplayGDI.h>
#include <visp3/gui/vpDisplayOpenCV.h>
#include <visp3/gui/vpDisplayX.h>
#include <visp3/io/vpImageIo.h>
#include <visp3/robot/vpImageSimulator.h>
 
int main()
{
#ifdef ENABLE_VISP_NAMESPACE
  using namespace VISP_NAMESPACE_NAME;
#endif
  try {
    vpImage<unsigned char> target;
    vpImageIo::read(target, "./target_square.pgm");
 
    vpColVector X[4];
    for (int i = 0; i < 4; i++)
      X[i].resize(3);
    // Top left     Top right       Bottom right   Bottom left
    X[0][0] = -0.1;
    X[1][0] = 0.1;
    X[2][0] = 0.1;
    X[3][0] = -0.1;
    X[0][1] = -0.1;
    X[1][1] = -0.1;
    X[2][1] = 0.1;
    X[3][1] = 0.1;
    X[0][2] = 0;
    X[1][2] = 0;
    X[2][2] = 0;
    X[3][2] = 0;
 
    vpImage<unsigned char> I(480, 640);
    vpCameraParameters cam(840, 840, I.getWidth() / 2, I.getHeight() / 2);
    vpHomogeneousMatrix cMo(0, 0, 0.35, 0, vpMath::rad(30), vpMath::rad(15));
 
    vpImageSimulator sim;
    sim.setInterpolationType(vpImageSimulator::BILINEAR_INTERPOLATION);
    sim.init(target, X);
 
    // Get the new image of the projected planar image target
    sim.setCleanPreviousImage(true);
    sim.setCameraPosition(cMo);
    sim.getImage(I, cam);
 
    try {
      vpImageIo::write(I, "./rendered_image.jpg");
    }
    catch (...) {
      std::cout << "Unsupported image format" << std::endl;
    }
 
#if defined(VISP_HAVE_X11)
    vpDisplayX d(I);
#elif defined(VISP_HAVE_GDI)
    vpDisplayGDI d(I);
#elif defined(HAVE_OPENCV_HIGHGUI)
    vpDisplayOpenCV d(I);
#else
    std::cout << "No image viewer is available..." << std::endl;
#endif
 
    vpDisplay::setTitle(I, "Planar image projection");
    vpDisplay::display(I);
    vpDisplay::flush(I);
    std::cout << "A click to quit..." << std::endl;
    vpDisplay::getClick(I);
  }
  catch (const vpException &e) {
    std::cout << "Catch an exception: " << e << std::endl;
  }
}

The result of this program is shown in the next image.

Resulting projection of the planar image at a given camera position.

The provide hereafter the explanation of the new lines that were introduced.

#include <visp3/robot/vpImageSimulator.h>

Include the header of the vpImageSimulator class that allows to project an image to a given camera position.

Then in the main() function we create an instance of a gray level image that corresponds to the image of the planar target, and then we read the image from the disk.

    vpImage<unsigned char> target;
    vpImageIo::read(target, "./target_square.pgm");

Since the previous image corresponds to a 20cm by 20cm target, we initialize the 3D coordinates of each corner in the plane Z=0. Each

    vpColVector X[4];
    for (int i = 0; i < 4; i++)
      X[i].resize(3);
    // Top left     Top right       Bottom right   Bottom left
    X[0][0] = -0.1;
    X[1][0] = 0.1;
    X[2][0] = 0.1;
    X[3][0] = -0.1;
    X[0][1] = -0.1;
    X[1][1] = -0.1;
    X[2][1] = 0.1;
    X[3][1] = 0.1;
    X[0][2] = 0;
    X[1][2] = 0;
    X[2][2] = 0;
    X[3][2] = 0;

Then we create an instance of the image I that will contain the rendered image from a given camera position.

    vpImage<unsigned char> I(480, 640);

Since the projection depends on the camera, we set its intrinsic parameters.

    vpCameraParameters cam(840, 840, I.getWidth() / 2, I.getHeight() / 2);

We also set the render position of the camera as an homogeneous transformation between the camera frame and the target frame.

    vpHomogeneousMatrix cMo(0, 0, 0.35, 0, vpMath::rad(30), vpMath::rad(15));

We create here an instance of the planar image projector, set the interpolation to bilinear and initialize the projector with the image of the target and the coordinates of its corners.

    vpImageSimulator sim;
    sim.setInterpolationType(vpImageSimulator::BILINEAR_INTERPOLATION);
    sim.init(target, X);

Now to retrieve the rendered image we first clean the content of the image to render, set the camera position, and finally get the image using the camera parameters.

    sim.setCleanPreviousImage(true);
    sim.setCameraPosition(cMo);
    sim.getImage(I, cam);

Then, if libjpeg is available, the rendered image is saved in the same directory then the executable.

    try {
      vpImageIo::write(I, "./rendered_image.jpg");
    }
    catch (...) {
      std::cout << "Unsupported image format" << std::endl;
    }

Finally, as in Tutorial: How to create and build a project that uses ViSP and CMake on Unix or Windows we open a window to display the rendered image.

Note that this planar image projection capability has been also introduced in vpVirtualGrabber class exploited in tutorial-ibvs-4pts-image-tracking.cpp. Thus the next Tutorial: Image-based visual servo shows how to use it in order to introduce an image processing that does the tracking of the target during a visual-servo simulation.