Visual Servoing Platform  version 3.6.1 under development (2024-06-20)
Tutorial: Pose estimation from QRcode

Introduction

This tutorial shows how to use Tutorial: Bar code detection and Tutorial: Pose estimation from points in order to estimate the pose of a QRcode. After a first step that enables QRcode detection, the pose estimation process is achieved from the location of the four QRcode corners. The 3D coordinates of the corners are set knowing the size of the QRcode while their 2D coordinates are extracted from the image and transformed in the image plane thanks to camera intrinsic parameters.

Note that all the material (source code and image) described in this tutorial is part of ViSP source code (in tutorial/computer-vision folder) and could be found in https://github.com/lagadic/visp/tree/master/tutorial/computer-vision.

Note also that ViSP has to be built with zbar 3rd party library that enables barcode detection. See zbar quick installation guide.

Pose estimation of a QRcode

In this section we consider the case of an image that may contain a 12 by 12 cm square QRcode. The camera should be calibrated (see Tutorial: Camera intrinsic calibration). For each QRcode that is detected thanks to vpDetectorQRCode class, we update the coordinates of the four corners as a vpPoint object and compute the pose from the four points thanks to vpPose class. This process is replicated in a while loop. The end of the loop is reached when the user click in the image that is displayed.

The corresponding source code also provided in tutorial-pose-from-qrcode-image.cpp is the following.

#include <visp3/core/vpConfig.h>
#include <visp3/core/vpIoTools.h>
#include <visp3/core/vpPixelMeterConversion.h>
#include <visp3/detection/vpDetectorQRCode.h>
#include <visp3/gui/vpDisplayGDI.h>
#include <visp3/gui/vpDisplayOpenCV.h>
#include <visp3/gui/vpDisplayX.h>
#include <visp3/io/vpImageIo.h>
#include <visp3/vision/vpPose.h>
#include "pose_helper.h"
int main(int, char *argv[])
{
#if defined(VISP_HAVE_ZBAR)
#ifdef ENABLE_VISP_NAMESPACE
using namespace VISP_NAMESPACE_NAME;
#endif
try {
vpImageIo::read(I, vpIoTools::getParent(argv[0]) + "/data/bar-code.pgm");
#if defined(VISP_HAVE_X11)
vpDisplayX d(I);
#elif defined(VISP_HAVE_GDI)
#elif defined(HAVE_OPENCV_HIGHGUI)
#endif
// Camera parameters should be adapted to your camera
vpCameraParameters cam(840, 840, I.getWidth() / 2, I.getHeight() / 2);
// 3D model of the QRcode: here we consider a 12cm by 12cm QRcode
std::vector<vpPoint> point;
point.push_back(vpPoint(-0.06, -0.06, 0)); // QRcode point 0 3D coordinates in plane Z=0
point.push_back(vpPoint(0.06, -0.06, 0)); // QRcode point 1 3D coordinates in plane Z=0
point.push_back(vpPoint(0.06, 0.06, 0)); // QRcode point 2 3D coordinates in plane Z=0
point.push_back(vpPoint(-0.06, 0.06, 0)); // QRcode point 3 3D coordinates in plane Z=0
bool init = true;
vpDetectorQRCode detector;
while (1) {
vpImageIo::read(I, vpIoTools::getParent(argv[0]) + "/data/bar-code.pgm");
bool status = detector.detect(I);
std::ostringstream legend;
legend << detector.getNbObjects() << " bar code detected";
vpDisplay::displayText(I, (int)I.getHeight() - 30, 10, legend.str(), vpColor::red);
if (status) { // true if at least one QRcode is detected
for (size_t i = 0; i < detector.getNbObjects(); i++) {
std::vector<vpImagePoint> p = detector.getPolygon(i); // get the four corners location in the image
for (size_t j = 0; j < p.size(); j++) {
std::ostringstream number;
number << j;
vpDisplay::displayText(I, p[j] + vpImagePoint(15, 5), number.str(), vpColor::blue);
}
computePose(point, p, cam, init, cMo); // resulting pose is available in cMo var
std::cout << "Pose translation (meter): " << cMo.getTranslationVector().t() << std::endl
<< "Pose rotation (quaternion): " << vpQuaternionVector(cMo.getRotationMatrix()).t() << std::endl;
vpDisplay::displayFrame(I, cMo, cam, 0.05, vpColor::none, 3);
}
}
vpDisplay::displayText(I, (int)I.getHeight() - 15, 10, "A click to quit...", vpColor::red);
if (vpDisplay::getClick(I, false))
break;
}
}
catch (const vpException &e) {
std::cout << "Catch an exception: " << e.getMessage() << std::endl;
}
#else
(void)argv;
std::cout << "ViSP is not build with zbar 3rd party." << std::endl;
#endif
}
Generic class defining intrinsic camera parameters.
static const vpColor red
Definition: vpColor.h:213
static const vpColor none
Definition: vpColor.h:225
static const vpColor blue
Definition: vpColor.h:219
std::vector< std::vector< vpImagePoint > > & getPolygon()
size_t getNbObjects() const
bool detect(const vpImage< unsigned char > &I) vp_override
Display for windows using GDI (available on any windows 32 platform).
Definition: vpDisplayGDI.h:132
The vpDisplayOpenCV allows to display image using the OpenCV library. Thus to enable this class OpenC...
Use the X11 console to display images on unix-like OS. Thus to enable this class X11 should be instal...
Definition: vpDisplayX.h:131
static bool getClick(const vpImage< unsigned char > &I, bool blocking=true)
static void display(const vpImage< unsigned char > &I)
static void displayFrame(const vpImage< unsigned char > &I, const vpHomogeneousMatrix &cMo, const vpCameraParameters &cam, double size, const vpColor &color=vpColor::none, unsigned int thickness=1, const vpImagePoint &offset=vpImagePoint(0, 0), const std::string &frameName="", const vpColor &textColor=vpColor::black, const vpImagePoint &textOffset=vpImagePoint(15, 15))
static void displayCross(const vpImage< unsigned char > &I, const vpImagePoint &ip, unsigned int size, const vpColor &color, unsigned int thickness=1)
static void flush(const vpImage< unsigned char > &I)
static void displayText(const vpImage< unsigned char > &I, const vpImagePoint &ip, const std::string &s, const vpColor &color)
error that can be emitted by ViSP classes.
Definition: vpException.h:60
const char * getMessage() const
Definition: vpException.cpp:65
Implementation of an homogeneous matrix and operations on such kind of matrices.
vpRotationMatrix getRotationMatrix() const
vpTranslationVector getTranslationVector() const
static void read(vpImage< unsigned char > &I, const std::string &filename, int backend=IO_DEFAULT_BACKEND)
Definition: vpImageIo.cpp:147
Class that defines a 2D point in an image. This class is useful for image processing and stores only ...
Definition: vpImagePoint.h:82
unsigned int getWidth() const
Definition: vpImage.h:246
unsigned int getHeight() const
Definition: vpImage.h:185
static std::string getParent(const std::string &pathname)
Definition: vpIoTools.cpp:2101
Class that defines a 3D point in the object frame and allows forward projection of a 3D point in the ...
Definition: vpPoint.h:79
Implementation of a rotation vector as quaternion angle minimal representation.
vpRowVector t() const
vpRowVector t() const
void init(VISP_NAMESPACE_ADDRESSING vpImage< unsigned char > &Iinput, VISP_NAMESPACE_ADDRESSING vpImage< unsigned char > &IcannyVisp, VISP_NAMESPACE_ADDRESSING vpImage< unsigned char > *p_dIx, VISP_NAMESPACE_ADDRESSING vpImage< unsigned char > *p_dIy, VISP_NAMESPACE_ADDRESSING vpImage< unsigned char > *p_IcannyimgFilter)
Initialize the different displays.
VISP_EXPORT int wait(double t0, double t)

Here is a screen shot of the resulting program:

More source code explanations could be found in Tutorial: Bar code detection and Tutorial: Pose estimation from points.

Note that adapting this tutorial to process images coming from a camera live stream consists in replacing the lines

vpImageIo::read(I, "bar-code.pgm");

by

g.aquire(I);

where g is nothing more than a grabber instantiated as a vp1394TwoGrabber, vpFlyCaptureGrabber, vpV4l2Grabber or vpRealSense. How to implement framegrabbing is explained in Tutorial: Image frame grabbing.

Next tutorial

You are now ready to see the next Tutorial: Homography estimation from points that shows how to estimate an homography from points.

If you have an RGB-D camera, you may also continue with Tutorial: Planar object pose estimation using RGB-D data..