Visual Servoing Platform  version 3.1.0 under development (2017-09-20)
Tutorial: Bar code detection

Introduction

This tutorial shows how to detect one or more barcodes with ViSP. To this end we provide two classes that are wrapper over 3rd party libraries:

These classes inherit from vpDetectorBase class, a generic class dedicated to detection. For each detected bar code, it allows to retrieve some characteristics such as the bar code message if it exists, and in the image the polygon that contains the bar code, the bounding box or the center of gravity of the bar code.

In the next sections you will find examples that show how to detect codes in a single image and or in images acquired from a camera connected to your computer.

Note that all the material (source code and image) described in this tutorial is part of ViSP source code and could be downloaded using the following command:

$ svn export https://github.com/lagadic/visp.git/trunk/tutorial/detection/barcode

Bar code detection (single image)

The following example also available in tutorial-barcode-detector.cpp allows to detect either a QR code or Data Matrix on a single image. The user can select with –code-type option which code to detect.

#include <visp3/detection/vpDetectorDataMatrixCode.h>
#include <visp3/detection/vpDetectorQRCode.h>
#include <visp3/gui/vpDisplayGDI.h>
#include <visp3/gui/vpDisplayOpenCV.h>
#include <visp3/gui/vpDisplayX.h>
#include <visp3/io/vpImageIo.h>
int main(int argc, const char** argv)
{
#if (defined(VISP_HAVE_ZBAR) || defined(VISP_HAVE_DMTX)) && (defined(VISP_HAVE_X11) || defined(VISP_HAVE_GDI) || defined(VISP_HAVE_OPENCV))
try {
vpImageIo::read(I, "bar-code.pgm");
#ifdef VISP_HAVE_X11
vpDisplayX d(I);
#elif defined(VISP_HAVE_GDI)
#elif defined(VISP_HAVE_OPENCV)
#endif
vpDetectorBase *detector = NULL;
#if (defined(VISP_HAVE_ZBAR) && defined(VISP_HAVE_DMTX))
int opt_barcode = 0; // 0=QRCode, 1=DataMatrix
for (int i=0; i<argc; i++) {
if (std::string(argv[i]) == "--code-type")
opt_barcode = atoi(argv[i+1]);
else if (std::string(argv[i]) == "--help") {
std::cout << "Usage: " << argv[0]
<< " [--code-type <0 for QRcode | 1 for DataMatrix>] [--help]"
<< std::endl;
return 0;
}
}
if (opt_barcode == 0)
detector = new vpDetectorQRCode;
else
detector = new vpDetectorDataMatrixCode;
#elif defined(VISP_HAVE_ZBAR)
detector = new vpDetectorQRCode;
(void)argc;
(void)argv;
#elif defined(VISP_HAVE_DMTX)
detector = new vpDetectorDataMatrixCode;
(void)argc;
(void)argv;
#endif
bool status = detector->detect(I);
std::ostringstream legend;
legend << detector->getNbObjects() << " bar code detected";
vpDisplay::displayText(I, (int)I.getHeight()-30, 10, legend.str(), vpColor::red);
if (status) {
for(size_t i=0; i < detector->getNbObjects(); i++) {
std::vector<vpImagePoint> p = detector->getPolygon(i);
vpRect bbox = detector->getBBox(i);
vpDisplay::displayText(I, (int)(bbox.getTop()-10), (int)bbox.getLeft(),
"Message: \"" + detector->getMessage(i) + "\"",
vpColor::red);
for(size_t j=0; j < p.size(); j++) {
std::ostringstream number;
number << j;
vpDisplay::displayText(I, p[j]+vpImagePoint(15,5), number.str(), vpColor::blue);
}
}
vpDisplay::displayText(I, (int)I.getHeight()-15, 10, "A click to quit...", vpColor::red);
}
delete detector;
}
catch(const vpException &e) {
std::cout << "Catch an exception: " << e.getMessage() << std::endl;
}
#else
(void)argc;
(void)argv;
#endif
}

To detect a QR code just run:

$ ./tutorial-barcode-detector --code-type 0

You will get the following result:

img-detection-qrcode.png

To detect a Data Matrix code just run:

$ ./tutorial-barcode-detector --code-type 1

You will get the following result:

img-detection-datamatrix.png

Now we explain the main lines of the source.

First we have to include the header of the two classes that allow to detect either the QR code or the Data Matrix code.

#include <visp3/detection/vpDetectorDataMatrixCode.h>
#include <visp3/detection/vpDetectorQRCode.h>

Then in the main() function before going further we need to check if at least one of the third party (zbar or dmtx libraries) were used to build ViSP. We also check if ViSP is able to display images using either X11, or the Graphical Device Interface (GDI) under Windows, or OpenCV.

#if (defined(VISP_HAVE_ZBAR) || defined(VISP_HAVE_DMTX)) && (defined(VISP_HAVE_X11) || defined(VISP_HAVE_GDI) || defined(VISP_HAVE_OPENCV))

After reading the input image bar-code.pgm and creation of a display device in order to visualize the image, we create a NULL pointer to a detector base class.

vpDetectorBase *detector = NULL;

Since the classes that allow to detect QR codes and Data Matrix codes inherit from vpDetectorBase, using the variable opt_barcode we are able to construct the requested detector.

if (opt_barcode == 0)
detector = new vpDetectorQRCode;
else
detector = new vpDetectorDataMatrixCode;

We are now ready to detect the bar code in the image.

bool status = detector->detect(I);

If one or more bar codes are detected, the variable status is set to true. In that case, we can retrieve the number of detected bar codes in order to create a for loop over the codes.

if (status) {
for(size_t i=0; i < detector->getNbObjects(); i++) {

For each code, we can then get the location of the 4 points that define the polygon that contains the code, but also the bounding box.

std::vector<vpImagePoint> p = detector->getPolygon(i);
vpRect bbox = detector->getBBox(i);

And finally, if it exists, we are also able to get the message that is coded in the bar code.

vpDisplay::displayText(I, (int)(bbox.getTop()-10), (int)bbox.getLeft(),
"Message: \"" + detector->getMessage(i) + "\"",
vpColor::red);

Bar code detection (live camera)

This other example also available in tutorial-barcode-detector-live.cpp shows how to couple the bar code detector to an image grabber in order to detect bar codes on each new image acquired by a camera connected to your computer.

#include <visp3/core/vpConfig.h>
#include <visp3/gui/vpDisplayGDI.h>
#include <visp3/gui/vpDisplayOpenCV.h>
#include <visp3/gui/vpDisplayX.h>
#include <visp3/core/vpImageConvert.h>
#include <visp3/detection/vpDetectorDataMatrixCode.h>
#include <visp3/detection/vpDetectorQRCode.h>
#ifdef VISP_HAVE_MODULE_SENSOR
#include <visp3/sensor/vpV4l2Grabber.h>
#endif
int main(int argc, const char** argv)
{
#if (VISP_HAVE_OPENCV_VERSION >= 0x020100) && (defined(VISP_HAVE_ZBAR) || defined(VISP_HAVE_DMTX))
int opt_device = 0;
int opt_barcode = 0; // 0=QRCode, 1=DataMatrix
for (int i=0; i<argc; i++) {
if (std::string(argv[i]) == "--device")
opt_device = atoi(argv[i+1]);
else if (std::string(argv[i]) == "--code-type")
opt_barcode = atoi(argv[i+1]);
else if (std::string(argv[i]) == "--help") {
std::cout << "Usage: " << argv[0]
<< " [--device <camera number>] [--code-type <0 for QRcode | 1 for DataMatrix>] [--help]"
<< std::endl;
return 0;
}
}
std::cout << "Use device: " << opt_device << std::endl;
try {
vpImage<unsigned char> I; // for gray images
#if defined(VISP_HAVE_V4L2)
std::ostringstream device;
device << "/dev/video" << opt_device;
g.setDevice(device.str());
g.setScale(1);
g.acquire(I);
#elif defined(VISP_HAVE_OPENCV)
cv::VideoCapture cap(opt_device); // open the default camera
if(!cap.isOpened()) { // check if we succeeded
std::cout << "Failed to open the camera" << std::endl;
return -1;
}
cv::Mat frame;
cap >> frame; // get a new frame from camera
#endif
#if defined(VISP_HAVE_X11)
vpDisplayX d(I);
#elif defined(VISP_HAVE_GDI)
#elif defined(VISP_HAVE_OPENCV)
#endif
vpDisplay::setTitle(I, "ViSP viewer");
vpDetectorBase *detector = NULL;
#if (defined(VISP_HAVE_ZBAR) && defined(VISP_HAVE_DMTX))
if (opt_barcode == 0)
detector = new vpDetectorQRCode;
else
detector = new vpDetectorDataMatrixCode;
#elif defined(VISP_HAVE_ZBAR)
detector = new vpDetectorQRCode;
(void)opt_barcode;
#elif defined(VISP_HAVE_DMTX)
detector = new vpDetectorDataMatrixCode;
(void)opt_barcode;
#endif
for(;;) {
#if defined(VISP_HAVE_V4L2)
g.acquire(I);
#else
cap >> frame; // get a new frame from camera
#endif
bool status = detector->detect(I);
std::ostringstream legend;
legend << detector->getNbObjects() << " bar code detected";
vpDisplay::displayText(I, 10, 10, legend.str(), vpColor::red);
if (status) {
for(size_t i=0; i < detector->getNbObjects(); i++) {
std::vector<vpImagePoint> p = detector->getPolygon(i);
vpRect bbox = detector->getBBox(i);
vpDisplay::displayText(I, (int)bbox.getTop()-20, (int)bbox.getLeft(), "Message: \"" + detector->getMessage(i) + "\"", vpColor::red);
for(size_t j=0; j < p.size(); j++) {
std::ostringstream number;
number << j;
vpDisplay::displayText(I, p[j]+vpImagePoint(10,0), number.str(), vpColor::blue);
}
}
}
vpDisplay::displayText(I, (int)I.getHeight()-25, 10, "Click to quit...", vpColor::red);
if (vpDisplay::getClick(I, false)) // a click to exit
break;
}
delete detector;
}
catch(vpException &e) {
std::cout << "Catch an exception: " << e << std::endl;
}
#else
(void)argc;
(void)argv;
#endif
}

The usage of this example is similar to the previous one:

  • with option –code-type you select if you want to detect a QR code (use –code-type 0) or a Data Matrix (use –code-type 1).
  • if more than one camera is connected to you computer, with option –device you can select which camera to use. The first camera that is found has number 0.

To detect QR codes on images acquired by a second camera connected to your computer use:

$ ./tutorial-barcode-detector-live --code-type 0 --device 1

The source code of this example is very similar to the previous one except that here we use camera framegrabber devices (see Tutorial: Image frame grabbing). Two different grabber may be used:

  • If ViSP was build with Video For Linux (V4L2) support available for example on Fedora or Ubuntu distribution, VISP_HAVE_V4L2 macro is defined. In that case, images coming from an USB camera are acquired using vpV4l2Grabber class.
  • If ViSP wasn't build with V4L2 support, but with OpenCV we use cv::VideoCapture class to grab the images. Notice that when images are acquired with OpenCV there is an additional conversion from cv::Mat to vpImage.
#if defined(VISP_HAVE_V4L2)
std::ostringstream device;
device << "/dev/video" << opt_device;
g.setDevice(device.str());
g.setScale(1);
g.acquire(I);
#elif defined(VISP_HAVE_OPENCV)
cv::VideoCapture cap(opt_device); // open the default camera
if(!cap.isOpened()) { // check if we succeeded
std::cout << "Failed to open the camera" << std::endl;
return -1;
}
cv::Mat frame;
cap >> frame; // get a new frame from camera
#endif

Then in the while loop, at each iteration we acquire a new image

#if defined(VISP_HAVE_V4L2)
g.acquire(I);
#else
cap >> frame; // get a new frame from camera
#endif

This new image is then given as input to the bar code detector.

Next tutorial

Now you can follow Tutorial: Pose estimation from points that explains how to compute a 3D pose from points. This tutorial could be useful if you want to compute the pose of a QR code from it’s 4 corner coordinates in the image that could be retrieved using getPolygon(), it’s 3D size and camera parameters.

You are now also ready to see Tutorial: AprilTag pattern detection, that illustrates how to detect AprilTag patterns in an image and compute the 3D pose of each pattern.