Tutorial: Moving-edges tracking

Table of Contents

• Introduction
• Line tracking
• Ellipse tracking
• Next tutorial

Introduction

ViSP moving-edges tracker provide real-time tracking capabilities of points normal to the object contours. Such a tracker allow to track a line, an ellipse or more complex objects using model-based approaches.

All the material (source code and video) described in this tutorial is part of ViSP source code and could be downloaded using the following command:

$ svn export https://github.com/lagadic/visp.git/trunk/tutorial/tracking/moving-edges

Line tracking

With ViSP you can track a line using moving edges. The following example code available in tutorial-me-line-tracker.cpp shows how to use ViSP vpMeLine class to this end.

#include <visp3/core/vpConfig.h>
#ifdef VISP_HAVE_MODULE_SENSOR
#include <visp3/sensor/vp1394CMUGrabber.h>
#include <visp3/sensor/vp1394TwoGrabber.h>
#include <visp3/sensor/vpV4l2Grabber.h>
#endif
#include <visp3/gui/vpDisplayGDI.h>
#include <visp3/gui/vpDisplayOpenCV.h>
#include <visp3/gui/vpDisplayX.h>
#include <visp3/me/vpMeLine.h>

int main()
{
#if (defined(VISP_HAVE_DC1394) || defined(VISP_HAVE_CMU1394) || defined(VISP_HAVE_V4L2))
  try {
    vpImage<unsigned char> I;

#if defined(VISP_HAVE_DC1394)
    vp1394TwoGrabber g(false);
#elif defined(VISP_HAVE_CMU1394)
    vp1394CMUGrabber g;
#elif defined(VISP_HAVE_V4L2)
    vpV4l2Grabber g;
#elif defined(VISP_HAVE_OPENCV)
  cv::VideoCapture g(0); // open the default camera
  if(!g.isOpened()) { // check if we succeeded
    std::cout << "Failed to open the camera" << std::endl;
    return -1;
  }
  cv::Mat frame;
  g >> frame; // get a new frame from camera
  vpImageConvert::convert(frame, I);
#endif
    g.open(I);
    g.acquire(I);

#if defined(VISP_HAVE_X11)
    vpDisplayX d(I, 0, 0, "Camera view");
#elif defined(VISP_HAVE_GDI)
    vpDisplayGDI d(I, 0, 0, "Camera view");
#elif defined(VISP_HAVE_OPENCV)
    vpDisplayOpenCV d(I, 0, 0, "Camera view");
#else
    std::cout << "No image viewer is available..." << std::endl;
#endif

    vpDisplay::display(I);
    vpDisplay::flush(I);

    vpMe me;
    me.setRange(25);
    me.setThreshold(15000);
    me.setSampleStep(10);

    vpMeLine line;
    line.setMe(&me);
    line.setDisplay(vpMeSite::RANGE_RESULT);
    line.initTracking(I);

    while(1) {
#if defined(VISP_HAVE_DC1394) || defined(VISP_HAVE_V4L2) || defined(VISP_HAVE_CMU1394)
      g.acquire(I);
#elif defined(VISP_HAVE_OPENCV)
      g >> frame;
      vpImageConvert::convert(frame, I);
#endif
      vpDisplay::display(I);
      line.track(I);
      line.display(I, vpColor::red);
      vpDisplay::flush(I);
    }
  }
  catch(vpException &e) {
    std::cout << "Catch an exception: " << e << std::endl;
  }
#endif
}

The video shows the result of the tracking:

Here after we explain line by line the program.

Images that are processed could be acquired from a firewire camera on Unix or Windows, of from an usb camera under Unix. That is allowed by including the grabber headers.

#include <visp3/sensor/vp1394CMUGrabber.h>
#include <visp3/sensor/vp1394TwoGrabber.h>
#include <visp3/sensor/vpV4l2Grabber.h>

To display these images we then include the headers of the viewers.

#include <visp3/core/vpDisplayGDI.h>
#include <visp3/core/vpDisplayX.h>

The Graphical Display Interface (GDI) allows to display images under Windows, while X11 allows this feature under unix-like systems.

Finally, to track a line with the moving edges, we include the header of the vpMeLine class.

#include <visp3/core/vpMeLine.h>

In the main() function, The source code is build only if one of the grabbers and one of the viewers is available.

#if (defined(VISP_HAVE_DC1394) || defined(VISP_HAVE_CMU1394) || defined(VISP_HAVE_V4L2)) && (defined(VISP_HAVE_X11) || defined(VISP_HAVE_GDI))

Here we create a gray level image container I that will contain the images acquired by our camera.

vpImage<unsigned char> I;

Then, we create a grabber instance, first for a firewire camera under Unix if libdc1394 3rd party is installed, secondly for a firewire camera under Windows if CMU1394 3rd party is installed, and lastly for an usb camera under Unix if none of the previous 3rd party are installed, and if libv4l is installed. The Tutorial: Image frame grabbing gives more details concerning the framegrabbing.

#if defined(VISP_HAVE_DC1394)
  vp1394TwoGrabber g(false);
#elif defined(VISP_HAVE_CMU1394)
  vp1394CMUGrabber g;
#elif defined(VISP_HAVE_V4L2)
  vpV4l2Grabber g;     
#endif

We then open the connection with the grabber and acquire an image in I.

g.open(I);
g.acquire(I);

To be able to display image I and the tracking results in overlay in a window, we create a display instance.

#if defined(VISP_HAVE_X11)
  vpDisplayX d(I, 0, 0, "Camera view");
#else
  vpDisplayGDI d(I, 0, 0, "Camera view");
#endif

Then we display the image in the window created previously.

vpDisplay::display(I);
vpDisplay::flush(I);

We then initialize the moving edges parameters used later by the tracker. From the previous position of a moving edge, we are tracking its new position along the normal of the contour with a range of 25 pixels. For each pixel along the normal we will compute the oriented convolution. The pixel that will be selected by the moving edges algorithm will be the one that has a convolution higher than 15000. Between two consecutive moving edges on the contour we keep a space of 10 pixels.

vpMe me;
me.setRange(25);
me.setThreshold(15000);
me.setSampleStep(10);

We then, create an instance of the vpMeTracker class that will track our line. We initialize the tracker with the previous moving-egdes parameters. We allow also the tracker to display additional information on the viewer overlay. The user has than to initialize the tracker on image I by clicking on two points located on the line to track.

vpMeLine line;
line.set Me(&me);
line.set Display(vpMeSite::RANGE_RESULT);
line.initTracking(I);

Once the tracker is initialized, we enter in a while loop where we successively acquire a new image, display it, track the line, display the tracking results and finally flush the overlay drawings in the viewer.

while(1) {
  g.acquire(I); 
  vpDisplay::display(I);
  line.track(I);
  line.display(I, vpColor::red);
  vpDisplay::flush(I);
}

Ellipse tracking

With ViSP you can also track an ellipse using moving edges. The following example code available in tutorial-me-ellipse-tracker.cpp shows how to use ViSP vpMeEllipse class to this end.

#include <visp3/core/vpConfig.h>
#ifdef VISP_HAVE_MODULE_SENSOR
#include <visp3/sensor/vp1394CMUGrabber.h>
#include <visp3/sensor/vp1394TwoGrabber.h>
#include <visp3/sensor/vpV4l2Grabber.h>
#endif
#include <visp3/gui/vpDisplayGDI.h>
#include <visp3/gui/vpDisplayOpenCV.h>
#include <visp3/gui/vpDisplayX.h>
#include <visp3/me/vpMeEllipse.h>

int main()
{
#if (defined(VISP_HAVE_DC1394) || defined(VISP_HAVE_CMU1394) || defined(VISP_HAVE_V4L2) || (VISP_HAVE_OPENCV_VERSION >= 0x020100))
  try {
    vpImage<unsigned char> I;

#if defined(VISP_HAVE_DC1394)
    vp1394TwoGrabber g(false);
#elif defined(VISP_HAVE_CMU1394)
    vp1394CMUGrabber g;
#elif defined(VISP_HAVE_V4L2)
    vpV4l2Grabber g;
#elif defined(VISP_HAVE_OPENCV)
    cv::VideoCapture g(0); // open the default camera
    if(!g.isOpened()) { // check if we succeeded
      std::cout << "Failed to open the camera" << std::endl;
      return -1;
    }
    cv::Mat frame;
#endif

#if defined(VISP_HAVE_DC1394) || defined(VISP_HAVE_V4L2) || defined(VISP_HAVE_CMU1394)
    g.acquire(I);
#elif defined(VISP_HAVE_OPENCV)
    g >> frame; // get a new frame from camera
    vpImageConvert::convert(frame, I);
#endif

#if defined(VISP_HAVE_X11)
    vpDisplayX d(I, 0, 0, "Camera view");
#elif defined(VISP_HAVE_GDI)
    vpDisplayGDI d(I, 0, 0, "Camera view");
#elif defined(VISP_HAVE_OPENCV)
    vpDisplayOpenCV d(I, 0, 0, "Camera view");
#else
    std::cout << "No image viewer is available..." << std::endl;
#endif

    vpDisplay::display(I);
    vpDisplay::flush(I);

    vpMe me;
    me.setRange(25);
    me.setThreshold(15000);
    me.setSampleStep(10);

    vpMeEllipse ellipse;
    ellipse.setMe(&me);
    ellipse.setDisplay(vpMeSite::RANGE_RESULT);
    ellipse.initTracking(I);

    while(1) {
#if defined(VISP_HAVE_DC1394) || defined(VISP_HAVE_V4L2) || defined(VISP_HAVE_CMU1394)
      g.acquire(I);
#elif defined(VISP_HAVE_OPENCV)
      g >> frame;
      vpImageConvert::convert(frame, I);
#endif
      vpDisplay::display(I);
      ellipse.track(I);
      ellipse.display(I, vpColor::red);
      vpDisplay::flush(I);
    }
  }
  catch(vpException &e) {
    std::cout << "Catch an exception: " << e << std::endl;
  }
#endif
}

The video shows the result of the tracking:

This example is very similar to the one presented in Line tracking. It differs only in the name of the class and its header that is used to allow ellipse tracking:

vpMeEllipse ellipse;
ellipse.setMe(&me);
ellipse.setDisplay(vpMeSite::RANGE_RESULT);
ellipse.initTracking(I);

Note here that the user has to initialize the tracker on image I by clicking on five points located on the ellipse to track.

Next tutorial

You are now ready to see the next Tutorial: Markerless model-based tracking.