ViSP  2.8.0
Tutorial: Keypoint tracking

Table of Contents

KLT tracker

With ViSP it is possible to track keypoints using OpenCV KLT tracker, an implementation of the Kanade-Lucas-Tomasi feature tracker. The following example code available in tutorial-klt-tracker.cpp shows how to use ViSP vpKltOpencv class to this end. This class is a wrapper over the OpenCV KLT tracker implementation.

#include <visp/vpImageConvert.h>
#include <visp/vpKltOpencv.h>
#include <visp/vpDisplayOpenCV.h>
#include <visp/vpVideoReader.h>
int main()
{
#if (VISP_HAVE_OPENCV_VERSION >= 0x010100) && defined(VISP_HAVE_FFMPEG)
vpVideoReader reader;
reader.setFileName("video-postcard.mpeg");
reader.acquire(I);
IplImage * cvI = NULL;
// Display initialisation
vpDisplayOpenCV d(I, 0, 0, "Klt tracking");
vpKltOpencv tracker;
// Set tracker parameters
tracker.setMaxFeatures(200);
tracker.setWindowSize(10);
tracker.setQuality(0.01);
tracker.setMinDistance(15);
tracker.setHarrisFreeParameter(0.04);
tracker.setBlockSize(9);
tracker.setUseHarris(1);
tracker.setPyramidLevels(3);
// Initialise the tracking
tracker.initTracking(cvI);
while ( ! reader.end() )
{
reader.acquire(I);
tracker.track(cvI);
tracker.display(I, vpColor::red);
}
cvReleaseImage(&cvI);
return 0;
#endif
}

The video shows the result of the tracking:

And here is the line by line explanation of the source :

#include <visp/vpImageConvert.h>
#include <visp/vpKltOpencv.h>
#include <visp/vpDisplayOpenCV.h>
#include <visp/vpVideoReader.h>

We include here the headers that define the corresponding classes. vpImageConvert class will be used to convert ViSP images implemented in vpImage class into OpenCV IplImage structure used as an entry by the KLT tracker. Then we include the header of vpKltOpencv class which is the wrapper over OpenCV KLT tracker implementation. We need also to include a device to display the images. We retain vpDisplayOpenCV that works on Unix and Windows since OpenCV is mandatory by the tracker. Finally we include vpVideoReader header that will be used to read an mpeg input stream.

#if (VISP_HAVE_OPENCV_VERSION >= 0x010100) && defined(VISP_HAVE_FFMPEG)

We use the previous macro to ensure that OpenCV at least version 1.1.0 requested by the tracker and the image viewer, and ffmpeg requested to read the video stream are available.

reader.setFileName("video-postcard.mpeg");

The program starts by the creation of a vpVideoReader instance able to extract all the images of the video file video-postcard.mpeg. Here, the video should be in the same folder than the binary.

Returns the first image of the video in the gray level ViSP image container I.

IplImage * cvI = NULL;

Then we convert I into cvI, an image in OpenCV IplImage format that will be used by the tracker.

vpDisplayOpenCV d(I, 0, 0, "Klt tracking");

Create a window associated to I, at position (0,0) in the screen, with "Klt tracking" as title, and display image I.

vpKltOpencv tracker;
tracker.setMaxFeatures(200);
tracker.setWindowSize(10);
tracker.setQuality(0.01);
tracker.setMinDistance(15);
tracker.setHarrisFreeParameter(0.04);
tracker.setBlockSize(9);
tracker.setUseHarris(1);
tracker.setPyramidLevels(3);
tracker.initTracking(cvI);

Create an instance of the tracker, set parameters of the Harris keypoint detector and finally initialize the tracker on cvI image.

while ( ! reader.end() )
{
reader.acquire(I);
tracker.track(cvI);
tracker.display(I, vpColor::red);
}

Until the end of the video is not reached, we get I the next image in ViSP format, display and convert it in IplImage format. Then we track the Harris keypoints using KLT tracker before displaying the keypoints that are tracked with a red cross.

We are waiting a mouse click event on image I to end the program.

With the following line, we release the memory allocated for the OpenCV IplImage cvI before ending the program.

cvReleaseImage(&cvI);

You are now ready to see the next Tutorial: Moving-edges tracking.