MeLine

class MeLine(*args, **kwargs)

Bases: MeTracker

Class that tracks in an image a line moving edges.

In this class the line is defined by its equation in the \((i,j) = (line,column)\) image plane. Two kinds of parametrization are available to describe a 2D line. The first one corresponds to the following equation

\[ai + bj + c = 0 \]

where \(i\) and \(j\) are the coordinates of the points belonging to the line. The line features are \((a, b, c)\) .

The second way to write the line equation is to consider polar coordinates

\[i \; cos(\theta) + j \; sin(\theta) - \rho = 0 \]

where \(i\) and \(j\) are still the coordinates of the points belonging to the line. But now the line features are \((\rho, \theta)\) . The computation of \(\rho\) and \(\theta\) is easy thanks to \((a, b, c)\) .

\[\theta = arctan(b/a) \]
\[\rho = -c/\sqrt{a^2+b^2} \]

The value of \(\theta\) is between \(0\) and \(2\pi\) . And the value of \(\rho\) can be positive or negative. The conventions to find the right values of the two features are illustrated in the following pictures.

<unparsed image <doxmlparser.compound.docImageType object at 0x7f745ec955d0>>

The angle \(\theta\) is computed thanks to the direction of the arrow. The arrow points to the side of the line which is darker.

The example below available in tutorial-me-line-tracker.cpp and described in tutorial-tracking-me shows how to use this class.

#include <iostream>

#include <visp3/core/vpConfig.h>

// #undef VISP_HAVE_V4L2
// #undef VISP_HAVE_DC1394
// #undef VISP_HAVE_CMU1394
// #undef VISP_HAVE_FLYCAPTURE
// #undef VISP_HAVE_REALSENSE2
// #undef HAVE_OPENCV_HIGHGUI
// #undef HAVE_OPENCV_VIDEOIO

#if (defined(VISP_HAVE_V4L2) || defined(VISP_HAVE_DC1394) || defined(VISP_HAVE_CMU1394) || \
   defined(VISP_HAVE_FLYCAPTURE) || defined(VISP_HAVE_REALSENSE2) || \
   ((VISP_HAVE_OPENCV_VERSION < 0x030000) && defined(HAVE_OPENCV_HIGHGUI)) || \
   ((VISP_HAVE_OPENCV_VERSION >= 0x030000) && defined(HAVE_OPENCV_VIDEOIO))) && \
  ((VISP_HAVE_OPENCV_VERSION < 0x050000) && defined(HAVE_OPENCV_CALIB3D) && defined(HAVE_OPENCV_FEATURES2D)) || \
  ((VISP_HAVE_OPENCV_VERSION >= 0x050000) && defined(HAVE_OPENCV_3D) && defined(HAVE_OPENCV_FEATURES))

#ifdef VISP_HAVE_MODULE_SENSOR
#include <visp3/sensor/vp1394CMUGrabber.h>
#include <visp3/sensor/vp1394TwoGrabber.h>
#include <visp3/sensor/vpFlyCaptureGrabber.h>
#include <visp3/sensor/vpRealSense2.h>
#include <visp3/sensor/vpV4l2Grabber.h>
#endif
#include <visp3/gui/vpDisplayGDI.h>
#include <visp3/gui/vpDisplayOpenCV.h>
#include <visp3/gui/vpDisplayX.h>
#include <visp3/me/vpMeLine.h>

#if (VISP_HAVE_OPENCV_VERSION < 0x030000) && defined(HAVE_OPENCV_HIGHGUI)
#include <opencv2/highgui/highgui.hpp> // for cv::VideoCapture
#elif (VISP_HAVE_OPENCV_VERSION >= 0x030000) && defined(HAVE_OPENCV_VIDEOIO)
#include <opencv2/videoio/videoio.hpp> // for cv::VideoCapture
#endif

int main()
{
#ifdef ENABLE_VISP_NAMESPACE
  using namespace VISP_NAMESPACE_NAME ;
#endif
  try {
    vpImage<unsigned char>  I;

    int opt_device = 0; // For OpenCV and V4l2 grabber to set the camera device
#if defined(VISP_HAVE_V4L2)
    vpV4l2Grabber  g;
    std::ostringstream device;
    device << "/dev/video" << opt_device;
    std::cout << "Use Video 4 Linux grabber on device " << device.str() << std::endl;
    g. setDevice (device.str());
    g. setScale (1);
    g. open (I);
#elif defined(VISP_HAVE_DC1394)
    (void)opt_device;         // To avoid non used warning
    std::cout << "Use DC1394 grabber" << std::endl;
    vp1394TwoGrabber  g;
    g. open (I);
#elif defined(VISP_HAVE_CMU1394)
    (void)opt_device; // To avoid non used warning
    std::cout << "Use CMU1394 grabber" << std::endl;
    vp1394CMUGrabber  g;
    g. open (I);
#elif defined(VISP_HAVE_FLYCAPTURE)
    (void)opt_device; // To avoid non used warning
    std::cout << "Use FlyCapture grabber" << std::endl;
    vpFlyCaptureGrabber  g;
    g. open (I);
#elif defined(VISP_HAVE_REALSENSE2)
    (void)opt_device; // To avoid non used warning
    std::cout << "Use Realsense 2 grabber" << std::endl;
    vpRealSense2  g;
    rs2::config config;
    config.disable_stream(RS2_STREAM_DEPTH);
    config.disable_stream(RS2_STREAM_INFRARED);
    config.enable_stream(RS2_STREAM_COLOR, 640, 480, RS2_FORMAT_RGBA8, 30);
    g. open (config);
    g. acquire (I);
#elif ((VISP_HAVE_OPENCV_VERSION < 0x030000) && defined(HAVE_OPENCV_HIGHGUI))|| ((VISP_HAVE_OPENCV_VERSION >= 0x030000) && defined(HAVE_OPENCV_VIDEOIO))
    std::cout << "Use OpenCV grabber on device " << opt_device << std::endl;
    cv::VideoCapture g(opt_device); // Open the default camera
    if (!g.isOpened()) {            // Check if we succeeded
      std::cout << "Failed to open the camera" << std::endl;
      return EXIT_FAILURE;
    }
    cv::Mat frame;
    g >> frame; // get a new frame from camera
    vpImageConvert::convert (frame, I);
#endif
#if defined(VISP_HAVE_V4L2) || defined(VISP_HAVE_DC1394) || defined(VISP_HAVE_CMU1394) || defined(VISP_HAVE_FLYCAPTURE) || defined(VISP_HAVE_REALSENSE2)
    g. acquire (I);
#elif ((VISP_HAVE_OPENCV_VERSION < 0x030000) && defined(HAVE_OPENCV_HIGHGUI))|| ((VISP_HAVE_OPENCV_VERSION >= 0x030000) && defined(HAVE_OPENCV_VIDEOIO))
    g >> frame; // get a new frame from camera
    vpImageConvert::convert (frame, I);
#endif

#if defined(VISP_HAVE_X11)
    vpDisplayX d(I, 0, 0, "Camera view");
#elif defined(VISP_HAVE_GDI)
    vpDisplayGDI  d(I, 0, 0, "Camera view");
#elif defined(HAVE_OPENCV_HIGHGUI)
    vpDisplayOpenCV  d(I, 0, 0, "Camera view");
#else
    std::cout << "No image viewer is available..." << std::endl;
#endif
    vpDisplay::display (I);
    vpDisplay::flush (I);

    vpMe  me;
    me. setRange (25);
    me. setLikelihoodThresholdType ( vpMe::NORMALIZED_THRESHOLD );
    me. setThreshold (20);
    me. setSampleStep (10);

    vpMeLine  line;
    line. setMe (&me);
    line. setDisplay ( vpMeSite::RANGE_RESULT );
    line. initTracking (I);

    bool quit = false;
    while (!quit) {
#if defined(VISP_HAVE_V4L2) || defined(VISP_HAVE_DC1394) || defined(VISP_HAVE_CMU1394) || defined(VISP_HAVE_FLYCAPTURE) || defined(VISP_HAVE_REALSENSE2)
      g. acquire (I);
#elif ((VISP_HAVE_OPENCV_VERSION < 0x030000) && defined(HAVE_OPENCV_HIGHGUI))|| ((VISP_HAVE_OPENCV_VERSION >= 0x030000) && defined(HAVE_OPENCV_VIDEOIO))
      g >> frame;
      vpImageConvert::convert (frame, I);
#endif
      vpDisplay::display (I);
      vpDisplay::displayText (I, 20, 20, "Click to quit", vpColor::red );
      line. track (I);
      line. display (I, vpColor::red );
      if ( vpDisplay::getClick (I, false)) {
        quit = true;
      }
      vpDisplay::flush (I);
    }
  }
  catch (const vpException  &e) {
    std::cout << "Catch an exception: " << e << std::endl;
  }
}

#else

int main()
{
#if defined(VISP_HAVE_OPENCV)
  std::cout << "Install a 3rd party dedicated to frame grabbing (dc1394, cmu1394, v4l2, OpenCV, FlyCapture, "
    << "Realsense2), configure and build ViSP again to use this tutorial."
    << std::endl;
#else
  std::cout << "Install OpenCV 3rd party, configure and build ViSP again to use this tutorial." << std::endl;
#endif
  return EXIT_SUCCESS;
}
#endif

The code below shows how to use this class.

#include <visp3/core/vpConfig.h>
#include <visp3/core/vpImage.h>
#include <visp3/core/vpImagePoint.h>
#include <visp3/me/vpMeLine.h>

#ifdef ENABLE_VISP_NAMESPACE
using namespace VISP_NAMESPACE_NAME;
#endif

int main()
{
  vpImage<unsigned char> I(240, 320);

  // Fill the image with a black rectangle
  I = 0u;
  for (int i = 100; i < 180; i ++) {
    for (int j = 120; j < 250; j ++) {
      I[i][j] = 255;
    }
  }

  // Set the moving-edges tracker parameters
  vpMe me;
  me.setRange(25);
  me.setLikelihoodThresholdType(vpMe::NORMALIZED_THRESHOLD);
  me.setThreshold(20);
  me.setSampleStep(10);

  // Initialize the moving-edges line tracker parameters
  vpMeLine line;
  line.setMe(&me);

  // Initialize the location of the vertical line to track
  vpImagePoint ip1, ip2; // Two points belonging to the line to track
  ip1.set_i( 120 );
  ip1.set_j( 119 );
  ip2.set_i( 170 );
  ip2.set_j( 122 );

  line.initTracking(I, ip1, ip2);

  while ( 1 )
  {
    // ... Here the code to read or grab the next image.

    // Track the line.
    line.track(I);
  }
  return 0;
}

Note

It is possible to display the line as an overlay. For that you must use the display function of the class vpMeLine .

Overloaded function.

  1. __init__(self: visp._visp.me.MeLine) -> None

Basic constructor that calls the constructor of the class vpMeTracker .

  1. __init__(self: visp._visp.me.MeLine, meline: visp._visp.me.MeLine) -> None

Copy constructor.

Methods

__init__

Overloaded function.

computeRhoSignFromIntensity

This method allows to turn off the computation of the sign of the rho attribute based on the intensity near the middle point of the line.

computeRhoTheta

Compute the two parameters \((\rho, \theta)\) of the line.

display

Overloaded function.

displayLine

Overloaded function.

getA

Gets parameter a of the line equation a*i + b*j + c = 0

getB

Gets parameter b of the line equation a*i + b*j + c = 0

getC

Gets parameter c of the line equation a*i + b*j + c = 0

getEquationParam

Gets the equation parameters of the line

getExtremities

Get the extremities of the line.

getRho

Get the value of \(\rho\) , the distance between the origin and the point on the line with belong to the normal to the line crossing the origin.

getTheta

Get the value of the angle \(\theta\) .

initTracking

Overloaded function.

intersection

Computes the intersection point of two lines.

leastSquare

Least squares method used to make the tracking more robust.

reSample

Resample the line if the number of sample is less than 80% of the expected value.

sample

Construct a list of vpMeSite moving edges at a particular sampling step between the two extremities of the line.

seekExtremities

Seek along the line defined by its equation, the two extremities of the line.

setExtremities

Seek in the list of available points the two extremities of the line.

suppressPoints

Suppression of the points which belong no more to the line.

track

Overloaded function.

updateDelta

Set the alpha value of the different vpMeSite to the value of delta.

Inherited Methods

getMeList

Overloaded function.

setDisplay

Set type of moving-edges display.

setInitRange

Set the initial range.

setMeList

Set the list of moving edges.

numberOfSignal

Return number of moving-edges that are tracked.

cPAvailable

get_cP

Return object parameters expressed in the 3D camera frame.

cP

p

init

Initialize the tracker.

getInitRange

Return the initial range.

get_p

Return object parameters expressed in the 2D image plane computed by perspective projection.

outOfImage

Overloaded function.

reset

Reset the tracker by removing all the moving edges.

setMask

Set the mask.

totalNumberOfSignal

Return the total number of moving-edges.

getNbPoints

Return the number of points that has not been suppressed.

Operators

__doc__

__init__

Overloaded function.

__module__

Attributes

__annotations__

cP

cPAvailable

p

__init__(*args, **kwargs)

Overloaded function.

  1. __init__(self: visp._visp.me.MeLine) -> None

Basic constructor that calls the constructor of the class vpMeTracker .

  1. __init__(self: visp._visp.me.MeLine, meline: visp._visp.me.MeLine) -> None

Copy constructor.

computeRhoSignFromIntensity(self, useIntensityForRho: bool) → None

This method allows to turn off the computation of the sign of the rho attribute based on the intensity near the middle point of the line. This is usually done to distinguish between a black/white and a white/black edge but it may be source of problem (ex. for a servoing example) when this point can be occluded.

Parameters:
useIntensityForRho: bool

new value of the flag.

computeRhoTheta(self, I: visp._visp.core.ImageGray) → None

Compute the two parameters \((\rho, \theta)\) of the line.

Parameters:
I: visp._visp.core.ImageGray

Image in which the line appears.

display(*args, **kwargs)

Overloaded function.

  1. display(self: visp._visp.me.MeLine, I: visp._visp.core.ImageGray, color: visp._visp.core.Color, thickness: int = 1) -> None

Display line.

Warning

To effectively display the line a call to vpDisplay::flush() is needed.

Parameters:
I

Image in which the line appears.

color

Color of the displayed line. Note that a moving edge that is considered as an outlier is displayed in green.

thickness

Drawings thickness.

  1. display(self: visp._visp.me.MeTracker, I: visp._visp.core.ImageGray) -> None

Display the moving edge sites with a color corresponding to their state.

  • If green : The vpMeSite is a good point.

  • If blue : The point is removed because of the vpMeSite tracking phase (contrast problem).

  • If purple : The point is removed because of the vpMeSite tracking phase (threshold problem).

  • If red : The point is removed because of the robust method in the virtual visual servoing (M-Estimator problem).

  • If cyan : The point is removed because it’s too close to another.

  • Yellow otherwise.

Parameters:
I

The image.

  1. display(self: visp._visp.me.MeTracker, I: visp._visp.core.ImageRGBa) -> None

Display the moving edge sites with a color corresponding to their state.

  • If green : The vpMeSite is a good point.

  • If blue : The point is removed because of the vpMeSite tracking phase (contrast problem).

  • If purple : The point is removed because of the vpMeSite tracking phase (threshold problem).

  • If red : The point is removed because of the robust method in the virtual visual servoing (M-Estimator problem).

  • If cyan : The point is removed because it’s too close to another.

  • Yellow otherwise.

Parameters:
I

The image.

static displayLine(*args, **kwargs)

Overloaded function.

  1. displayLine(I: visp._visp.core.ImageGray, PExt1: visp._visp.me.MeSite, PExt2: visp._visp.me.MeSite, A: float, B: float, C: float, color: visp._visp.core.Color = vpColor::green, thickness: int = 1) -> None

Display of a moving line thanks to its equation parameters and its extremities.

Parameters:
I

The image used as background.

PExt1

First extremity

PExt2

Second extremity

A

Parameter a of the line equation a*i + b*j + c = 0

B

Parameter b of the line equation a*i + b*j + c = 0

C

Parameter c of the line equation a*i + b*j + c = 0

color

Color used to display the line.

thickness

Thickness of the line.

  1. displayLine(I: visp._visp.core.ImageRGBa, PExt1: visp._visp.me.MeSite, PExt2: visp._visp.me.MeSite, A: float, B: float, C: float, color: visp._visp.core.Color = vpColor::green, thickness: int = 1) -> None

Display of a moving line thanks to its equation parameters and its extremities.

Parameters:
I

The image used as background.

PExt1

First extremity

PExt2

Second extremity

A

Parameter a of the line equation a*i + b*j + c = 0

B

Parameter b of the line equation a*i + b*j + c = 0

C

Parameter c of the line equation a*i + b*j + c = 0

color

Color used to display the line.

thickness

Thickness of the line.

  1. displayLine(I: visp._visp.core.ImageGray, PExt1: visp._visp.me.MeSite, PExt2: visp._visp.me.MeSite, site_list: list[visp._visp.me.MeSite], A: float, B: float, C: float, color: visp._visp.core.Color = vpColor::green, thickness: int = 1) -> None

Display of a moving line thanks to its equation parameters and its extremities with all the site list.

Parameters:
I

The image used as background.

PExt1

First extremity

PExt2

Second extremity

site_list

vpMeSite list

A

Parameter a of the line equation a*i + b*j + c = 0

B

Parameter b of the line equation a*i + b*j + c = 0

C

Parameter c of the line equation a*i + b*j + c = 0

color

Color used to display the line.

thickness

Thickness of the line.

  1. displayLine(I: visp._visp.core.ImageRGBa, PExt1: visp._visp.me.MeSite, PExt2: visp._visp.me.MeSite, site_list: list[visp._visp.me.MeSite], A: float, B: float, C: float, color: visp._visp.core.Color = vpColor::green, thickness: int = 1) -> None

Display of a moving line thanks to its equation parameters and its extremities with all the site list.

Parameters:
I

The image used as background.

PExt1

First extremity

PExt2

Second extremity

site_list

vpMeSite list

A

Parameter a of the line equation a*i + b*j + c = 0

B

Parameter b of the line equation a*i + b*j + c = 0

C

Parameter c of the line equation a*i + b*j + c = 0

color

Color used to display the line.

thickness

Thickness of the line.

getA(self) → float

Gets parameter a of the line equation a*i + b*j + c = 0

getB(self) → float

Gets parameter b of the line equation a*i + b*j + c = 0

getC(self) → float

Gets parameter c of the line equation a*i + b*j + c = 0

getEquationParam(self, A: float, B: float, C: float) → tuple[float, float, float]

Gets the equation parameters of the line

Returns:

A tuple containing:

  • A

  • B

  • C

getExtremities(self, ip1: visp._visp.core.ImagePoint, ip2: visp._visp.core.ImagePoint) → None

Get the extremities of the line.

Parameters:
ip1: visp._visp.core.ImagePoint

Coordinates of the first extremity.

ip2: visp._visp.core.ImagePoint

Coordinates of the second extremity.

getInitRange(self) → int

Return the initial range.

Returns:

Value of init_range.

getMeList(*args, **kwargs)

Overloaded function.

  1. getMeList(self: visp._visp.me.MeTracker) -> list[visp._visp.me.MeSite]

Return the list of moving edges

Returns:

List of Moving Edges.

  1. getMeList(self: visp._visp.me.MeTracker) -> list[visp._visp.me.MeSite]

Return the list of moving edges

Returns:

List of Moving Edges.

getNbPoints(self) → int

Return the number of points that has not been suppressed.

Returns:

Number of good points.

getRho(self) → float

Get the value of \(\rho\) , the distance between the origin and the point on the line with belong to the normal to the line crossing the origin.

Depending on the convention described at the beginning of this class, \(\rho\) is signed.

getTheta(self) → float

Get the value of the angle \(\theta\) .

get_cP(self) → visp._visp.core.ColVector

Return object parameters expressed in the 3D camera frame.

get_p(self) → visp._visp.core.ColVector

Return object parameters expressed in the 2D image plane computed by perspective projection.

init(self) → None

Initialize the tracker.

initTracking(*args, **kwargs)

Overloaded function.

  1. initTracking(self: visp._visp.me.MeLine, I: visp._visp.core.ImageGray) -> None

Initialization of the tracking. Ask the user to click on two points from the line to track.

Parameters:
I

Image in which the line appears.

  1. initTracking(self: visp._visp.me.MeLine, I: visp._visp.core.ImageGray, ip1: visp._visp.core.ImagePoint, ip2: visp._visp.core.ImagePoint) -> None

Initialization of the tracking. The line is defined thanks to the coordinates of two points.

Parameters:
I

Image in which the line appears.

ip1

Coordinates of the first point.

ip2

Coordinates of the second point.

  1. initTracking(self: visp._visp.me.MeTracker, I: visp._visp.core.ImageGray) -> None

Virtual function that is called by lower classes vpMeEllipse , vpMeLine and vpMeNurbs .

static intersection(line1: visp._visp.me.MeLine, line2: visp._visp.me.MeLine, ip: visp._visp.core.ImagePoint) → bool

Computes the intersection point of two lines. The result is given in the (i,j) frame.

Parameters:
line1: visp._visp.me.MeLine

The first line.

line2: visp._visp.me.MeLine

The second line.

ip: visp._visp.core.ImagePoint

The coordinates of the intersection point.

Returns:

Returns a boolean value which depends on the computation success. True means that the computation ends successfully.

leastSquare(self) → None

Least squares method used to make the tracking more robust. It ensures that the points taken into account to compute the right equation belong to the line.

numberOfSignal(self) → int

Return number of moving-edges that are tracked.

outOfImage(*args, **kwargs)

Overloaded function.

  1. outOfImage(self: visp._visp.me.MeTracker, i: int, j: int, border: int, nrows: int, ncols: int) -> bool

Parameters:
i

Pixel coordinates.

j

Pixel coordinates.

border

Number of pixels along the image border to exclude. When border is set to 0, consider the complete image.

nrows

Size of the image.

ncols

Size of the image.

Returns:

true when the pixel is inside the image minus the border size, false otherwise.

  1. outOfImage(self: visp._visp.me.MeTracker, iP: visp._visp.core.ImagePoint, border: int, nrows: int, ncols: int) -> bool

Parameters:
iP

Pixel coordinates.

border

Number of pixels along the image border to exclude. When border is set to 0, consider the complete image.

nrows

Size of the image.

ncols

Size of the image.

Returns:

true when the pixel is inside the image minus the border size, false otherwise.

reSample(self, I: visp._visp.core.ImageGray) → None

Resample the line if the number of sample is less than 80% of the expected value.

Note

The expected value is computed thanks to the length of the line and the parameter which indicates the number of pixel between two points (vpMe::sample_step).

Parameters:
I: visp._visp.core.ImageGray

Image in which the line appears.

reset(self) → None

Reset the tracker by removing all the moving edges.

sample(self, I: visp._visp.core.ImageGray, doNotTrack: bool = false) → None

Construct a list of vpMeSite moving edges at a particular sampling step between the two extremities of the line.

Parameters:
I: visp._visp.core.ImageGray

Image in which the line appears.

doNotTrack: bool = false

Inherited parameter, not used.

seekExtremities(self, I: visp._visp.core.ImageGray) → None

Seek along the line defined by its equation, the two extremities of the line. This function is useful in case of translation of the line.

Parameters:
I: visp._visp.core.ImageGray

Image in which the line appears.

setDisplay(self, select: visp._visp.me.MeSite.MeSiteDisplayType) → None

Set type of moving-edges display.

Parameters:
select: visp._visp.me.MeSite.MeSiteDisplayType

Display type selector.

setExtremities(self) → None

Seek in the list of available points the two extremities of the line.

setInitRange(self, r: int) → None

Set the initial range.

Parameters:
r: int

initial range.

setMask(self: visp._visp.me.MeTracker, mask: vpImage<bool>) → None

Set the mask.

Parameters:
mask

Mask.

setMeList(self, meList: list[visp._visp.me.MeSite]) → None

Set the list of moving edges.

Parameters:
meList: list[visp._visp.me.MeSite]

List of Moving Edges.

suppressPoints(self) → None

Suppression of the points which belong no more to the line.

totalNumberOfSignal(self) → int

Return the total number of moving-edges.

track(*args, **kwargs)

Overloaded function.

  1. track(self: visp._visp.me.MeLine, I: visp._visp.core.ImageGray) -> None

Track the line in the image I.

Parameters:
I

Image in which the line appears.

  1. track(self: visp._visp.me.MeTracker, I: visp._visp.core.ImageGray) -> None

Track moving-edges.

Parameters:
I

Image.

updateDelta(self) → None

Set the alpha value of the different vpMeSite to the value of delta.