Visual Servoing Platform  version 3.2.0 under development (2018-08-20)

#include <visp3/vision/vpPlanarObjectDetector.h>

Public Member Functions

 vpPlanarObjectDetector ()
 
 vpPlanarObjectDetector (const std::string &dataFile, const std::string &objectName)
 
virtual ~vpPlanarObjectDetector ()
 
unsigned int buildReference (const vpImage< unsigned char > &I)
 
unsigned int buildReference (const vpImage< unsigned char > &I, const vpImagePoint &iP, unsigned int height, unsigned int width)
 
unsigned int buildReference (const vpImage< unsigned char > &I, const vpRect &rectangle)
 
bool matchPoint (const vpImage< unsigned char > &I)
 
bool matchPoint (const vpImage< unsigned char > &I, const vpImagePoint &iP, const unsigned int height, const unsigned int width)
 
bool matchPoint (const vpImage< unsigned char > &I, const vpRect &rectangle)
 
void recordDetector (const std::string &objectName, const std::string &dataFile)
 
void load (const std::string &dataFilename, const std::string &objName)
 
void display (vpImage< unsigned char > &I, bool displayKpts=false)
 
void display (vpImage< unsigned char > &Iref, vpImage< unsigned char > &Icurrent, bool displayKpts=false)
 
std::vector< vpImagePointgetDetectedCorners () const
 
vpFernClassifiergetFernClassifier ()
 
void getHomography (vpHomography &_H) const
 
unsigned int getNbRefPoints ()
 
void getReferencePoint (const unsigned int _i, vpImagePoint &_imPoint)
 
void getMatchedPoints (const unsigned int _index, vpImagePoint &_referencePoint, vpImagePoint &_currentPoint)
 
void setMinNbPointValidation (const unsigned int _min)
 
unsigned int getMinNbPointValidation () const
 

Protected Member Functions

virtual void init ()
 
void computeRoi (vpImagePoint *ip, const unsigned int nbpt)
 
void initialiseRefCorners (const cv::Rect &_modelROI)
 

Protected Attributes

vpFernClassifier fern
 
vpHomography homography
 
cv::Mat H
 
std::vector< cv::Point2f > dst_corners
 
bool isCorrect
 
std::vector< cv::Point2f > ref_corners
 
cv::Rect modelROI
 
std::vector< vpImagePointcurrentImagePoints
 
std::vector< vpImagePointrefImagePoints
 
unsigned int minNbMatching
 

Detailed Description

Class used to detect a planar surface.

Deprecated:
This class is deprecated with OpenCV 3.0.0 or more recent.

This class allows to learn and recognise a surface in an image based on the Fern Classifier or any other point of interest matching class.

It uses the class vpFernClassifier to extract points of interest in a reference image. These points are recorded and a classifier is trained to recognises them.

In this class the points detected are assumed to belong to a planar surface. Therefore an homography can be computed between the reference image and the current image if the object is detected in the image.

A robust method (RANSAC) is used to remove outliers in the matching process.

The following example shows how to use the class.

#include <visp3/core/vpConfig.h>
#include <visp3/core/vpDisplay.h>
#include <visp3/core/vpImage.h>
#include <visp3/vision/vpPlanarObjectDetector.h>
#if VISP_HAVE_OPENCV_VERSION >= 0x020000 // Surf Fern classifier only
available since 2.1.0 int main()
{
//First grab the reference image Ireference
//Select a part of the image by clincking on two points which define a rectangle
vpImagePoint corners[2];
for (int i=0 ; i < 2 ; i++) {
vpDisplay::getClick(Ireference, corners[i]);
}
//Build the reference points (and train the classifier).
int nbrRef;
unsigned int height, width;
height = (unsigned int)(corners[1].get_i() - corners[0].get_i());
width = (unsigned int)(corners[1].get_j() - corners[0].get_j());
nbrRef = planar.buildReference(Ireference, corners[0], height, width);
//Then grab another image which represents the current image Icurrent
//Match points between the reference points and the current points computed in the current image.
bool isDetected; height = (unsigned int)(corners[1].get_i() - corners[0].get_i());
width = (unsigned int)(corners[1].get_j() - corners[0].get_j());
isDetected = planar.matchPoint(Icurrent, corners[0], height, width);
//Display the matched points
if(isDetected){
planar.display(Ireference, Icurrent);
planar.getHomography(homography);
}
else{
std::cerr << "planar surface not detected in the current image" << std::endl;
}
return(0);
}
#else
int main() {}
#endif
Examples:
planarObjectDetector.cpp.

Definition at line 141 of file vpPlanarObjectDetector.h.

Constructor & Destructor Documentation

vpPlanarObjectDetector::vpPlanarObjectDetector ( )

Basic constructor

Definition at line 59 of file vpPlanarObjectDetector.cpp.

vpPlanarObjectDetector::vpPlanarObjectDetector ( const std::string &  _dataFile,
const std::string &  _objectName 
)

Basic constructor, load data from a file.

Parameters
_dataFile: the name of the file.
_objectName: the name of the object to load.

Definition at line 73 of file vpPlanarObjectDetector.cpp.

References load().

vpPlanarObjectDetector::~vpPlanarObjectDetector ( )
virtual

Basic destructor

Definition at line 90 of file vpPlanarObjectDetector.cpp.

Member Function Documentation

unsigned int vpPlanarObjectDetector::buildReference ( const vpImage< unsigned char > &  _I)

Train the classifier from the entire image (it is therefore assumed that the image is planar).

Depending on the parameters, the training can take up to several minutes.

Parameters
_I: The image use to train the classifier.
Returns
The number of reference points.
Examples:
planarObjectDetector.cpp.

Definition at line 141 of file vpPlanarObjectDetector.cpp.

References vpFernClassifier::buildReference(), fern, vpImage< Type >::getHeight(), vpImage< Type >::getWidth(), initialiseRefCorners(), and modelROI.

unsigned int vpPlanarObjectDetector::buildReference ( const vpImage< unsigned char > &  _I,
const vpImagePoint _iP,
unsigned int  _height,
unsigned int  _width 
)

Train the classifier on a region of the entire image. The region is a rectangle defined by its top left corner, its height and its width. The parameters of this rectangle must be given in pixel. It also includes the training of the fern classifier.

Parameters
_I: The image use to train the classifier.
_iP: The top left corner of the rectangle defining the region of interest (ROI).
_height: The height of the ROI.
_width: The width of the ROI.
Returns
the number of reference points

Definition at line 166 of file vpPlanarObjectDetector.cpp.

References vpFernClassifier::buildReference(), fern, vpImagePoint::get_u(), vpImagePoint::get_v(), initialiseRefCorners(), and modelROI.

unsigned int vpPlanarObjectDetector::buildReference ( const vpImage< unsigned char > &  _I,
const vpRect _rectangle 
)

Train the classifier on a region of the entire image. The region is a rectangle. The parameters of this rectangle must be given in pixel. It also includes the training of the fern classifier.

Parameters
_I: The image use to train the classifier.
_rectangle: The rectangle defining the region of interest (ROI).
Returns
The number of reference points.

Definition at line 190 of file vpPlanarObjectDetector.cpp.

References vpFernClassifier::buildReference(), fern, vpImagePoint::get_u(), vpImagePoint::get_v(), vpRect::getHeight(), vpRect::getTopLeft(), vpRect::getWidth(), initialiseRefCorners(), and modelROI.

void vpPlanarObjectDetector::computeRoi ( vpImagePoint ip,
const unsigned int  nbpt 
)
protected

Compute the rectangular ROI from at least 4 points and set the region of interest on the current image.

Parameters
ip: the list of image point.
nbpt: the number of point.

Definition at line 99 of file vpPlanarObjectDetector.cpp.

References vpException::badValue, and vpImagePoint::get_j().

void vpPlanarObjectDetector::display ( vpImage< unsigned char > &  I,
bool  displayKpts = false 
)

Display the result of the matching. The plane is displayed in red and the points of interests detected in the image are shown by a red dot surrounded by a green circle (the radius of the circle depends on the octave at which it has been detected).

Parameters
I: The gray scaled image for the display.
displayKpts: The flag to display keypoints in addition to the surface.
Examples:
planarObjectDetector.cpp.

Definition at line 348 of file vpPlanarObjectDetector.cpp.

References currentImagePoints, vpDisplay::displayCross(), vpDisplay::displayLine(), dst_corners, modelROI, and vpColor::red.

Referenced by display().

void vpPlanarObjectDetector::display ( vpImage< unsigned char > &  Iref,
vpImage< unsigned char > &  Icurrent,
bool  displayKpts = false 
)

This function displays the matched reference points and the matched points computed in the current image. The reference points are displayed in the image Ireference and the matched points coming from the current image are displayed in the image Icurrent. It is possible to set Ireference and Icurrent with the same image when calling the method.

Parameters
Iref: The image where the matched reference points are displayed.
Icurrent: The image where the matched points computed in the current image are displayed.
displayKpts: The flag to display keypoints in addition to the surface.

Definition at line 382 of file vpPlanarObjectDetector.cpp.

References display(), vpDisplay::displayCross(), vpColor::green, and refImagePoints.

std::vector< vpImagePoint > vpPlanarObjectDetector::getDetectedCorners ( ) const

Return the positions of the detected corners.

Returns
a vector of vpImagePoint containing the position of the corners of the planar surface in the current image.

Return the last positions of the detected corners.

Returns
The vectors of corners' postions.

Definition at line 427 of file vpPlanarObjectDetector.cpp.

References dst_corners, and vpImagePoint::set_uv().

vpFernClassifier& vpPlanarObjectDetector::getFernClassifier ( )
inline

Return a reference to the classifier.

Returns
The fern classifier.

Definition at line 216 of file vpPlanarObjectDetector.h.

void vpPlanarObjectDetector::getHomography ( vpHomography _H) const
inline

Return the computed homography between the reference image and the current image.

Parameters
_H: The computed homography.
Examples:
planarObjectDetector.cpp.

Definition at line 224 of file vpPlanarObjectDetector.h.

void vpPlanarObjectDetector::getMatchedPoints ( const unsigned int  _index,
vpImagePoint _referencePoint,
vpImagePoint _currentPoint 
)

Get the nth couple of reference point and current point which have been matched. These points are copied in the vpImagePoint instances given in argument.

Parameters
_index: The index of the desired couple of reference point and current point . The index must be between 0 and the number of matched points - 1.
_referencePoint: The coordinates of the desired reference point are copied here.
_currentPoint: The coordinates of the desired current point are copied here.

Definition at line 474 of file vpPlanarObjectDetector.cpp.

References currentImagePoints, vpException::fatalError, and refImagePoints.

unsigned int vpPlanarObjectDetector::getMinNbPointValidation ( ) const
inline

Get the threshold for the minimal number of point to validate the homography. Default value is 10.

Returns
: the current threshold.

Definition at line 270 of file vpPlanarObjectDetector.h.

unsigned int vpPlanarObjectDetector::getNbRefPoints ( )
inline

Return the number of reference points

Returns
Number of reference points.

Definition at line 231 of file vpPlanarObjectDetector.h.

void vpPlanarObjectDetector::getReferencePoint ( const unsigned int  _i,
vpImagePoint _imPoint 
)

Get the i-th reference point.

Exceptions
vpExceptionif _i is out if bound.
Parameters
_i: index of the point to get
_imPoint: image point returned by the

Definition at line 466 of file vpPlanarObjectDetector.cpp.

References vpException::fatalError, and refImagePoints.

void vpPlanarObjectDetector::init ( void  )
protectedvirtual

Initialise stuff. For the moment does nothing.

Definition at line 83 of file vpPlanarObjectDetector.cpp.

void vpPlanarObjectDetector::initialiseRefCorners ( const cv::Rect &  _modelROI)
protected

Initialise the internal reference corners from the rectangle.

Parameters
_modelROI: The rectangle defining the region of interest.

Definition at line 445 of file vpPlanarObjectDetector.cpp.

References ref_corners.

Referenced by buildReference(), and load().

void vpPlanarObjectDetector::load ( const std::string &  dataFilename,
const std::string &  objName 
)

Load the Fern classifier.

Load and initialize the Fern classifier and load the 3D and 2D keypoints. It can take up to sevral seconds.

Parameters
dataFilename: The name of the data filename (very large text file). It can have any file extension.
objName: The name of the object.
Examples:
planarObjectDetector.cpp.

Definition at line 403 of file vpPlanarObjectDetector.cpp.

References fern, vpFernClassifier::getModelROI(), initialiseRefCorners(), vpFernClassifier::load(), and modelROI.

Referenced by vpPlanarObjectDetector().

bool vpPlanarObjectDetector::matchPoint ( const vpImage< unsigned char > &  I)

Compute the points of interest in the current image and try to recognise them using the trained classifier. The matched pairs can be found with the getMatchedPointByRef function. The homography between the two planar surfaces is also computed.

Parameters
I: The gray scaled image where the points are computed.
Returns
True if the surface has been found.
Examples:
planarObjectDetector.cpp.

Definition at line 216 of file vpPlanarObjectDetector.cpp.

References currentImagePoints, dst_corners, fern, vpFernClassifier::getCurPt(), vpFernClassifier::getRefPt(), H, homography, isCorrect, vpFernClassifier::matchPoint(), minNbMatching, modelROI, ref_corners, refImagePoints, vpImagePoint::set_i(), and vpImagePoint::set_j().

Referenced by matchPoint().

bool vpPlanarObjectDetector::matchPoint ( const vpImage< unsigned char > &  I,
const vpImagePoint iP,
const unsigned int  height,
const unsigned int  width 
)

Compute the points of interest in the specified region of the current image and try to recognise them using the trained classifier. The matched pairs can be found with the getMatchedPointByRef function. The homography between the two planar surfaces is also computed.

Parameters
I: The gray scaled image where the points are computed.
iP: the top left corner of the rectangle defining the ROI
height: the height of the ROI
width: the width of the ROI
Returns
true if the surface has been found.

Definition at line 304 of file vpPlanarObjectDetector.cpp.

References vpImageTools::crop(), vpImagePoint::get_i(), vpImagePoint::get_j(), vpImage< Type >::getHeight(), vpImage< Type >::getWidth(), matchPoint(), vpImageException::notInTheImage, and vpTRACE.

bool vpPlanarObjectDetector::matchPoint ( const vpImage< unsigned char > &  I,
const vpRect rectangle 
)

Compute the points of interest in the specified region of the current image and try to recognise them using the trained classifier. The matched pairs can be found with the getMatchedPointByRef function. The homography between the two planar surfaces is also computed.

Parameters
I: The gray scaled image where the points are computed.
rectangle: The rectangle defining the ROI.
Returns
True if the surface has been found.

Definition at line 330 of file vpPlanarObjectDetector.cpp.

References vpRect::getHeight(), vpRect::getLeft(), vpRect::getTop(), vpRect::getWidth(), matchPoint(), vpImagePoint::set_i(), and vpImagePoint::set_j().

void vpPlanarObjectDetector::recordDetector ( const std::string &  objectName,
const std::string &  dataFile 
)

Record the Ferns classifier in the text file.

Parameters
objectName: The name of the object to store in the data file.
dataFile: The name of the data filename (very large text file). It can have any file extension.
Examples:
planarObjectDetector.cpp.

Definition at line 417 of file vpPlanarObjectDetector.cpp.

References fern, and vpFernClassifier::record().

void vpPlanarObjectDetector::setMinNbPointValidation ( const unsigned int  _min)
inline

Set the threshold for the minimal number of point to validate the homography. Default value is 10.

Parameters
_min: the new threshold.

Definition at line 262 of file vpPlanarObjectDetector.h.

Member Data Documentation

std::vector<vpImagePoint> vpPlanarObjectDetector::currentImagePoints
protected

Vector of the image point in the current image that match after the deletion of the outliers with the RANSAC.

Definition at line 168 of file vpPlanarObjectDetector.h.

Referenced by display(), getMatchedPoints(), and matchPoint().

std::vector<cv::Point2f> vpPlanarObjectDetector::dst_corners
protected

The estimated new coordinates of the corners (reprojected using the homography).

Definition at line 155 of file vpPlanarObjectDetector.h.

Referenced by display(), getDetectedCorners(), and matchPoint().

vpFernClassifier vpPlanarObjectDetector::fern
protected

Fern Classifier used to match the points between a reference image and the current image.

Definition at line 146 of file vpPlanarObjectDetector.h.

Referenced by buildReference(), load(), matchPoint(), and recordDetector().

cv::Mat vpPlanarObjectDetector::H
protected

Computed homography in the OpenCV format.

Definition at line 151 of file vpPlanarObjectDetector.h.

Referenced by matchPoint().

vpHomography vpPlanarObjectDetector::homography
protected

Computed homography in the ViSP format.

Definition at line 149 of file vpPlanarObjectDetector.h.

Referenced by matchPoint().

bool vpPlanarObjectDetector::isCorrect
protected

Flag to indicate wether the last computed homography is correct or not.

Definition at line 158 of file vpPlanarObjectDetector.h.

Referenced by matchPoint().

unsigned int vpPlanarObjectDetector::minNbMatching
protected

Minimal number of point to after the ransac needed to suppose that the homography has been correctly computed.

Definition at line 175 of file vpPlanarObjectDetector.h.

Referenced by matchPoint().

cv::Rect vpPlanarObjectDetector::modelROI
protected

The ROI for the reference image.

Definition at line 164 of file vpPlanarObjectDetector.h.

Referenced by buildReference(), display(), load(), and matchPoint().

std::vector<cv::Point2f> vpPlanarObjectDetector::ref_corners
protected

The corners in the reference image.

Definition at line 161 of file vpPlanarObjectDetector.h.

Referenced by initialiseRefCorners(), and matchPoint().

std::vector<vpImagePoint> vpPlanarObjectDetector::refImagePoints
protected

Vector of the image point in the reference image that match after the deletion of the outliers with the RANSAC.

Definition at line 171 of file vpPlanarObjectDetector.h.

Referenced by display(), getMatchedPoints(), getReferencePoint(), and matchPoint().