Visual Servoing Platform  version 3.2.0 under development (2018-12-13)
vpDetectorAprilTag Class Reference

#include <visp3/detection/vpDetectorAprilTag.h>

+ Inheritance diagram for vpDetectorAprilTag:

Public Types

enum  vpAprilTagFamily {
  TAG_36h11, TAG_36h10, TAG_36ARTOOLKIT, TAG_25h9,
  TAG_25h7, TAG_16h5
}
 
enum  vpPoseEstimationMethod {
  HOMOGRAPHY, HOMOGRAPHY_VIRTUAL_VS, DEMENTHON_VIRTUAL_VS, LAGRANGE_VIRTUAL_VS,
  BEST_RESIDUAL_VIRTUAL_VS
}
 

Public Member Functions

 vpDetectorAprilTag (const vpAprilTagFamily &tagFamily=TAG_36h11, const vpPoseEstimationMethod &poseEstimationMethod=HOMOGRAPHY_VIRTUAL_VS)
 
virtual ~vpDetectorAprilTag ()
 
bool detect (const vpImage< unsigned char > &I)
 
bool detect (const vpImage< unsigned char > &I, const double tagSize, const vpCameraParameters &cam, std::vector< vpHomogeneousMatrix > &cMo_vec)
 
bool getPose (size_t tagIndex, const double tagSize, const vpCameraParameters &cam, vpHomogeneousMatrix &cMo)
 
vpPoseEstimationMethod getPoseEstimationMethod () const
 
void setAprilTagNbThreads (const int nThreads)
 
void setAprilTagPoseEstimationMethod (const vpPoseEstimationMethod &poseEstimationMethod)
 
void setAprilTagQuadDecimate (const float quadDecimate)
 
void setAprilTagQuadSigma (const float quadSigma)
 
void setAprilTagRefineDecode (const bool refineDecode)
 
void setAprilTagRefineEdges (const bool refineEdges)
 
void setAprilTagRefinePose (const bool refinePose)
 
void setDisplayTag (const bool display, const vpColor &color=vpColor::none, const unsigned int thickness=2)
 
void setZAlignedWithCameraAxis (bool zAlignedWithCameraFrame)
 
Inherited functionalities from vpDetectorBase
vpRect getBBox (size_t i) const
 
vpImagePoint getCog (size_t i) const
 
std::vector< std::string > & getMessage ()
 
std::string & getMessage (size_t i)
 
size_t getNbObjects () const
 
std::vector< std::vector< vpImagePoint > > & getPolygon ()
 
std::vector< vpImagePoint > & getPolygon (size_t i)
 

Protected Attributes

bool m_displayTag
 
vpColor m_displayTagColor
 
unsigned int m_displayTagThickness
 
vpPoseEstimationMethod m_poseEstimationMethod
 
vpAprilTagFamily m_tagFamily
 
bool m_zAlignedWithCameraFrame
 
std::vector< std::vector< vpImagePoint > > m_polygon
 
std::vector< std::string > m_message
 
size_t m_nb_objects
 

Detailed Description

Base class for AprilTag detector. This class is a wrapper over AprilTag. There is no need to download and install AprilTag from source code or existing pre-built packages since the source code is embedded in ViSP. Reference papers are AprilTag: A robust and flexible visual fiducial system ([31]) and AprilTag 2: Efficient and robust fiducial detection ([41]).

The detect() function allows to detect multiple tags in an image. Once detected, for each tag it is possible to retrieve the location of the corners using getPolygon(), the encoded message using getMessage(), the bounding box using getBBox() and the center of gravity using getCog().

If camera parameters and the size of the tag are provided, you can also estimate the 3D pose of the tag in terms of position and orientation wrt the camera considering 2 cases:

The following sample code shows how to use this class to detect the location of 36h11 AprilTag patterns in an image.

#include <visp3/detection/vpDetectorAprilTag.h>
#include <visp3/io/vpImageIo.h>
int main()
{
#ifdef VISP_HAVE_APRILTAG
vpImageIo::read(I, "image-tag36h11.pgm");
bool status = detector.detect(I);
if (status) {
for(size_t i=0; i < detector.getNbObjects(); i++) {
std::cout << "Tag code " << i << ":" << std::endl;
std::vector<vpImagePoint> p = detector.getPolygon(i);
for(size_t j=0; j < p.size(); j++)
std::cout << " Point " << j << ": " << p[j] << std::endl;
std::cout << " Message: \"" << detector.getMessage(i) << "\"" << std::endl;
}
}
#endif
}

The previous example may produce results like:

Tag code 0:
Point 0: 124.008, 442.226
Point 1: 194.614, 441.237
Point 2: 184.833, 540.386
Point 3: 111.948, 533.634
Message: "36h11 id: 0"
Tag code 1:
Point 0: 245.327, 438.801
Point 1: 338.116, 437.221
Point 2: 339.341, 553.539
Point 3: 238.954, 543.855
Message: "36h11 id: 1"

This other example shows how to estimate the 3D pose of 36h11 AprilTag patterns considering that all the tags have the same size (in our example 0.053 m).

#include <visp3/detection/vpDetectorAprilTag.h>
#include <visp3/io/vpImageIo.h>
int main()
{
#ifdef VISP_HAVE_APRILTAG
vpImageIo::read(I, "image-tag36h11.pgm");
std::vector<vpHomogeneousMatrix> cMo;
cam.initPersProjWithoutDistortion(615.1674805, 615.1675415, 312.1889954, 243.4373779);
double tagSize = 0.053;
bool status = detector.detect(I, tagSize, cam, cMo);
if (status) {
for(size_t i=0; i < detector.getNbObjects(); i++) {
std::cout << "Tag number " << i << ":" << std::endl;
std::cout << " Message: \"" << detector.getMessage(i) << "\"" << std::endl;
std::cout << " Pose: " << vpPoseVector(cMo[i]).t() << std::endl;
std::size_t tag_id_pos = detector.getMessage(i).find("id: ");
if (tag_id_pos != std::string::npos) {
std::string tag_id = detector.getMessage(i).substr(tag_id_pos + 4);
std::cout << " Tag Id: " << tag_id << std::endl;
}
}
}
#endif
}

The previous example may produce results like:

Tag number 0:
Message: "36h11 id: 0"
Pose: 0.1015061088 -0.05239057228 0.3549037285 1.991474322 2.04143538 -0.9412360063
Tag Id: 0
Tag number 1:
Message: "36h11 id: 1"
Pose: 0.08951250829 0.02243780207 0.306540622 1.998073197 2.061488008 -0.8699567948
Tag Id: 1

In this other example we estimate the 3D pose of 36h11 AprilTag patterns considering that tag 36h11 with id 0 (in that case the tag message is "36h11 id: 0") has a size of 0.040 m, while all the others have a size of 0.053m.

#include <visp3/detection/vpDetectorAprilTag.h>
#include <visp3/io/vpImageIo.h>
int main()
{
#ifdef VISP_HAVE_APRILTAG
vpImageIo::read(I, "image-tag36h11.pgm");
cam.initPersProjWithoutDistortion(615.1674805, 615.1675415, 312.1889954, 243.4373779);
double tagSize_id_0 = 0.04;
double tagSize_id_others = 0.053;
bool status = detector.detect(I);
if (status) {
for(size_t i=0; i < detector.getNbObjects(); i++) {
std::cout << "Tag code " << i << ":" << std::endl;
std::cout << " Message: \"" << detector.getMessage(i) << "\"" << std::endl;
if (detector.getMessage(i) == std::string("36h11 id: 0")) {
if (! detector.getPose(i, tagSize_id_0, cam, cMo)) {
std::cout << "Unable to get tag index " << i << " pose!" << std::endl;
}
}
else {
if (! detector.getPose(i, tagSize_id_others, cam, cMo)) {
std::cout << "Unable to get tag index " << i << " pose!" << std::endl;
}
}
std::cout << " Pose: " << vpPoseVector(cMo).t() << std::endl;
}
}
#endif
}

With respect to the previous example, this example may now produce a different pose for tag with id 0:

Tag code 0:
Message: "36h11 id: 0"
Pose: 0.07660838403 -0.03954005455 0.2678518706 1.991474322 2.04143538 -0.9412360063
Tag code 1:
Message: "36h11 id: 1"
Pose: 0.08951250829 0.02243780207 0.306540622 1.998073197 2.061488008 -0.8699567948

Other examples are also provided in tutorial-apriltag-detector.cpp and tutorial-apriltag-detector-live.cpp

Examples:
mbot-apriltag-2D-half-vs.cpp, mbot-apriltag-ibvs.cpp, mbot-apriltag-pbvs.cpp, servoFrankaPBVS.cpp, testAprilTag.cpp, tutorial-apriltag-detector-live.cpp, tutorial-apriltag-detector.cpp, tutorial-mb-generic-tracker-apriltag-live-realsense2.cpp, and tutorial-mb-generic-tracker-apriltag-live-webcam.cpp.

Definition at line 213 of file vpDetectorAprilTag.h.

Member Enumeration Documentation

Enumerator
TAG_36h11 

AprilTag 36h11 pattern (recommended)

TAG_36h10 

AprilTag 36h10 pattern

TAG_36ARTOOLKIT 

ARToolKit pattern.

TAG_25h9 

AprilTag 25h9 pattern

TAG_25h7 

AprilTag 25h7 pattern

TAG_16h5 

AprilTag 16h5 pattern

Examples:
mbot-apriltag-2D-half-vs.cpp, mbot-apriltag-ibvs.cpp, mbot-apriltag-pbvs.cpp, tutorial-apriltag-detector-live.cpp, tutorial-apriltag-detector.cpp, tutorial-mb-generic-tracker-apriltag-live-realsense2.cpp, and tutorial-mb-generic-tracker-apriltag-live-webcam.cpp.

Definition at line 217 of file vpDetectorAprilTag.h.

Enumerator
HOMOGRAPHY 

Pose from homography

HOMOGRAPHY_VIRTUAL_VS 

Non linear virtual visual servoing approach initialized by the homography approach

DEMENTHON_VIRTUAL_VS 

Non linear virtual visual servoing approach initialized by the Dementhon approach

LAGRANGE_VIRTUAL_VS 

Non linear virtual visual servoing approach initialized by the Lagrange approach

BEST_RESIDUAL_VIRTUAL_VS 

Non linear virtual visual servoing approach initialized by the approach that gives the lowest residual

Examples:
tutorial-apriltag-detector-live.cpp, and tutorial-apriltag-detector.cpp.

Definition at line 237 of file vpDetectorAprilTag.h.

Constructor & Destructor Documentation

vpDetectorAprilTag::vpDetectorAprilTag ( const vpAprilTagFamily tagFamily = TAG_36h11,
const vpPoseEstimationMethod poseEstimationMethod = HOMOGRAPHY_VIRTUAL_VS 
)

Default constructor.

Definition at line 385 of file vpDetectorAprilTag.cpp.

vpDetectorAprilTag::~vpDetectorAprilTag ( )
virtual

Destructor that desallocate memory.

Definition at line 396 of file vpDetectorAprilTag.cpp.

Member Function Documentation

bool vpDetectorAprilTag::detect ( const vpImage< unsigned char > &  I)
virtual
bool vpDetectorAprilTag::detect ( const vpImage< unsigned char > &  I,
const double  tagSize,
const vpCameraParameters cam,
std::vector< vpHomogeneousMatrix > &  cMo_vec 
)

Detect AprilTag tags in the image and compute the corresponding tag poses considering that all the tags have the same size.

If tags with different sizes have to be considered, you may use getPose().

Parameters
I: Input image.
tagSize: Tag size in meter corresponding to the external width of the pattern.
cam: Camera intrinsic parameters.
cMo_vec: List of tag poses.
Returns
true if at least one tag is detected.
See also
getPose()

Definition at line 432 of file vpDetectorAprilTag.cpp.

References m_displayTag, m_displayTagColor, m_displayTagThickness, vpDetectorBase::m_message, vpDetectorBase::m_nb_objects, and vpDetectorBase::m_polygon.

vpImagePoint vpDetectorBase::getCog ( size_t  i) const
inherited

Return the center of gravity location of the ith object.

Examples:
mbot-apriltag-2D-half-vs.cpp, and mbot-apriltag-ibvs.cpp.

Definition at line 74 of file vpDetectorBase.cpp.

References vpDetectorBase::m_polygon.

std::vector<std::string>& vpDetectorBase::getMessage ( void  )
inlineinherited
std::string & vpDetectorBase::getMessage ( size_t  i)
inherited

Returns the contained message of the ith object if there is one.

Definition at line 62 of file vpDetectorBase.cpp.

References vpException::badValue, vpDetectorBase::m_message, and vpDetectorBase::m_polygon.

std::vector<std::vector<vpImagePoint> >& vpDetectorBase::getPolygon ( )
inlineinherited
std::vector< vpImagePoint > & vpDetectorBase::getPolygon ( size_t  i)
inherited

Returns ith object container box as a vector of points.

Definition at line 50 of file vpDetectorBase.cpp.

References vpException::badValue, and vpDetectorBase::m_polygon.

bool vpDetectorAprilTag::getPose ( size_t  tagIndex,
const double  tagSize,
const vpCameraParameters cam,
vpHomogeneousMatrix cMo 
)

Get the pose of a tag depending on its size and camera parameters. This function is useful to get the pose of tags with different sizes, while detect(const vpImage<unsigned char> &, const double, const vpCameraParameters &, std::vector<vpHomogeneousMatrix> &) considers that all the tags have the same size.

Parameters
[in]tagIndex: Index of the tag. Value should be in range [0, nb tags-1] with nb_tags = getNbObjects().
[in]tagSize: Tag size in meter corresponding to the external width of the pattern.
[in]cam: Camera intrinsic parameters.
[out]cMo: Pose of the tag.
Returns
true if success, false otherwise.

The following code shows how to use this function:

vpDetectorAprilTag detector(tagFamily);
detector.detect(I);
for (size_t i = 0; i < detector.getNbObjects(); i++) {
double tagSize;
detector.getPose(i, tagSize, cam, cMo);
}
See also
detect(const vpImage<unsigned char> &, const double, const vpCameraParameters &, std::vector<vpHomogeneousMatrix> &)

Definition at line 475 of file vpDetectorAprilTag.cpp.

vpPoseEstimationMethod vpDetectorAprilTag::getPoseEstimationMethod ( ) const
inline

Return the pose estimation method.

Definition at line 263 of file vpDetectorAprilTag.h.

void vpDetectorAprilTag::setAprilTagNbThreads ( const int  nThreads)
void vpDetectorAprilTag::setAprilTagPoseEstimationMethod ( const vpPoseEstimationMethod poseEstimationMethod)

Set the method to use to compute the pose,

See also
vpPoseEstimationMethod
Parameters
poseEstimationMethod: The method to used.
Examples:
mbot-apriltag-2D-half-vs.cpp, mbot-apriltag-pbvs.cpp, servoFrankaPBVS.cpp, tutorial-apriltag-detector-live.cpp, and tutorial-apriltag-detector.cpp.

Definition at line 496 of file vpDetectorAprilTag.cpp.

References m_poseEstimationMethod.

void vpDetectorAprilTag::setAprilTagQuadDecimate ( const float  quadDecimate)

From the AprilTag code:

detection of quads can be done on a lower-resolution image, improving speed at a cost of pose accuracy and a slight decrease in detection rate. Decoding the binary payload is still done at full resolution.

Default is 1.0, increase this value to reduce the computation time.

Parameters
quadDecimate: Value for quad_decimate.
Examples:
mbot-apriltag-2D-half-vs.cpp, mbot-apriltag-ibvs.cpp, mbot-apriltag-pbvs.cpp, servoFrankaPBVS.cpp, tutorial-apriltag-detector-live.cpp, tutorial-apriltag-detector.cpp, tutorial-mb-generic-tracker-apriltag-live-realsense2.cpp, and tutorial-mb-generic-tracker-apriltag-live-webcam.cpp.

Definition at line 514 of file vpDetectorAprilTag.cpp.

void vpDetectorAprilTag::setAprilTagQuadSigma ( const float  quadSigma)

From the AprilTag code:

What Gaussian blur should be applied to the segmented image (used for quad detection?) Parameter is the standard deviation in pixels. Very noisy images benefit from non-zero values (e.g. 0.8).

Default is 0.0.

Parameters
quadSigma: Value for quad_sigma.

Definition at line 528 of file vpDetectorAprilTag.cpp.

void vpDetectorAprilTag::setAprilTagRefineDecode ( const bool  refineDecode)

From the AprilTag code:

when non-zero, detections are refined in a way intended to increase the number of detected tags. Especially effective for very small tags near the resolution threshold (e.g. 10px on a side).

Default is 0.

Parameters
refineDecode: If true, set refine_decode to 1.

Definition at line 542 of file vpDetectorAprilTag.cpp.

void vpDetectorAprilTag::setAprilTagRefineEdges ( const bool  refineEdges)

From the AprilTag code:

When non-zero, the edges of the each quad are adjusted to "snap to" strong gradients nearby. This is useful when decimation is employed, as it can increase the quality of the initial quad estimate substantially. Generally recommended to be on (1). Very computationally inexpensive. Option is ignored if quad_decimate = 1.

Default is 1.

Parameters
refineEdges: If true, set refine_edges to 1.

Definition at line 558 of file vpDetectorAprilTag.cpp.

void vpDetectorAprilTag::setAprilTagRefinePose ( const bool  refinePose)

From the AprilTag code:

when non-zero, detections are refined in a way intended to increase the accuracy of the extracted pose. This is done by maximizing the contrast around the black and white border of the tag. This generally increases the number of successfully detected tags, though not as effectively (or quickly) as refine_decode. This option must be enabled in order for "goodness" to be computed.

Default is 0.

Parameters
refinePose: If true, set refine_pose to 1.

Definition at line 576 of file vpDetectorAprilTag.cpp.

void vpDetectorAprilTag::setDisplayTag ( const bool  display,
const vpColor color = vpColor::none,
const unsigned int  thickness = 2 
)
inline

Allow to enable the display of overlay tag information in the windows (vpDisplay) associated to the input image.

Examples:
mbot-apriltag-2D-half-vs.cpp, mbot-apriltag-ibvs.cpp, mbot-apriltag-pbvs.cpp, servoFrankaPBVS.cpp, tutorial-apriltag-detector-live.cpp, and tutorial-apriltag-detector.cpp.

Definition at line 275 of file vpDetectorAprilTag.h.

void vpDetectorAprilTag::setZAlignedWithCameraAxis ( bool  zAlignedWithCameraFrame)

Modify the resulting tag pose returned by getPose() in order to get a pose where z-axis is aligned when the camera plane is parallel to the tag.

Parameters
zAlignedWithCameraFrame: Flag to get a pose where z-axis is aligned with the camera frame.
Examples:
tutorial-apriltag-detector.cpp.

Definition at line 583 of file vpDetectorAprilTag.cpp.

References m_zAlignedWithCameraFrame.

Member Data Documentation

bool vpDetectorAprilTag::m_displayTag
protected

Definition at line 285 of file vpDetectorAprilTag.h.

Referenced by detect().

vpColor vpDetectorAprilTag::m_displayTagColor
protected

Definition at line 286 of file vpDetectorAprilTag.h.

Referenced by detect().

unsigned int vpDetectorAprilTag::m_displayTagThickness
protected

Definition at line 287 of file vpDetectorAprilTag.h.

Referenced by detect().

std::vector<std::string> vpDetectorBase::m_message
protectedinherited
size_t vpDetectorBase::m_nb_objects
protectedinherited
std::vector<std::vector<vpImagePoint> > vpDetectorBase::m_polygon
protectedinherited
vpPoseEstimationMethod vpDetectorAprilTag::m_poseEstimationMethod
protected

Definition at line 288 of file vpDetectorAprilTag.h.

Referenced by setAprilTagPoseEstimationMethod().

vpAprilTagFamily vpDetectorAprilTag::m_tagFamily
protected

Definition at line 289 of file vpDetectorAprilTag.h.

bool vpDetectorAprilTag::m_zAlignedWithCameraFrame
protected

Definition at line 290 of file vpDetectorAprilTag.h.

Referenced by setZAlignedWithCameraAxis().