Visual Servoing Platform  version 3.2.1 under development (2019-06-26)
vpDetectorAprilTag Class Reference

#include <visp3/detection/vpDetectorAprilTag.h>

+ Inheritance diagram for vpDetectorAprilTag:

Public Types

enum  vpAprilTagFamily {
  TAG_36h11, TAG_36h10, TAG_36ARTOOLKIT, TAG_25h9,
  TAG_25h7, TAG_16h5, TAG_CIRCLE21h7, TAG_CIRCLE49h12,
  TAG_CUSTOM48h12, TAG_STANDARD41h12, TAG_STANDARD52h13
}
 
enum  vpPoseEstimationMethod {
  HOMOGRAPHY, HOMOGRAPHY_VIRTUAL_VS, DEMENTHON_VIRTUAL_VS, LAGRANGE_VIRTUAL_VS,
  BEST_RESIDUAL_VIRTUAL_VS, HOMOGRAPHY_ORTHOGONAL_ITERATION
}
 

Public Member Functions

 vpDetectorAprilTag (const vpAprilTagFamily &tagFamily=TAG_36h11, const vpPoseEstimationMethod &poseEstimationMethod=HOMOGRAPHY_VIRTUAL_VS)
 
virtual ~vpDetectorAprilTag ()
 
bool detect (const vpImage< unsigned char > &I)
 
bool detect (const vpImage< unsigned char > &I, const double tagSize, const vpCameraParameters &cam, std::vector< vpHomogeneousMatrix > &cMo_vec, std::vector< vpHomogeneousMatrix > *cMo_vec2=NULL, std::vector< double > *projErrors=NULL, std::vector< double > *projErrors2=NULL)
 
bool getPose (size_t tagIndex, const double tagSize, const vpCameraParameters &cam, vpHomogeneousMatrix &cMo, vpHomogeneousMatrix *cMo2=NULL, double *projError=NULL, double *projError2=NULL)
 
vpPoseEstimationMethod getPoseEstimationMethod () const
 
void setAprilTagDecodeSharpening (const double decodeSharpening)
 
void setAprilTagNbThreads (const int nThreads)
 
void setAprilTagPoseEstimationMethod (const vpPoseEstimationMethod &poseEstimationMethod)
 
void setAprilTagQuadDecimate (const float quadDecimate)
 
void setAprilTagQuadSigma (const float quadSigma)
 
void setAprilTagRefineDecode (const bool refineDecode)
 
void setAprilTagRefineEdges (const bool refineEdges)
 
void setAprilTagRefinePose (const bool refinePose)
 
void setDisplayTag (const bool display, const vpColor &color=vpColor::none, const unsigned int thickness=2)
 
void setZAlignedWithCameraAxis (bool zAlignedWithCameraFrame)
 
Inherited functionalities from vpDetectorBase
vpRect getBBox (size_t i) const
 
vpImagePoint getCog (size_t i) const
 
std::vector< std::string > & getMessage ()
 
std::string & getMessage (size_t i)
 
size_t getNbObjects () const
 
std::vector< std::vector< vpImagePoint > > & getPolygon ()
 
std::vector< vpImagePoint > & getPolygon (size_t i)
 

Protected Attributes

bool m_displayTag
 
vpColor m_displayTagColor
 
unsigned int m_displayTagThickness
 
vpPoseEstimationMethod m_poseEstimationMethod
 
vpAprilTagFamily m_tagFamily
 
std::vector< std::vector< vpImagePoint > > m_polygon
 
std::vector< std::string > m_message
 
size_t m_nb_objects
 

Detailed Description

Base class for AprilTag detector. This class is a wrapper over AprilTag. There is no need to download and install AprilTag from source code or existing pre-built packages since the source code is embedded in ViSP. Reference papers are AprilTag: A robust and flexible visual fiducial system ([32]), AprilTag 2: Efficient and robust fiducial detection ([42]) and Flexible Layouts for Fiducial Tags (Under Review) ([21]).

The detect() function allows to detect multiple tags in an image. Once detected, for each tag it is possible to retrieve the location of the corners using getPolygon(), the encoded message using getMessage(), the bounding box using getBBox() and the center of gravity using getCog().

If camera parameters and the size of the tag are provided, you can also estimate the 3D pose of the tag in terms of position and orientation wrt the camera considering 2 cases:

  • 1. If all the tags have the same size use detect(const vpImage<unsigned char> &, const double, const vpCameraParameters &, std::vector<vpHomogeneousMatrix> &)
  • 2. If tag sizes differ, use rather getPose()

The following sample code shows how to use this class to detect the location of 36h11 AprilTag patterns in an image.

#include <visp3/detection/vpDetectorAprilTag.h>
#include <visp3/io/vpImageIo.h>
int main()
{
#ifdef VISP_HAVE_APRILTAG
vpImageIo::read(I, "image-tag36h11.pgm");
bool status = detector.detect(I);
if (status) {
for(size_t i=0; i < detector.getNbObjects(); i++) {
std::cout << "Tag code " << i << ":" << std::endl;
std::vector<vpImagePoint> p = detector.getPolygon(i);
for(size_t j=0; j < p.size(); j++)
std::cout << " Point " << j << ": " << p[j] << std::endl;
std::cout << " Message: \"" << detector.getMessage(i) << "\"" << std::endl;
}
}
#endif
}

The previous example may produce results like:

Tag code 0:
Point 0: 124.008, 442.226
Point 1: 194.614, 441.237
Point 2: 184.833, 540.386
Point 3: 111.948, 533.634
Message: "36h11 id: 0"
Tag code 1:
Point 0: 245.327, 438.801
Point 1: 338.116, 437.221
Point 2: 339.341, 553.539
Point 3: 238.954, 543.855
Message: "36h11 id: 1"

This other example shows how to estimate the 3D pose of 36h11 AprilTag patterns considering that all the tags have the same size (in our example 0.053 m).

#include <visp3/detection/vpDetectorAprilTag.h>
#include <visp3/io/vpImageIo.h>
int main()
{
#ifdef VISP_HAVE_APRILTAG
vpImageIo::read(I, "image-tag36h11.pgm");
std::vector<vpHomogeneousMatrix> cMo;
cam.initPersProjWithoutDistortion(615.1674805, 615.1675415, 312.1889954, 243.4373779);
double tagSize = 0.053;
bool status = detector.detect(I, tagSize, cam, cMo);
if (status) {
for(size_t i=0; i < detector.getNbObjects(); i++) {
std::cout << "Tag number " << i << ":" << std::endl;
std::cout << " Message: \"" << detector.getMessage(i) << "\"" << std::endl;
std::cout << " Pose: " << vpPoseVector(cMo[i]).t() << std::endl;
std::size_t tag_id_pos = detector.getMessage(i).find("id: ");
if (tag_id_pos != std::string::npos) {
std::string tag_id = detector.getMessage(i).substr(tag_id_pos + 4);
std::cout << " Tag Id: " << tag_id << std::endl;
}
}
}
#endif
}

The previous example may produce results like:

Tag number 0:
Message: "36h11 id: 0"
Pose: 0.1015061088 -0.05239057228 0.3549037285 1.991474322 2.04143538 -0.9412360063
Tag Id: 0
Tag number 1:
Message: "36h11 id: 1"
Pose: 0.08951250829 0.02243780207 0.306540622 1.998073197 2.061488008 -0.8699567948
Tag Id: 1

In this other example we estimate the 3D pose of 36h11 AprilTag patterns considering that tag 36h11 with id 0 (in that case the tag message is "36h11 id: 0") has a size of 0.040 m, while all the others have a size of 0.053m.

#include <visp3/detection/vpDetectorAprilTag.h>
#include <visp3/io/vpImageIo.h>
int main()
{
#ifdef VISP_HAVE_APRILTAG
vpImageIo::read(I, "image-tag36h11.pgm");
cam.initPersProjWithoutDistortion(615.1674805, 615.1675415, 312.1889954, 243.4373779);
double tagSize_id_0 = 0.04;
double tagSize_id_others = 0.053;
bool status = detector.detect(I);
if (status) {
for(size_t i=0; i < detector.getNbObjects(); i++) {
std::cout << "Tag code " << i << ":" << std::endl;
std::cout << " Message: \"" << detector.getMessage(i) << "\"" << std::endl;
if (detector.getMessage(i) == std::string("36h11 id: 0")) {
if (! detector.getPose(i, tagSize_id_0, cam, cMo)) {
std::cout << "Unable to get tag index " << i << " pose!" << std::endl;
}
}
else {
if (! detector.getPose(i, tagSize_id_others, cam, cMo)) {
std::cout << "Unable to get tag index " << i << " pose!" << std::endl;
}
}
std::cout << " Pose: " << vpPoseVector(cMo).t() << std::endl;
}
}
#endif
}

With respect to the previous example, this example may now produce a different pose for tag with id 0:

Tag code 0:
Message: "36h11 id: 0"
Pose: 0.07660838403 -0.03954005455 0.2678518706 1.991474322 2.04143538 -0.9412360063
Tag code 1:
Message: "36h11 id: 1"
Pose: 0.08951250829 0.02243780207 0.306540622 1.998073197 2.061488008 -0.8699567948

Other examples are also provided in tutorial-apriltag-detector.cpp and tutorial-apriltag-detector-live.cpp

Examples:
mbot-apriltag-2D-half-vs.cpp, mbot-apriltag-ibvs.cpp, mbot-apriltag-pbvs.cpp, servoFrankaPBVS.cpp, testAprilTag.cpp, tutorial-apriltag-detector-live.cpp, tutorial-apriltag-detector.cpp, tutorial-mb-generic-tracker-apriltag-rs2.cpp, and tutorial-mb-generic-tracker-apriltag-webcam.cpp.

Definition at line 214 of file vpDetectorAprilTag.h.

Member Enumeration Documentation

Enumerator
TAG_36h11 

AprilTag 36h11 pattern (recommended)

TAG_36h10 

DEPRECATED.

TAG_36ARTOOLKIT 

DEPRECATED AND WILL NOT DETECT ARTOOLKIT TAGS.

TAG_25h9 

AprilTag 25h9 pattern.

TAG_25h7 

DEPRECATED AND POOR DETECTION PERFORMANCE.

TAG_16h5 

AprilTag 16h5 pattern.

TAG_CIRCLE21h7 

AprilTag Circle21h7 pattern.

TAG_CIRCLE49h12 

AprilTag Circle49h12 pattern.

TAG_CUSTOM48h12 

AprilTag Custom48h12 pattern.

TAG_STANDARD41h12 

AprilTag Standard41h12 pattern.

TAG_STANDARD52h13 

AprilTag Standard52h13 pattern.

Examples:
mbot-apriltag-2D-half-vs.cpp, mbot-apriltag-ibvs.cpp, mbot-apriltag-pbvs.cpp, tutorial-apriltag-detector-live.cpp, tutorial-apriltag-detector.cpp, tutorial-mb-generic-tracker-apriltag-rs2.cpp, and tutorial-mb-generic-tracker-apriltag-webcam.cpp.

Definition at line 217 of file vpDetectorAprilTag.h.

Enumerator
HOMOGRAPHY 

Pose from homography

HOMOGRAPHY_VIRTUAL_VS 

Non linear virtual visual servoing approach initialized by the homography approach

DEMENTHON_VIRTUAL_VS 

Non linear virtual visual servoing approach initialized by the Dementhon approach

LAGRANGE_VIRTUAL_VS 

Non linear virtual visual servoing approach initialized by the Lagrange approach

BEST_RESIDUAL_VIRTUAL_VS 

Non linear virtual visual servoing approach initialized by the approach that gives the lowest residual

HOMOGRAPHY_ORTHOGONAL_ITERATION 

Pose from homography followed by a refine by Orthogonal Iteration

Examples:
testAprilTag.cpp, tutorial-apriltag-detector-live.cpp, and tutorial-apriltag-detector.cpp.

Definition at line 231 of file vpDetectorAprilTag.h.

Constructor & Destructor Documentation

vpDetectorAprilTag::vpDetectorAprilTag ( const vpAprilTagFamily tagFamily = TAG_36h11,
const vpPoseEstimationMethod poseEstimationMethod = HOMOGRAPHY_VIRTUAL_VS 
)

Definition at line 588 of file vpDetectorAprilTag.cpp.

vpDetectorAprilTag::~vpDetectorAprilTag ( )
virtual

Definition at line 596 of file vpDetectorAprilTag.cpp.

Member Function Documentation

bool vpDetectorAprilTag::detect ( const vpImage< unsigned char > &  I)
virtual
bool vpDetectorAprilTag::detect ( const vpImage< unsigned char > &  I,
const double  tagSize,
const vpCameraParameters cam,
std::vector< vpHomogeneousMatrix > &  cMo_vec,
std::vector< vpHomogeneousMatrix > *  cMo_vec2 = NULL,
std::vector< double > *  projErrors = NULL,
std::vector< double > *  projErrors2 = NULL 
)

Detect AprilTag tags in the image and compute the corresponding tag poses considering that all the tags have the same size.

If tags with different sizes have to be considered, you may use getPose().

Parameters
[in]I: Input image.
[in]tagSize: Tag size in meter corresponding to the external width of the pattern.
[in]cam: Camera intrinsic parameters.
[out]cMo_vec: List of tag poses.
[out]cMo_vec2: Optional second list of tag poses, since there are 2 solutions for planar pose estimation.
[out]projErrors: Optional (sum of squared) projection errors in the normalized camera frame.
[out]projErrors2: Optional (sum of squared) projection errors for the 2nd solution in the normalized camera frame.
Returns
true if at least one tag is detected.
See also
getPose()

Definition at line 637 of file vpDetectorAprilTag.cpp.

References m_displayTag, m_displayTagColor, m_displayTagThickness, vpDetectorBase::m_message, vpDetectorBase::m_nb_objects, and vpDetectorBase::m_polygon.

vpImagePoint vpDetectorBase::getCog ( size_t  i) const
inherited

Return the center of gravity location of the ith object.

Examples:
mbot-apriltag-2D-half-vs.cpp, and mbot-apriltag-ibvs.cpp.

Definition at line 74 of file vpDetectorBase.cpp.

References vpDetectorBase::m_polygon.

std::vector<std::string>& vpDetectorBase::getMessage ( void  )
inlineinherited
std::string & vpDetectorBase::getMessage ( size_t  i)
inherited

Returns the contained message of the ith object if there is one.

Definition at line 62 of file vpDetectorBase.cpp.

References vpException::badValue, vpDetectorBase::m_message, and vpDetectorBase::m_polygon.

std::vector<std::vector<vpImagePoint> >& vpDetectorBase::getPolygon ( )
inlineinherited
std::vector< vpImagePoint > & vpDetectorBase::getPolygon ( size_t  i)
inherited

Returns ith object container box as a vector of points.

Definition at line 50 of file vpDetectorBase.cpp.

References vpException::badValue, and vpDetectorBase::m_polygon.

bool vpDetectorAprilTag::getPose ( size_t  tagIndex,
const double  tagSize,
const vpCameraParameters cam,
vpHomogeneousMatrix cMo,
vpHomogeneousMatrix cMo2 = NULL,
double *  projError = NULL,
double *  projError2 = NULL 
)

Get the pose of a tag depending on its size and camera parameters. This function is useful to get the pose of tags with different sizes, while detect(const vpImage<unsigned char> &, const double, const vpCameraParameters &, std::vector<vpHomogeneousMatrix> &) considers that all the tags have the same size.

Parameters
[in]tagIndex: Index of the tag. Value should be in range [0, nb tags-1] with nb_tags = getNbObjects().
[in]tagSize: Tag size in meter corresponding to the external width of the pattern.
[in]cam: Camera intrinsic parameters.
[out]cMo: Pose of the tag.
[out]cMo2: Optional second pose of the tag.
[out]projError: Optional (sum of squared) projection errors in the normalized camera frame.
[out]projError2: Optional (sum of squared) projection errors for the 2nd solution in the normalized camera frame.
Returns
true if success, false otherwise.

The following code shows how to use this function:

vpDetectorAprilTag detector(tagFamily);
detector.detect(I);
for (size_t i = 0; i < detector.getNbObjects(); i++) {
double tagSize;
detector.getPose(i, tagSize, cam, cMo);
}
See also
detect(const vpImage<unsigned char> &, const double, const vpCameraParameters &, std::vector<vpHomogeneousMatrix> &)

Definition at line 684 of file vpDetectorAprilTag.cpp.

vpPoseEstimationMethod vpDetectorAprilTag::getPoseEstimationMethod ( ) const
inline

Return the pose estimation method.

Definition at line 261 of file vpDetectorAprilTag.h.

void vpDetectorAprilTag::setAprilTagDecodeSharpening ( const double  decodeSharpening)

Definition at line 691 of file vpDetectorAprilTag.cpp.

void vpDetectorAprilTag::setAprilTagNbThreads ( const int  nThreads)
void vpDetectorAprilTag::setAprilTagPoseEstimationMethod ( const vpPoseEstimationMethod poseEstimationMethod)

Set the method to use to compute the pose,

See also
vpPoseEstimationMethod
Parameters
poseEstimationMethod: The method to used.
Examples:
mbot-apriltag-2D-half-vs.cpp, mbot-apriltag-pbvs.cpp, servoFrankaPBVS.cpp, tutorial-apriltag-detector-live.cpp, and tutorial-apriltag-detector.cpp.

Definition at line 712 of file vpDetectorAprilTag.cpp.

References m_poseEstimationMethod.

void vpDetectorAprilTag::setAprilTagQuadDecimate ( const float  quadDecimate)

From the AprilTag code:

detection of quads can be done on a lower-resolution image, improving speed at a cost of pose accuracy and a slight decrease in detection rate. Decoding the binary payload is still done at full resolution.

Default is 1.0, increase this value to reduce the computation time.

Parameters
quadDecimate: Value for quad_decimate.
Examples:
mbot-apriltag-2D-half-vs.cpp, mbot-apriltag-ibvs.cpp, mbot-apriltag-pbvs.cpp, servoFrankaPBVS.cpp, tutorial-apriltag-detector-live.cpp, tutorial-apriltag-detector.cpp, tutorial-mb-generic-tracker-apriltag-rs2.cpp, and tutorial-mb-generic-tracker-apriltag-webcam.cpp.

Definition at line 730 of file vpDetectorAprilTag.cpp.

void vpDetectorAprilTag::setAprilTagQuadSigma ( const float  quadSigma)

From the AprilTag code:

What Gaussian blur should be applied to the segmented image (used for quad detection?) Parameter is the standard deviation in pixels. Very noisy images benefit from non-zero values (e.g. 0.8).

Default is 0.0.

Parameters
quadSigma: Value for quad_sigma.

Definition at line 747 of file vpDetectorAprilTag.cpp.

vp_deprecated void vpDetectorAprilTag::setAprilTagRefineDecode ( const bool  refineDecode)

Deprecated parameter from AprilTag 2 version.

Definition at line 756 of file vpDetectorAprilTag.cpp.

void vpDetectorAprilTag::setAprilTagRefineEdges ( const bool  refineEdges)

From the AprilTag code:

When non-zero, the edges of the each quad are adjusted to "snap to" strong gradients nearby. This is useful when decimation is employed, as it can increase the quality of the initial quad estimate substantially. Generally recommended to be on (1). Very computationally inexpensive. Option is ignored if quad_decimate = 1.

Default is 1.

Parameters
refineEdges: If true, set refine_edges to 1.

Definition at line 775 of file vpDetectorAprilTag.cpp.

vp_deprecated void vpDetectorAprilTag::setAprilTagRefinePose ( const bool  refinePose)

Deprecated parameter from AprilTag 2 version.

Definition at line 784 of file vpDetectorAprilTag.cpp.

void vpDetectorAprilTag::setDisplayTag ( const bool  display,
const vpColor color = vpColor::none,
const unsigned int  thickness = 2 
)
inline

Allow to enable the display of overlay tag information in the windows (vpDisplay) associated to the input image.

Examples:
mbot-apriltag-2D-half-vs.cpp, mbot-apriltag-ibvs.cpp, mbot-apriltag-pbvs.cpp, servoFrankaPBVS.cpp, tutorial-apriltag-detector-live.cpp, and tutorial-apriltag-detector.cpp.

Definition at line 274 of file vpDetectorAprilTag.h.

void vpDetectorAprilTag::setZAlignedWithCameraAxis ( bool  zAlignedWithCameraFrame)

Modify the resulting tag pose returned by getPose() in order to get a pose where z-axis is aligned when the camera plane is parallel to the tag.

Parameters
zAlignedWithCameraFrame: Flag to get a pose where z-axis is aligned with the camera frame.
Examples:
tutorial-apriltag-detector-live.cpp, and tutorial-apriltag-detector.cpp.

Definition at line 795 of file vpDetectorAprilTag.cpp.

Member Data Documentation

bool vpDetectorAprilTag::m_displayTag
protected

Definition at line 284 of file vpDetectorAprilTag.h.

Referenced by detect().

vpColor vpDetectorAprilTag::m_displayTagColor
protected

Definition at line 285 of file vpDetectorAprilTag.h.

Referenced by detect().

unsigned int vpDetectorAprilTag::m_displayTagThickness
protected

Definition at line 286 of file vpDetectorAprilTag.h.

Referenced by detect().

std::vector<std::string> vpDetectorBase::m_message
protectedinherited
size_t vpDetectorBase::m_nb_objects
protectedinherited
std::vector<std::vector<vpImagePoint> > vpDetectorBase::m_polygon
protectedinherited
vpPoseEstimationMethod vpDetectorAprilTag::m_poseEstimationMethod
protected

Definition at line 287 of file vpDetectorAprilTag.h.

Referenced by setAprilTagPoseEstimationMethod().

vpAprilTagFamily vpDetectorAprilTag::m_tagFamily
protected

Definition at line 288 of file vpDetectorAprilTag.h.