Visual Servoing Platform  version 3.3.0 under development (2020-02-17)
Tutorial: Bridge over OpenCV

Table of Contents

ViSP is interfaced with OpenCV third party. In this tutorial we explain how to convert data such as camera parameters or images from ViSP to OpenCV or vice versa.

Camera parameters conversions

ViSP camera parameters are implemented in vpCameraParameters class. If you want to calibrate a camera with ViSP tools follow Tutorial: Camera intrinsic calibration.

Let us recall the pinhole camera model implemented in ViSP. In this model, a scene view is formed by projecting 3D points into the image plane using a perspective transformation.

\[ \left[ \begin{array}{c} u \\ v \\ 1 \end{array}\right] = \left[ \begin{array}{ccc} p_x & 0 & u_0 \\ 0 & p_y & v_0 \\ 0 & 0 & 1 \end{array}\right] \left[ \begin{array}{c} X_c \\ Y_c \\ Z_c \end{array}\right] \]

where:

  • $(X_c,Y_c,Z_c)$ are the coordinates of a 3D point in the camera frame
  • $(u,v)$ are the coordinates in pixels of the projected 3D point
  • $(u_0,v_0)$ is a principal point that is usually near the image center
  • $(p_x,p_y)$ are the focal lengths expressed in pixel units.

When $Z_c \neq 0$, the previous equation si equivalent to the following:

\[ \begin{array}{lcl} x &=& X_c / Z_c \\ y &=& Y_c / Z_c \\ u &=& u_0 + x \; p_x \\ v &=& v_0 + y \; p_y \end{array} \]

Real lenses usually have some radial distortion. So, the above model is extended as:

\[ \begin{array}{lcl} x &=& X_c / Z_c \\ y &=& Y_c / Z_c \\ x^{'} &=& x (1 + k_{ud} r^2) \\ y^{'} &=& y (1 + k_{ud} r^2) \\ r^2 &=& x^2 + y^2 \\ u &=& u_0 + x^{'} \; p_x \\ v &=& v_0 + y^{'} \; p_y \end{array} \]

where $k_{ud}$ is the first order radial distorsion. Higher order distorsion coefficients are not considered in ViSP.

Note
In ViSP we introduce an extra parameter named $k_{du}$ which is the radial first order distorsion that allows to transform pixels in meters. If this parameter is unknown as in OpenCV a good approximation is to consider $k_{du} = - k_{ud}$

Even if OpenCV notations are different, this model is exactly the same then the one used in OpenCV and described here where higher order OpenCV distorsion parameters are turned to 0.

The following table gives the correspondances between ViSP and OpenCV parameters:

\[ \begin{array}{l|l} ViSP & OpenCV \\ \hline u_0 & c_x\\ v_0 & c_y \\ p_x & f_x \\ p_y & f_y \\ k_{ud} & k_1 \\ k_{du} & -k_1 \\ & k_2 = 0 \\ & k_3 = 0 \\ & k_4 = 0 \\ & k_5 = 0 \\ & k_6 = 0 \\ & p_1 = 0 \\ & p_2 = 0 \end{array} \]

From a coding point of view, let us consider the following code also available in tutorial-bridge-opencv.cpp where we initialize camera parameters using ViSP:

double u0 = 326.6;
double v0 = 215.0;
double px = 582.7;
double py = 580.6;
double kud = -0.3372;
double kdu = 0.4021;
vpCameraParameters cam(px, py, u0, v0, kud, kdu);

These parameters could be used to initialize OpenCV camera parameters:

cv::Mat K = (cv::Mat_<double>(3, 3) << cam.get_px(), 0, cam.get_u0(), 0, cam.get_py(), cam.get_v0(), 0, 0, 1);
cv::Mat D = (cv::Mat_<double>(4, 1) << cam.get_kud(), 0, 0, 0);

Image conversions

ViSP image is implemented in vpImage class, while OpenCV images in cv::Mat class. All the functions that allow image conversion from ViSP to OpenCV or vice versa are implemented in vpImageConvert.

Some examples are available in tutorial-bridge-opencv.cpp.

For example, the following code allow to read an image with ViSP:

vpImage<unsigned char> I;
vpImageIo::read(I, "chessboard.pgm");

and then convert the image in OpenCV using:

cv::Mat image;
vpImageConvert::convert(I, image);