Camera parameters conversions

ViSP camera parameters are implemented in vpCameraParameters class. If you want to calibrate a camera with ViSP tools follow Tutorial: Camera intrinsic calibration.

Let us recall the pinhole camera model implemented in ViSP. In this model, a scene view is formed by projecting 3D points into the image plane using a perspective transformation.

$\left[ \begin{array}{c} u \\ v \\ 1 \end{array}\right] = \left[ \begin{array}{ccc} p_x & 0 & u_0 \\ 0 & p_y & v_0 \\ 0 & 0 & 1 \end{array}\right] \left[ \begin{array}{c} X_c \\ Y_c \\ Z_c \end{array}\right]$

where:

are the coordinates of a 3D point in the camera frame
are the coordinates in pixels of the projected 3D point
is a principal point that is usually near the image center
are the focal lengths expressed in pixel units.

When $Z_c \neq 0$ , the previous equation is equivalent to the following:

$\begin{array}{lcl} x &=& X_c / Z_c \\ y &=& Y_c / Z_c \\ u &=& u_0 + x \; p_x \\ v &=& v_0 + y \; p_y \end{array}$

Real lenses usually have some radial distortion. So, the above model is extended as:

$\begin{array}{lcl} x &=& X_c / Z_c \\ y &=& Y_c / Z_c \\ x^{'} &=& x (1 + k_{ud} r^2) \\ y^{'} &=& y (1 + k_{ud} r^2) \\ r^2 &=& x^2 + y^2 \\ u &=& u_0 + x^{'} \; p_x \\ v &=& v_0 + y^{'} \; p_y \end{array}$

where $k_{ud}$ is the first order radial distortion. Higher order distortion coefficients are not considered in ViSP.

Note: In ViSP we introduce an extra parameter named $k_{du}$ which is the radial first order distortion that allows to transform pixels in meters. If this parameter is unknown as in OpenCV a good approximation is to consider $k_{du} = - k_{ud}$

Even if OpenCV notations are different, this model is exactly the same then the one used in OpenCV and described here where higher order OpenCV distortion parameters are turned to 0.

The following table gives the correspondences between ViSP and OpenCV parameters:

$\begin{array}{l|l} ViSP & OpenCV \\ \hline u_0 & c_x\\ v_0 & c_y \\ p_x & f_x \\ p_y & f_y \\ k_{ud} & k_1 \\ k_{du} & -k_1 \\ & k_2 = 0 \\ & k_3 = 0 \\ & k_4 = 0 \\ & k_5 = 0 \\ & k_6 = 0 \\ & p_1 = 0 \\ & p_2 = 0 \end{array}$

From a coding point of view, let us consider the following code also available in tutorial-bridge-opencv-camera-param.cpp where we initialize camera parameters using ViSP:

  double u0 = 326.6;
  double v0 = 215.0;
  double px = 582.7;
  double py = 580.6;
  double kud = -0.3372;
  double kdu = 0.4021;
  vpCameraParameters cam(px, py, u0, v0, kud, kdu);

These parameters could be used to initialize OpenCV camera parameters:

cv::Mat K = (cv::Mat_<double>(3, 3) << cam.get_px(), 0, cam.get_u0(), 0, cam.get_py(), cam.get_v0(), 0, 0, 1);

cv::Mat D = (cv::Mat_<double>(4, 1) << cam.get_kud(), 0, 0, 0);

Image conversions

ViSP image is implemented in vpImage class, while OpenCV images in cv::Mat class. All the functions that allow image conversion from ViSP to OpenCV or vice versa are implemented in vpImageConvert.

Some image conversion examples could be found in tutorial-bridge-opencv-camera-param.cpp. There is also tutorial-bridge-opencv-image.cpp from where the following snippets are extracted.

From ViSP to OpenCV image conversion

Color image conversion

The following code allows to read a color image with ViSP:
vpImage<vpRGBa> Irgba;

vpImageIo::read(Irgba, "monkey.jpeg");

vpImageIo::read
static void read(vpImage< unsigned char > &I, const std::string &filename, int backend=IO_DEFAULT_BACKEND)
Definition: vpImageIo.cpp:147

vpImage< vpRGBa >

and then convert the image in OpenCV using:
cv::Mat cv_img_color;

vpImageConvert::convert(Irgba, cv_img_color);

vpImageConvert::convert
static void convert(const vpImage< unsigned char > &src, vpImage< vpRGBa > &dest)
Definition: vpImageConvert.cpp:73

before saving the converted color image
cv::imwrite("monkey-cv-color.jpeg", cv_img_color);
Grey scale image conversion

The following code allows to read a grey scale image with ViSP. Note here that if the input image (ie. monkey.jpeg) is a color image, vpImageIo::read() converts implicitly the color image in a grey scale image:
vpImage<unsigned char> Igrey;

vpImageIo::read(Igrey, "monkey.jpeg");

and then convert the image in OpenCV using:
cv::Mat cv_img_grey;

vpImageConvert::convert(Igrey, cv_img_grey);

before saving the converted grey image
cv::imwrite("monkey-cv-grey.jpeg", cv_img_grey);

From OpenCV to ViSP image conversion

Color image conversion

The following code allows to read a color image with OpenCV:
#if VISP_HAVE_OPENCV_VERSION >= 0x030000

cv::Mat cv_img_color = cv::imread("monkey.jpeg", cv::IMREAD_COLOR);

#else

cv::Mat cv_img_color = cv::imread("monkey.jpeg", CV_LOAD_IMAGE_COLOR);

#endif

and then convert the image in ViSP using:
vpImage<vpRGBa> Irgba;

vpImageConvert::convert(cv_img_color, Irgba);

before saving the converted color image
vpImageIo::write(Irgba, "monkey-vp-color.jpeg");

vpImageIo::write
static void write(const vpImage< unsigned char > &I, const std::string &filename, int backend=IO_DEFAULT_BACKEND)
Definition: vpImageIo.cpp:291
Grey scale image conversion

The following code allows to read a grey scale image with OpenCV:
#if VISP_HAVE_OPENCV_VERSION >= 0x030000

cv::Mat cv_img_grey = cv::imread("monkey.jpeg", cv::IMREAD_GRAYSCALE);

#else

cv::Mat cv_img_grey = cv::imread("monkey.jpeg", CV_LOAD_IMAGE_GRAYSCALE);

#endif

and then convert the image in ViSP using:
vpImage<unsigned char> Igrey;

vpImageConvert::convert(cv_img_grey, Igrey);

before saving the converted grey image
vpImageIo::write(Igrey, "monkey-vp-grey.jpeg");

Matrix conversions

ViSP matrices are implemented in different classes:

vpMatrix for a non specific matrix
vpHomogeneousMatrix for a 4-by-4 homogeneous matrix
vpHomography for a 3-by-3 homography matrix
vpRotationMatrix for a 3-by-3 rotation matrix

while OpenCV matrices are always implemented as a cv::Mat.

Up to now, there is no specific function that does the conversion between ViSP and OpenCv matrices.

Note that each element of a ViSP matrix is a double and that ViSP matrices are row-major.

In the following subsections we illustrate how to convert a vpMatrix to a cv::Mat, but the code could be easily adapted to other ViSP specific matrices as long as the corresponding OpenCV matrix contains double elements and that it has the same size as the ViSP specific matrix.

From OpenCV to ViSP matrix conversion

The following code also available in tutorial-bridge-opencv-matrix.cpp shows how to create an OpenCV row-major matrix with size 3 rows by 4 cols that contains double elements.
cv::Mat M_cv = (cv::Mat_<double>(3, 4) << 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12);

std::cout << "M_cv: \n" << M_cv << std::endl;
Deep copy conversion

To convert the previous matrix in a vpMatrix using a deep copy, you may use:
vpMatrix M(static_cast<unsigned int>(M_cv.rows), static_cast<unsigned int>(M_cv.cols));

memcpy(M.data, M_cv.data, sizeof(double) * static_cast<size_t>(M_cv.rows * M_cv.cols));

std::cout << "M: \n" << M << std::endl;

vpMatrix
Implementation of a matrix and operations on matrices.
Definition: vpMatrix.h:169

The content of the matrices is now the following:
```
M_cv: 
[1, 2, 3, 4;
 5, 6, 7, 8;
 9, 10, 11, 12]
M: 
1  2  3  4
5  6  7  8
9  10  11  12
```

From ViSP to OpenCV conversion

The following code also available in tutorial-bridge-opencv-matrix.cpp shows how to create a ViSP row-major matrix with size 3 rows by 4 cols that contains double elements.
vpMatrix M(3, 4, { 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12 });

std::cout << "M: \n" << M << std::endl;
Deep copy conversion

To convert the previous matrix in a cv::Mat using a deep copy, you may use:
cv::Mat tmp(static_cast<int>(M.getRows()), static_cast<int>(M.getCols()), CV_64F, static_cast<void *>(M.data));

cv::Mat M_cv_deep = tmp.clone();

std::cout << "M_cv_deep: \n" << M_cv_deep << std::endl;

The content of the matrices is now the following:
```
M: 
1  2  3  4
5  6  7  8
9  10  11  12
M_cv_deep: 
[1, 2, 3, 4;
 5, 6, 7, 8;
 9, 10, 11, 12]
```
Pointer assignment, without deep copy conversion

For performance issues, there is also the possibility to just copy the data pointer. To this end, you may use:
cv::Mat M_cv(static_cast<int>(M.getRows()), static_cast<int>(M.getCols()), CV_64F, static_cast<void *>(M.data));

std::cout << "M_cv: \n" << M_cv << std::endl;

The content of the matrices is now the following:
```
M: 
1  2  3  4
5  6  7  8
9  10  11  12
M_cv: 
[1, 2, 3, 4;
 5, 6, 7, 8;
 9, 10, 11, 12]
```
As a side effect, you may know that modifying one of the matrices (ie. M like below or M_cv) will affect the content of both matrices. Let us illustrate this behavior, where the diagonal of M is set to 1:
std::cout << "Set M = eye" << std::endl;

M.eye();

std::cout << "M: \n" << M << std::endl;

std::cout << "M_cv: \n" << M_cv << std::endl;

As expected, the content of M_cv matrix is also changed:
```
Set M = eye
M: 
1  0  0  0
0  1  0  0
0  0  1  0
M_cv: 
[1, 0, 0, 0;
 0, 1, 0, 0;
 0, 0, 1, 0]
```

Table of Contents