Visual Servoing Platform  version 3.2.0 under development (2019-01-22)
Practical example: count the number of coins in an image

Table of Contents

Introduction

This tutorial will show you how to count the number of coins in an image. To do this, we will reuse some of the image processing techniques presented before and some other image processing methods:

The two sample images to test the image processing pipeline are:

img-tutorial-count-coins-coins1.png
Sample image 1
img-tutorial-count-coins-coins2.png
Sample image 2

The second image can be downloaded here (by Atacanus (Own work) [Public domain], via Wikimedia Commons).

We assume similar lighting condition and coins placement in the image to build the image processing pipeline.

Example code

The example code also available in tutorial-count-coins.cpp is:

#include <cstdlib>
#include <iostream>
#include <visp3/core/vpImage.h>
#include <visp3/gui/vpDisplayGDI.h>
#include <visp3/gui/vpDisplayOpenCV.h>
#include <visp3/gui/vpDisplayX.h>
#include <visp3/io/vpImageIo.h>
#if defined(VISP_HAVE_MODULE_IMGPROC)
#include <visp3/core/vpMomentObject.h>
#include <visp3/imgproc/vpImgproc.h>
#endif
int main(int argc, char *argv[])
{
#if defined(VISP_HAVE_MODULE_IMGPROC) && (defined(VISP_HAVE_X11) || defined(VISP_HAVE_GDI) || defined(VISP_HAVE_OPENCV))
std::string input_filename = "coins1.pgm";
bool white_foreground = false;
for (int i = 1; i < argc; i++) {
if (std::string(argv[i]) == "--input" && i + 1 < argc) {
input_filename = std::string(argv[i + 1]);
} else if (std::string(argv[i]) == "--method" && i + 1 < argc) {
method = (vp::vpAutoThresholdMethod)atoi(argv[i + 1]);
} else if (std::string(argv[i]) == "--white_foreground") {
white_foreground = true;
} else if (std::string(argv[i]) == "--help" || std::string(argv[i]) == "-h") {
std::cout << "Usage: " << argv[0]
<< " [--input <input image>]"
" [--method <0: Huang, 1: Intermodes, 2: IsoData, 3: "
"Mean, 4: Otsu, 5: Triangle>]"
" [--white_foreground]"
" [--help]"
<< std::endl;
return EXIT_SUCCESS;
}
}
vpImageIo::read(I, input_filename);
#ifdef VISP_HAVE_X11
vpDisplayX d, d2, d3, d4, d5;
#elif defined(VISP_HAVE_GDI)
vpDisplayGDI d, d2, d3, d4, d5;
#elif defined(VISP_HAVE_OPENCV)
vpDisplayOpenCV d, d2, d3, d4, d5;
#endif
d.init(I, 0, 0, "Coins");
vpImage<unsigned char> I_bin, I_fill;
I_bin = I;
vp::autoThreshold(I_bin, method, white_foreground ? 0 : 255, white_foreground ? 255 : 0);
d2.init(I_bin, I.getWidth(), 0, "Binarisation");
I_fill = I_bin;
vp::fillHoles(I_fill);
d3.init(I_fill, 0, I.getHeight() + 80, "Fill holes");
vpImage<unsigned char> I_open = I_fill;
vpImage<unsigned char> I_close = I_open;
d4.init(I_close, I.getWidth(), I.getHeight() + 80, "Closing");
vpImage<unsigned char> I_contours(I_close.getHeight(), I_close.getWidth());
for (unsigned int cpt = 0; cpt < I_close.getSize(); cpt++)
I_contours.bitmap[cpt] = I_close.bitmap[cpt] ? 1 : 0;
vp::vpContour vp_contours;
std::vector<std::vector<vpImagePoint> > contours;
vp::findContours(I_contours, vp_contours, contours, vp::CONTOUR_RETR_EXTERNAL);
vpImage<vpRGBa> I_draw_contours(I_contours.getHeight(), I_contours.getWidth(), vpRGBa());
vp::drawContours(I_draw_contours, contours, vpColor::red);
d5.init(I_draw_contours, 0, 2 * I.getHeight() + 80, "Contours");
vpDisplay::display(I_draw_contours);
int nb_coins = 0;
for (size_t i = 0; i < contours.size(); i++) {
std::vector<vpPoint> vec_p;
for (size_t j = 0; j < contours[i].size(); j++) {
vpPoint pt;
pt.set_x(contours[i][j].get_u());
pt.set_y(contours[i][j].get_v());
vec_p.push_back(pt);
}
obj.fromVector(vec_p);
// sign(m00) depends of the contour orientation (clockwise or
// counter-clockwise) that's why we use fabs
if (std::fabs(obj.get(0, 0)) >= I.getSize() / 200) {
nb_coins++;
std::stringstream ss;
ss << "Coin " << nb_coins;
int centroid_x = (int)std::fabs(obj.get(1, 0) / obj.get(0, 0));
int centroid_y = (int)std::fabs(obj.get(0, 1) / obj.get(0, 0));
vpDisplay::displayText(I_draw_contours, centroid_y, centroid_x - 20, ss.str(), vpColor::red);
}
}
vpDisplay::displayText(I_draw_contours, 20, 20, "Click to quit.", vpColor::red);
vpDisplay::flush(I_close);
vpDisplay::flush(I_draw_contours);
vpDisplay::getClick(I_draw_contours);
return EXIT_SUCCESS;
#else
(void)argc;
(void)argv;
return 0;
#endif
}

To run the demo code for the sample image 1:

$ ./tutorial-count-coins

To run the demo code for the sample image 2:

$ ./tutorial-count-coins --input coins2.pgm --white_foreground

The functions we will use needs the following includes:

#include <visp3/core/vpMomentObject.h>
#include <visp3/imgproc/vpImgproc.h>

The image processing functions in the imgproc module are declared in the vp:: namespace.

The first thing to do is to read the image using:

vpImageIo::read(I, input_filename);

As we assume the coins are placed on an uniform background with a distinct color, we can use an automatic thresholding method to binarize the image:

I_bin = I;
vp::autoThreshold(I_bin, method, white_foreground ? 0 : 255, white_foreground ? 255 : 0);

We use an option to switch between dark or bright background. The coins will be represented with 255 values after the binarisation, as you can see in the following images:

img-tutorial-count-coins-binarisation1.png
Otsu's thresholding for the sample image 1
img-tutorial-count-coins-binarisation2.png
Otsu's thresholding for the sample image 2

You can notice some "holes" in the binarisation due to some shiny parts of the coin for the first case and to some dark parts of the coin in the second case.

We can now use a function to fill the holes in the binary images:

I_fill = I_bin;
vp::fillHoles(I_fill);

The fill holes algorithm is basic:

  • flood fill the binary image using a seed point just outside of the image
img-tutorial-count-coins-mask.png
Left: binary image, right: result of the flood fill operation
  • substract the flood fill image from a white image to get only the holes
img-tutorial-count-coins-white-holes.png
Top left: white image, bottom left: flood fill image, right: I_holes = I_white - I_flood_fill
  • add the holes image to the binary image to get an image without holes
img-tutorial-count-coins-fill-holes.png
Top left: binary image, bottom left: holes image, right: I_fill = I_bin + I_holes

Similarly for the sample image 1, the binary image with holes filled is:

img-tutorial-count-coins-fill1.png
Binary image with holes filled for the sample image 1

To "clean" the binary image, we will now perform some morphological operations:

The result image for the sample image 1 is (the morphological operations do not improve the binarisation for the sample 2):

img-tutorial-count-coins-close1.png
Binary image after an opening and a closing

Now that we have properly binarized the images to get only the coins, we can extract the contours. As the function expects a binary image with values 0/1, we need to create a new image accordingly:

vpImage<unsigned char> I_contours(I_close.getHeight(), I_close.getWidth());
for (unsigned int cpt = 0; cpt < I_close.getSize(); cpt++)
I_contours.bitmap[cpt] = I_close.bitmap[cpt] ? 1 : 0;
vp::vpContour vp_contours;
std::vector<std::vector<vpImagePoint> > contours;
vp::findContours(I_contours, vp_contours, contours, vp::CONTOUR_RETR_EXTERNAL);

To display the extracted contours, we can use:

vpImage<vpRGBa> I_draw_contours(I_contours.getHeight(), I_contours.getWidth(), vpRGBa());
vp::drawContours(I_draw_contours, contours, vpColor::red);

To count the number of coins, we use the number of extracted contours. But to be robust to some remaining bad binarized pixels, we will measure the area of the contour ( $ m_{00} $) and discard too small area contours. The image moments are used to compute the area of the contour and the centroid ( $ x_{centroid}=\frac{m_{10}}{m_{00}} $, $ y_{centroid}=\frac{m_{01}}{m_{00}} $) of the contour to display some texts:

int nb_coins = 0;
for (size_t i = 0; i < contours.size(); i++) {
std::vector<vpPoint> vec_p;
for (size_t j = 0; j < contours[i].size(); j++) {
vpPoint pt;
pt.set_x(contours[i][j].get_u());
pt.set_y(contours[i][j].get_v());
vec_p.push_back(pt);
}
obj.fromVector(vec_p);
// sign(m00) depends of the contour orientation (clockwise or
// counter-clockwise) that's why we use fabs
if (std::fabs(obj.get(0, 0)) >= I.getSize() / 200) {
nb_coins++;
std::stringstream ss;
ss << "Coin " << nb_coins;
int centroid_x = (int)std::fabs(obj.get(1, 0) / obj.get(0, 0));
int centroid_y = (int)std::fabs(obj.get(0, 1) / obj.get(0, 0));
vpDisplay::displayText(I_draw_contours, centroid_y, centroid_x - 20, ss.str(), vpColor::red);
}
}

The final result images are:

img-tutorial-count-coins-count-coins1.png
9 coins have been detected for the sample image 1
img-tutorial-count-coins-count-coins2.png
11 coins have been detected for the sample image 2

This tutorial showed you how some basic image processing techniques can be used to create an application to count the number of coins in an image. Some assumptions must be made to guarantee that the image processing pipeline will work:

  • the coins are placed on an uniform background with a different color (to be able to automatically threshold the image)
  • the coins must be isolated from each other (to be able to extract the contours)
  • the image must be clean (to avoid to use too much some morphological operations)
  • the size of the coins in the image is more or less defined (to be able to discard contours that are not coins using the contour area)