Visual Servoing Platform  version 3.5.1 under development (2022-10-03)
Practical example: count the number of coins in an image

Table of Contents

Introduction

This tutorial will show you how to count the number of coins in an image. To do this, we will reuse some of the image processing techniques presented before and some other image processing methods:

The two sample images to test the image processing pipeline are:

Sample image 1
Sample image 2

The second image can be downloaded here (by Atacanus (Own work) [Public domain], via Wikimedia Commons).

We assume similar lighting condition and coins placement in the image to build the image processing pipeline.

Example code

The example code also available in tutorial-count-coins.cpp is:

#include <cstdlib>
#include <iostream>
#include <visp3/core/vpImage.h>
#include <visp3/gui/vpDisplayGDI.h>
#include <visp3/gui/vpDisplayOpenCV.h>
#include <visp3/gui/vpDisplayX.h>
#include <visp3/io/vpImageIo.h>
#if defined(VISP_HAVE_MODULE_IMGPROC)
#include <visp3/core/vpMomentObject.h>
#include <visp3/imgproc/vpImgproc.h>
#endif
int main(int argc, char *argv[])
{
#if defined(VISP_HAVE_MODULE_IMGPROC) && (defined(VISP_HAVE_X11) || defined(VISP_HAVE_GDI) || defined(VISP_HAVE_OPENCV))
std::string input_filename = "coins1.pgm";
bool white_foreground = false;
for (int i = 1; i < argc; i++) {
if (std::string(argv[i]) == "--input" && i + 1 < argc) {
input_filename = std::string(argv[i + 1]);
} else if (std::string(argv[i]) == "--method" && i + 1 < argc) {
method = (vp::vpAutoThresholdMethod)atoi(argv[i + 1]);
} else if (std::string(argv[i]) == "--white_foreground") {
white_foreground = true;
} else if (std::string(argv[i]) == "--help" || std::string(argv[i]) == "-h") {
std::cout << "Usage: " << argv[0]
<< " [--input <input image>]"
" [--method <0: Huang, 1: Intermodes, 2: IsoData, 3: "
"Mean, 4: Otsu, 5: Triangle>]"
" [--white_foreground]"
" [--help]"
<< std::endl;
return EXIT_SUCCESS;
}
}
vpImageIo::read(I, input_filename);
#ifdef VISP_HAVE_X11
vpDisplayX d, d2, d3, d4, d5;
#elif defined(VISP_HAVE_GDI)
vpDisplayGDI d, d2, d3, d4, d5;
#elif defined(VISP_HAVE_OPENCV)
vpDisplayOpenCV d, d2, d3, d4, d5;
#endif
d.init(I, 0, 0, "Coins");
vpImage<unsigned char> I_bin, I_fill;
I_bin = I;
vp::autoThreshold(I_bin, method, white_foreground ? 0 : 255, white_foreground ? 255 : 0);
d2.init(I_bin, I.getWidth(), 0, "Binarisation");
I_fill = I_bin;
vp::fillHoles(I_fill);
d3.init(I_fill, 0, I.getHeight() + 80, "Fill holes");
vpImage<unsigned char> I_open = I_fill;
vpImage<unsigned char> I_close = I_open;
d4.init(I_close, I.getWidth(), I.getHeight() + 80, "Closing");
vpImage<unsigned char> I_contours(I_close.getHeight(), I_close.getWidth());
for (unsigned int cpt = 0; cpt < I_close.getSize(); cpt++)
I_contours.bitmap[cpt] = I_close.bitmap[cpt] ? 1 : 0;
vp::vpContour vp_contours;
std::vector<std::vector<vpImagePoint> > contours;
vp::findContours(I_contours, vp_contours, contours, vp::CONTOUR_RETR_EXTERNAL);
vpImage<vpRGBa> I_draw_contours(I_contours.getHeight(), I_contours.getWidth(), vpRGBa());
vp::drawContours(I_draw_contours, contours, vpColor::red);
d5.init(I_draw_contours, 0, 2 * I.getHeight() + 80, "Contours");
vpDisplay::display(I_draw_contours);
int nb_coins = 0;
for (size_t i = 0; i < contours.size(); i++) {
std::vector<vpPoint> vec_p;
for (size_t j = 0; j < contours[i].size(); j++) {
vpPoint pt;
pt.set_x(contours[i][j].get_u());
pt.set_y(contours[i][j].get_v());
vec_p.push_back(pt);
}
obj.fromVector(vec_p);
// sign(m00) depends of the contour orientation (clockwise or
// counter-clockwise) that's why we use fabs
if (std::fabs(obj.get(0, 0)) >= I.getSize() / 200) {
nb_coins++;
std::stringstream ss;
ss << "Coin " << nb_coins;
int centroid_x = (int)std::fabs(obj.get(1, 0) / obj.get(0, 0));
int centroid_y = (int)std::fabs(obj.get(0, 1) / obj.get(0, 0));
vpDisplay::displayText(I_draw_contours, centroid_y, centroid_x - 20, ss.str(), vpColor::red);
}
}
vpDisplay::displayText(I_draw_contours, 20, 20, "Click to quit.", vpColor::red);
vpDisplay::flush(I_close);
vpDisplay::flush(I_draw_contours);
vpDisplay::getClick(I_draw_contours);
return EXIT_SUCCESS;
#else
(void)argc;
(void)argv;
return 0;
#endif
}
static const vpColor red
Definition: vpColor.h:217
Display for windows using GDI (available on any windows 32 platform).
Definition: vpDisplayGDI.h:129
The vpDisplayOpenCV allows to display image using the OpenCV library. Thus to enable this class OpenC...
Use the X11 console to display images on unix-like OS. Thus to enable this class X11 should be instal...
Definition: vpDisplayX.h:135
void init(vpImage< unsigned char > &I, int win_x=-1, int win_y=-1, const std::string &win_title="")
static bool getClick(const vpImage< unsigned char > &I, bool blocking=true)
static void display(const vpImage< unsigned char > &I)
static void flush(const vpImage< unsigned char > &I)
static void displayText(const vpImage< unsigned char > &I, const vpImagePoint &ip, const std::string &s, const vpColor &color)
static void read(vpImage< unsigned char > &I, const std::string &filename, int backend=IO_DEFAULT_BACKEND)
Definition: vpImageIo.cpp:148
static void dilatation(vpImage< Type > &I, Type value, Type value_out, vpConnexityType connexity=CONNEXITY_4)
static void erosion(vpImage< Type > &I, Type value, Type value_out, vpConnexityType connexity=CONNEXITY_4)
unsigned int getWidth() const
Definition: vpImage.h:246
unsigned int getSize() const
Definition: vpImage.h:227
Type * bitmap
points toward the bitmap
Definition: vpImage.h:143
unsigned int getHeight() const
Definition: vpImage.h:188
Class for generic objects.
Class that defines a 3D point in the object frame and allows forward projection of a 3D point in the ...
Definition: vpPoint.h:82
void set_x(double x)
Set the point x coordinate in the image plane.
Definition: vpPoint.cpp:511
void set_y(double y)
Set the point y coordinate in the image plane.
Definition: vpPoint.cpp:513
Definition: vpRGBa.h:67
VISP_EXPORT void findContours(const vpImage< unsigned char > &I_original, vpContour &contours, std::vector< std::vector< vpImagePoint > > &contourPts, const vpContourRetrievalType &retrievalMode=vp::CONTOUR_RETR_TREE)
Definition: vpContours.cpp:300
VISP_EXPORT void drawContours(vpImage< unsigned char > &I, const std::vector< std::vector< vpImagePoint > > &contours, unsigned char grayValue=255)
Definition: vpContours.cpp:250
VISP_EXPORT void fillHoles(vpImage< unsigned char > &I)
Definition: vpMorph.cpp:54
VISP_EXPORT unsigned char autoThreshold(vpImage< unsigned char > &I, const vp::vpAutoThresholdMethod &method, const unsigned char backgroundValue=0, const unsigned char foregroundValue=255)
@ CONTOUR_RETR_EXTERNAL
Definition: vpContours.h:168
vpAutoThresholdMethod
Definition: vpImgproc.h:58
@ AUTO_THRESHOLD_OTSU
Definition: vpImgproc.h:75

To run the demo code for the sample image 1:

$ ./tutorial-count-coins

To run the demo code for the sample image 2:

$ ./tutorial-count-coins --input coins2.pgm --white_foreground

The functions we will use needs the following includes:

#include <visp3/core/vpMomentObject.h>
#include <visp3/imgproc/vpImgproc.h>

The image processing functions in the imgproc module are declared in the vp:: namespace.

The first thing to do is to read the image using:

vpImageIo::read(I, input_filename);

As we assume the coins are placed on an uniform background with a distinct color, we can use an automatic thresholding method to binarize the image:

I_bin = I;
vp::autoThreshold(I_bin, method, white_foreground ? 0 : 255, white_foreground ? 255 : 0);

We use an option to switch between dark or bright background. The coins will be represented with 255 values after the binarisation, as you can see in the following images:

Otsu's thresholding for the sample image 1
Otsu's thresholding for the sample image 2

You can notice some "holes" in the binarisation due to some shiny parts of the coin for the first case and to some dark parts of the coin in the second case.

We can now use a function to fill the holes in the binary images:

I_fill = I_bin;
vp::fillHoles(I_fill);

The fill holes algorithm is basic:

  • flood fill the binary image using a seed point just outside of the image
Left: binary image, right: result of the flood fill operation
  • substract the flood fill image from a white image to get only the holes
Top left: white image, bottom left: flood fill image, right: I_holes = I_white - I_flood_fill
  • add the holes image to the binary image to get an image without holes
Top left: binary image, bottom left: holes image, right: I_fill = I_bin + I_holes

Similarly for the sample image 1, the binary image with holes filled is:

Binary image with holes filled for the sample image 1

To "clean" the binary image, we will now perform some morphological operations:

The result image for the sample image 1 is (the morphological operations do not improve the binarisation for the sample 2):

Binary image after an opening and a closing

Now that we have properly binarized the images to get only the coins, we can extract the contours. As the function expects a binary image with values 0/1, we need to create a new image accordingly:

vpImage<unsigned char> I_contours(I_close.getHeight(), I_close.getWidth());
for (unsigned int cpt = 0; cpt < I_close.getSize(); cpt++)
I_contours.bitmap[cpt] = I_close.bitmap[cpt] ? 1 : 0;
vp::vpContour vp_contours;
std::vector<std::vector<vpImagePoint> > contours;
vp::findContours(I_contours, vp_contours, contours, vp::CONTOUR_RETR_EXTERNAL);

To display the extracted contours, we can use:

vpImage<vpRGBa> I_draw_contours(I_contours.getHeight(), I_contours.getWidth(), vpRGBa());
vp::drawContours(I_draw_contours, contours, vpColor::red);

To count the number of coins, we use the number of extracted contours. But to be robust to some remaining bad binarized pixels, we will measure the area of the contour ( $ m_{00} $) and discard too small area contours. The image moments are used to compute the area of the contour and the centroid ( $ x_{centroid}=\frac{m_{10}}{m_{00}} $, $ y_{centroid}=\frac{m_{01}}{m_{00}} $) of the contour to display some texts:

int nb_coins = 0;
for (size_t i = 0; i < contours.size(); i++) {
std::vector<vpPoint> vec_p;
for (size_t j = 0; j < contours[i].size(); j++) {
vpPoint pt;
pt.set_x(contours[i][j].get_u());
pt.set_y(contours[i][j].get_v());
vec_p.push_back(pt);
}
obj.fromVector(vec_p);
// sign(m00) depends of the contour orientation (clockwise or
// counter-clockwise) that's why we use fabs
if (std::fabs(obj.get(0, 0)) >= I.getSize() / 200) {
nb_coins++;
std::stringstream ss;
ss << "Coin " << nb_coins;
int centroid_x = (int)std::fabs(obj.get(1, 0) / obj.get(0, 0));
int centroid_y = (int)std::fabs(obj.get(0, 1) / obj.get(0, 0));
vpDisplay::displayText(I_draw_contours, centroid_y, centroid_x - 20, ss.str(), vpColor::red);
}
}

The final result images are:

9 coins have been detected for the sample image 1
11 coins have been detected for the sample image 2

This tutorial showed you how some basic image processing techniques can be used to create an application to count the number of coins in an image. Some assumptions must be made to guarantee that the image processing pipeline will work:

  • the coins are placed on an uniform background with a different color (to be able to automatically threshold the image)
  • the coins must be isolated from each other (to be able to extract the contours)
  • the image must be clean (to avoid to use too much some morphological operations)
  • the size of the coins in the image is more or less defined (to be able to discard contours that are not coins using the contour area)