Tutorial: Image frame grabbing

Table of Contents

• Introduction
• Frame grabbing using FlyCapture SDK
  • Source code explained
  • How to acquire images
• Frame grabbing using libdc1394 SDK
  • Source code explained
  • How to acquire images
• Frame grabbing using libv4l2 SDK
  • How to acquire images
• Frame grabbing using Pylon SDK
  • How to acquire images
• Frame grabbing using Realsense SDK
  • How to acquire images
  • How to acquire images from 2 Realsense devices
• Frame grabbing using Occipital Structure SDK
  • How to acquire images
• RGBD frame grabbing from RealSense D435 and Structure Core
• Frame grabbing using OpenCV
  • How to acquire images
• Frame grabbing using CMU1394 SDK
• Frame grabbing using Parrot Bebop 2 drone
  • How to acquire images
• Images from a video stream
  • Source code explained
  • How to acquire images
• Next tutorial

Introduction

In this tutorial you will learn how to grab images with ViSP, either from cameras or from a video stream.

Grabbing images from a real camera is only possible if you have installed the corresponding 3rd party. The complete list of 3rd parties supported by ViSP and dedicated to framegrabbing is given here. From this page you will also found useful information to install these 3rd parties.

Note that all the material (source code and videos) described in this tutorial is part of ViSP source code (in tutorial/grabber folder) and could be found in https://github.com/lagadic/visp/tree/master/tutorial/grabber.

Frame grabbing using FlyCapture SDK

After ViSP 3.0.0, we introduce vpFlyCaptureGrabber class, a wrapper over PointGrey FlyCapture SDK that allows to grab images from any PointGrey camera. This grabber was tested under Ubuntu and Windows with the following cameras:

• Flea3 USB 3.0 cameras (FL3-U3-32S2M-CS, FL3-U3-13E4C-C)
• Flea2 firewire camera (FL2-03S2C)
• Dragonfly2 firewire camera (DR2-COL)

It should also work with GigE PGR cameras.

The following example also available in tutorial-grabber-flycapture.cpp shows how to use vpFlyCaptureGrabber to capture grey level images from a PointGrey camera under Ubuntu or Windows. The following example suppose that a window renderer (libX11 on Ubuntu or GDI on Windows) and FlyCapture SDK 3rd party are available throw VISP.

Source code explained

#include <visp3/core/vpConfig.h>
#include <visp3/core/vpImage.h>
#include <visp3/gui/vpDisplayGDI.h>
#include <visp3/gui/vpDisplayOpenCV.h>
#include <visp3/gui/vpDisplayX.h>
#include <visp3/io/vpImageStorageWorker.h>
#include <visp3/sensor/vpFlyCaptureGrabber.h>
 
void usage(const char *argv[], int error)
{
  std::cout << "SYNOPSIS" << std::endl
    << "  " << argv[0] << " [--change-settings]"
    << " [--seqname <sequence name>]"
    << " [--record <mode>]"
    << " [--no-display]"
    << " [--help] [-h]" << std::endl
    << std::endl;
  std::cout << "DESCRIPTION" << std::endl
    << "  --change-settings" << std::endl
    << "    Force camera settings to auto shutter, auto gain and 640x480 MONO8 image" << std::endl
    << "    acquisition at 60 fps." << std::endl
    << std::endl
    << "  --seqname <sequence name>" << std::endl
    << "    Name of the sequence of image to create (ie: /tmp/image%04d.jpg)." << std::endl
    << "    Default: empty." << std::endl
    << std::endl
    << "  --record <mode>" << std::endl
    << "    Allowed values for mode are:" << std::endl
    << "      0: record all the captures images (continuous mode)," << std::endl
    << "      1: record only images selected by a user click (single shot mode)." << std::endl
    << "    Default mode: 0" << std::endl
    << std::endl
    << "  --no-display" << std::endl
    << "    Disable displaying captured images." << std::endl
    << "    When used and sequence name specified, record mode is internally set to 1 (continuous mode)."
    << std::endl
    << std::endl
    << "  --help, -h" << std::endl
    << "    Print this helper message." << std::endl
    << std::endl;
  std::cout << "USAGE" << std::endl
    << "  Example to visualize images:" << std::endl
    << "    " << argv[0] << std::endl
    << std::endl
    << "  Examples to record a sequence:" << std::endl
    << "    " << argv[0] << " --seqname I%04d.png" << std::endl
    << "    " << argv[0] << " --seqname folder/I%04d.png --record 0" << std::endl
    << std::endl
    << "  Examples to record single shot images:\n"
    << "    " << argv[0] << " --seqname I%04d.png --record 1\n"
    << "    " << argv[0] << " --seqname folder/I%04d.png --record 1" << std::endl
    << std::endl;
 
  if (error) {
    std::cout << "Error" << std::endl
      << "  "
      << "Unsupported parameter " << argv[error] << std::endl;
  }
}
 
int main(int argc, const char *argv[])
{
#if defined(VISP_HAVE_FLYCAPTURE) && defined(VISP_HAVE_THREADS)
#ifdef ENABLE_VISP_NAMESPACE
  using namespace VISP_NAMESPACE_NAME;
#endif
  try {
    std::string opt_seqname;
    int opt_record_mode = 0;
    bool opt_change_settings = false;
    bool opt_display = true;
 
    for (int i = 1; i < argc; i++) {
      if (std::string(argv[i]) == "--change-settings") {
        opt_change_settings = true;
      }
      else if (std::string(argv[i]) == "--seqname") {
        opt_seqname = std::string(argv[i + 1]);
        i++;
      }
      else if (std::string(argv[i]) == "--record") {
        opt_record_mode = std::atoi(argv[i + 1]);
        i++;
      }
      else if (std::string(argv[i]) == "--no-display") {
        opt_display = false;
      }
      else if (std::string(argv[i]) == "--help" || std::string(argv[i]) == "-h") {
        usage(argv, 0);
        return EXIT_SUCCESS;
      }
      else {
        usage(argv, i);
        return EXIT_FAILURE;
      }
    }
 
    if ((!opt_display) && (!opt_seqname.empty())) {
      opt_record_mode = 0;
    }
 
    std::cout << "Settings   : " << (opt_change_settings ? "modified" : "current") << std::endl;
    std::cout << "Recording  : " << (opt_seqname.empty() ? "disabled" : "enabled") << std::endl;
    std::cout << "Display    : " << (opt_display ? "enabled" : "disabled") << std::endl;
 
    std::string text_record_mode =
      std::string("Record mode: ") + (opt_record_mode ? std::string("single") : std::string("continuous"));
 
    if (!opt_seqname.empty()) {
      std::cout << text_record_mode << std::endl;
      std::cout << "Record name: " << opt_seqname << std::endl;
    }
 
    vpImage<vpRGBa> I; // Create a gray level image container
    vpFlyCaptureGrabber g; // Create a grabber based on FlyCapture SDK third party lib
 
    if (opt_change_settings) {
      try {
        g.setShutter(true); // Turn auto shutter on
        g.setGain(true);    // Turn auto gain on
        g.setVideoModeAndFrameRate(FlyCapture2::VIDEOMODE_640x480Y8, FlyCapture2::FRAMERATE_60);
      }
      catch (...) { // If settings are not available just catch execption to
     // continue with default settings
        std::cout << "Warning: cannot modify camera settings" << std::endl;
      }
    }
    g.open(I);
 
    std::cout << "Image size : " << I.getWidth() << " " << I.getHeight() << std::endl;
 
    vpDisplay *d = nullptr;
    if (opt_display) {
#if !(defined(VISP_HAVE_X11) || defined(VISP_HAVE_GDI) || defined(VISP_HAVE_OPENCV))
      std::cout << "No image viewer is available..." << std::endl;
      opt_display = false;
#endif
    }
    if (opt_display) {
#ifdef VISP_HAVE_X11
      d = new vpDisplayX(I);
#elif defined(VISP_HAVE_GDI)
      d = new vpDisplayGDI(I);
#elif defined(HAVE_OPENCV_HIGHGUI)
      d = new vpDisplayOpenCV(I);
#endif
    }
 
    vpImageQueue<vpRGBa> image_queue(opt_seqname, opt_record_mode);
    vpImageStorageWorker<vpRGBa> image_storage_worker(std::ref(image_queue));
    std::thread image_storage_thread(&vpImageStorageWorker<vpRGBa>::run, &image_storage_worker);
 
    bool quit = false;
    while (!quit) {
      double t = vpTime::measureTimeMs();
      g.acquire(I);
      vpDisplay::display(I);
      quit = image_queue.record(I);
      std::stringstream ss;
      ss << "Acquisition time: " << std::setprecision(3) << vpTime::measureTimeMs() - t << " ms";
      vpDisplay::displayText(I, I.getHeight() - 20, 10, ss.str(), vpColor::red);
      vpDisplay::flush(I);
    }
    image_queue.cancel();
    image_storage_thread.join();
 
    if (d) {
      delete d;
    }
  }
  catch (const vpException &e) {
    std::cout << "Catch an exception: " << e.getStringMessage() << std::endl;
  }
#else
  (void)argc;
  (void)argv;
#ifndef VISP_HAVE_FLYCAPTURE
  std::cout << "Install Flycapture SDK, configure and build ViSP again to use this example" << std::endl;
#endif
#if (VISP_CXX_STANDARD < VISP_CXX_STANDARD_11)
  std::cout << "This tutorial should be built with c++11 support" << std::endl;
#endif
#endif
}

Here after we explain the source code.

First an instance of the frame grabber is created.

    vpFlyCaptureGrabber g; // Create a grabber based on FlyCapture SDK third party lib

Once the grabber is created, we turn auto shutter and auto gain on and set the camera image size, color coding, and framerate. These settings are enclosed in a try/catch to be able to continue if one of these settings are not supported by the camera.

    if (opt_change_settings) {
      try {
        g.setShutter(true); // Turn auto shutter on
        g.setGain(true);    // Turn auto gain on
        g.setVideoModeAndFrameRate(FlyCapture2::VIDEOMODE_640x480Y8, FlyCapture2::FRAMERATE_60);
      }
      catch (...) { // If settings are not available just catch execption to
     // continue with default settings
        std::cout << "Warning: cannot modify camera settings" << std::endl;
      }
    }

Then the grabber is initialized using:

    g.open(I);

From now the grey level image I is also initialized with the size corresponding to the grabber settings.

Then we enter in a while loop where image acquisition is simply done by:

      g.acquire(I);

This image is then displayed using libX11 or GDI renderer:

      vpDisplay::display(I);

Depending on the command line options we are recording a sequence of images, or single shot images. We are also waiting for a non blocking mouse event to quit the while loop.

      quit = image_queue.record(I);

How to acquire images

Once tutorial-grabber-flycapture.cpp is built, the binary usage is given by:

./tutorial-grabber-flycapture --help

To just view images, run:

./tutorial-grabber-flycapture

To grab single shot images (for camera calibration for example), you may use:

./tutorial-grabber-flycapture --seqname I%04d.pgm --record 1

To grab a sequence of images, you may rather use:

./tutorial-grabber-flycapture --seqname I%04d.pgm --record 0

Frame grabbing using libdc1394 SDK

The next example also available in tutorial-grabber-1394.cpp shows how to use a framegrabber to acquire color images from a firewire or USB3 camera under Unix. The following example suppose that libX11 and libdc1394-2 3rd party are available.

Source code explained

The source code is the following:

#include <visp3/core/vpConfig.h>
#include <visp3/core/vpImage.h>
#include <visp3/gui/vpDisplayOpenCV.h>
#include <visp3/gui/vpDisplayX.h>
#include <visp3/io/vpImageStorageWorker.h>
#include <visp3/sensor/vp1394TwoGrabber.h>
 
void usage(const char *argv[], int error)
{
  std::cout << "SYNOPSIS" << std::endl
    << "  " << argv[0] << " [--change-settings]"
    << " [--seqname <sequence name>]"
    << " [--record <mode>]"
    << " [--no-display]"
    << " [--help] [-h]" << std::endl
    << std::endl;
  std::cout << "DESCRIPTION" << std::endl
    << "  --change-settings" << std::endl
    << "    Force camera settings to acquire 640x480 images in M0NO8 at 60 fps." << std::endl
    << std::endl
    << "  --seqname <sequence name>" << std::endl
    << "    Name of the sequence of image to create (ie: /tmp/image%04d.jpg)." << std::endl
    << "    Default: empty." << std::endl
    << std::endl
    << "  --record <mode>" << std::endl
    << "    Allowed values for mode are:" << std::endl
    << "      0: record all the captures images (continuous mode)," << std::endl
    << "      1: record only images selected by a user click (single shot mode)." << std::endl
    << "    Default mode: 0" << std::endl
    << std::endl
    << "  --no-display" << std::endl
    << "    Disable displaying captured images." << std::endl
    << "    When used and sequence name specified, record mode is internally set to 1 (continuous mode)."
    << std::endl
    << std::endl
    << "  --help, -h" << std::endl
    << "    Print this helper message." << std::endl
    << std::endl;
  std::cout << "USAGE" << std::endl
    << "  Example to visualize images:" << std::endl
    << "    " << argv[0] << std::endl
    << std::endl
    << "  Examples to record a sequence:" << std::endl
    << "    " << argv[0] << " --seqname I%04d.png" << std::endl
    << "    " << argv[0] << " --seqname folder/I%04d.png --record 0" << std::endl
    << std::endl
    << "  Examples to record single shot images:\n"
    << "    " << argv[0] << " --seqname I%04d.png --record 1\n"
    << "    " << argv[0] << " --seqname folder/I%04d.png --record 1" << std::endl
    << std::endl;
 
  if (error) {
    std::cout << "Error" << std::endl
      << "  "
      << "Unsupported parameter " << argv[error] << std::endl;
  }
}
 
int main(int argc, const char *argv[])
{
#if defined(VISP_HAVE_DC1394) && defined(VISP_HAVE_THREADS)
#ifdef ENABLE_VISP_NAMESPACE
  using namespace VISP_NAMESPACE_NAME;
#endif
  try {
    std::string opt_seqname;
    int opt_record_mode = 0;
    bool opt_change_settings = false;
    bool opt_display = true;
 
    for (int i = 1; i < argc; i++) {
      if (std::string(argv[i]) == "--change-settings") {
        opt_change_settings = true;
      }
      else if (std::string(argv[i]) == "--seqname") {
        opt_seqname = std::string(argv[i + 1]);
        i++;
      }
      else if (std::string(argv[i]) == "--record") {
        opt_record_mode = std::atoi(argv[i + 1]);
        i++;
      }
      else if (std::string(argv[i]) == "--no-display") {
        opt_display = false;
      }
      else if (std::string(argv[i]) == "--help" || std::string(argv[i]) == "-h") {
        usage(argv, 0);
        return EXIT_SUCCESS;
      }
      else {
        usage(argv, i);
        return EXIT_FAILURE;
      }
    }
 
    if ((!opt_display) && (!opt_seqname.empty())) {
      opt_record_mode = 0;
    }
 
    std::cout << "Settings   : " << (opt_change_settings ? "modified" : "current") << std::endl;
    std::cout << "Recording  : " << (opt_seqname.empty() ? "disabled" : "enabled") << std::endl;
    std::cout << "Display    : " << (opt_display ? "enabled" : "disabled") << std::endl;
 
    std::string text_record_mode =
      std::string("Record mode: ") + (opt_record_mode ? std::string("single") : std::string("continuous"));
 
    if (!opt_seqname.empty()) {
      std::cout << text_record_mode << std::endl;
      std::cout << "Record name: " << opt_seqname << std::endl;
    }
 
    vpImage<vpRGBa> I;  // Create a color image container
    bool reset = false; // Disable bus reset during construction (default)
    vp1394TwoGrabber g(reset); // Create a grabber based on libdc1394-2.x third party lib
 
    if (opt_change_settings) {
      try {
        g.setVideoMode(vp1394TwoGrabber::vpVIDEO_MODE_640x480_MONO8);
        g.setFramerate(vp1394TwoGrabber::vpFRAMERATE_60);
      }
      catch (...) { // If settings are not available just catch execption to
     // continue with default settings
        std::cout << "Warning: cannot modify camera settings" << std::endl;
      }
    }
    g.open(I);
 
    std::cout << "Image size : " << I.getWidth() << " " << I.getHeight() << std::endl;
 
    vpDisplay *d = nullptr;
    if (opt_display) {
#if !(defined(VISP_HAVE_X11) || defined(VISP_HAVE_OPENCV))
      std::cout << "No image viewer is available..." << std::endl;
      opt_display = false;
#endif
    }
    if (opt_display) {
#ifdef VISP_HAVE_X11
      d = new vpDisplayX(I);
#elif defined(HAVE_OPENCV_HIGHGUI)
      d = new vpDisplayOpenCV(I);
#endif
    }
 
    vpImageQueue<vpRGBa> image_queue(opt_seqname, opt_record_mode);
    vpImageStorageWorker<vpRGBa> image_storage_worker(std::ref(image_queue));
    std::thread image_storage_thread(&vpImageStorageWorker<vpRGBa>::run, &image_storage_worker);
 
    bool quit = false;
    while (!quit) {
      double t = vpTime::measureTimeMs();
      g.acquire(I);
      vpDisplay::display(I);
      quit = image_queue.record(I);
      std::stringstream ss;
      ss << "Acquisition time: " << std::setprecision(3) << vpTime::measureTimeMs() - t << " ms";
      vpDisplay::displayText(I, I.getHeight() - 20, 10, ss.str(), vpColor::red);
      vpDisplay::flush(I);
    }
    image_queue.cancel();
    image_storage_thread.join();
 
    if (d) {
      delete d;
    }
  }
  catch (const vpException &e) {
    std::cout << "Catch an exception: " << e << std::endl;
}
#else
  (void)argc;
  (void)argv;
#ifndef VISP_HAVE_DC1394
  std::cout << "Install libdc1394, configure and build ViSP again to use this example" << std::endl;
#endif
#if (VISP_CXX_STANDARD < VISP_CXX_STANDARD_11)
  std::cout << "This tutorial should be built with c++11 support" << std::endl;
#endif
#endif
}

Here after we explain the new lines that are introduced.

First an instance of the frame grabber is created. During the creating a bus reset is send. If you don't want to reset the firewire bus, just turn reset to false.

    vp1394TwoGrabber g(reset); // Create a grabber based on libdc1394-2.x third party lib

Once the grabber is created, we set the camera image size, color coding, and framerate.

    if (opt_change_settings) {
      try {
        g.setVideoMode(vp1394TwoGrabber::vpVIDEO_MODE_640x480_MONO8);
        g.setFramerate(vp1394TwoGrabber::vpFRAMERATE_60);
      }
      catch (...) { // If settings are not available just catch execption to
     // continue with default settings
        std::cout << "Warning: cannot modify camera settings" << std::endl;
      }
    }

Note that here you can specify some other settings such as the firewire transmission speed. For a more complete list of settings see vp1394TwoGrabber class.

g.setIsoTransmissionSpeed(vp1394TwoGrabber::vpISO_SPEED_800);

Then the grabber is initialized using:

    g.open(I);

From now the color image I is also initialized with the size corresponding to the grabber settings.

Then we enter in a while loop where image acquisition is simply done by:

      g.acquire(I);

As in the previous example, depending on the command line options we are recording a sequence of images, or single shot images. We are also waiting for a non blocking mouse event to quit the while loop.

      quit = image_queue.record(I);

In the previous example we use vp1394TwoGrabber class that works for firewire cameras under Unix. If you are under Windows, you may use vp1394CMUGrabber class. A similar example is provided in tutorial-grabber-CMU1394.cpp.

How to acquire images

Once tutorial-grabber-1394.cpp is built, the binary usage is given by:

./tutorial-grabber-1394 --help

To just view images, run:

./tutorial-grabber-1394

To grab single shot images (for camera calibration for example), you may use:

./tutorial-grabber-1394 --seqname I%04d.pgm --record 1

To grab a sequence of images, you may rather use:

./tutorial-grabber-1394 --seqname I%04d.pgm --record 0

Frame grabbing using libv4l2 SDK

If you want to grab images from an usb camera under Unix, you may use vpV4l2Grabber class that is a wrapper over Video For Linux SDK. To this end libv4l should be installed. An example is provided in tutorial-grabber-v4l2.cpp.

How to acquire images

Once tutorial-grabber-v4l2.cpp is built, the binary usage is given by:

./tutorial-grabber-v4l2 --help

To just view images, run:

./tutorial-grabber-v4l2

To grab single shot images (for camera calibration for example), you may use:

./tutorial-grabber-v4l2 --seqname I%04d.pgm --record 1

To grab a sequence of images, you may rather use:

./tutorial-grabber-v4l2 --seqname I%04d.pgm --record 0

Frame grabbing using Pylon SDK

It is also possible to grab images using Pylon, the SDK for Basler cameras. You may find an example in tutorial-grabber-basler-pylon.cpp.

How to acquire images

Once tutorial-grabber-basler-pylon.cpp is built, the binary usage is given by:

./tutorial-grabber-basler-pylon --help

To just view images, run:

./tutorial-grabber-basler-pylon

To grab single shot images (for camera calibration for example), you may use:

./tutorial-grabber-basler-pylon --seqname I%04d.pgm --record 1

To grab a sequence of images, you may rather use:

./tutorial-grabber-basler-pylon --seqname I%04d.pgm --record 0

Frame grabbing using Realsense SDK

It is also possible to grab images using librealsense, the SDK provided for Intel Realsense RDB-D cameras. You may find an example in tutorial-grabber-realsense.cpp.

How to acquire images

Once tutorial-grabber-realsense.cpp is built, the binary usage is given by:

./tutorial-grabber-realsense --help

To just view images, run:

./tutorial-grabber-realsense

To grab single shot images (for camera calibration for example), you may use:

./tutorial-grabber-realsense --seqname I%04d.pgm --record 1

To grab a sequence of images, you may rather use:

./tutorial-grabber-realsense --seqname I%04d.pgm --record 0

How to acquire images from 2 Realsense devices

Once tutorial-grabber-multiple-realsense.cpp is built, the binary usage is given by:

./tutorial-grabber-multiple-realsense --help

To acquire images from 2 T265 devices with serial numbers 11622110511 and 11622110433 respectively, run:

./tutorial-grabber-multiple-realsense --T265 11622110511 --T265 11622110433

To acquire images from 1 T265 device (Serial Number:11622110511) and 1 D435 device (Serial Number: 752112070408), you may use:

./tutorial-grabber-multiple-realsense --T265 11622110511 --D435 752112070408

Note

There is getRealSense2Info.cpp in example/device/framegrabber folder that could be used to get the device serial number.

$ ./getRealSense2Info
RealSense characteristics:
Intel RealSense T265          11622110409         0.2.0.951
 Device info:
    Name                          :     Intel RealSense T265
    Serial Number                 :     11622110409
    Firmware Version              :     0.2.0.951
    Physical Port                 :     2-3-2
    Product Id                    :     0B37
    Usb Type Descriptor           :     3.1
    Product Line                  :     T200

Frame grabbing using Occipital Structure SDK

If you have a Structure Core RGB-D camera, it is also possible to grab images using libStructure the cross-platform library that comes with Occipital Structure SDK. You may find an example in tutorial-grabber-structure-core.cpp. It allows to save visible and depth images.

How to acquire images

Once tutorial-grabber-structure-core.cpp is built, the binary usage is given by:

./tutorial-grabber-structure-core --help

To just view images, run:

./tutorial-grabber-structure-core

To grab single shot images (for camera calibration for example), you may use:

./tutorial-grabber-structure-core --record 1

To grab a sequence of images, you may rather use:

./tutorial-grabber-structure-core --record 0

RGBD frame grabbing from RealSense D435 and Structure Core

If you have both Intel RealSense D435 and Occipital Structure Core, you can acquire color and depth frames simultaneously from both sensors.

Once tutorial-grabber-rgbd-D435-structurecore.cpp is built, you just need to run:

./tutorial-grabber-rgbd-D435-structurecore

You should see something similar to the following:

Frame grabbing using OpenCV

It is also possible to grab images using OpenCV. You may find an example in tutorial-grabber-opencv.cpp.

How to acquire images

Once tutorial-grabber-opencv.cpp is built, the binary usage is given by:

./tutorial-grabber-opencv --help

To just view images, run:

./tutorial-grabber-opencv

To grab single shot images (for camera calibration for example), you may use:

./tutorial-grabber-opencv --seqname I%04d.pgm --record 1

To grab a sequence of images, you may rather use:

./tutorial-grabber-opencv --seqname I%04d.pgm --record 0

Frame grabbing using CMU1394 SDK

It is also possible to grab images using CMU1394 SDK if you want to use a firewire camera under Windows. You may find an example in tutorial-grabber-CMU1394.cpp.

Frame grabbing using Parrot Bebop 2 drone

It is also possible to grab images using a Parrot Bebop 2 drone. You may find an example in tutorial-grabber-bebop2.cpp.

Warning

Image grabbing with Bebop 2 drone only works if ViSP was built with Parrot ARSDK3 and OpenCV support.

How to acquire images

Once tutorial-grabber-bebop2.cpp is built, the binary usage is given by:

./tutorial-grabber-bebop2 --help

To just view images, run:

./tutorial-grabber-bebop2

To grab single shot images (for camera calibration for example), you may use:

./tutorial-grabber-bebop2 --seqname I%04d.pgm --record 1

To grab a sequence of images, you may rather use:

./tutorial-grabber-bebop2 --seqname I%04d.pgm --record 0

You can chose to record HD 720p pictures from the drone (instead of default 480p) using –hd_resolution option :

./tutorial-grabber-bebop2 --seqname I%04d.pgm --record 0 --hd_resolution

Images from a video stream

With ViSP it also possible to get images from an input video stream. Supported formats are *.avi, *.mp4, *.mov, *.ogv, .flv and many others... To this end we exploit OpenCV 3rd party.

The example below available in tutorial-video-reader.cpp shows how to consider an mpeg video stream.

Warning

We recall that this example only works if ViSP was built with OpenCV support.

Source code explained

The source code is the following:

#include <visp3/core/vpConfig.h>
#include <visp3/gui/vpDisplayFactory.h>
#include <visp3/core/vpTime.h>
#include <visp3/io/vpVideoReader.h>
 
int main(int argc, char **argv)
{
#if defined(VISP_HAVE_OPENCV) && defined(HAVE_OPENCV_HIGHGUI) && defined(HAVE_OPENCV_VIDEOIO)
#ifdef ENABLE_VISP_NAMESPACE
  using namespace VISP_NAMESPACE_NAME;
#endif
  try {
    std::string videoname = "video.mp4";
 
    for (int i = 0; i < argc; i++) {
      if (std::string(argv[i]) == "--name")
        videoname = std::string(argv[i + 1]);
      else if (std::string(argv[i]) == "--help" || std::string(argv[i]) == "-h") {
        std::cout << "\nUsage: " << argv[0] << " [--name <video name> (default: " << videoname << ")]"
          << " [--help] [-h]\n"
          << std::endl;
        return EXIT_SUCCESS;
      }
    }
 
    vpImage<unsigned char> I;
    vpVideoReader g;
    g.setFileName(videoname);
    g.open(I);
    std::cout << "Video name     : " << videoname << std::endl;
    std::cout << "Video framerate: " << g.getFramerate() << "Hz" << std::endl;
    std::cout << "Video dimension: " << I.getWidth() << " " << I.getHeight() << std::endl;
 
#ifdef VISP_HAVE_DISPLAY
    vpDisplay *d = vpDisplayFactory::allocateDisplay(I);
#else
    std::cout << "No image viewer is available..." << std::endl;
#endif
    vpDisplay::setTitle(I, "Video reader");
 
    unsigned cpt = 1;
    bool quit = false;
    while ((!g.end()) && (!quit)) {
      double t = vpTime::measureTimeMs();
      g.acquire(I);
      vpDisplay::display(I);
      vpDisplay::displayText(I, 20, 20, "Click to quit", vpColor::red);
      std::stringstream ss;
      ss << "Frame: " << ++cpt;
      std::cout << "Read " << ss.str() << std::endl;
      vpDisplay::displayText(I, 40, 20, ss.str(), vpColor::red);
      vpDisplay::flush(I);
      if (vpDisplay::getClick(I, false)) {
        quit = true;
      }
      vpTime::wait(t, 1000. / g.getFramerate());
    }
    if (!quit) {
      std::cout << "End of video was reached" << std::endl;
    }
#ifdef VISP_HAVE_DISPLAY
    delete d;
#endif
  }
  catch (const vpException &e) {
    std::cout << e.getMessage() << std::endl;
  }
#else
  (void)argc;
  (void)argv;
  std::cout << "Install OpenCV and rebuild ViSP to use this example." << std::endl;
#endif
  return EXIT_SUCCESS;
}

We explain now the new lines that were introduced.

#include <visp3/core/vpTime.h>
#include <visp3/io/vpVideoReader.h>

Include the header of the vpTime class that allows to measure time, and of the vpVideoReader class that allows to read a video stream.

    vpVideoReader g;

Create an instance of a video reader.

    g.setFileName(videoname);

Set the name of the video stream. Here videoname corresponds to a video file name location. For example we provide the file video.mpg located in the same folder than the executable.

The vpVideoReader class can also handle a sequence of images. For example, to read the following images:

% ls *.png
image0000.png
image0001.png
image0002.png
image0003.png
image0004.png
...

you may use the following:

g.setFileName("./image%04d.png");

where you specify that each image number is coded with 4 digits. Here, we will use libpng or OpenCV to read PNG images. Supported image formats are PPM, PGM, PNG and JPEG.

Then as for any other grabber, you have to initialize the frame grabber using:

    g.open(I);

Then we enter in the while loop until the last image was not reached:

    while ((!g.end()) && (!quit)) {

To get the next image in the stream, we just use:

      g.acquire(I);

To synchronize the video decoding with the video framerate, we measure the beginning time of each loop iteration:

      double t = vpTime::measureTimeMs();

The synchronization is done by waiting from the beginning of the iteration the corresponding time expressed in milliseconds by using:

      vpTime::wait(t, 1000. / g.getFramerate());

How to acquire images

Once tutorial-video-reader.cpp is built, the binary usage is given by:

./tutorial-video-reader --help

To read the default video video.mpg, run:

./tutorial-video-reader

To read an other video, let say my-video.mpg, you may use:

./tutorial-grabber-realsense --name my-video.mpg

Next tutorial

You are now ready to see how to continue with:

• a simple image processing that shows how to track blobs explained in Tutorial: Blob tracking.
• There is also the Tutorial: How to manipulate a video or a sequence of images that could be useful when you need to visualize, rename or change the format of a captured video or sequence of successive images, typically for deep learning purpose.