 |
Visual Servoing Platform
version 3.5.0 under development (2022-02-15)
|
|
Visual Servoing Platform
version 3.5.0 under development (2022-02-15)
|
After ViSP 3.0.0, we introduce a new cross-platform vpThread class that allows to execute a function in a separate thread. We also improve vpMutex class useful to protect shared data by mutexes to be cross-platform.
The vpThread and vpMutex classes are wrappers over native pthread functionality when pthread is available. This is the case for all unix-like OS, including OSX and MinGW under Windows. If pthread is not available, we use Windows native functionality instead.
To use vpThread class you have first to include the corresponding header.
With vpThread the prototype of the function vpThread::Fn that could be executed in a separate thread is the following:
where arguments passed to the function are of type vpThread::Args. This function should return a vpThread::Return type.
Then to create the thread that executes this function, you have just to construct a vpThread object indicating which is the function to execute.
To illustrate this behavior, see testThread.cpp.
To use vpMutex class you have first to include the corresponding header.
Then protecting a shared var from concurrent access could be done like:
To illustrate this usage, see testMutex.cpp.
There is also a more elegant way using vpMutex::vpScopedLock. The previous example becomes:
Here, the vpMutex::vpScopedLock constructor locks the mutex, while the destructor unlocks. Using vpMutex::vpScopedLock, the scope of the portion of code that is protected is defined inside the brackets. To illustrate this usage, see tutorial-grabber-opencv-threaded.cpp.
This section will show you one convenient way to pass multiple arguments to a vpThread and retrieve multiple return values at the end of the computation. This example (testThread2.cpp) uses a functor class to do that.
Basically, you declare a class that will act like a function by defining the operator() that will do the computation in a dedicated thread. In the following toy example, we want to compute the element-wise addition ( ![$ v_{add}\left [ i \right ] = v_1 \left [ i \right ] + v_2 \left [ i \right ] $](form_1322.png)
) and the element-wise multiplication ( ![$ v_{mul}\left [ i \right ] = v_1 \left [ i \right ] \times v_2 \left [ i \right ] $](form_1323.png)
) of two vectors.
Each thread will process a subset of the input vectors and the partial results will be stored in two vectors (one for the addition and the other one for the multiplication).
The required arguments needed by the constructor are the two input vectors, the start index and the end index that will define the portion of the vector to be processed by the current thread. Two getters are used to retrieve the results at the end of the computation.
Let's see now how to create and initialize the threads:
The pointer to the routine arithmThread() called by the thread is defined as the following:
This routine is called by the threading library. We cast the argument passed to the thread routine and we call the function that needs to be executed by the thread.
To get the results:
After joining the threads, the partial results from one thread can be obtained by a call to the appropriate getter function.
nor as the following:
as theses lines of code create a temporary vpThread object that will be copied to the vector and after destructed. The destructor of the vpThread calls automatically the join() function and thus it will result that the threads will be created, started and joined sequentially as soon as the temporary vpThread object will be destructed.
Note that all the material (source code) described in this section is part of ViSP source code and could be downloaded using the following command:
The following example implemented in tutorial-grabber-opencv-threaded.cpp shows how to implement a multi-threaded application, where image capture is executed in one thread and image display in an other one. The capture is here performed thanks to OpenCV cv::VideoCapture class. It could be easily adapted to deal with other framegrabbers available in ViSP. In tutorial-grabber-v4l2-threaded.cpp you will find the same example using vpV4l2Grabber. To adapt the code to other framegrabbers see Tutorial: Image frame grabbing.
Hereafter we explain how tutorial-grabber-opencv-threaded.cpp works.
First we include all ViSP headers corresponding to the classes we will use; vpImageConvert to convert OpenCV images in ViSP images, vpMutex to protect shared data between the threads, vpThread to create the threads, vpTime to handle the time, vpDisplayX to display images under unix-like OS, and vpDisplayGDI to display the images under Windows.
Then if OpenCV 2.1.0 or higher is found we include OpenCV highgui.hpp header that brings cv::VideoCapture class that will be used in this example for image capture.
We declare then the shared data with variable names prefixed by "s_" (s_capture_state, indicating if capture is in progress or is stopped, s_frame the image that is currently captured and s_mutex_capture, the mutex that will be used to protect from concurrent access to these shared variables).
Then we implement captureFunction(), the capture function that we want to run in a separate thread. As argument this function receives a reference over cv::VideoCapture object that was created in the Main thread.
We check if the capture is able to found a camera thanks to cap.isOpened(), and start a 30 seconds capture loop that will fill frame_ with the image from the camera. The capture could be stopped before 30 seconds if stop_capture_ boolean is turned to true. Once an image is captured, with the mutex we update the shared data. After the while loop, we also update the capture state to capture_stopped to finish the display thread.
We implement then displayFunction() used to display the captured images. This function doesn't exploit any argument. Depending on the OS we create a display pointer over the class that we want to use (vpDisplayX or vpDisplayGDI). We enter then in a while loop that will end when the capture is stopped, meaning that the Capture thread is finished.
In the display loop, with the mutex we create a copy of the shared variables s_capture_state in order to use if just after. When capture is started we convert the OpenCV cv::mat image into a local ViSP image I. Since we access to the shared s_frame data, the conversion is protected by the mutex. Then with the first available ViSP image I we initialize the display and turn display_initialized_ boolean to false indicating that the display is already initialized. Next we update the display with the content of the image. When we capture is not started, we just sleep for 2 milli-seconds.
The main thread is the one that is implemented in the main() function. We manage first the command line option "--device <camera device>" to allow the user to select a specific camera when more then one camera are connected. Then as explained in Capture thread we need the create cv::VideoCapture object in the main(). Finally, captureFunction() and displayFunction() are started as two separate threads, one for the capture, an other one for the display using vpThread constructor.
The call to join() is here to wait until capture and display thread ends to return from the main().
Once build, to run this tutorial just run in a terminal:
where "--device 0" could be avoided since it is the default option.
Note that all the material (source code) described in this section is part of ViSP source code and could be downloaded using the following command:
The example given in the previous section Multi-threaded capture and display could be extended to introduce an image processing. In this section, we illustrate the case of the face detection described in Tutorial: Face detection and implemented in tutorial-face-detector-live.cpp as a single main thread. Now we propose to extend this example using multi-threading where face detection is achieved in a separate thread. The complete source code is given in tutorial-face-detector-live-threaded.cpp.
Here after we give the changes that we introduce in tutorial-face-detector-live-threaded.cpp to add a new thread dedicated to the face detection.
The function that does the face detection is implemented in detectionFunction(). We first instantiate an object of type vpDetectorFace. Then in the while loop, we call the face detection function using face_detector_.detect() when a new image is available. When faces are found, we retrieve the bounding box of the first face that is the largest in the image. We update the shared s_face_bbox var with the bounding box. This var is then exploited in the display thread and displayed as a rectangle.
The main() is modified to call the detectionFunction() in a third thread.
To run the binary just open a terminal and run:
1.8.13