| OpenCV 5.0.0alpha Open Source Computer Vision

Loading...
Searching...
No Matches
Detecting colorcheckers using neural network

In this tutorial you will learn how to use the neural network to boost up the accuracy of the chart detection algorithm.

# Building

When building OpenCV, run the following command to build all the contrib module:

cmake -D OPENCV_EXTRA_MODULES_PATH=<opencv_contrib>/modules/

Or only build the mcc module:

cmake -D OPENCV_EXTRA_MODULES_PATH=<opencv_contrib>/modules/mcc

Or make sure you check the mcc module in the GUI version of CMake: cmake-gui.

# Source Code of the sample

You can run the sample code by doing

<path_of_your_opencv_build_directory>/bin/example_mcc_chart_detection_with_network -t=<type_of_chart> -m=<path_to_neural_network> -pb=<path_to_models_pbtxt> -v=<optional_path_to_video_if_not_provided_webcam_will_be_used.mp4> --ci=<optional_camera_id_needed_only_if_video_not_provided> --nc=<optional_maximum_number_of_charts_in_image> --use_gpu <optional_should_gpu_be_used>
``'
* -t=# is the chart type where 0 (Standard), 1 (DigitalSG), 2 (Vinyl)
* --ci=# is the camera ID where 0 (default is the main camera), 1 (secondary camera) etc
* --nc=# By default its values is 1 which means only the best chart will be detected
Example:

Simple run on CPU (GPU wont be used) /home/opencv/build/bin/example_mcc_chart_detection_with_network -t=0 -m=/home/model.pb –pb=/home/model.pbtxt -v=mcc24.mp4

To run on GPU /home/opencv/build/bin/example_mcc_chart_detection_with_network -t=0 -m=/home/model.pb –pb=/home/model.pbtxt -v=mcc24.mp4 –use_gpu

To run on GPU and detect the best 5 charts (Detections can be less than 5 but not more than 5) /home/opencv/build/bin/example_mcc_chart_detection_with_network -t=0 -m=/home/model.pb –pb=/home/model.pbtxt -v=mcc24.mp4 –use_gpu –nc=5 ```

1#include <opencv2/core.hpp>
2
3#include <opencv2/highgui.hpp>
4#include <opencv2/mcc.hpp>
5#include <opencv2/dnn.hpp>
6#include <iostream>
7
8 using namespace std;
9 using namespace cv;
10 using namespace mcc;
11
12 const char *about = "Basic chart detection using neural network";
13 const char *keys = {
14 "{ help h usage ? | | show this message }"
15 "{t | 0 | chartType: 0-Standard, 1-DigitalSG, 2-Vinyl, default:0}"
16 "{m | | File path of model, if you don't have the model you can \
17 find the link in the documentation}"
18 "{pb | | File path of pbtxt file, available along with with the model \
19 file }"
20 "{v | | Input from video file, if ommited, input comes from camera }"
21 "{ci | 0 | Camera id if input doesnt come from video (-v) }"
22 "{nc | 1 | Maximum number of charts in the image }"
23 "{use_gpu | | Add this flag if you want to use gpu}"};
24
25 int main(int argc, char *argv[])
26{
27 // ----------------------------------------------------------
28 // Scroll down a bit (~50 lines) to find actual relevant code
29 // ----------------------------------------------------------
30
31 CommandLineParser parser(argc, argv, keys);
32 parser.about(about);
33
34 if (parser.has("help"))
35 {
36 parser.printMessage();
37 return -1;
38 }
39
40 int t = parser.get<int>("t");
41
42 CV_Assert(0 <= t && t <= 2);
43 TYPECHART chartType = TYPECHART(t);
44
45 string model_path = parser.get<string>("m");
46 string pbtxt_path = parser.get<string>("pb");
47
48 int camId = parser.get<int>("ci");
49 int nc = parser.get<int>("nc");
50
51 String video;
52
53 if (parser.has("v"))
54 video = parser.get<String>("v");
55
56 bool use_gpu = parser.has("use_gpu");
57
58 if (!parser.check())
59 {
60 parser.printErrors();
61 return 0;
62 }
63
64 VideoCapture inputVideo;
65 int waitTime;
66 if (!video.empty())
67 {
68 inputVideo.open(video);
69 waitTime = 10;
70 }
71 else
72 {
73 inputVideo.open(camId);
74 waitTime = 10;
75 }
76
77 //--------------------------------------------------------------------------
78 //-------------------------Actual Relevant Code-----------------------------
79 //--------------------------------------------------------------------------
80
81 //load the network
82
83 cv::dnn::Net net = cv::dnn::readNetFromTensorflow(model_path, pbtxt_path);
84
85 if (use_gpu)
86 {
87 net.setPreferableBackend(dnn::DNN_BACKEND_CUDA);
88 net.setPreferableTarget(dnn::DNN_TARGET_CUDA);
89 }
90
91 Ptr<CCheckerDetector> detector = CCheckerDetector::create();
92 if (!detector->setNet(net))
93 {
94 cout << "Loading Model failed: Aborting" << endl;
95 return 0;
96 }
97
98 while (inputVideo.grab())
99 {
100 Mat image, imageCopy;
101 inputVideo.retrieve(image);
102
103 imageCopy = image.clone();
104
105 // Marker type to detect
106 if (!detector->process(image, chartType, nc, true))
107 {
108 printf("ChartColor not detected \n");
109 }
110 else
111 {
112
113 // get checker
114 std::vector<Ptr<mcc::CChecker>> checkers = detector->getListColorChecker();
115 for (Ptr<mcc::CChecker> checker : checkers)
116 {
117 // current checker
118
119 Ptr<CCheckerDraw> cdraw = CCheckerDraw::create(checker);
120 cdraw->draw(image);
121 }
122 }
123
124 imshow("image result | q or esc to quit", image);
125 imshow("original", imageCopy);
126 char key = (char)waitKey(waitTime);
127 if (key == 27)
128 break;
129 }
130
131 return 0;
132}
cv::CommandLineParser
Designed for command line parsing.
Definition utility.hpp:890
cv::Mat
n-dimensional dense array class
Definition mat.hpp:951
cv::Mat::clone
CV_NODISCARD_STD Mat clone() const
Creates a full copy of the array and the underlying data.
cv::VideoCapture
Class for video capturing from video files, image sequences or cameras.
Definition videoio.hpp:727
cv::VideoCapture::open
virtual bool open(const String &filename, int apiPreference=CAP_ANY)
Opens a video file or a capturing device or an IP video stream for video capturing.
cv::VideoCapture::retrieve
virtual bool retrieve(OutputArray image, int flag=0)
Decodes and returns the grabbed video frame.
cv::VideoCapture::grab
virtual bool grab()
Grabs the next frame from video file or capturing device.
cv::dnn::Net
This class allows to create and manipulate comprehensive artificial neural networks.
Definition dnn.hpp:535
cv::dnn::Net::setPreferableBackend
void setPreferableBackend(int backendId)
Ask network to use specific computation backend where it supported.
cv::dnn::Net::setPreferableTarget
void setPreferableTarget(int targetId)
Ask network to make computations on specific target device.
dnn.hpp
cv::String
std::string String
Definition cvstd.hpp:151
cv::Ptr
std::shared_ptr< _Tp > Ptr
Definition cvstd_wrapper.hpp:23
CV_Assert
#define CV_Assert(expr)
Checks a condition at runtime and throws exception if it fails.
Definition exception.hpp:198
cv::dnn::readNetFromTensorflow
Net readNetFromTensorflow(CV_WRAP_FILE_PATH const String &model, CV_WRAP_FILE_PATH const String &config=String(), int engine=ENGINE_AUTO, const std::vector< String > &extraOutputs=std::vector< String >())
Reads a network model stored in TensorFlow framework's format.
cv::imshow
void imshow(const String &winname, InputArray mat)
Displays an image in the specified window.
cv::waitKey
int waitKey(int delay=0)
Waits for a pressed key.
highgui.hpp
main
int main(int argc, char *argv[])
Definition highgui_qt.cpp:3
mcc.hpp
core.hpp
cv
Definition core.hpp:107
std
STL namespace.

# Explanation

1. Set header and namespaces

   #include <opencv2/mcc.hpp>
   using namespace std;
   using namespace cv;
   using namespace mcc;

   If you want you can set the namespace like the code above.

2. Create the detector object

   Ptr<CCheckerDetector> detector = CCheckerDetector::create();

   This is just to create the object.

3. Load the model

   cv::dnn::Net net = cv::dnn::readNetFromTensorflow(model_path, pbtxt_path);

   Load the model, here the model supplied with model was trained in tensorflow so we are loading it in tensorflow, but if you have some other model trained in some other framework you can use that also.

4. (Optional) Set the dnn backend to CUDA

   net.setPreferableBackend(dnn::DNN_BACKEND_CUDA);
   net.setPreferableTarget(dnn::DNN_TARGET_CUDA);

   Models run much faster on CUDA, so use CUDA if possible.

5. Run the detector

   detector->process(image, chartType, max_number_charts_in_image, true);

   If the detector successfully detects atleast one chart, it return true otherwise it returns false. In the above given code we print a failure message if no chart were detected. Otherwise if it were successful, the list of colorcharts is stored inside the detector itself, we will see in the next step on how to extract it. The fourth parameter is for deciding whether to use the net or not.

6. Get List of ColorCheckers

   std::vector<cv::Ptr<mcc::CChecker>> checkers;
   detector->getListColorChecker(checkers);

   All the colorcheckers that were detected are now stored in the 'checkers' vector.

7. Draw the colorcheckers back to the image

   for(Ptr<mcc::CChecker> checker : checkers)
   {
   // current checker
   Ptr<CCheckerDraw> cdraw = CCheckerDraw::create(checker);
   cdraw->draw(image);
   }

   Loop through all the checkers one by one and then draw them.