KR20210157194A - Crop growth measurement device using image processing and method thereof - Google Patents

Abstract

The present invention relates to a growth measurement device capable of measuring the growth amount of a crop through image processing. The growth measurement device according to one embodiment of the present application comprises: a recognition unit for recognizing a reference area, first and second growth point stickers through an image; a calculation unit for calculating the color of the reference area and the first and second growth point stickers based on the image as a color model; and an extraction unit for extracting an outline of at least one of the reference area and the first and second growth point stickers, and extracting a growth amount of a crop based on outline information of the first and second growth point stickers, thereby capable of increasing the accuracy and convenience of measuring crop growth.

Description

Apparatus and method for measuring crop growth using image processing

The present invention relates to a growth amount measuring device capable of measuring the growth amount of a crop through image processing.

Recently, the Internet, short-range wireless communication technology, image processing, and various sensing ICT (Information & Communication Technology) complex technologies are being applied to crop cultivation and growth monitoring systems and are being developed. In addition, greenhouse monitoring and management systems that can manage the growing environment of crops are also developing together.

When measuring the height of crops, if the height of some crops exceeds 2m, the angle of view is not captured, so there is a problem that the height cannot be determined after a certain growing period. In addition, in image recognition after photographing, if crops that are dense by colony are photographed, when crops overlap, it is difficult to accurately recognize crop organs. In addition, there is no system capable of predicting the change in crop growth or diagnosing crop conditions by collecting currently measured crop data.

An object of the present application is to develop an image-based growth measurement apparatus capable of increasing the accuracy and convenience of measuring the growth amount of a crop based on a captured image.

A growth measurement apparatus according to an embodiment of the present application includes a recognition unit for recognizing a reference sphere and first and second growth point stickers through an image, and the color of the reference sphere and the first and second growth point stickers based on the image a calculation unit that calculates as a color model, extracting the outline of at least one of the reference sphere, and the first and second growth point stickers, and extracting the growth amount of a crop based on the outline information of the first and second growth point stickers includes an extractor.

In one embodiment of the present application, the reference sphere is located within a certain distance of the crop to determine the position of the crop, the first growth point sticker is located at the previous growth point of the crop, and the second growth point The sticker is located at the current growth point of the crop.

In an embodiment of the present application, a noise removing unit for removing noise from at least one of the reference sphere and the first and second growth point stickers extracted based on the color model is further included, wherein the noise removing unit removes the image from the image. Convert to black and white image.

In an embodiment of the present application, the noise remover applies at least one of a Gaussian blur and a median blur to the black-and-white image to perform blur processing.

In one embodiment of the present application, the noise removing unit removes the noise of the first and second growth rate stickers by binarizing the blur-processed black-and-white image.

In an embodiment of the present application, the noise removing unit further includes removing the noise of the first and second growth rate stickers through erosion and expansion calculations.

In an embodiment of the present application, the noise removing unit removes the noise of the reference sphere by Hough-transforming the blurred black-and-white image.

In an embodiment of the present application, the difference between the positions of the first growth amount sticker and the second growth amount sticker is the growth amount of the crop.

In one embodiment of the present application, the growth amount is the degree of growth of the crop, and the growth amount further includes maturity of the crop, disease, and fruit shape.

The method of operating the apparatus for measuring growth according to an embodiment of the present application includes the steps of: a recognition unit recognizing a reference sphere, first and second growth point stickers through an image; and a calculator based on the image, the reference sphere, the first and calculating the color of the second growth point sticker as a color model; removing noise from at least one of the reference sphere and the first and second growth point stickers extracted based on the color model by the noise removing unit; and extracting an outline of at least one of a reference sphere and the first and second growth point stickers, and extracting a growth amount of a crop based on outline information of the first and second growth point stickers.

In one embodiment of the present application, the step of removing the noise of the first and second growth stickers by the noise removing unit includes converting the image to a black and white image by the noise removing unit, Gaussian blur and intermediate to the black and white image applying at least one of value blur and binarizing the blurred black-and-white image.

In an embodiment of the present application, the step of removing the noise of the reference sphere by the noise removing unit includes at least one of converting the image to a black and white image by the noise removing unit, the Gaussian blur and the median blur in the black and white image It includes applying one and Hough transforming the blurred black-and-white image.

In an embodiment of the present application, the growth amount measurement system includes a growth amount measurement device, an image photographing device for taking an image to provide an image to the growth amount measurement device, and the growth amount measurement device includes a reference phrase, a first A recognition unit for recognizing the first and second growth point stickers, a calculation unit for calculating the colors of the reference sphere and the first and second growth point stickers as a color model based on the image, and the reference sphere, the first and second growth points and an extraction unit for extracting an outline of at least one of the stickers and extracting a growth amount of a crop based on the outline information of the first and second growth point stickers.

The image-based growth amount measuring apparatus according to an embodiment of the present application may increase the accuracy and convenience of measuring the growth amount of a crop.

1 is a block diagram of an apparatus for measuring growth according to an embodiment of the present application.
2A to 2I are photographs illustrating a plurality of noise removal methods for extracting an outline according to an embodiment of the present application.
3 is a flowchart illustrating an operating method of an apparatus for measuring growth according to an embodiment of the present application.
4 is a diagram of a growth amount measurement system according to an embodiment of the present application.
5 is a view showing an example of applying the growth measurement system according to an embodiment of the present application to a smart farm.

Hereinafter, the present disclosure will be described with reference to the accompanying drawings. As the present disclosure is capable of various changes and may have various embodiments, specific embodiments are illustrated in the drawings and the related detailed description is set forth. However, this is not intended to limit the present disclosure to specific embodiments, and should be understood to include all modifications and/or equivalents or substitutes included in the spirit and scope of the present disclosure. In connection with the description of the drawings, like reference numerals have been used for like elements.

Terms used in the present disclosure are used only to describe specific embodiments, and are not intended to limit the present disclosure. The singular expression includes the plural expression unless the context clearly dictates otherwise.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. Terms such as those defined in a commonly used dictionary should be interpreted as having a meaning consistent with the meaning in the context of the related art, and should not be interpreted in an ideal or excessively formal meaning unless explicitly defined in the present disclosure. does not

1 is a block diagram of an apparatus 100 for measuring growth according to an embodiment of the present application.

Referring to FIG. 1 , the apparatus 100 for measuring growth of the present application may include a recognition unit 120 , a calculation unit 140 , a noise removal unit 160 , and an extraction unit 180 .

The recognition unit 120 recognizes the reference sphere and the first and second growth point stickers through the image. In an embodiment of the present application, the recognition unit 120 captures a planting area of a crop such as a greenhouse or a plastic house through an image photographing device, and recognizes a reference phrase and first and second stickers in the image. The crop may be a plant that grows horizontally relative to the ground, such as tomatoes, eggplants, peppers, and the like.

In one embodiment of the present application, the reference sphere is located within a certain distance of the crop to determine the position of the crop, the first growth point sticker is located at the previous growth point of the crop, and the second growth point sticker is the current growth point of the crop is located in

In an embodiment of the present application, the growth point sticker may have a rectangular shape. The growth point sticker may be positioned at the growth point of the crop, and may be positioned as a first growth point sticker positioned at a previous growth point or a second growth point sticker positioned at a current growth point. However, the shape and number of growth point stickers are not limited thereto, and may be modified and used as necessary.

The calculator 140 calculates the color of the reference sphere and the first and second growth point stickers as a color model based on the image. In order to find the reference sphere and the first and second growth point stickers, an HSV color model that can be expressed intuitively may be used. The HSV color model can be calculated as an HSV value using a formula for an RGB image acquired with a general camera.

R, G, and B have a range of 0 to 1, and when the existing RGB range is 0 to 255, divide by 255. H ranges from 0 to 360, and V and S ranges from 0 to 1. If H is less than 0, add 360 to get the final H. However, since only a single color may not appear depending on the environment, the HSV color is calculated based on the above formula and extracted in a similar region.

The noise removing unit 160 removes noise from an image so as to provide a smooth, unsmoothly extracted image. If the extracted image is smooth, the noise removing unit 160 may be omitted. The noise removing unit 160 removes noise from at least one of the reference sphere and the first and second growth point stickers extracted based on the color model.

In order to remove the noise, the noise removing unit 160 converts the image captured by the recognition unit 120 into a black and white image. In order to remove the noise on the first and second growth point stickers, the black and white image is blurred by applying at least one of Gaussian blur and median blur to the black and white image. The noise of the first and second growth rate stickers is removed by binarizing the blurred black-and-white image. After the noise is removed, erosion and dilation operations are performed on the image. Erosion and dilation operations may be a process for restoring the size of an object in an image that has been reduced in order to remove noise on the image.

로 표현할 수 있다.To remove the noise on the reference sphere, Hough transform the black and white image to remove the noise. The Hough transform is a method of finding shapes in an image. It can find the shape of an image and restore missing or broken areas. Hough transform is implemented as a function based on mathematical theory. The Hough transform uses the equation of a straight line. The equation for innumerable straight lines passing through a point can be expressed as y = mx + c, and if this is transformed using a trigonometric function,

can be expressed as

값을 1 ~ 180까지 변화를 하면서 원점에서 (x,y)까지의 거리(r)을 구한다. 그러면 (

, r)로 구성된 180개의 2차원 배열을 구할 수 있습니다. 동일한 방법으로 두번째 점에 대해서도

값을 변화해 가면서 2차원 배열을 구한다. 이렇게 해서 구해서 2차원 배열을 다시 그래프로 표현하면 사인파 그래프로 표현이 된다. 3개의 점에 대한 3개의 방정식의 만나는 점이 바로 직선인 확률이 높은 점이다.For example, if there are 3 points and you need to find the first straight line among them, use a trigonometric function for each point (x,y)

Find the distance (r) from the origin to (x,y) while changing the value from 1 to 180. then (

, r) of 180 two-dimensional arrays. In the same way, for the second point

A two-dimensional array is obtained by changing the values. If this is obtained and the two-dimensional array is expressed as a graph again, it is expressed as a sine wave graph. The point where the three equations for three points meet is a point with a high probability of being a straight line.

The extraction unit 180 extracts an outline of at least one of the noise-removed reference sphere and the first and second growth point stickers, and extracts the growth amount of the crop based on the outline.

The distance between the centers of the two extracted growth point stickers is obtained in pixels. Calculate the number of pixels of the reference sphere per 10mm by calculating the pixel value of the diameter of the reference sphere. Calculate the distance between crops by calculating the distance between centers in mm using the pixel value of the reference sphere diameter.

The growth amount is a degree of growth of a crop, and may be a difference in positions between the first growth amount sticker and the second growth amount sticker. The growth amount according to an embodiment of the present application may further include the maturity of the crop, the disease, and the shape of the fruit.

In one embodiment of the present application, the position of the crop may be grasped even when the crops are overlapped by using the reference phrase. For example, the position of each crop may be grasped by differentiating the color or shape of each reference sphere spaced apart from each crop by a predetermined distance.

In an embodiment of the present application, the growth amount of some crops may be checked through the first and second growth point stickers even if the height of some crops is high.

As described above, it is possible to increase the accuracy and convenience of measuring the growth amount of a crop through the apparatus 100 for measuring the growth amount including the reference phrase and the first and second growth point stickers.

On the other hand, it will be understood that the above description is exemplary, and the technical spirit of the present application is not limited thereto. For example, the growth amount measuring apparatus 100 of FIG. 1 has been described as including the noise removing unit 160 . However, this is only an example, and in another embodiment of the present application, the growth measurement apparatus 100 may not include the noise removing unit 160 .

2A to 2I are images illustrating a plurality of noise removal methods for extracting an outline according to an embodiment of the present application.

Referring to FIGS. 2A to 2I , in FIG. 2A , a red rectangular sticker for identifying the position of a crop is attached as an original image.

2B is an image when Gaussian blur is applied after conversion from an original image to a black-and-white image. Gaussian blur blurs an image using a Gaussian function. Gaussian blur is used in graphics software to reduce image noise and reduce detail.

FIG. 2C is an image when an intermediate value blur is applied after converting an original image to a black-and-white image. Median blur is a method of selecting a median value from a rectangular area around a pixel and replacing it with the value of the corresponding pixel. A simple blur using an average value can be sensitive to noise that appears independently of itself, because pixels with a large bouncing value in a small number of pixels can have a large effect on the average value calculation. Therefore, the median blur that selects and blurs the median value can solve the problem of simple blur.

2D is an image when image binarization is applied after converting an original image to a black-and-white image. In image binarization, all pixels are expressed only in black and white.

2E is an image when Gaussian blur and image binarization are applied after converting the original image to a black-and-white image. After blurring the image through Gaussian blur, all pixels are expressed only in black and white through image binarization, so that noise in the image can be more effectively removed.

FIG. 2f is an image when the median blur and image binarization are applied after converting the original image to a black and white image. Image noise can be removed more effectively by equalizing the image through median blur and expressing in black and white through image binarization.

2G is an image when the median blur and OTSU binarization are applied after converting the original image to a black-and-white image. OTSU binarization is to find a valley point and use that point as the threshold value (T), assuming that the histogram shape of the image is bimodal. That is, when a set is divided into two classes (classes), the relatively crowded parts are classified into the same class (class). Since OTSU binarization captures the median value in the double peaks (two histograms), it is possible to catch the threshold value relatively accurately. Through OTSU binarization and median blur, it is possible to effectively remove artifacts from the image.

2H is an image when erosion and dilation operations are applied after converting an original image to a black-and-white image. Erosion and dilation operations may be a process for restoring the size of an object in an image that has been reduced in order to remove noise on the image.

2I is an image in which the outline of a red rectangular sticker in the original image is extracted using the various methods above. For contour extraction, one of the above methods may be used, and several methods may be applied simultaneously.

The growth measuring apparatus having the above configuration may be operated in the following way.

3 is a flowchart illustrating an operating method of an apparatus for measuring growth according to an embodiment of the present application.

Referring to FIG. 3 , the operation method of the growth amount measuring apparatus includes the steps of recognizing a reference sphere and first and second growth amount stickers by the recognition unit (S11), calculating the recognized color as a color model by the calculator (S13), removing noise A step of removing the noise of the additional reference sphere and the first and second growth point stickers (S15), the extraction unit extracting outlines of the reference sphere and the second growth point sticker (S17), and the extraction unit based on the extracted outlines It includes the step of extracting the growth (S19).

In the step (S11) of the recognition unit recognizing the reference sphere, the first and second growth rate stickers, the plantation of crops such as a greenhouse or a plastic house is photographed through an image photographing device, and the reference sphere and the first and second stickers in the image are captured. Recognize.

In the step (S13) of the calculation unit calculating the recognized color as the color model, the color of the reference sphere and the first and second growth point stickers is calculated as the color model based on the image. As the color model, an HSV color model that can be intuitively expressed in order to find the reference sphere and the first and second growth point stickers may be used. The HSV color model can be calculated as an HSV value using a formula for an RGB image acquired with a general camera.

In the step (S15) of the noise removal unit removing the noise of the reference sphere and the first and second growth point stickers, the noise of the image is removed so that the image that is not smoothly extracted through various methods can be smoothly provided. If the extracted image is smooth, step S15 can be omitted.

In an embodiment of the present application, a method for the noise removing unit to remove the noise of the first and second growth amount stickers is as follows. The noise canceling unit converts the captured image into a black-and-white image. Blur is performed by applying at least one of Gaussian blur and median blur to the converted black-and-white image. Removes noise by binarizing the blurred black-and-white image.

In an embodiment of the present application, a method for the noise removing unit to remove the noise of the reference sphere is as follows. The noise canceling unit converts the image into a black-and-white image. Blur is performed by applying at least one of Gaussian blur and median blur to the converted black-and-white image. Noise is removed by Hough transforming the blurred black-and-white image.

In the step (S17) of the extraction unit extracting the outlines of the reference sphere and the second growth point sticker, at least one outline of the noise-removed reference sphere and the first and second growth point stickers is extracted.

The extracting unit extracting the growth amount of the crop based on the extracted outline (S19) extracts the growth amount of the crop based on the extracted outline. The growth amount is a degree of growth of a crop, and may be a difference in positions between the first growth amount sticker and the second growth amount sticker. The growth amount according to an embodiment of the present application may further include the maturity of the crop, the disease, and the shape of the fruit.

In one embodiment of the present application, the position of the crop may be grasped even when the crops are overlapped by using the reference phrase. For example, the position of each crop may be grasped by differentiating the color or shape of each reference sphere spaced apart from each crop by a predetermined distance.

In an embodiment of the present application, the growth amount of some crops may be checked through the first and second growth point stickers even if the height of some crops is high.

As described above, it is possible to increase the accuracy and convenience of measuring the growth amount of a crop through the apparatus 100 for measuring the growth amount including the reference phrase and the first and second growth point stickers.

4 is a diagram of a growth measurement system 1000 according to an embodiment of the present application.

Referring to FIG. 4 , the growth amount measurement system 1000 includes an image capturing apparatus 200_1 and a growth amount measurement apparatus 100_1 . The image capturing apparatus 200_1 captures an image to provide the image to the growth measurement apparatus 100_1 . The image capturing apparatus 200_1 may be one of electronic devices in which a camera capable of capturing an image is mounted.

For example, the video recording device 200_1 is an IP camera, an HD-SDI camera, an analog camera, a fire detection color camera, a thermal imager, an HD (1920x5080, HD5080p) camera, IP at the resolution of the SD (720x486, NTSC) level. It may be at least one of a zoom speed camera or a CCTV camera.

The growth amount measuring apparatus 100_1 may be the same as the growth amount measuring apparatus described in FIG. 1 . Therefore, detailed description will be omitted.

The growth measurement apparatus 100_1 includes a recognition unit 120_1 , a calculation unit 140_1 , and an extraction unit 180_1 . The recognition unit 120_1 recognizes the reference sphere and the first and second growth point stickers through the image. The calculator 140_1 calculates the color of the reference sphere and the first and second growth point stickers as a color model based on the image. The extraction unit 180_1 extracts an outline of at least one of the reference sphere and the first and second growth point stickers, and extracts the growth amount of the crop based on the outline information of the first and second growth point stickers.

In an embodiment of the present application, the growth measurement apparatus 100_1 may further include a noise removing unit 160_1. When the captured image is not smooth, the noise removing unit 160_1 blurs the image by selecting at least one of various blur processing methods.

5 is a view showing an example of applying the growth measurement system according to an embodiment of the present application to a smart farm.

Referring to FIG. 5 , the system for measuring growth is also applicable to the smart farm 2000 . The smart farm 2000 may include a growth measurement system, a server 300 , and a user terminal 400 . The growth measurement system includes a growth measurement device 100_2 and an image photographing device 200_2, and the growth amount measurement apparatus 100_2 and the image photographing apparatus 200_2 are the same as the growth amount measurement apparatus and image photographing apparatus disclosed in FIG. A description will be omitted.

In one embodiment of the present application, the growth measurement system may be disposed in a crop cultivation facility, such as a greenhouse or a greenhouse. The growth measurement system may communicate with the user terminal 400 through the server 300 .

For example, the user terminal 400 is a smart phone (Smartphone), a smart pad (Smart Pad), a tablet PC, a wearable device, PCS (Personal Communication System), GSM (Global System for Mobile communication), PDC (Personal Digital Cellular) , PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), Wibro (Wireless Broadband Internet) It may be any kind of wireless communication device such as a terminal, a desktop computer, and a stationary terminal such as a smart TV.

The server 300 connects the user terminal 400 and the growth measurement system to enable communication with each other. The server 300 may provide the data of the growth measurement apparatus 100_2 to the user terminal 400 .

In an embodiment of the present application, the growth measurement apparatus 100_2 may be disposed outside a greenhouse or a plastic house.

Embodiments disclosed in the present specification and drawings are merely provided for specific examples to easily explain the contents of the present disclosure and help understanding, and are not intended to limit the scope of the present disclosure. Therefore, the scope of the present disclosure should be construed as including all changes or modifications derived from the technical spirit of the present disclosure in addition to the embodiments disclosed herein as being included in the scope of the present disclosure.

100: growth measuring device
100_1: Growth measuring device
100_2: Growth measuring device
120: recognition unit
120_1: Recognition unit
140: calculator
140_1 : Calculator
160: noise canceling unit
160_1: noise canceling unit
180: extraction unit
180_1 : Extraction part
200_1 : video recording device
200_2 : video recording device
300 : server
400: user terminal
1000: growth measurement system
2000 : Smart Farm

Claims (13)

a recognition unit for recognizing a reference sphere, first and second growth point stickers through an image;
a calculation unit for calculating the colors of the reference sphere and the first and second growth point stickers as a color model based on the image; and
and an extractor configured to extract an outline of at least one of the reference sphere and the first and second growth point stickers, and extract a growth amount of a crop based on outline information of the first and second growth point stickers. According to claim 1,
The reference sphere is located within a certain distance of the crop to determine the position of the crop,
The first growth point sticker is located at the previous growth point of the crop,
The second growth point sticker is a growth amount measuring device located at the current growth point of the crop. According to claim 1,
and a noise removing unit for removing noise from at least one of the reference sphere and the first and second growth point stickers extracted based on the color model, wherein the noise removing unit converts the image into a black and white image. 4. The method of claim 3,
The noise removing unit applies at least one of a Gaussian blur and a median blur to the black-and-white image to perform blur processing. 5. The method of claim 4,
The noise removing unit binarizes the black-and-white image that has been blurred to remove noise from the first and second growth amount stickers. 6. The method of claim 5,
The noise removal unit further comprises removing the noise of the first and second growth amount stickers through erosion and expansion calculations. 4. The method of claim 3,
The noise removing unit Hough-transforms the blur-processed black-and-white image to remove the noise of the reference sphere. According to claim 1,
A growth amount measuring device in which a position difference between the first growth amount sticker and the second growth amount sticker is a growth amount of a crop. 9. The method of claim 8,
The growth rate is the degree of growth of the crop,
The growth amount is a growth amount measuring device further comprising the maturity level of the crop, diseases, and the shape of the fruit. In the operating method of the growth measurement device,
Recognizing the reference sphere, first and second growth point stickers through the recognition unit;
calculating, by a calculator, the colors of the reference sphere and the first and second growth point stickers as a color model based on the image;
removing, by a noise removing unit, noise from at least one of the reference sphere and the first and second growth point stickers extracted based on the color model; and
The extracting unit extracting an outline of at least one of the reference sphere and the first and second growth point stickers, and extracting the growth amount of the crop based on the outline information of the first and second growth point stickers; Growth measurement comprising a; How the device works. 11. The method of claim 10,
The step of the noise removing unit removing the noise of the first and second growth rate stickers,
converting the image into a black-and-white image by the noise removing unit;
applying at least one of a Gaussian blur and a median blur to the black-and-white image; and
A method of operating a growth measurement apparatus comprising; binarizing the blurred black-and-white image. 11. The method of claim 10,
The step of removing the noise of the reference sphere by the noise removing unit comprises:
converting the image into a black-and-white image by the noise removing unit;
applying at least one of the Gaussian blur and the median blur to the black-and-white image; and
Hough transforming the blur-processed black-and-white image; Method of operation of a growth measurement apparatus comprising a. growth measuring device;
and an image photographing apparatus for photographing an image to provide an image to the growth measurement apparatus,
The growth measurement device,
a recognition unit for recognizing a reference sphere, first and second growth point stickers through an image;
a calculation unit for calculating the color of the reference sphere and the first and second growth point stickers as a color model based on the image; and
An extraction unit for extracting an outline of at least one of the reference sphere and the first and second growth point stickers, and extracting a growth amount of a crop based on the outline information of the first and second growth point stickers; Growth measurement system comprising a; .

Images