JP6225443B2 - Image processing apparatus, image processing method, and image processing program - Google Patents

Description

  The present invention relates to an image processing apparatus and the like.

  For example, if a product is displayed in a disordered manner on the sales floor, the sales of the product will decrease. For this reason, the store clerk checks whether or not the product is disturbed while moving in the sales floor, and appropriately arranges the product.

  In addition, the prior art includes a technique for determining whether or not a book stored in a bookshelf needs to be organized. In this prior art, the degree of inclination of a book is calculated from the outline of the book stored in the bookshelf, and it is determined whether or not ordering is necessary based on the ratio of books whose degree of inclination is a predetermined angle.

JP 2000-306077 A JP-A-10-229550

  However, the above-described conventional technique has a problem that it is impossible to determine the disturbance of the monitored object.

  In one aspect, an object is to provide an image processing apparatus, an image processing method, and an image processing program capable of determining disturbance of an object to be monitored.

  In the first plan, the image processing apparatus includes an extraction unit, a classification unit, and a determination unit. The extraction unit extracts the contour of the monitored object from the image data. The classification unit classifies the contours extracted by the extraction unit into a plurality of groups based on the directions of the contours. The determination unit determines whether or not the monitored object is disturbed based on the classification result of the classification unit.

  According to one embodiment of the present invention, there is an effect that it is possible to determine the disturbance of the monitored object.

FIG. 1 is a diagram illustrating the configuration of the image processing apparatus according to the first embodiment. FIG. 2 is a diagram illustrating an example of a data structure of edge information. FIG. 3 is a diagram illustrating an example of a data structure of the frequency distribution information. FIG. 4 is a diagram for explaining the angle of the edge. FIG. 5 is a diagram illustrating an example of image data when the monitored object is not disturbed. FIG. 6 is a diagram (1) showing the relationship between the angle distribution and the number of edges. FIG. 7 is a diagram (1) illustrating an example of the angular distribution rearranged in descending order. FIG. 8 is a diagram illustrating an example of image data when the monitored object is disturbed. FIG. 9 is a diagram (2) showing the relationship between the angle distribution and the number of edges. FIG. 10 is a diagram (2) illustrating an example of the angle distribution rearranged in descending order. FIG. 11 is a flowchart illustrating the processing procedure of the image processing apparatus according to the first embodiment. FIG. 12 is a flowchart illustrating the processing procedure of the extraction unit according to the first embodiment. FIG. 13 is a diagram illustrating an example of divided image data. FIG. 14 is a diagram illustrating the configuration of the image processing apparatus according to the second embodiment. FIG. 15 is a diagram illustrating an example of a data structure of corner information. FIG. 16 is a diagram (1) illustrating an example of the corner detection result. FIG. 17 is a diagram (2) illustrating an example of the corner detection result. FIG. 18 is a flowchart illustrating the processing procedure of the image processing apparatus according to the second embodiment. FIG. 19 is a diagram illustrating an example of a computer that executes an image processing program.

  Embodiments of an image processing apparatus, an image processing method, and an image processing program disclosed in the present application will be described below in detail with reference to the drawings. Note that the present invention is not limited to the embodiments.

  A configuration of the image processing apparatus according to the first embodiment will be described. FIG. 1 is a diagram illustrating the configuration of the image processing apparatus according to the first embodiment. As illustrated in FIG. 1, the image processing apparatus 100 includes a communication unit 105, an imaging unit 110, a storage unit 120, and a control unit 130.

  The communication unit 105 is a processing unit that executes data communication with an external device. For example, the communication unit 105 corresponds to a communication card or a communication device. A control unit 130 described below exchanges data with an external device via the communication unit 105.

  The imaging unit 110 is an apparatus that captures an image in an imaging range. It is assumed that the imaging range is set in advance by the user so as to include monitored objects such as products. The imaging unit 110 outputs the video data to the control unit 130. For example, the imaging unit 110 corresponds to a video camera.

  The storage unit 120 stores edge information 121 and frequency distribution information 122. The storage unit 120 corresponds to a storage device such as a semiconductor memory element such as a random access memory (RAM), a read only memory (ROM), and a flash memory.

  The edge information 121 is edge information extracted from the image data. The edge corresponds to the contour of the monitored object, for example. FIG. 2 is a diagram illustrating an example of a data structure of edge information. As shown in FIG. 2, the edge information 121 holds the coordinates of the start point and the coordinates of the end point in association with each edge.

  The frequency distribution information 122 holds information that classifies edges according to the angle of the edges. FIG. 3 is a diagram illustrating an example of a data structure of the frequency distribution information. As shown in FIG. 3, the frequency distribution information 122 associates an identification number, a range, and a number. The identification number is information for uniquely identifying the group of classified edges. The range is information indicating the angle of the edge assigned to the corresponding group. The number is information indicating the number of edges assigned to the corresponding group.

  For example, the first line in FIG. 3 will be described. The edge assigned to the group having the identification number “1” is an edge whose angle is “0 to 10”. Then, “2” edges are assigned to the group having the identification number “1”.

  Returning to the description of FIG. The control unit 130 includes an extraction unit 131, a classification unit 132, and a determination unit 133. The control unit 130 corresponds to an integrated device such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). The control unit 130 corresponds to an electronic circuit such as a CPU or MPU (Micro Processing Unit).

  The extraction unit 131 is a processing unit that extracts image data serving as a still image from the video data output from the imaging unit 110 and extracts an edge from the extracted image data. Hereinafter, the processing of the extraction unit 131 will be specifically described.

  First, the extraction unit 131 compares the image data Is acquired at the first time with the image data Ic acquired at the second time, and determines whether the image data Is and the image data Ic have changed. To do. For example, the second time is a time that is a predetermined time after the first time. The extraction unit 131 determines to extract an edge from the image data Is when the image data Is and the image data Ic have not changed.

  The extraction unit 131 may determine whether or not the image data Is and the image data Ic are changed using any conventional technique. For example, the extraction unit 131 generates a difference image between the binary image of the image data Is and the binary image of the image data Ic, and among the generated pixels of the difference image, the number of pixels that are 0 is a predetermined pixel. When the number is greater than or equal to the number, it may be determined that the image data Is and the image data Ic have not changed.

  The extraction unit 131 obtains a differential image from the image data Is to be subjected to edge extraction, and extracts an edge by applying Hough transform or the like. The extraction unit 131 registers information on the start point and end point of the edge in the edge information 121 for each edge.

  The extraction unit 131 removes an edge having a predetermined edge length from the extracted edges, and registers information on the remaining edges in the edge information 121. That is, in the edge information 121, information on an edge having a predetermined length or more is registered.

  The classification unit 132 is a processing unit that calculates an edge angle for each edge based on the edge information 121 and generates the frequency distribution information 122 based on the angle of each edge. Below, the process of the classification | category part 132 is demonstrated concretely.

  The classification unit 132 calculates an angle formed by the reference horizontal component and the edge as an edge angle. FIG. 4 is a diagram for explaining the angle of the edge. As shown in FIG. 4, the classification unit 132 sets the angle of the edge 10 as a counterclockwise angle from the horizontal component 11 to the edge 10 when the horizontal component 11 is 0 degree.

  After calculating the angle of each edge, the classification unit 132 obtains the edge frequency distribution c [i] with a predetermined angular width d. The angle width d is a value set in advance by the user and is a value of 360 or less. I is a value of 1 to 360 / d. The edge frequency distribution c [i] is the number of edges at which the edge angle θ is not less than (i−1) × d and less than i × d.

  The classification unit 132 obtains the frequency distributions c [1], c [2],..., C [n], and registers the obtained results in the frequency distribution information 122. For example, when the value of the frequency distribution c [1] is “2”, the number corresponding to the identification information “1” of the frequency distribution information 122 illustrated in FIG. 3 is set to “2”.

  Returning to the description of FIG. The determination unit 133 is a processing unit that determines whether the monitored object is disturbed based on the frequency distribution. If the determination unit 133 determines that the monitored object is disturbed, the determining unit 133 notifies the external device of information indicating that the monitored object is disturbed. Below, the process of the determination part 133 is demonstrated concretely.

  The determination unit 133 uses c ′ [1], c ′ [2], c ′ [1], c ′ [2], c ′ [2],. ..., c '[n] is obtained. Then, the determination unit 133 obtains the minimum number min (k) of k that satisfies the condition of Expression (1), and calculates the obtained k as the number of peaks. In Expression (1), a is a value set in advance by the user, and is a value between 0 and 1. N is the total number of edges.

  c ′ [1] + c [2] +... + c [k]> a × N (1)

  When the number of peaks is larger than a predetermined threshold, the determination unit 133 determines that the monitored object is disturbed and notifies the external device. For example, 2 or 3 is set as the threshold for comparison with the number of peaks.

  Next, an example of calculating the number of peaks when the monitored object is not disturbed and when it is disturbed will be described.

  FIG. 5 is a diagram illustrating an example of image data when the monitored object is not disturbed. When the edge angle distribution is obtained from the image data of the monitored object shown in FIG. 5, the result is shown in FIG. FIG. 6 is a diagram (1) showing the relationship between the angle distribution and the number of edges. The horizontal axis in FIG. 6 indicates the angle, and the vertical axis indicates the number of edges. When the angle distribution shown in FIG. 6 is rearranged in descending order, the result is shown in FIG. FIG. 7 is a diagram (1) illustrating an example of the angular distribution rearranged in descending order. The horizontal axis in FIG. 7 indicates the order of angular distribution, and the vertical axis indicates the number of edges. In the example illustrated in FIG. 7, the condition of Expression (1) is satisfied when the value of k becomes “2”. For this reason, the number of peaks is 2.

  As shown in FIG. 5, when the monitored object is not disturbed, most of the edge angles are in a small number of sections, so the number of peaks is small. The determination unit 133 determines that the monitored object is not disturbed because the number of peaks is equal to or less than a predetermined threshold.

  FIG. 8 is a diagram illustrating an example of image data when the monitored object is disturbed. When the angular distribution of the edge is obtained from the image data of the monitored object shown in FIG. 8, it is as shown in FIG. FIG. 9 is a diagram (2) showing the relationship between the angle distribution and the number of edges. In FIG. 9, the horizontal axis indicates the angle, and the vertical axis indicates the number of edges. When the angle distribution shown in FIG. 9 is rearranged in descending order, the result is shown in FIG. FIG. 10 is a diagram (2) illustrating an example of the angle distribution rearranged in descending order. The horizontal axis in FIG. 7 indicates the order of angular distribution, and the vertical axis indicates the number of edges. In the example illustrated in FIG. 10, the condition of Expression (1) is satisfied when the value of k becomes “8”. For this reason, the number of peaks is 8.

  As shown in FIG. 8, when the monitored object is disturbed, the angles of the edges are various angles, and a number of peaks appear in the angle distribution. As a result, the number of peaks becomes a large value. The determination unit 133 notifies that the monitored object is disturbed because the number of peaks is greater than a predetermined threshold.

  Next, a processing procedure of the image processing apparatus 100 according to the first embodiment will be described. FIG. 11 is a flowchart illustrating the processing procedure of the image processing apparatus according to the first embodiment. The process illustrated in FIG. 11 is executed at predetermined time intervals, for example. As shown in FIG. 11, the image processing apparatus 100 acquires image data (step S101) and extracts an edge (step S102).

  The image processing apparatus 100 calculates the angle of the edge (step S103) and obtains the angle distributions c [1], c [2],... C [n] (step S104). The image processing apparatus 100 rearranges the angle distribution c [i] in descending order (step S105). The image processing apparatus 100 calculates the number of peaks that satisfy the condition of Expression (1) (step S106).

  The image processing apparatus 100 determines whether or not the number of peaks is greater than a threshold value (step S107). If the number of peaks is less than the threshold value (No at Step S107), the image processing apparatus 100 ends the process. On the other hand, when the number of peaks is larger than the threshold value (step S107, Yes), the image processing apparatus 100 notifies that the product of the monitored object is disturbed (step S108).

  Next, the processing procedure of the extraction unit according to the first embodiment will be described. FIG. 12 is a flowchart illustrating the processing procedure of the extraction unit according to the first embodiment. The process shown in FIG. 12 corresponds to a specific process of steps S101 and S102 of FIG. As illustrated in FIG. 12, the extraction unit 131 acquires image data Is captured at the start time Ts (step S150).

  The extraction unit 131 determines whether or not the time T has elapsed from the time Ts (step S151). If the time T has elapsed from the time Ts (step S151, Yes), the extraction unit 131 extracts an edge from the image data Is (step S152).

  On the other hand, when the time T has not elapsed since the time Ts (step S151, No), the extraction unit 131 acquires the image data Ic photographed at the current time (step S153). The extraction unit 131 compares the image data Ic and the image data Is to determine whether or not the image data Is has changed (step S154).

  If the image data Is has not changed (No at Step S155), the extraction unit 131 proceeds to Step S151. On the other hand, when the image data Is has changed (step S155, Yes), the extraction unit 131 proceeds to step S150.

  Next, effects of the image processing apparatus 100 according to the first embodiment will be described. The image processing apparatus 100 classifies the edges into a plurality of groups based on the direction of each edge extracted from the image data, and determines whether a monitored object such as a product is disturbed based on the classification result. judge. Thereby, according to the image processing apparatus 100, the disturbance of the monitored object can be determined.

  Further, according to the image processing apparatus 100, each edge is classified into a plurality of groups on the basis of the relative angle between each edge of the monitored object and a reference horizontal component, and the number of groups is a threshold value. If it is larger than that, it is determined that the monitored object is disturbed. By executing such a process, for example, even when the monitored object is arranged on a movable shelf such as a wagon, or even when it is in a flat state, the disturbance of the monitored object is prevented. Can be judged.

  Moreover, according to the image processing apparatus 100, the outline of less than a predetermined length is removed from the edges of the monitored object extracted from the image data. For this reason, it becomes possible to remove a noise component, and the accuracy of the determination regarding the disturbance of the monitored object can be improved.

  Here, other processes of the image processing apparatus 100 shown in the first embodiment will be described. In the following, a process for determining the disturbance of the monitored object using entropy and a process for determining the disturbance of the monitored object using the Gini coefficient will be described. In addition, a process for determining the disturbance of the monitored object from the connection relationship of each edge and a process for determining the disturbance of the monitored object for each divided image area will be described.

(1) Processing for determining disturbance of monitored object using entry pea The determination unit 133 of the image processing apparatus 100 calculates entropy based on the frequency distribution information 122, and based on entropy, It may be determined whether or not the monitoring target is disturbed.

  The determination unit 133 calculates the entropy e using Expression (2). N in equation (2) corresponds to the total number of edges. Further, n in the formula (2) is the number of angular distributions. When the number of angular distributions is n, it indicates that there are angular distributions c [1], c [2],..., C [n].

  The determination unit 133 determines that the monitored object is disturbed when the value of entropy e is larger than a predetermined value. Thus, the image processing apparatus 100 can determine the disturbance of the monitored object with high accuracy by calculating the entropy and determining whether or not the monitored object is disturbed.

(2) Processing for determining disturbance of monitored object using Gini coefficient The determination unit 133 of the image processing apparatus 100 calculates a Gini coefficient based on the frequency distribution information 122, and based on the Gini coefficient. It may be determined whether or not the monitored object is disturbed.

  The determination unit 133 calculates the Gini coefficient g using Expression (3). N in equation (3) corresponds to the total number of edges. In addition, c ′ [j] in Expression (3) is c ′ [that is rearranged in descending order based on the values of the edge angle distributions c [1], c [2],..., C [n]. 1], c ′ [2],..., C ′ [n].

  The determination unit 133 determines that the monitored object is disturbed when the value of the Gini coefficient g is equal to or less than a predetermined value. Thus, the image processing apparatus 100 can determine the disturbance of the monitored object with high accuracy by calculating the Gini coefficient and determining whether or not the monitored object is disturbed.

(3) Processing for Determining Disturbance of Monitored Object from Connection Relationship of Each Edge Based on the edge information 121, the determination unit 133 of the image processing apparatus 100 is a combination of edges whose edge ends are connected. Select multiple items. The determination unit 133 classifies the corner formed by the set of edges in the connection relationship as one of the first to fourth corners of the monitored object. For example, the first corner corresponds to the upper right corner of the monitored object. The second corner corresponds to the upper left corner of the monitored object. The third corner corresponds to the lower left of the monitored object. The fourth corner corresponds to the lower right of the monitored object.

  For example, the determination unit 133 classifies the edge set as the first angle in order that the angle formed by the edge set is directed to the first quadrant with reference to a predetermined point on the image data. In order for the angle formed by the edge set to be in the second quadrant direction with a predetermined point on the image data as a reference, the edge set is classified as the second angle. In order for the angle formed by the edge set to face the third quadrant with a predetermined point on the image data as a reference, the edge set is classified as the third angle. In order for the angle formed by the edge set to face the fourth quadrant with a predetermined point on the image data as a reference, the edge set is classified as the fourth angle.

  The determination unit 133 determines the disturbance of the monitored object based on the number of sets of edges classified into each corner. The determination unit 133 determines that the monitored object is not disturbed when the number of sets of edges classified into each corner is equal. On the other hand, the determination unit 133 determines that the monitored object is disturbed when the number of sets of edges classified into each corner is not uniform. For example, the determination unit 133 calculates an average value of a set of edges classified into the first to fourth corners. The determination unit 133 determines that the number of sets of edges classified into any of the first to fourth corners is not equal when the number of sets differs from the average value by a predetermined number. As described above, the image processing apparatus 100 can determine the disturbance of the monitored object with high accuracy by determining the disturbance of the monitored object from the connection relation of each edge.

(4) Processing for Determining Disturbance of Monitored Object for Each Divided Image Area The image processing apparatus 100 divides the image data extracted by the extraction unit 131 into a plurality of areas, and for each divided image area. It is determined whether or not the monitored object is disturbed. FIG. 13 is a diagram illustrating an example of divided image data. In the example illustrated in FIG. 13, the image processing apparatus 100 illustrates a case where the image data 20 is divided into a plurality of regions 20 a to 20 p.

  The image processing apparatus 100 determines the disturbance of the monitored object for each divided area. Alternatively, the image processing apparatus 100 may determine the disturbance of the monitored object for only some of the divided areas. For example, the image processing apparatus 100 determines the disturbance of the monitored object for each of the areas 20k to 20f among the divided areas 20a to 20p. The process in which the image processing apparatus 100 determines the disturbance of the monitored object may be the same process as in the first embodiment, or may calculate the entropy or Gini coefficient to determine the disturbance of the monitored object. .

  In this way, the image processing apparatus 100 divides the image data into a plurality of areas, and determines the disturbance of the monitored object for each area, thereby notifying the partial display disorder of the wide product shelf, for example. can do. In addition, when multiple products are displayed, the correspondence between the display range for each product type and the disordered display area identifies the products that are often disturbed, that is, the shoppers are highly interested and can be easily obtained. You can also For example, the image processing apparatus 100 may count the number of times it is determined that the monitored object is disturbed for each region, and store the count in association with each region.

  Next, the configuration of the image processing apparatus 200 according to the second embodiment will be described. FIG. 14 is a diagram illustrating the configuration of the image processing apparatus according to the second embodiment. As illustrated in FIG. 14, the image processing apparatus 200 includes a communication unit 205, an imaging unit 210, a storage unit 220, and a control unit 230.

  The communication unit 205 is a processing unit that performs data communication with an external device. For example, the communication unit 205 corresponds to a communication card or a communication device. A control unit 230 described later exchanges data with an external device via the communication unit 205.

  The imaging unit 210 is a device that captures an image in the imaging range. It is assumed that the imaging range is set in advance by the user so as to include monitored objects such as products. The imaging unit 210 outputs the video data to the control unit 230. For example, the imaging unit 210 corresponds to a video camera.

  The storage unit 220 stores corner information 221. The storage unit 220 corresponds to a storage device such as a semiconductor memory element such as a RAM, a ROM, or a flash memory.

  Corner information 221 is corner information extracted from image data. FIG. 15 is a diagram illustrating an example of a data structure of corner information. As illustrated in FIG. 15, the corner information 221 holds detection times, the number of corners, and coordinates in association with each other. The detection time is the time when the corner is detected. The number of corners is the number of detected corners. The coordinates are the coordinates of each detected corner.

  Returning to the description of FIG. The control unit 230 includes a corner detection unit 231 and a determination unit 232. The control unit 230 corresponds to, for example, an integrated device such as an ASIC or FPGA. Moreover, the control part 230 respond | corresponds to electronic circuits, such as CPU and MPU, for example.

  The corner detection unit 231 is a processing unit that extracts image data serving as a still image from the video data output from the imaging unit 210 and detects a corner from the extracted image data. The corner detection unit 231 registers the detected corner information in the corner information 221. That is, the corner detection unit 231 registers the detection time, the number of corners, and the coordinates of each corner in the corner information 221 in association with each other.

  For example, the corner detection unit 231 detects a pattern that can be regarded as a corner from image data using a known technique. For example, the corner detection unit 231 may detect a corner using a Harris detector, or may detect the corner based on a FAST algorithm or the like.

  16 and 17 are diagrams illustrating examples of corner detection results. From the image data 30 shown in FIG. 16, corners 30a, 30b, 30c, and 30d are detected, and the number of corners is four. From the image data 40 shown in FIG. 17, corners 40a, 40b, 40c, 40c, 40d, 40e, 40f, 40g, and 40h are detected, and the number of corners is eight. When the monitored object is disturbed, there is a feature that many corners are detected.

  The determination unit 232 is a processing unit that determines whether the monitored object is disturbed based on the corner information 221. For example, the determination unit 232 calculates, as the reference value, a value obtained by multiplying the number of corners detected at a predetermined time by one or more real numbers. For example, the predetermined time is a time before the store opens. The determination unit 232 determines that the monitored object is disturbed when the number of corners is larger than the reference value, and notifies that the product of the monitored object is disturbed. On the other hand, when the number of corners is equal to or less than the reference value, the determination unit 232 determines that the product of the monitored object is not disturbed.

  Next, a processing procedure of the image processing apparatus 200 according to the second embodiment will be described. FIG. 18 is a flowchart illustrating the processing procedure of the image processing apparatus according to the second embodiment. The process shown in FIG. 18 is repeatedly executed at predetermined time intervals, for example. As shown in FIG. 18, the image processing apparatus 200 acquires image data (step S201) and detects a corner (step S202).

  The image processing apparatus 200 calculates the number of corners (step S203), and determines whether the number of corners is larger than a reference value (step S204). If the number of corners is equal to or less than the reference value (No at Step S204), the image processing apparatus 200 ends the process. On the other hand, when the number of corners is larger than the reference value (step S204, Yes), the image processing apparatus 200 notifies that the product is disturbed (step S205).

  Next, effects of the image processing apparatus 200 according to the second embodiment will be described. The image processing apparatus 200 detects a corner from the image data, and calculates a reference value from the number of corners detected at the first time. The image processing apparatus 200 compares the number of corners detected at the second time after the first time with the reference value, and based on the comparison result, determines whether the product of the monitored object is disturbed. judge. For this reason, according to the image processing apparatus 200, for example, it is possible to determine and notify the disorder of the merchandise display without being affected by the rotation of the merchandise shelf. Further, even when only the direction of the product at the time of flat stacking is deviated, it is possible to determine the display disorder.

  Next, an example of a computer that executes an image processing program that realizes the same function as the image processing apparatus shown in the above embodiment will be described. FIG. 19 is a diagram illustrating an example of a computer that executes an image processing program.

  As illustrated in FIG. 19, the computer 300 includes a CPU 301 that executes various arithmetic processes, an input device 302 that receives input of data from a user, and a display 303. The computer 300 also includes a reading device 304 that reads a program or the like from a storage medium, and an interface device 305 that exchanges data with other computers via a network. The computer 300 also includes a RAM 306 that temporarily stores various types of information and a hard disk device 307. The devices 301 to 307 are connected to the bus 308.

  The hard disk device 307 includes, for example, an extraction program 307a, a classification program 307b, and a determination program 307c. The CPU 301 reads out each program 307 a to 307 c and develops it in the RAM 306.

  The extraction program 307a functions as an extraction process 306a. The classification program 307b functions as a classification process 306b. The determination program 307c functions as a determination process 306c.

  For example, the extraction process 306 a corresponds to the extraction unit 131. The classification process 306 b corresponds to the classification unit 132. The determination process 306c corresponds to the determination unit 133.

  Note that the programs 307a to 307c are not necessarily stored in the hard disk device 307 from the beginning. For example, each program is stored in a “portable physical medium” such as a flexible disk (FD), a CD-ROM, a DVD disk, a magneto-optical disk, and an IC card inserted into the computer 300. Then, the computer 300 may read the programs 307a to 307c from these and execute them. The corner detection unit 231 and the determination unit 232 shown in the second embodiment are also executed by the computer 300. Although a detailed description is omitted, a corner detection program and a determination program are stored in the hard disk device 307 and read out by the CPU 301.

  The following supplementary notes are further disclosed with respect to the embodiments including the above examples.

(Additional remark 1) The extraction part which extracts the outline of the to-be-monitored target object from image data,
A classification unit that classifies each contour extracted by the extraction unit into a plurality of groups based on the direction of each contour;
An image processing apparatus comprising: a determination unit that determines whether or not the monitored object is disturbed based on a classification result of the classification unit.

(Additional remark 2) The said classification | category part classify | categorizes each outline into a some group based on the relative angle of each outline of the said to-be-monitored target object, and the reference | standard line segment, The said determination part, The image processing apparatus according to claim 1, wherein when the number of groups is larger than a threshold value, the monitored object is determined to be disturbed.

(Supplementary Note 3) The determination unit calculates entropy based on a relative angle between each contour of the monitored object and a reference line segment, and when the value of entropy is larger than a threshold value, The image processing apparatus according to appendix 1, wherein it is determined that the monitored object is disturbed.

(Supplementary Note 4) When the determination unit calculates a Gini coefficient based on a relative angle between each contour of the monitored object and a reference line segment, and the value of the Gini coefficient is less than a threshold value, The image processing apparatus according to appendix 1, wherein it is determined that the monitored object is disturbed.

(Additional remark 5) The said classification | category part determines from the connection relation of each outline of the said to-be-monitored target object which corner of the to-be-monitored object, and classifies each outline for every corner, The said determination part has each The image processing apparatus according to appendix 1, wherein the monitored object is determined to be disordered based on whether or not the numbers classified into corners are equal.

(Supplementary Note 6) The image according to any one of Supplementary Notes 1 to 5, wherein the extraction unit removes a contour having a length less than a predetermined length from the contour of the monitored object extracted from the image data. Processing equipment.

(Supplementary Note 7) When the image data photographed at the first time and the image data photographed at the second time have not changed, the extraction unit or the first data captured at the first time The image processing apparatus according to any one of appendices 1 to 6, wherein the contour of the monitored object is extracted from an image taken at two times.

(Supplementary note 8) The extraction unit divides the image data into a plurality of regions, extracts the outline of the monitored object for each of the divided regions, and the determination unit performs the processing for each of the divided regions. The image processing apparatus according to any one of appendices 1 to 7, wherein it is determined whether or not the monitoring target is disturbed.

(Supplementary note 9) a corner detection unit for detecting a corner of a monitored object from image data;
The number of corners detected at the first time by the corner detection unit is compared with the number of corners detected at the second time, and based on the comparison result, is the monitored object disturbed? An image processing apparatus comprising: a determination unit that determines whether or not.

(Supplementary Note 10) An image processing method executed by a computer,
Extract the contour of the monitored object from the image data,
Classify each extracted contour into multiple groups based on the direction of each contour,
An image processing method comprising: executing each process of determining whether or not the monitored object is disturbed based on a classification result.

(Additional remark 11) The said classification | category process classify | categorizes each outline into a some group based on the relative angle of each outline of the said to-be-monitored target object, and a reference | standard line segment, and the said determination process The image processing method according to claim 10, wherein when the number of groups is larger than a threshold value, it is determined that the monitored object is disturbed.

(Supplementary note 12) The determination process calculates entropy based on a relative angle between each contour of the monitored object and a reference line segment, and when the value of entropy is larger than a threshold value, The image processing method according to appendix 10, wherein it is determined that the monitored object is disturbed.

(Supplementary note 13) The determination process is performed when a Gini coefficient is calculated based on a relative angle between each contour of the monitored object and a reference line segment, and the value of the Gini coefficient is less than a threshold value. The image processing method according to appendix 10, wherein it is determined that the monitored object is disturbed.

(Additional remark 14) The said classification | category process determines the corner of a to-be-monitored target from the connection relation of each outline of the to-be-monitored target, classifies each outline for every corner, The image processing method according to appendix 10, wherein it is determined that the monitored object is disturbed based on whether or not the number classified into each corner is equal.

(Additional remark 15) The said process to extract removes the outline less than predetermined length among the outlines of the to-be-monitored target object extracted from the said image data, The additional description 10 to 14 characterized by the above-mentioned. Image processing method.

(Additional remark 16) When the image data image | photographed at the 1st time and the image data image | photographed at the 2nd time are not changing, the process to extract is the image data image | photographed at the said 1st time, or the said data The image processing method according to any one of appendices 10 to 15, wherein the contour of the monitored object is extracted from an image photographed at the second time.

(Supplementary Note 17) The process of extracting divides the image data into a plurality of areas, extracts the outline of the monitored object for each of the divided areas, and the process of determining the process for each of the divided areas. The image processing method according to any one of appendices 10 to 16, wherein it is determined whether or not the monitored object is disturbed.

(Supplementary note 18) An image processing method executed by a computer,
Detect corners of monitored objects from image data,
Each process of comparing the number of corners detected at the first time with the number of corners detected at the second time, and determining whether the monitored object is disturbed based on the comparison result An image processing method characterized in that

(Supplementary note 19)
Extract the contour of the monitored object from the image data,
Classify each extracted contour into multiple groups based on the direction of each contour,
An image processing program for executing each process for determining whether or not the monitored object is disturbed based on a classification result.

(Additional remark 20) The process to classify | categorize each outline into a some group based on the relative angle of each outline of the to-be-monitored target object, and a reference line segment, and the process to determine The image processing program according to appendix 19, wherein when the number of groups is greater than a threshold, it is determined that the monitored object is disturbed.

(Supplementary note 21) In the determination process, entropy is calculated based on a relative angle between each contour of the monitored object and a reference line segment, and when the value of entropy is larger than a threshold value, The image processing program according to appendix 19, wherein it is determined that the monitored object is disturbed.

(Supplementary Note 22) The determination process is performed when a Gini coefficient is calculated based on a relative angle between each contour of the monitored object and a reference line segment, and the value of the Gini coefficient is less than a threshold value. The image processing program according to appendix 19, wherein it is determined that the monitored object is disturbed.

(Additional remark 23) The said classification | category process determines the corner of a to-be-monitored target from the connection relation of each outline of the to-be-monitored target, classifies each outline for every corner, The image processing program according to appendix 19, wherein the monitored object is determined to be disordered based on whether or not the numbers classified into each corner are equal.

(Additional remark 24) The said process to extract removes the outline less than predetermined length among the outlines of the to-be-monitored target extracted from the said image data, The additional description 19 to 23 characterized by the above-mentioned Image processing program.

(Additional remark 25) The process to extract is the image data photographed at the first time when the image data photographed at the first time and the image data photographed at the second time are not changed, or 25. The image processing program according to any one of appendices 19 to 24, wherein the contour of the monitored object is extracted from an image photographed at the second time.

(Supplementary Note 26) The extracting process divides the image data into a plurality of areas, extracts the contour of the monitored object for each divided area, and the determining process is performed for each of the divided areas. The image processing program according to any one of appendices 19 to 25, wherein it is determined whether or not the monitored object is disturbed.

(Supplementary note 27)
Detect corners of monitored objects from image data,
Each process of comparing the number of corners detected at the first time with the number of corners detected at the second time, and determining whether the monitored object is disturbed based on the comparison result An image processing program for executing

DESCRIPTION OF SYMBOLS 100 Image processing apparatus 105 Communication part 110 Imaging part 120 Storage part 121 Edge information 122 Frequency distribution information 130 Control part 131 Extraction part 132 Classification part 133 Determination part

Claims (6)

An extraction unit for extracting the contour of the monitored object from the image data;
From the connection relationship of each contour of the monitored object, it is determined which corner of the monitored object, and a classification unit that classifies each contour for each corner;
An image processing apparatus comprising: a determination unit that determines that the display of the monitored object is disturbed based on whether or not the number classified into each corner is equal . The extraction unit captures the image data captured at the first time or the second time when the image data captured at the first time and the image data captured at the second time have not changed. The image processing apparatus according to claim 1, wherein an outline of the monitored object is extracted from the obtained image data. The extraction unit is configured to divide the image data into a plurality of regions, said extracting a contour of the monitored object in each divided region, the determination unit, for each of the divided regions, the monitored object the image processing apparatus according to claim 1 or 2 of the display and judging whether or not disturbed. An extraction unit for extracting the contour of the monitored object from the image data;
A corner detection unit for detecting a corner of the monitored object from image data;
A classification unit that classifies each contour extracted by the extraction unit into a plurality of groups based on the direction of each contour;
Based on the classification result of the classification unit, the process for determining whether or not the display of the monitored object is disturbed, the number of corners detected at the first time by the corner detection unit, A determination unit that compares the number of corners detected at time 2 and determines whether the display of the monitored object is disturbed based on the comparison result;
An image processing apparatus comprising: An image processing method executed by a computer,
Extract the contour of the monitored object from the image data,
From the connection relationship of each contour of the monitored object, determine which corner of the monitored object, and classify each contour for each corner,
An image processing method, comprising: executing each process for determining that the display of the monitored object is disturbed based on whether or not the number classified into each corner is equal . On the computer,
Extract the contour of the monitored object from the image data,
From the connection relationship of each contour of the monitored object, determine which corner of the monitored object, and classify each contour for each corner,
An image processing program for executing each process for determining that the display of the monitored object is disturbed based on whether or not the number classified into each corner is equal .

Images