JP2007140729A - Method and apparatus for detecting the position and orientation of an article - Google Patents

Abstract

A method and apparatus for detecting the position and orientation of an article that can accurately and reliably detect the position and orientation of a target article from an image and that can perform high-speed processing applicable to an FA or the like.
An image 3 in which a plurality of identical articles 1 are arranged on the same plane is captured, and the position and posture 9 of the article are detected from the images. Template generation processing S1 for generating a template 5 indicating the outline of an article on the image from the shape data 4 of the article, edge enhancement processing S2 for generating a binary edge image 6 in which the shape of the article is emphasized from the image, and a template A rough search process S3 for creating article candidate data 7 based on vector correlation of edge images, a line segment approximation process S4 for creating line segment data 8 constituting an edge from an edge image, and a line segment from article candidate data and line segment data It consists of a fine search process S5 for determining the exact position and posture 9 of the article by searching.
[Selection] Figure 2

Description

  The present invention relates to a method and apparatus for detecting the position and orientation of an article.

As means for detecting the position and posture of an article on an image, template matching, a region division method, and the like are known.
As schematically shown in FIG. 6, the template matching is performed by sequentially shifting the local areas Sx, y having the same size as the template starting from the position (x, y), while sequentially shifting the template image Gi, j and the search image Fi, The correlation value Mx, y of j is calculated, and the position (x, y) with the highest correlation value is output as the position where the template is found.
The area division method estimates an article position from a feature for each area by dividing an image into areas.

Furthermore, various image processing methods have already been proposed in connection with template matching and region division methods (for example, Patent Documents 1 to 5).
The “image processing apparatus” of Patent Document 1 performs matching using the characteristics of each area after dividing the area.
Similarly to Patent Document 1, the means of Patent Document 2 also performs matching with the features of each region after region division.
The “calculation method and storage medium based on image recognition” of Patent Document 3 relates to a rotation search in which a template is rotated around the center of gravity after the center of gravity of an object is roughly obtained.
Patent Document 4 “Article Position Detection Method and Device” prepares a two-dimensional appearance model of an article and detects a three-dimensional position by matching the feature portion with each stereo image. Is.
The “image processing apparatus” in Patent Document 5 performs template matching on a ternary image using a rotating coarse template, and then performs a precise search using a partial template.

Japanese Patent Publication No. 07-066440, “Image processing device” JP-A-11-203461 Japanese Patent Application Laid-Open No. 11-167640, “Calculation Method by Image Recognition and Storage Medium” Japanese Patent Application Laid-Open No. 08-136220, “Article Position Detection Method and Apparatus” Japanese Patent No. 3212777, “Image Processing Device”

The template matching described above takes a long processing time, particularly when the posture of the article on the screen is unknown. For this reason, it is difficult to apply to fields where high-speed processing is required, such as FA.
In addition, although the area dividing unit is relatively fast in estimating the article position, there is a problem that an appropriate result cannot be obtained if the area dividing unit fails in the area dividing itself.

  The present invention has been made to solve the various problems described above. That is, an object of the present invention is to provide a method and apparatus for detecting the position and orientation of an article that can accurately and reliably detect the position and orientation of the target article from an image, and that can perform high-speed processing applicable to an FA or the like. There is to do.

According to the present invention, a method of capturing an image in which a plurality of identical articles are arranged on the same plane and detecting the position and orientation of the article from the images,
A template generation process for generating a template indicating the outline of the article on the image from the shape data of the article;
Edge enhancement processing for creating a binary edge image in which the shape of the article is enhanced from the image;
A rough search process for creating article candidate data by vector correlation between the template and the edge image;
Line segment approximation processing for creating line segment data constituting an edge from the edge image;
There is provided a method for detecting the position and orientation of an article, comprising: a precise search process for determining an accurate position and orientation of an article by line search from the article candidate data and line segment data.

  According to a preferred embodiment of the present invention, the template is a closed fold line or curve showing an outline of the same scale as the image based on the shape data of the article, and a plurality of outlines rotated about the centroid. Consists of

  In the coarse search process, a vector correlation value based on a template is obtained from the edge image, and a position and orientation on the image in which the vector correlation value is greater than a predetermined threshold value are used as article candidate data.

  In the precise search process, the accurate position and orientation of the article are accurately detected by line segment search from line segment data located in the vicinity of the article candidate data.

In addition, according to the present invention, an apparatus that captures an image in which a plurality of identical articles are arranged on the same plane and detects the position and orientation of the article from the images,
An imaging device that captures the image; and an image processing device that performs image processing on the image;
The image processing apparatus includes:
A template generation process for generating a template indicating the outline of the article on the image from the shape data of the article;
Edge enhancement processing for creating a binary edge image in which the shape of the article is enhanced from the image;
A rough search process for creating article candidate data by vector correlation between the template and the edge image;
Line segment approximation processing for creating line segment data constituting an edge from the edge image;
An apparatus for detecting the position and orientation of an article is provided, which performs a precise search process for determining an accurate position and orientation of an article by line segment search from the article candidate data and line segment data. .

  According to the method and apparatus of the present invention, since a template indicating the outline of an article on an image is generated from the article shape data by template generation processing, the template can be automatically created from very simple shape data. Thereby, it can respond to many articles | goods, without putting the burden of template registration to a user.

  In addition, the rough search process for creating article candidate data based on the vector correlation between the template and the edge image allows the entire screen to be searched with a small amount of processing, thereby enabling a high-speed search without oversight.

By adding a line search process to the vicinity of the article candidate position obtained as a result of the coarse search, a precise search process that determines the exact position and orientation of the article by line search from the article candidate data and line segment data. Can accurately detect the position and posture.
Therefore, an image processing apparatus for extracting an article at high speed and accurately can be realized by the means of the present invention.

  Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings. In addition, the same code | symbol is attached | subjected to the common part in each figure, and the overlapping description is abbreviate | omitted.

  FIG. 1 is an overall configuration diagram of an apparatus according to the present invention. As shown in the figure, the apparatus of the present invention includes an imaging device 12 that captures an image 3 in which a plurality of identical articles 1 are arranged on the same plane, and an image processing device 14 that performs image processing on the captured image 3. .

  The article 1 is laminated on the pallet 2 in this example. The apparatus of the present invention is intended for only the uppermost article among the articles on the pallet, and detects the position and orientation of the uppermost article from the image 3.

The imaging device 12 is, for example, a digital still camera or a digital video camera that can acquire a digital image. Note that these cameras have a resolution capable of imaging at least all the uppermost articles and identifying each article.
The imaging device 12 may be a stereo camera with a certain interval so that the distance to the article can be detected. Note that the present invention does not particularly limit the method for distinguishing the uppermost article from other articles.

  In this example, the image processing apparatus 14 is an image processing PC (computer), and is connected to the host computer 15 and the case information database 16 via a LAN. The case information database 16 is preliminarily input with article shape data.

  The image processing device 14 erases images other than the uppermost article from the image taken by the imaging device 12 on the basis of the distance to the article or the like. Next, the image processing apparatus 14 of the present invention executes template generation processing, edge enhancement processing, rough search processing, line segment approximation processing, and fine search processing, which will be described later.

  FIG. 2 is an overall flow diagram of the method of the present invention, and FIG. 3 is a schematic diagram of the method of the present invention. The method of the present invention is a method of capturing an image 3 in which a plurality of identical articles 1 are arranged on the same plane, and detecting the position and orientation of the article from this image 3. The image 3 of the article as shown in FIG. 3A becomes an image as shown in FIG. 3B by extracting the uppermost article and its vicinity.

  In FIG. 2, the method of the present invention comprises a template generation process S1, an edge enhancement process S2, a rough search process S3, a line segment approximation process S4, and a fine search process S5.

In the template generation process S1, a template 5 (see FIG. 3D) indicating the outline of the article on the image 3 is generated from the shape data 4 of the article 1.
In the edge enhancement processing S2, a binary edge image 6 (see FIG. 3C) in which the shape of the article is enhanced is created from the image 3.
In the coarse search process S3, the article candidate data 7 is created by the vector correlation between the template 5 and the edge image 6.
In the line segment approximation process S4, line segment data 8 constituting an edge is created from the edge image 6.
In the precise search process S5, as shown in FIG. 3E, an accurate position and orientation 9 of the article are determined by line segment search from the article candidate data 7 and the line segment data 8.

FIG. 4 is an explanatory diagram of vector correlation in the present invention.
Vector correlation is one of the techniques for searching for a position of a geometric shape similar to a template from a binary image (edge image). The outline of the algorithm is as follows.
(1) Create a vector starting from the contour position of the template image to be searched (for example, FIG. 4A) and ending at one point (point C in FIG. 4A) such as the center of gravity of the figure (FIG. 4B).
(2) Prepare a voting image having the same size as the edge image to be searched. The voting image has a pixel value that is an integer, and is used for cumulatively storing the results of voting operations described later. The initial value of the voting image is set to 0.
(3) On the edge image (point P in FIG. 4C) in the edge image to be searched, all the vectors are arranged with the point P as the start point of the plurality of vectors created in (1) ( FIG. 4C).
(4) The end point position of the arranged vector is set as a voting position, and voting is performed on the pixels on the voting image corresponding to the position (1 is added to the pixel value).
(5) The operations (3) to (4) are performed for all edge positions on the edge image to be searched.
(6) The pixel position having the largest value (the largest number of voted water jets) in the voting map after the voting ends corresponds to the position of one point such as the center of gravity given in (1) of the template image.
(7) When there are a plurality of search target figures on the edge image to be searched, a plurality of peak portions with a large number of votes are generated. They correspond to the article candidate positions.

  In the coarse search process S3, the position and orientation on the image in which the vector correlation value is larger than a predetermined threshold value are set as the article candidate data 7. For example, in the above example, when the total sum S is 8 or close to 8, the figure on the template 5 and the image 3 are almost the same.

By scanning the above-described rough search process S3 for an arbitrary point on the image 3 for the entire screen, all the article candidate data that matches the template 5 existing on the image can be detected.
Further, by performing this operation for a plurality of templates shown in FIG. 4A, all the article candidate data in a state where the templates are rotated can be detected.
Further, as described above, in the coarse search process S3, since the vector correlation value is used and matching is performed only for a plurality of points, the entire screen can be searched with a small amount of processing, and thereby, a high-speed that is not overlooked. Search can be performed, and high-speed processing applicable to FA and the like can be performed.

FIG. 5 is an explanatory diagram of line segment search in the present invention.
In line segment approximation processing S4, line segment data 8 constituting an edge is created from the edge image 6.
In the fine search process S5, as shown in FIG. 5, the article candidate data 7 is set as a search reference polygon, and areas having search widths inside and outside thereof are set as search ranges.
Subsequently, a straight line formed by the line segment data 8 is determined for each search range corresponding to each side of the search reference polygon. This straight line is determined by obtaining a straight line (minimum square line) that minimizes the sum of squares of the distance from the end point of each line segment to the straight line. At that time, in order to suppress the influence of the line segment having a short length, weighting by the length of the line segment is performed.
This process is performed for each side of the search reference polygon, and the position of the polygon is precisely determined by combining a plurality of straight lines obtained as a result.
Further, as the certainty factor, (the line segment length projected onto the polygon determined by the fine search) / (the perimeter length of the polygon decided by the fine search) is defined, and this certainty factor is high (for example, 0.75 or more). In this case, it is possible to prevent erroneous detection of an article by determining that the article exists at that position.

  According to the method and apparatus of the present invention described above, the template 5 indicating the outline of the article on the image is generated from the article shape data 4 by the template generation process S1, and therefore the template 5 can be automatically created from very simple shape data. . Thereby, it can respond to many articles | goods, without putting the burden of template registration to a user (supposing tens of thousands or more types).

  Further, the rough search process S3 for creating the article candidate data 7 based on the vector correlation between the template 5 and the edge image 6 allows the entire screen to be searched with a small amount of processing, and thereby a high-speed search without oversight can be performed.

Further, a line search process is performed in the vicinity of the article candidate position obtained as a result of the rough search by the fine search process S5 for determining the accurate position and orientation of the article 1 by the line search from the article candidate data 7 and the line segment data 8. The position and posture of the article can be detected accurately.
Therefore, an image processing apparatus for extracting an article at high speed and accurately can be realized by the means of the present invention.

  In addition, this invention is not limited to the Example and embodiment mentioned above, Of course, it can change variously in the range which does not deviate from the summary of this invention.

It is a whole block diagram of the apparatus of this invention. 1 is an overall flow diagram of a method of the present invention. FIG. 2 is a schematic diagram of the method of the present invention. It is explanatory drawing of the vector correlation in this invention. It is explanatory drawing of the line segment search in this invention. It is explanatory drawing of the conventional template matching.

Explanation of symbols

1 article, 2 palettes, 3 images, 4 shape data, 5 templates,
6 edge image, 7 article candidate data, 8 line segment data, 9 position and orientation,
12 imaging device, 14 image processing device,
15 host computer, 16 case information database

Claims (5)

A method of capturing an image in which a plurality of identical articles are arranged on the same plane, and detecting the position and orientation of the article from the images,
A template generation process for generating a template indicating the outline of the article on the image from the shape data of the article;
Edge enhancement processing for creating a binary edge image in which the shape of the article is enhanced from the image;
A rough search process for creating article candidate data by vector correlation between the template and the edge image;
Line segment approximation processing for creating line segment data constituting an edge from the edge image;
A method for detecting the position and orientation of an article, comprising: a precise search process for determining an accurate position and orientation of an article from the article candidate data and line segment data by line segment search.   The template is a closed fold line or curve showing an outline of the same scale as the image based on the shape data of the article, and is composed of a plurality of outlines rotated about the centroid. A method for detecting the position and orientation of an article according to claim 1.   2. The rough search process, wherein a vector correlation value based on a template is obtained from an edge image, and a position and orientation on an image in which the vector correlation value is larger than a predetermined threshold value are used as article candidate data. Of detecting the position and posture of an article.   2. The article position according to claim 1, wherein in the precision search process, an accurate position and posture of the article are accurately detected by line segment search from line segment data located in the vicinity of the article candidate data. How to detect posture. An apparatus for capturing an image in which a plurality of identical articles are arranged on the same plane and detecting the position and orientation of the article from the images,
An imaging device that captures the image; and an image processing device that performs image processing on the image;
The image processing apparatus includes:
A template generation process for generating a template indicating the outline of the article on the image from the shape data of the article;
Edge enhancement processing for creating a binary edge image in which the shape of the article is enhanced from the image;
A rough search process for creating article candidate data by vector correlation between the template and the edge image;
Line segment approximation processing for creating line segment data constituting an edge from the edge image;
An apparatus for detecting the position and orientation of an article, comprising: executing an accurate search process for determining an accurate position and orientation of an article from the article candidate data and line segment data by a line segment search.

Images