KR20110010300A - How to get 3D shape information of panel - Google Patents

Abstract

The present invention relates to a method of obtaining three-dimensional shape information of a panel. The present invention comprises a step of irradiating a laser of the line (line) form on the panel; Detecting the irradiated laser beam through a visual sensor; A correction step of calculating two-dimensional coordinates by using the distorted coordinates sensed by the visual sensor; And converting the two-dimensional coordinates into three-dimensional coordinates representing a surface shape of the actual panel.

Therefore, according to the present invention, radiation distortion can be corrected, and two-dimensional image information can be converted into three-dimensional image information to obtain three-dimensional shape information of the panel.

Laser, Vision Sensor, Calibration, Geometry, Detection, Distortion, Polarization, Camera

Description

Method for getting 3-dimensional shape information of a panel}

The present invention relates to a method of obtaining three-dimensional shape information of a panel, and more particularly, to a method of obtaining three-dimensional shape information of a surface of a panel using image information obtained by using a laser.

With the rapid development of the manufacturing industry, automation has been applied to many production sites. Among them, welding has played a pivotal role in improving productivity and precision in major export industries such as automobiles, shipbuilding, and industrial machinery. In welding, robots have been used for a long time. The essential task of operating the welding robot in the welding operation is to identify the position and shape of the welding base material and to determine how to approach the welding robot to the base material. For this purpose, welding line tracking or robot control could be performed by using CAD information or by using various contact and non-contact sensors.

  Recently, many of these sensors use a visual sensor. Since the sight line is non-contact and lightweight, it does not have to worry about abrasion or impact and can be easily attached to the robot. However, visual sensors need attention to be applied to poor welding sites where heat or sparks are high due to the sensitive nature of lighting and environment. Furthermore, the visual sensor has a problem in that precision and calculation errors are large.

In particular, the radial distortion caused by the visual sensor is distorted by a nonlinear function according to the distance from the center of the visual sensor. If the information on the three-dimensional shape is obtained without correcting the distortion, a significant error occurs. There is a problem.

On the other hand, there is a method using two cameras, but due to the characteristics of the panel, it is difficult to find a corresponding point in the two images.

The present invention has been made to solve the above problems, the present invention is to provide a method for obtaining three-dimensional shape information of the panel that can accurately measure the profile of the surface of the panel using a visual sensor.

In addition, the present invention provides a method for obtaining three-dimensional shape information of a panel capable of correcting the distorted two-dimensional coordinates obtained by the visual sensor to an ideal two-dimensional coordinates and converting them into three-dimensional coordinates representing the shape of the panel. .

를 이용해서 2차원 좌표인

를 산출하는 보정단계; 및 상기 2차원 좌표를 실제 패널의 표면 형상을 표현하는 3차원 좌표인

로 변환하는 변환단계;를 포함하는 패널의 3차원 형상 정보를 얻는 방법을 제공한다.In order to achieve the above object, the present invention comprises a irradiation step of irradiating a line (line) laser to the panel; Detecting the irradiated laser beam through a visual sensor; Distorted coordinates detected by the visual sensor

Using two-dimensional coordinates

A correction step of calculating; And the two-dimensional coordinates are three-dimensional coordinates representing the surface shape of the actual panel.

It provides a method for obtaining the three-dimensional shape information of the panel comprising a conversion step of converting.

In particular, the laser is preferably irradiated in the normal direction of the panel.

In the correcting step, the distorted coordinates and the two-dimensional coordinates may have a relationship of Equation 1.

[Equation 1]

, 여기서,

은 카메라 중심

로부터의 거리로

의 관계가 있고,

, here,

Silver camera center

From distance

Has a relationship with

이다.Distortion function

to be.

과

는 차이(Tsai) 격자를 이용해서 테스트 점을 찾고, 비선형 최소 자승 최적화방법을 이용해서 반복적으로 계산해서 산출되는 것이 가능하다.Furthermore, the camera center which is a variable in the correction step

and

Can be calculated by finding the test point using the Tsai grid and iteratively calculating it using the nonlinear least-squares optimization method.

와 상기 3차원 좌표인

는 수학식 2의 관계로 이루어질 수 있다.Meanwhile, the two-dimensional coordinates in the conversion step

And the 3D coordinates

May be made in relation to Equation 2.

[Equation 2]

Further, the variables a to l in Equation 2 can be calculated using vertex information obtained by irradiating a laser to a calibration block in which left and right steps are formed.

When using a low-cost lens and a camera as a visual sensor, the radiation distortion may be increased at a short focal length. According to the present invention, the radial distortion may be prevented, thereby obtaining an incorrect profile information of the panel. have.

 In addition, the present invention can be implemented by deriving an algorithm for the correction step of eliminating the radiation distortion phenomenon and the conversion step for converting the two-dimensional coordinates obtained by the visual sensor to the actual three-dimensional coordinates.

By iterative calculation using nonlinear least-squares optimization method to derive coefficients to remove the radiation distortion, the initial value can be determined when the approximate range is known, and the converged results can be obtained faster. . Since the Tsai grid uses points that must be on a straight line, the radiation distortion can be corrected without knowing the exact dimensional information of the Tsai grid.

Homogeneous Transformation, which is a linear matrix used to obtain three-dimensional shape information of a panel, is widely used for image transformation. One equation can be used for projection, translation, rotation, and reduction of images. dilation) and the like.

Hereinafter, with reference to the accompanying drawings, preferred embodiments of the present invention that can specifically realize the above object will be described.

In the present invention, structured laser light is used to obtain three-dimensional information, and a linear laser is used. In the case of using a linear laser beam, it is easy to use the data in the case of a welding robot moving in a predetermined direction by acquiring a shape existing in one plane of view.

1 is a schematic structural diagram of an apparatus for implementing the present invention. A description with reference to FIG. 1 is as follows.

Line-type laser uses 660nm monochromatic laser light and passes only this wavelength band to minimize interference of external noise. The laser band in which the fan-shaped optical plane formed by the light irradiated by the laser intersects the surface of the panel represents the cross-sectional shape of the panel and is obtained as an image through a camera, that is, a visual sensor. By linearly transforming the image information thus obtained, three-dimensional information on the surface of the panel can be obtained.

On the other hand, the laser irradiated to the panel is preferably irradiated in the normal direction of the panel.

2 is a schematic flowchart of the present invention. Hereinafter, a process of correcting and converting a coordinate value will be briefly described with reference to FIG. 2.

First, a line laser is irradiated to the panel (S10). In Figure 1 it can be seen that the laser shown on the upper surface of the panel forms a line.

In order to reduce the image processing error sensed by the camera, various filters such as a bandpass filter that passes only the irradiated laser and a polarization filter for removing diffuse reflection are applied. Although not clearly shown in FIG. 1, the filter is preferably installed in the light receiving portion of the camera. 8 to 10 show the principle and effect of the filter. In order to reduce noise during image processing, a bandpass filter or a polarization filter that passes only the laser light (wavelength 635 nm) irradiated to the front of the camera lens is applied. The polarizing filter is often used to reduce this component, paying attention to polarization when light is reflected on a reflective surface such as glass as shown in FIG. 9. It is used here to effectively remove laser light when a diffused reflection occurs in a metallic material. As shown in FIG. 10, it can be seen that the laser image can be better extracted after the polarization filter is applied.

Then, the irradiated laser is detected through the camera which is a visual sensor (S20).

를 이상적인 2차원 좌표

로 보정해서 산출한다(S30). 이때 보정시에 필요한 데이터는 차이(Tsai) 격자를 이용해서 얻는 것이 가능하다.Then, the distorted coordinates detected by the visual sensor

Ideal two-dimensional coordinates

It calculates by adjusting to (S30). At this time, data necessary for correction can be obtained by using a Tsai grid.

로 변환한다(S40). 이때 변환시에 필요한 데이터는 단차가 형성된 교정블록을 이용한다.Finally, since the information to be obtained through the present invention is a three-dimensional profile of the panel, the two-dimensional coordinates are converted into three-dimensional coordinates.

Convert to (S40). At this time, the data required for the conversion uses a calibration block having a step.

3 to 5 show Tsai gratings. In particular, 3 is a distorted image measured by the visual sensor. The closer the focal length from the visual sensor to the panel, the more severe the radiation distortion.

4 is a diagram illustrating a method of finding vertices of a grid using a Harris corner extractor. Hereinafter, a method of correcting distorted coordinates to ideal two-dimensional coordinates will be described with reference to FIG. 4.

The correction method proposed by R.Y.Tsai is R.Y. Tsai, "A versatile camera calibration technique for high accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses", IEEE J. Robotics Automat, Vol. 3, P323-P344, August 1987 and others.

과 왜곡 함수의 계수를

반복적으로 구할 수 있다.If you use the Tsai grid, you will find more test points than the number of variables you need to find. Using the coefficients obtained for all the test points, the coefficients are determined so that the sum of the distances between the shifted coordinates and the coordinates of the original difference grid is the minimum when transformed. This method is called nonlinear least-squares optimization and is also called Levenberg-Marquardt method. In other words, the camera center by the method described above

And the coefficients of the distortion function

Can be obtained repeatedly.

, 여기서,

은 카메라 중심

로부터의 거리로

이고, 왜곡함수는

이다.At this time

, here,

Silver camera center

From distance

And the distortion function

to be.

On the other hand, when using Harris's corner extractor to automatically find the vertices of the grid and transform the points that should be on a straight line, the sum of the distances away from the straight line is used as an optimization function to distort the camera center and distortion. The coefficient of the function can be found. The corrected image can be derived as shown in FIG. 5 based on the coefficient and the camera center value thus obtained.

Meanwhile, as shown in FIG. 4, four groups of up, down, left, and right corners positioned at the edge of the screen may be designated using a mouse, and the distortion may be found by automatically performing this process. As a result, the following results can be obtained.

FIG. 6 is a diagram illustrating a calibration block, and FIG. 7 is a diagram illustrating a screen of a program driven by an algorithm of a conversion method according to the present invention. A description with reference to FIGS. 6 and 7 is as follows.

After the correction of the radiation distortion, the conversion to the three-dimensional shape requires a camera calibration step, that is, a conversion step. As a result, two-dimensional coordinates, which are ideal coordinates, are converted into a three-dimensional coordinate system to obtain three-dimensional shape information of the panel.

Since the camera image is a projection of a three-dimensional object onto a two-dimensional plane, it becomes ambiguous at what point on the ray toward the projection center is actually formed. However, since the present invention knows that the point is on the plane of the structured laser light, the point can be uniquely determined.

In the transformation step of the present invention, a method using a homogeneous coordinate is added by adding one dimension to the actual dimension. At this time, the point O (u, v, w) in the object space and the point I (x, y) on the image plane corresponding thereto have a linear relationship T as follows.

Homogeneous Transformation, a linear matrix used to obtain three-dimensional shape information of a panel, is widely used for image transformation, and the projection, translation, rotation, and reduction ( dilation) and the like.

In the linear transformation T, there are 12 unknowns from a to l. If one is fixed, there are 11 unknowns. When one corresponding point is known, three equations appear. If the corresponding point of at least 4 points is known, all unknowns can be obtained. When the corresponding points are given, the equations obtained are summarized for the unknown, and the least square root is obtained using the pseudo inverse. In this case, a sufficient number of matching points can be given to reduce the effect of the error. This method is called DLT (Direct Linear Transformation).

On the other hand, it is preferable to use the calibration block shown in Figure 6 to find the corresponding point. This block can be manufactured to have a step of 3mm left and right and 2mm above and below. After acquiring the laser irradiation image of the calibration block, the vertices are extracted, the actual coordinates are given to each vertex, and the transformation matrix T is obtained by using the points acquired through the image. Once this matrix is obtained, the measured two-dimensional coordinates can be converted into three-dimensional coordinates, and finally, information on the actual shape of the panel can be obtained.

As illustrated in FIG. 7, a camera calibration operation may be performed using 13 corresponding points (four or more required points) among vertices through a program using the above-described conversion method as an algorithm. The root mean squared error (RSME) was found to be 0.302mm when the error between the coordinates calculated for the calibration block and the actual coordinates was calculated using the obtained variable values. This is an average of the results of 10 runs.

In this way, the overall surface profile can be obtained by integrating the obtained cross-sectional information of the panel while moving the robot to which the visual sensor is attached. In particular, normal information and gap information necessary for robot control can be obtained by calculation from the obtained shape profile.

The present invention is not limited to the above-described embodiments, and as can be seen in the appended claims, modifications can be made by those skilled in the art to which the invention pertains, and such modifications are within the scope of the present invention.

1 is a schematic structural diagram of an apparatus for implementing the present invention.

2 is a schematic flowchart of the invention.

3 to 5 show a Tsai grating, FIG. 3 is a distorted image measured by a visual sensor, and FIG. 4 is a view showing a vertex of a grating using a Harris corner extractor. 5 shows the result of correcting a distorted image.

6 shows a calibration block;

7 is a diagram showing a part of a screen of a program using an algorithm as a conversion method according to the present invention;

8A-8C show the effect of a band pass filter;

9 shows the principle of a polarizing filter and a polarizing filter.

10 is a diagram showing the results before and after applying a polarizing filter.

Claims (6)

An irradiation step of irradiating a line-shaped laser to the panel; Detecting the irradiated laser beam through a visual sensor; Distorted coordinates detected by the visual sensor

Using two-dimensional coordinates

A correction step of calculating; And The two-dimensional coordinates are three-dimensional coordinates representing the surface shape of the actual panel

A conversion step of converting the information to a panel; The method of claim 1, The laser is irradiated in the normal direction of the panel, A method for obtaining three-dimensional shape information of a panel, characterized in that it detects only the irradiated laser by applying one of a band pass filter or a polarization filter for removing diffuse reflection. The method of claim 2, And in the correcting step, the distorted coordinates and the two-dimensional coordinates have a relationship of Equation 1.

, Where is relative to the distance from the camera center, Distortion function

to be. The method of claim 3, Camera center which is a variable in the correction step

and

The method for obtaining three-dimensional shape information of a panel, characterized in that it is calculated by finding a test point using a Tsai grid and iteratively calculated using a nonlinear least-squares optimization method. The method of claim 2, The two-dimensional coordinates in the conversion step

And the 3D coordinates

Is a relationship of Equation 2, and the method of obtaining three-dimensional shape information of the panel, characterized in that using a linear matrix homogeneous transformation.

The method of claim 5, The variables (a to l) of Equation (2) are calculated using vertex information obtained by irradiating a laser to a calibration block in which left and right steps are formed, wherein the three-dimensional shape information of the panel is obtained.

Images