CN109827502B - High-precision calibration method for line-structured light vision sensor for calibration point image compensation - Google Patents

Abstract

The invention relates to a high-precision calibration method of a line structured light vision sensor for calibrating point image compensation, which comprises the following steps: calibrating the internal parameters of a camera in the structured light vision sensor; adopting a plane metal target embedded with LED luminous characteristic points, and shooting a plane target image with light bars by a camera; extracting target characteristic points and coordinates of light bar calibration points; respectively calculating the positioning uncertainty of the target characteristic point coordinates and the light strip calibration points; solving the positioning deviation of all the characteristic points in a nonlinear optimization mode by taking the positioning uncertainty of the characteristic points as constraint; compensating the coordinates of the target characteristic points and the light strip calibration points; moving the target more than twice, acquiring three-dimensional coordinates of all light bar calibration points under the camera coordinates, fitting the three-dimensional coordinate points to solve a light plane equation, and completing calibration; the invention is suitable for the field complex light environment, and can still finish the high-precision calibration of the line structure vision sensor even under the condition of larger image noise or characteristic point positioning deviation.

Description

A high-precision calibration method of line structured light vision sensor with calibration point image compensation

technical field

The invention relates to the technical field of sensor calibration, in particular to a high-precision calibration method for line structured light vision sensors with image compensation of calibration points.

Background technique

As an important means of 3D data acquisition, linear structured light vision sensor has the advantages of large range, non-contact, high speed and high precision, and is being widely used in the field of online dynamic measurement. Such as online dynamic measurement of train wheelset size, online measurement of pantograph wear, and online detection of train body. The above-mentioned on-site measurement environment is complex and diverse, and sensor calibration and measurement are easily affected by factors such as sensor layout and external light, so it is difficult to meet the ideal calibration conditions and the calibration accuracy is low, which becomes a difficult problem restricting the measurement accuracy. At present, the structured light calibration method based on planar target is widely used in field calibration due to its advantages of high precision, low cost, flexibility and convenience. However, its calibration accuracy is still easily affected by the on-site calibration environment, making it difficult for image feature points or light bar calibration points to satisfy the optimal imaging at the same time, and it is easy to generate positioning deviation. However, relying on image processing method can only reduce the positioning deviation, but cannot eliminate it completely, which makes this method unable to realize the calibration in the complex environment of the scene. Therefore, researching a method that is not affected by image noise and can achieve calibration under complex light conditions has become a new direction for the development of structured light vision sensors.

Linear structured light vision sensor calibration includes two parts: camera internal parameter calibration and light plane parameter calibration. For a long time, there have been relatively many studies on the calibration of camera internal parameters, among which there are many studies on the calibration of camera internal parameters, so the focus is on the calibration process of light plane parameters. There are many calibration methods for the light plane equation at present. Dewar R. proposed a wire drawing calibration method in the article "Self-generated targets for spatial calibration of structured lightoptical sectioning sensors with respect to an external coordinate system". Due to the uneven brightness distribution of the bright spots themselves or high-brightness reflections, the measurement equipment is used in the space It is difficult to strictly correspond to the bright spots in the image and the bright spots in the image, so the calibration points obtained by this method are few and the calibration accuracy is low. For at least three collinear points with known coordinates on the three-dimensional target, the coordinates of the intersection of the structured light strip and the straight line where the known three points are located are obtained by using the constant intersection ratio. This method can obtain high-precision calibration points on the light plane, which is suitable for on-site calibration. However, a high-precision three-dimensional target composed of at least two mutually perpendicular planes is required. At the same time, it is difficult to obtain high-quality calibration images due to the mutual occlusion of light between planes, and the number of calibration points is also limited. In the article "novelcalibration method for multi-sensor visual measurement system based on structured light", Liu et al. proposed a method of using the plucker's equation to represent the straight line of the light bar. Compared with the method using fewer calibration points, this method can effectively improve the calibration accuracy. In the article "a novel 1D target-based calibration method with unknown orientation for structured light vision sensor", Wei et al. proposed a one-dimensional target-based structured light vision sensor calibration method. The three-dimensional coordinates of the intersection of the light plane and the one-dimensional target are solved by using the distance between the feature points of the one-dimensional target. The light plane equation is obtained by fitting multiple intersection points. In recent years, spatial geometry has been used as an auxiliary constraint for the calibration of structured light vision sensors with filters in complex on-site environments. In the article "calibration method for line-structured light vision sensor based a single ball target", Liu et al. proposed a calibration method for structured light vision sensor based on ball target. This method needs to extract the outer contour edge of the ball target, and then obtain the orientation of the ball target in the camera coordinate system. Combined with the intersection of the light plane and the spherical target, the conic contour is obtained to solve the light plane equation. In this method, the obtained contour of the spherical target is not affected by the placement angle of the target, but the outer contour of the target needs to be extracted, which is easily disturbed by the background or the outside world. And in "An on-site calibration of line-structured light vision in complex light environment", it is proposed to use parallel double cylindrical targets to achieve on-site calibration of structured light parameters when the camera is equipped with filters and the external environment is complex. Using the corresponding relationship between the space parallel ellipse plane and the image plane generated by the intersection of the light plane and the double cylinder, and taking the cylinder radius equal to the length of the short axis of the space ellipse as the constraint, the light plane equation was further optimized. At the same time, some scholars have proposed methods to improve the calibration accuracy. For example, Zhou et al. proposed a method for calibrating light plane parameters based on planar targets in the article "complete calibration of a structured lightstripe vison sensor through planar target of unknown orientations". The calibration point in the light plane is obtained by the invariance of the cross ratio, and the three-dimensional coordinates of the calibration point in the light plane are obtained by repeated movement of the plane target, and the light plane equation is obtained by fitting. This method is widely used in the field of high precision measurement due to its low cost, flexibility and high precision. However, it still cannot get rid of the calibration error caused by the positioning deviation caused by image noise, and cannot achieve higher-precision calibration.

Analyzing the current calibration methods, the extraction of target feature points or light bar calibration points is used as the true value of the actual image point coordinates, and the linear solution is obtained, and then the optimal solution is obtained by minimizing the projection error of the feature point. At the same time, try to use the calibration space as the calibration mode of the measurement space. However, during the on-site calibration, it is inevitable that due to complex light, laser quality, target surface processing roughness, target placement angle, image out-of-focus blur, image noise, characteristic Due to the influence of factors such as the extraction deviation of points and light strips, the calibration accuracy is reduced. When only taking care of the imaging quality of target feature points during on-site calibration, it is easy to cause the brightness of the light bar to decrease or become thick; , cannot meet the best image quality at the same time. Therefore, it has become an urgent problem to study the high-precision calibration method of structured light vision sensor under complex outdoor conditions, which is universal and not disturbed by image noise.

SUMMARY OF THE INVENTION

The technical solution of the present invention is to overcome the deficiencies of the prior art and provide a high-precision calibration method for a line structured light vision sensor with image compensation of calibration points, which can especially be blurred, out-of-focus and noise images in the case of complex light environment on site. High-precision calibration in the case of interference.

In order to achieve the above object, the technical scheme of the present invention is achieved in this way:

A high-precision calibration method for a line structured light vision sensor with calibration point image compensation, the method includes:

a. Calibrate the camera in the linear structured light vision sensor without turning on the laser;

b. Use a flat metal target inlaid with LED light-emitting feature points, and the camera shoots a flat target image with a light bar; extract the target feature point and the coordinates of the light bar calibration point;

c. Calculate the target feature point coordinates and the positioning uncertainty of the light bar calibration point respectively. Constrained by the positioning uncertainty of feature points, the positioning deviation of all feature points is solved by nonlinear optimization and compensated to obtain accurate feature point coordinates;

d. Move the target more than two times to obtain the three-dimensional coordinates of all the calibration points of the light bar under the camera coordinates, and then fit and solve the light plane equation after removing the noise points by RANSAC.

In step a, when the laser is not turned on, the camera in the linear structured light vision sensor is calibrated, and the improved Zhang Zhengyou calibration method proposed by Liu Zhen et al. is used to calibrate the internal parameters of the camera and the second-order radial distortion coefficient of the lens;

In step b, the image of the intersection of the light bar and the plane target is taken, and the image coordinates of the target feature point and the light bar calibration point are respectively extracted, and the method is as follows:

(b1) Adjusting the intersection of the metal target and the space light plane to ensure that the light bar does not pass through the target light-emitting feature points in the space;

(b2) Using the multi-scale light point center coordinate method to extract the image coordinates of the target feature point as the initial value of the target feature point coordinates; Extract the coordinates of the center point of the light bar, fit the list of target feature points perpendicular to the direction of the light bar as a straight line, and solve its The intersection point with the light bar is used as the initial value of the coordinates of the calibration point of the light bar.

In step c, the specific method for solving the positioning uncertainty of the target feature point and the light bar calibration point is as follows:

(c1) using a plurality of Gaussian convolution kernels to process the collected images respectively, solving a plurality of positioning coordinates of each feature point, and statistically obtaining the positioning uncertainty of each feature point;

(c2) Establish a mathematical model of the localization uncertainty of the local image of each light bar calibration point, use the mean filtering method to solve the image noise, and then obtain the positioning uncertainty of each light bar calibration point.

In step c, with the positioning uncertainty of the feature point as the constraint, the positioning deviation between the target feature point and the light bar calibration point is solved by the nonlinear optimization method, and the precise feature point coordinates are obtained after compensation.

The method of final sensor calibration in step d is as follows:

(d1) based on the compensated light bar calibration point coordinates, adopt the plane target calibration method to obtain the three-dimensional coordinates of the light bar calibration point under the camera coordinate system;

(d2) After obtaining the three-dimensional coordinates of all the calibration points of the light bar in the camera coordinates, use the RANSAC method to eliminate the noise points and then fit and solve the initial value of the light plane equation;

(d3) Use the Levenberg-Marquardt nonlinear optimization method to obtain the optimal solution of the light plane equation, and complete the calibration of the linear structured light vision sensor.

The advantages of the present invention compared with the prior art are:

The invention proposes a calibration method of a structured light vision sensor based on an uncertainty model. Firstly, the positioning uncertainty model of target feature points and light bar calibration points is established, and the positioning uncertainty of each feature point is solved. And the minimum distance from the target plane is the constraint to establish the target equation, and the positioning deviation of each feature point is obtained by the nonlinear optimization method; finally, after compensation, the plane target calibration method is substituted to solve the high-precision light plane equation. The invention can effectively make up for the positioning deviation caused by the extraction of target feature points and light bar calibration points, and is especially suitable for solving the problem of degraded calibration accuracy caused by poor imaging quality or out-of-focus, image noise and other influences in complex on-site environments. Calibration accuracy.

Description of drawings

Fig. 1 is the flow chart of the high-precision calibration method of structured light vision sensor based on uncertainty model of the present invention;

Fig. 2 is the schematic diagram of line structured light vision sensor calibration of the present invention;

FIG. 3 is a schematic diagram of the positioning uncertainty of the target feature point and the light bar calibration point according to the present invention.

Detailed ways

The basic idea of the present invention is: extracting the initial coordinates of the target feature point and the calibration point of the light bar, and determining the positioning uncertainty of all the feature points. Taking the positioning uncertainty of feature points as constraints, the positioning deviation of each feature point is obtained by nonlinear optimization. After compensation, the plane target calibration method is substituted again, and the three-dimensional coordinates of the calibration point of the light bar in the camera coordinate system are obtained. The RANSAC method is used to remove the noise, and the accurate solution of the light plane equation is solved by the optimization method, so as to realize the high-precision calibration of the structured light vision sensor.

The present invention will be described in further detail below by taking a line structured light vision sensor composed of a camera and a line laser as an example.

As shown in Figure 1, the high-precision calibration method of structured light vision sensor based on uncertainty model of the present invention mainly comprises the following steps:

Step 11: When the line laser is not turned on, calibrate the camera in the line structured light vision sensor.

The camera of the vision sensor is calibrated here, that is, the internal parameters of the camera are solved. The specific calibration method is in the article of the improved Zhang Zhengyou calibration method proposed by Liu Zhen et al. [Liu Z, Wu Q, Chen X, et al.High-accuracy calibration of low-cost camera using image disturbance factor [J]. Optics Express, 2016, 24(21):24321-24336.] is described in detail.

Step 12: Turn on the laser, place the flat target in front of the camera, so that the light plane projected by the line laser intersects the flat target, and the camera captures the flat target image with the light bar.

为靶标点连线与L_i的交点，用于后续光平面方程求解。p_1j p_2j p_3j分别为Q_1j、Q_2j、Q_3j对 应的图像点，

为

对应的图像点，定义为光条标定点。光平面方程可表示为 ax+by+cz+d＝0，其中

As shown in Figure 2, set O _c x _c y _c z _c as the camera coordinate system, O _t x _t y _t z _t as the target coordinate system, Y _t as the target plane, Y as the laser plane, and _Li as Y and Y The intersection line of _t , _{li is Li imaging, Q 1j , Q 2j , Q 3j represent the three} luminous points on the target row,

is the _intersection of the line connecting the target point and Li, which is used to solve the subsequent light plane equation. p _1j p _2j p _3j are the image points corresponding to Q _1j , Q _2j , and Q _3j respectively,

for

The corresponding image point is defined as the calibration point of the light bar. The light plane equation can be expressed as ax+by+cz+d=0, where

Step 13: Extract the image coordinates of the feature points, that is, the coordinates of the target feature points and the calibration point of the light bar.

Here, it specifically includes the following steps:

Step 131: Extract the image coordinates of the target calibration point in the captured image, and use a multi-scale extraction method (such as the article "Liu Zhen, Shang Yanna. High-precision positioning of the center of multi-scale light spot images [J]. Optical Precision Engineering, 2013, 21(6):1586-1591.”) to obtain the best image coordinates of all target feature points in the image.

The center of the light bar was extracted using the method described by Steger "Steger C. Unbiased Extraction of Curvilinear Structures from 2D and 3D Images [J]. 1998.".

Step 14: Calculate the positioning uncertainty of the target feature point and the light bar calibration point.

与标准差σ_u、σ_v。并将σ_u、σ_v为靶标特征点的定位不确定度。Step 141 : As shown in FIG. 3 , the present invention adopts a multi-scale extraction method to complete the initial value of target feature point location and the solution of uncertainty. The image is processed by multiple different Gaussian convolutions, the center point is extracted multiple times, and the statistical method is used to obtain the positioning uncertainty of the target feature point. At the same time, the coordinate corresponding to the best scale is selected as the initial value of the center coordinate of the feature point. Select m different Gaussian convolution kernels to process j feature points at position i respectively, and obtain m groups of target feature point coordinates pi _j and form a point set P _m , as shown in the red cross point in the feature points in Figure 2 . Then find the average coordinates of the target feature point set in the u, v direction

and standard deviation σ _u , σ _v . And σ _u , σ _v are the positioning uncertainty of the target feature point.

Step 142: During the on-site calibration, affected by the laser quality, power, target surface material and external illumination, the extraction of the center point of the light bar is easily affected by image noise, resulting in a positioning deviation. The invention provides a method for solving the uncertainty of the calibration point of the light bar in any direction. The following is the process of solving the coordinate uncertainty of the calibration point of the light bar.

表示。因此，光条截面曲线可以沿着边缘方向用(n_u,n_v)表 示为，The partial derivatives obtained after convolving the image with the Gaussian kernel are g _u , g _v , g _uu , g _uv , and g _vv , respectively. By calculating the Hessian matrix, the normal vector n(u,v) of the light strip is obtained. Let (u ₀ , v ₀ ) be the coordinates of any point in the image, the edge direction of the light bar is represented by (n _u , n _v ), and ||(n _u , n _v )||=1, the orthogonal direction is represented by

express. Therefore, the light-strip cross-section curve can be expressed as (n _u ,n _v ) along the edge direction as,

可得：For line edges, let

Available:

Therefore, the coordinates of the maximum value point of the image gray level, that is, the center point of the light bar are (p _u ,p _v )=((tn _u +u ₀ ),(tn _v +v ₀ )).

与(n_u,n_v)为坐标轴建立o-rc坐 标系，设(n_u,n_v)方向上灰度曲线为h(c,r)，因此求解(p_u,p_v)的不确定度转化为求解h(c,r)_r＝0处c₀的不确定度。设h(c,r)＝I(c,r)+N(c,r)，其中I(c,r)为理想图像，N(c,r)是均值为零， 方差为

的图像噪声。由[CSteger]可知，h(c₀,r₀)处一阶导数为零，则：Taking the center point (0,0) of the light bar corresponding to the noise-free image as the origin,

The o-rc coordinate system is established with (n _u , n _v ) as the coordinate axis, and the grayscale curve in the (n _u , n _v ) direction is set to be h(c, r), so the invariance of (p _u , p _v ) is solved. The degree of certainty translates into the uncertainty of solving for c ₀ at h(c,r) _r=0 . Let h(c,r)=I(c,r)+N(c,r), where I(c,r) is the ideal image, N(c,r) is zero mean, and the variance is

image noise. It can be known from [CSteger] that the first derivative at h(c ₀ ,r ₀ ) is zero, then:

分别为灰度分布、理想图像分布、图像噪声经过方差为

的高斯核卷积后 的数据，

分别为在(c₀,r₀) 处对c轴方向的一阶、二阶偏导数。对

在(0,0)处进行泰勒展开，in,

Respectively, gray distribution, ideal image distribution, and image noise through variance are

The Gaussian kernel convolved data,

are the first-order and second-order partial derivatives with respect to the c-axis direction at (c ₀ , r ₀ ), respectively. right

Taylor expansion at (0,0),

M为灰度最大值，σ_w为高斯核，且满足

且

求得：Since the ideal image can be expressed as

M is the maximum gray value, σ _w is the Gaussian kernel, and satisfies the

and

Get:

对应的方差

为，

Corresponding variance

for,

为，From equations 8 and 9, the center point positioning variance is obtained

for,

Substituting Equation (11) into Equation (10), the center point coordinate positioning variance is:

Among them, _σw can be obtained by taking the grayscale curve in the n _c direction and fitting it with the fittype() function in matlab, where the prototype of the fitting function is

分解到o-uv坐标系下，可以得出光条标定点点不确定度分量

Will

Decomposed into the o-uv coordinate system, the uncertainty component of the light bar calibration point can be obtained

Step 15: Based on the feature point positioning uncertainty determined in step 14, with the feature point uncertainty as a constraint, and establishing the minimum back-projection error between the target feature point image and the space target, the light bar calibration point is back-projected to the space target The plane and the minimum distance from point to plane are the objective equations, and the positioning deviation of feature points is obtained by nonlinear optimization method.

Step 151: Decompose the imaging process of target feature points, and establish a transformation equation between perspective projection points, distortion points and final noise points in the feature points.

Suppose p _u =[u _u ,v _u ,1] ^T , p _d =[u _d ,v _d ,1] ^T and p _n =[u _n ,v _n ,1] ^T are respectively the no distortion in the image coordinate system Point, distortion point and actual image point coordinates. It can be seen from the imaging optical path that the imaging of the spatial target point Q=[x,y,1] can be decomposed into three processes, the first is the perspective projection model, the second is the lens distortion model, and the third is image noise. Superposition model; where the perspective projection model can be expressed as:

where H _3×3 is the homography matrix between the target plane and the image plane, ρ is the constant coefficient, K is the camera internal parameter matrix, u ₀ and v ₀ are the principal point coordinates. γ is the tilt factor of the uv coordinate axis of the image. r ₁ , r ₂ , and t are the first two columns of the rotation matrix and the translation vector, respectively. The lens distortion model can be expressed as:

k₁，k₂镜头前两阶畸变系数，[x_n,y_n]表示归一化的图像坐标。根据实际 经验，前两阶径向畸变已经足够精确地描述镜头畸变，达到较高精度。设由于图像噪声等原 因导致的图像偏差为Δu,Δv，则畸变点到实际成像点可表示为：in

k ₁ , k ₂ lens front two-order distortion coefficients, [x _n , y _n ] represent normalized image coordinates. According to practical experience, the first two orders of radial distortion are sufficiently accurate to describe the lens distortion to achieve higher accuracy. Assuming that the image deviation due to image noise and other reasons is Δu, Δv, the distortion point to the actual imaging point can be expressed as:

p_ij通过式(14)，(15)计算出p_u(ij)，p_u(ij)与Q_j＝[x_j,y_j,1]^T通过 式13求解H_i矩阵，其中式(15)中Δu_ij,Δv_ij的初值为0。Under the ith placement position of the target, let the homogeneous coordinates of the jth point of the target in the target coordinate system and the image coordinate system be Q _j = [x _j , y _j , 1] ^T and

p _ij calculates p _u(ij) through equations (14) and (15), p _u(ij) and Q _j =[x _j ,y _j ,1] ^T solves the _Hi matrix by equation 13, where equation (15) ), the initial values of Δu _ij and Δv _ij are 0.

Step 152: Establish the minimum back-projection error between the target feature point image and the space target, the light bar calibration point is back-projected to the space target plane and the minimum distance from the point to the plane is the target equation, and the positioning deviation of the feature point is obtained by a nonlinear optimization method.

According to the mapping matrix H _i from the ith position target plane to the image plane, Q _j obtains the homogeneous coordinate p _{n(ij) of the projection point of the jth feature point of the target in the image coordinate system by formula (13)} . The first objective function is established with the minimum distance between p _ij and p _n(ij) and the minimum distance between the image statistical center points as follows:

D(p _ij , p _n(ij) ) represents the distance between p _ij and p _n(ij) , M represents the number of target placement positions, and N represents the number of target feature points.

以q_j与

之 间距离最小和靶标点与投影点统计中心距离最小为目标函数建立第二个目标函数：Calculate the homogeneous coordinates of p _ij back-projected to the point under the target coordinate system by formula (13)

with q _j and

The minimum distance and the minimum distance between the target point and the statistical center of the projection point are used to establish the second objective function as the objective function:

As an important constraint in projective geometry, the intersection ratio invariant constraint can better optimize the coordinates between image feature points with the help of target rigid constraints, and is widely used in camera parameter calibration and other links. The present invention establishes a cross ratio constraint between the target feature point image coordinates and the target, and establishes the third objective function as follows:

where CR _h and CR _v represent the cross ratio in the horizontal and vertical directions, respectively. In order to achieve the strength of the constraint and improve the calculation efficiency at the same time, four points are randomly selected in the horizontal direction to calculate the intersection ratio, and there are K kinds of combinations; The specific value is determined by the number of points, and it is enough to ensure that the target feature points evenly cover the entire image surface.

前向投影与靶标相交于空间三维点

且与p_ij在靶标坐标系下投影点的 齐次坐标

拟合的靶标平面距离最小，以点到平面之间距离最小建立第四个目标函数如下：Set light bar calibration point

The forward projection intersects the target at a three-dimensional point in space

and the homogeneous coordinates of the projected point of p _ij in the target coordinate system

The fitted target plane distance is the smallest, and the fourth objective function is established with the smallest distance between the point and the plane as follows:

表示点到平面之间距离。where F _i represents the plane equation fitted by the target point at the i-th position of the target,

Indicates the distance between the point and the plane.

中第k列拟合直线为

拟合直线为

与

交点为

以

与

之间 距离最小建立第五个目标函数如下：At the same time, set

The fitted straight line in the kth column is

The fitted line is

and

The intersection is

by

and

The minimum distance between the establishment of the fifth objective function is as follows:

Combine the four objective functions to get:

E(a)=e ₁ +e ₂ +e ₃ +e ₄ +e ₅ (21)

加入了优化范围约束，如式22所示：For Δu _ij , Δv _ij ,

An optimization range constraint is added, as shown in Equation 22:

σ_u(ij)和σ_v(ij)为靶标第i个摆放位置处第j 个点在图像中特征点定位不确定度，

为靶标i个位置处第k个光条标定点定位不 确定度。n为非零比例系数，根据大量反复试验，本发明设置为9。in

σ _u(ij) and σ _v(ij) are the positioning uncertainty of the feature point in the image of the jth point at the ith position of the target,

is the positioning uncertainty of the kth light bar calibration point at the i position of the target. n is a non-zero scale factor, which is set to 9 in the present invention based on extensive trial and error.

Step 16: Obtain the positioning deviation of each feature point based on the nonlinear optimization in Step 15, obtain accurate feature point positioning coordinates after compensation, and obtain undistorted coordinates through de-distortion.

Step 17: The homography matrix determined by the target feature points maps the calibration points of the light bar to the three-dimensional space, and obtains a list of three-dimensional coordinate points of the light plane. Use RANSAC to remove the coarse error points, and then use the least squares method to obtain the initial value of the light plane. Finally, the maximum likelihood solution of the light plane is obtained by nonlinear optimization.

为去除畸变后的靶标特征点坐标，Q_ij为靶标特征点坐标，

为靶标平面与图像平 面之间的单应矩阵；

为去除畸变后的光条标定点坐标，

为光条标定点在靶标坐标系点 坐标，

为光平面与靶标交线在相机坐标下三维点坐标。则图像点与靶标点之间满足

其中s为非零系数。根据单应矩阵映射的可逆性，可以得出：Assume

In order to remove the distorted target feature point coordinates, Q _ij is the target feature point coordinates,

is the homography matrix between the target plane and the image plane;

In order to remove the distortion of the light bar calibration point coordinates,

is the point coordinate of the light bar calibration point in the target coordinate system,

The three-dimensional point coordinates in the camera coordinates for the intersection of the light plane and the target. Then the relationship between the image point and the target point satisfies

where s is a non-zero coefficient. According to the invertibility of the homography matrix mapping, we can get:

可分解为旋转矩阵

与平移矢量

则Assume

can be decomposed into a rotation matrix

with translation vector

but

将

代 入RANSAC约束的最小二乘法中，得到光平面方程初解

基于光平面上点到平面距 离最小为约束，可建立如下目标函数，Let the light plane equation be expressed as ax+by+cz+d=0, where,

Will

Substitute into the least square method constrained by RANSAC to obtain the initial solution of the light plane equation

Based on the constraint of the minimum distance from the point to the plane on the light plane, the following objective function can be established:

Among them, a, b, c, d are the four coefficients of the light plane equation, x _ik = [x _ik , y _ik , zi _ik , 1] is the i target position, k represents the intersection of the light plane and the target on the light bar. The k calibration points of . S represents the number of target placements, and M represents the number of calibration points of the light bar obtained at each position. The optimal solutions of a, b, c, d can be obtained by maximum likelihood estimation of nonlinear optimization methods.

Claims (6)

1. A high-precision calibration method for a line structured light vision sensor with calibration point image compensation is characterized by comprising the following implementation steps:

step a, calibrating a camera in a line structure light vision sensor under the condition that a laser is not turned on;

b, adopting a planar metal target embedded with LED luminous characteristic points, and shooting a planar metal target image embedded with the LED luminous characteristic points printed with laser stripes by using a calibrated camera; calculating coordinates of all target LED characteristic points in the image and initial coordinate values of all light bar calibration points;

step c, solving the uncertainty of the target characteristic points and the light strip calibration points, solving the positioning deviation of the target characteristic points and the light strip calibration points by utilizing the uncertainty and compensating to obtain characteristic point coordinates;

and d, moving the plane metal target more than twice, acquiring the three-dimensional coordinates of all light strip calibration points under the camera coordinates, eliminating the miscellaneous points, fitting and solving a light plane equation, and finishing the calibration of the line-structure light vision sensor.

2. The high-precision calibration method for the line structured light vision sensor with calibration point image compensation according to claim 1, characterized in that: and a, calibrating a camera in the line structure light vision sensor under the condition that the laser is not started in the step a, and obtaining the internal parameters of the camera and the second-order radial distortion coefficient of the lens.

3. The high-precision calibration method for the line structured light vision sensor with calibration point image compensation according to claim 1, characterized in that: the method for shooting the image of the intersection of the light strip and the plane target in the step b, calculating the image coordinates of the target characteristic points and calculating the initial values of the image coordinates of the light strip calibration points by using the image coordinates of the target characteristic points comprises the following steps:

(b1) carrying out distortion correction on the shot image by using a second-order distortion coefficient of the lens obtained by calibration;

(b2) adjusting the intersection of the planar metal target and the spatial light plane to ensure that the laser light stripe does not pass through the target characteristic point in the space;

(b3) extracting image coordinates of the target characteristic points by adopting a multi-scale light spot center coordinate method to serve as initial values of the coordinates of the target characteristic points; and extracting coordinates of the central point of the light bar, fitting a point column straight line of the target characteristic point vertical to the direction of the light bar, and solving the intersection point of the point column straight line and the light bar as a coordinate initial value of the light bar calibration point.

4. The high-precision calibration method for the line structured light vision sensor with calibration point image compensation according to claim 1, characterized in that: the specific method for solving the positioning uncertainty of the target characteristic point and the light bar calibration point in the step c is as follows:

(c1) respectively processing the calibration image by utilizing a plurality of Gaussian convolution kernels, respectively solving a plurality of positioning coordinates of each characteristic point, and counting to obtain the positioning uncertainty of each characteristic point;

(c2) and establishing a positioning uncertainty mathematical model of the local image of each light strip calibration point, and solving the image noise to further obtain the positioning uncertainty of each light strip calibration point.

5. The high-precision calibration method for the line structured light vision sensor with calibration point image compensation according to claim 1, characterized in that: c, compensating the coordinates of the target characteristic points and the light bar calibration points based on the characteristic point positioning uncertainty constraint;

(c1) the uncertainty of the target characteristic point and the light strip calibration point is taken as constraint, and the positioning deviation of the target characteristic point and the light strip calibration point is solved through a nonlinear optimization method;

(c2) and compensating the coordinates of the target characteristic points and the light bar calibration points by using the positioning deviation.

6. The high-precision calibration method for the line structured light vision sensor with calibration point image compensation according to claim 1, characterized in that: the calibration method of the line structure optical vision sensor in the step d is as follows:

(d1) based on the compensated coordinates of the light strip calibration points, a plane target calibration method is adopted, and the internal parameters of the camera are obtained by calibration to obtain the three-dimensional coordinates of the light strip calibration points in the coordinate system of the camera;

(d2) after three-dimensional coordinates of all light bar calibration points under the camera coordinates are obtained, removing the impurity points by using a RANSAC method, and then fitting and solving an initial value of a light plane equation;

(d3) and (4) according to the initial value of the optical plane equation, obtaining the optimal solution of the optical plane equation by using a Levenberg-Marquardt nonlinear optimization method, and completing the calibration of the linear structure optical vision sensor.

Images