CN101901502B - A global optimal registration method for multi-view point cloud data in optical 3D measurement - Google Patents

Abstract

The invention relates to a global optimal registration method for multi-view point cloud data in optical three-dimensional measurement, which belongs to the technical field of digital image processing and solves the problem of deviation in the measurement of public marker points in the characteristic marker point measurement method. The data registration process is as follows: register the data to be registered to the target data, obtain the registration result data, and use the registration result data as the target data for the next registration; select the i-th viewpoint as the viewpoint to be registered, and the The two-dimensional measurement data of the object to be measured at the viewpoint is used as the data to be registered, and then the above data registration process is repeated until all viewpoints are used as the viewpoint to be converted to complete the data registration; during the registration process, the beam adjustment method is used to calculate the data to be optimized The coordinate transformation vector of . The invention realizes global optimal registration of multi-view point cloud data in optical three-dimensional measurement, and is suitable for three-dimensional measurement of objects to be measured.

Description

A global optimal registration method for multi-view point cloud data in optical 3D measurement

technical field

The invention belongs to the technical field of digital image processing, and in particular relates to a global optimization registration method for multi-view point cloud data in optical three-dimensional measurement.

Background technique

Optical three-dimensional measurement is the cutting-edge technology of information optics research, which has the advantages of non-contact, non-destructive, fast data acquisition, and simple operation. Therefore, optical three-dimensional measurement technology is widely used in machine vision, automatic processing, industrial online inspection, product quality control, physical objects It has been widely used in profiling, biomedicine and other fields. However, due to the limitation of the measurement environment, the measured object and the measuring equipment itself, a single measurement can only obtain the data of a limited area of the measured object. To obtain the complete data of the measured object profile, it is necessary to perform multiple measurements by changing different measurement angles , also known as multi-look measurement. Due to the change of each measurement angle, the obtained multi-view measurement data cannot be directly unified into one coordinate system. The process of unifying the measurement data of multiple viewpoints into the same coordinate system is called registration. At present, the realization of registration The quasi-mainly adopts the following methods:

(1) Shape feature extraction method: extract the corresponding features such as curvature or shape from the measured multi-view point cloud data, and use these features to realize the registration of multi-view data. However, this process has a large amount of calculation, slow calculation speed and low calculation accuracy, and some objects to be measured have no obvious characteristics. Therefore, the three-dimensional measurement of the object to be measured by this method has great limitations, and the reliability is not guaranteed.

(2) Auxiliary device measurement method: the relative motion between the measured object and the measuring equipment is detected by means of auxiliary devices such as articulated arms and motion platforms, and the coordinate transformation relationship between multi-view data is directly obtained. However, the use of auxiliary devices not only increases the cost of the entire measurement system, but also increases its complexity. Moreover, the working range of the measurement system is limited, and it is impossible to perform three-dimensional measurement on larger objects to be measured.

(3) Feature mark point registration method: make feature mark points on the surface of the measured object. In order to realize the coordinate unification of the two sets of measurement data in the same coordinate system, it is necessary to ensure that at least three common feature mark points are measured twice. can be identified. However, due to the influence of the observation angle, when the same public marker point is measured multiple times, there is a certain deviation in the recognition of the public marker point, which leads to error transmission and cumulative errors in the registration results.

Contents of the invention

The purpose of the present invention is to solve the problem of deviation in the measurement of common marker points in the characteristic marker point measurement method, and provide a global optimization registration method for multi-view point cloud data in optical three-dimensional measurement.

The present invention is realized through the following scheme, the global optimization registration method of multi-view point cloud data in optical three-dimensional measurement, the method is based on a hardware system platform including an acquisition device and a computer control device, and the object to be measured is pasted with The marked points are used as the characteristic mark points of the object to be measured, and the computer control device controls the acquisition device to collect the measurement data information of the object to be measured from N viewpoints, obtain the two-dimensional measurement data of N groups of objects to be measured, and extract N groups of objects to be measured from The two-dimensional measurement data of the characteristic marker points is restored to the three-dimensional measurement data of N groups of characteristic marker points of the object to be measured, wherein N is the number of viewpoints of the object to be measured collected by the acquisition device, and N is an integer greater than 1;

The method for registering the three-dimensional measurement data of the feature marker points of the object to be measured collected by N viewpoints:

Taking the first viewpoint as the target viewpoint and the second viewpoint as the viewpoint to be converted, from the three-dimensional measurement data of the characteristic mark points of the object to be measured at the target viewpoint and the three-dimensional measurement data of the characteristic mark points of the object to be measured at the viewpoint to be converted to identify M ₁ public feature mark points of the object to be measured, the three-dimensional measurement data of the public feature mark points of the object to be measured in the target viewpoint are used as target data, and the three-dimensional measurement data of the public feature mark points of the object to be measured in the viewpoint to be converted are used as the target data registration data;

The process of data registration is: register the data to be registered to the target data, obtain the registration result data, and use the registration result data as the target data for the next registration;

Select the i-th viewpoint as the viewpoint to be converted, identify M _i common features of the object to be measured from the three-dimensional measurement data of the characteristic marker points of the object to be measured at the target viewpoint and the three-dimensional measurement data of the characteristic marker points of the object to be measured at the viewpoint to be converted Marking points, the three-dimensional measurement data of the common feature marker points of the object to be measured at the viewpoint to be converted is used as the data to be registered, and then the above data registration process is repeated until all viewpoints are used as the viewpoint to be converted to complete the data registration, wherein, i=3, 4, ... N, M ₁ , M _i are integers greater than 2;

The method for registering the data to be registered to the target data is specifically as follows:

Step 1. According to the data to be registered and the target data, the computer control device calculates the rotation matrix and translation vector when registering the M _i public feature marker points from the data to be registered to the target data, and converts the rotation matrix to The unit quaternion is stored in the form of a unit quaternion, and the real number component of the unit quaternion is positive, and the unit quaternion and the translation vector form a coordinate transformation vector;

，该公共特征标记点在目标视点的二维测量数据的坐标为x，将步骤一中获得的坐标变换向量作为待优化的坐标变换向量p₀；Step 2: According to the data to be registered and the target data, project the data to be registered of the M _i public feature marker points onto the plane corresponding to the target viewpoint, and obtain the two-dimensional coordinates of the M _i common feature marker points of the projected image, Let the two-dimensional coordinates of the common feature marker points of a projected image be

, the coordinate of the public feature mark point in the two-dimensional measurement data of the target viewpoint is x, and the coordinate transformation vector obtained in step 1 is used as the coordinate transformation vector p ₀ to be optimized;

计算公共特征标记点在目标视点的二维测量数据的坐标为x与投影图像公共特征标记点的二维坐标为

之间的偏差为

其中，f(·)为投影函数；Step 3. According to the imaging principle, the two-dimensional coordinates of the public feature marker points of the projected image The relationship with the coordinate transformation vector p ₀ to be optimized is expressed as

Calculate the coordinates of the two-dimensional measurement data of the public feature marker point at the target viewpoint as x and the two-dimensional coordinates of the public feature marker point of the projection image as

The deviation between

Among them, f( ) is the projection function;

Step 4, judging whether the deviation e described in step 3 is within the preset allowable deviation range, if the judgment result is yes, perform step 6, if the judgment result is no, perform step 5;

Step 5: Use the beam adjustment method to calculate the variation δ _p of the coordinate transformation vector to be optimized, correct the coordinate transformation vector p ₀ to be optimized, the corrected coordinate transformation vector to be optimized p _k =p ₀ +δ _p , and Define the corrected coordinate transformation vector p _k to be optimized as the coordinate transformation vector p ₀ to be optimized, and return to step 3;

Step 6, define the coordinate transformation vector _p0 to be optimized as the optimized coordinate transformation vector p, and perform step 7;

Step 7. According to the optimized coordinate transformation vector p obtained in Step 6, register the data to be registered to obtain registration result data.

The number of identified M common feature marker points in the present invention is at least 3, so as to ensure that coordinates of multiple sets of measurement data can be unified.

The real number component of the unit quaternion described in step 3 of the present invention is positive, and the restriction method described in the present invention can ensure the one-to-one relationship between the rotation matrix represented by the quaternion and the original rotation matrix, and the original rotation matrix represents In the operation process of the method, the trigonometric functions of the rotation matrix have multi-value and singularity, which will easily lead to an increase in the number of iterative calculations, and even non-convergence.

The global optimization registration method described in the present invention combines the feature mark point registration method with the beam adjustment method, and in the registration process of each two sets of measurement data, the measurement deviation of the common mark points is eliminated to make it Within the allowable range, the cumulative error caused by error transmission in the registration process of multi-view point measurement data can be reduced. Adopting the method described in the present invention does not increase the complexity of the system, only uses the three-dimensional feature mark points to calculate the initial value of the coordinate transformation on the basis of the original optical three-dimensional measurement system, and uses the beam adjustment method to accurately calculate the coordinate transformation vector and the three-dimensional mark points coordinate. The invention has the advantages of small calculation amount, high reliability, unrestricted measurement range, simple and easy-to-implement measurement method, and the like.

Description of drawings

Fig. 1 is a flow chart of the method for registering the data to be registered to the target data described in the first embodiment; Fig. 2 is the calculation of the coordinate transformation vector to be optimized by using the bundle adjustment method described in the fourth embodiment Flowchart of the method for varying _δp .

Detailed ways

Specific Embodiment 1: The present embodiment will be specifically described below with reference to FIG. 1 . A global optimization registration method for multi-view point cloud data in optical three-dimensional measurement, the method is based on a hardware system platform including an acquisition device and a computer control device, marking points are pasted on the object to be measured as characteristic mark points of the object to be measured, and the computer The control device controls the acquisition device to collect the measurement data information of the object to be measured from N viewpoints respectively, obtains two-dimensional measurement data of N groups of objects to be measured, extracts the two-dimensional measurement data of N groups of characteristic mark points of the object to be measured, and converts them to Restoring the three-dimensional measurement data of N groups of feature marker points of the object to be measured, wherein N is the number of viewpoints of the object to be measured collected by the acquisition device, and N is an integer greater than 1;

The method for registering the three-dimensional measurement data of the feature marker points of the object to be measured collected by N viewpoints:

Taking the first viewpoint as the target viewpoint and the second viewpoint as the viewpoint to be converted, from the three-dimensional measurement data of the characteristic mark points of the object to be measured at the target viewpoint and the three-dimensional measurement data of the characteristic mark points of the object to be measured at the viewpoint to be converted to identify M ₁ public feature mark points of the object to be measured, the three-dimensional measurement data of the public feature mark points of the object to be measured in the target viewpoint are used as target data, and the three-dimensional measurement data of the public feature mark points of the object to be measured in the viewpoint to be converted are used as the target data registration data;

The process of data registration is: register the data to be registered to the target data, obtain the registration result data, and use the registration result data as the target data for the next registration;

Select the i-th viewpoint as the viewpoint to be converted, and identify M _i common features of the object to be measured from the three-dimensional measurement data of the characteristic marker points of the object to be measured at the target viewpoint and the three-dimensional measurement data of the characteristic marker points of the object to be measured at the viewpoint to be converted Marking points, the three-dimensional measurement data of the common feature marker points of the object to be measured at the viewpoint to be converted is used as the data to be registered, and then the above data registration process is repeated until all viewpoints are used as viewpoints to be converted to complete the data registration, wherein, i=3, 4, ... N, M ₁ , M _i are integers greater than 2;

The method for registering the data to be registered to the target data is specifically as follows:

Step 1. According to the data to be registered and the target data, the computer control device calculates the rotation matrix and translation vector when registering the M _i public feature marker points from the data to be registered to the target data, and converts the rotation matrix to The unit quaternion is stored in the form of a unit quaternion, and the real number component of the unit quaternion is positive, and the unit quaternion and the translation vector form a coordinate transformation vector;

该公共特征标记点在目标视点的二维测量数据的坐标为x，将步骤一中获得的坐标变换向量作为待优化的坐标变换向量p₀；Step 2: According to the data to be registered and the target data, project the data to be registered of the M _i public feature marker points onto the plane corresponding to the target viewpoint, and obtain the two-dimensional coordinates of the M _i common feature marker points of the projected image, Let the two-dimensional coordinates of the common feature marker points of a projected image be

The coordinates of the two-dimensional measurement data of the public feature mark point at the target viewpoint are x, and the coordinate transformation vector obtained in step 1 is used as the coordinate transformation vector p ₀ to be optimized;

与待优化的坐标变换向量p₀的关系表示为

计算公共特征标记点在目标视点的二维测量数据的坐标为x与投影图像公共特征标记点的二维坐标为

之间的偏差为

其中，f(·)为投影函数；Step 3. According to the imaging principle, the two-dimensional coordinates of the public feature marker points of the projected image

The relationship with the coordinate transformation vector p ₀ to be optimized is expressed as

Calculate the coordinates of the two-dimensional measurement data of the public feature marker point at the target viewpoint as x and the two-dimensional coordinates of the public feature marker point of the projection image as

The deviation between

Among them, f( ) is the projection function;

Step 4, judging whether the deviation e described in step 3 is within the preset allowable deviation range, if the judgment result is yes, perform step 6, if the judgment result is no, perform step 5;

Step 5: Use the beam adjustment method to calculate the variation δ _p of the coordinate transformation vector to be optimized, correct the coordinate transformation vector p ₀ to be optimized, the corrected coordinate transformation vector to be optimized p _k =p ₀ +δ _p , and Define the corrected coordinate transformation vector p _k to be optimized as the coordinate transformation vector p ₀ to be optimized, and return to step 3;

Step 6, define the coordinate transformation vector _p0 to be optimized as the optimized coordinate transformation vector p, and perform step 7;

Step 7. According to the optimized coordinate transformation vector p obtained in Step 6, register the data to be registered to obtain registration result data.

The number of identified M common feature marker points in this embodiment is at least 3, so as to ensure that coordinates of multiple sets of measurement data can be unified.

The real number component of the unit quaternion described in step 1 of this embodiment is positive, and the limited method described in this embodiment can ensure the one-to-one correspondence between the rotation matrix represented by the quaternion and the original rotation matrix, and the original rotation In the operation process of matrix representation, the trigonometric function of the rotation matrix has multi-value and singularity, which will easily lead to an increase in the number of iterative calculations, and even non-convergence.

The global optimization registration method described in this embodiment combines the feature mark point registration method with the beam adjustment method. During the registration process of each two sets of measurement data, the measurement deviation of the common mark points is eliminated, so that It is within the allowable range, thereby reducing the cumulative error caused by error transmission during the registration process of multi-view point measurement data. The method described in this embodiment does not increase the complexity of the system, only uses the three-dimensional feature mark points to calculate the initial value of the coordinate transformation on the basis of the original optical three-dimensional measurement system, and uses the beam adjustment method to accurately calculate the coordinate transformation vector and the three-dimensional mark point coordinates. This implementation mode has the advantages of small calculation amount, high reliability, unlimited measurement range, simple and easy-to-implement measurement method, and the like.

Embodiment 2: This embodiment is a further limitation of the global optimization registration method for multi-view point cloud data in optical three-dimensional measurement described in Embodiment 1, and the acquisition device is a CCD camera.

Specific embodiment three: this embodiment is a further limitation of the global optimal registration method for multi-view point cloud data in optical three-dimensional measurement described in specific embodiment one. In step one, the rotation matrix described is a 3×3 matrix, the translation vector is a 3×1 vector, the unit quaternion is a 4×1 vector, and the coordinate transformation vector composed of the unit quaternion and the translation vector is a 7× A vector of 1.

Specific Embodiment 4: The present embodiment will be specifically described below with reference to FIG. 2 . This embodiment is a further limitation of the global optimization registration method for multi-view point cloud data in the optical three-dimensional measurement described in the first specific embodiment. In step five, the calculation of the coordinate transformation vector to be optimized by using the beam adjustment method The method of changing δ _p is:

Step 51. At the coordinate transformation vector p ₀ to be optimized, for the variation δ _p of the coordinate transformation vector to be optimized, expand the projection function f(p ₀ +δ _p ) into a first-order Taylor polynomial:

f(p ₀ +δ _p )≈f(p ₀ )+Jδ _p

所述的投影函数f(p₀+δ_p)，即为修正后的待优化的坐标变换向量p_k在平面上的投影；Among them, J is the Jacobian matrix

The projection function f(p ₀ +δ _p ) is the projection on the plane of the modified coordinate transformation vector p _k to be optimized;

Step 52: Calculate the difference between the two-dimensional coordinate x of the target data of the public feature marker point and the projection f(p ₀ +δ _p ) of the revised coordinate transformation vector p _k to be optimized on the plane, ||xf( p ₀ +δ _p )||≈||xf(p ₀ )-Jδ _p ||＝||e-Jδ _p ||

Step 53. In order to minimize ||xf(p ₀ +δ _p )|| mentioned in step 52, set e-Jδ _p to be orthogonal to J, that is, J ^T (e-Jδ _p )=0;

Step five and four, arrange the equation J ^T (e-Jδ _p )=0 in step five and three, and obtain the following form: J ^T Jδ _p =J ^T e;

Step five and five, converting the sorted equation obtained in step five and four into an enhanced equation, (J ^T J+μI)δ _p =J ^T e,

Among them, I is the unit matrix, μ is the unit matrix coefficient, and μ>0;

Steps five and six, according to the enhanced equation obtained in steps seven and five, the variation δ _p of the coordinate transformation vector to be optimized is δ _p =-(J ^T J+μI) ^-1 J ^T e.

The method of calculating the variation δ _p of the coordinate transformation vector to be optimized by using the beam adjustment method described in this embodiment has the advantages of small calculation amount, easy implementation, high reliability, and error-free transmission.

Embodiment 5: This embodiment is a further supplementary description of the global optimization registration method for multi-view point cloud data in optical three-dimensional measurement described in Embodiment 4. In step 5, the following steps are also included:

Recording the number of operations for calculating the variation δ _p of the coordinate transformation vector to be optimized by using the beam adjustment method, and executing the end instruction when the number of operations exceeds a preset value.

In this embodiment, the beam adjustment method is used to calculate the variation δ _p of the coordinate transformation vector to be optimized. When the number of calculations for calculating the variation δ _p of the coordinate transformation vector to be optimized exceeds the preset value, the loop is jumped out.

Claims (3)

1. The global optimal registration method of multi-view point cloud data in optical three-dimensional measurement, described method is based on a hardware system platform that comprises acquisition device and computer control device, and the mark point is pasted on the object to be measured as the characteristic mark point of object to be measured , the computer control device controls the acquisition device to collect the measurement data information of the object to be measured from N viewpoints, obtain the two-dimensional measurement data of N groups of objects to be measured, and extract the two-dimensional measurement data of N groups of characteristic mark points of the object to be measured, Restoring it as three-dimensional measurement data of N groups of feature marker points of the object to be measured, wherein N is the number of viewpoints of the object to be measured collected by the acquisition device, and N is an integer greater than 1; The method for registering the three-dimensional measurement data of the feature marker points of the object to be measured collected by N viewpoints: Taking the first viewpoint as the target viewpoint and the second viewpoint as the viewpoint to be converted, from the three-dimensional measurement data of the characteristic mark points of the object to be measured at the target viewpoint and the three-dimensional measurement data of the characteristic mark points of the object to be measured at the viewpoint to be converted to identify M ₁ public feature mark points of the object to be measured, the three-dimensional measurement data of the public feature mark points of the object to be measured at the target viewpoint are used as target data, and the three-dimensional measurement data of the public feature mark points of the object to be measured at the viewpoint to be converted are used as the target data registration data; The process of data registration is: register the data to be registered to the target data, obtain the registration result data, and use the registration result data as the target data for the next registration; Select the i-th viewpoint as the viewpoint to be converted, identify M _i common features of the object to be measured from the three-dimensional measurement data of the characteristic marker points of the object to be measured at the target viewpoint and the three-dimensional measurement data of the characteristic marker points of the object to be measured at the viewpoint to be converted Marking points, the three-dimensional measurement data of the common feature marker points of the object to be measured at the viewpoint to be converted is used as the data to be registered, and then the above data registration process is repeated until all viewpoints are used as the viewpoint to be converted to complete the data registration, wherein, i=3, 4, ... N, M ₁ , M _i are integers greater than 2; It is characterized in that: the method for registering the data to be registered to the target data is specifically: Step 1. According to the data to be registered and the target data, the computer control device calculates the rotation matrix and translation vector when registering the M _i public feature marker points from the data to be registered to the target data, and converts the rotation matrix to The unit quaternion is stored in the form of a unit quaternion, and the real number component of the unit quaternion is positive, and the unit quaternion and the translation vector form a coordinate transformation vector; Step 2: According to the data to be registered and the target data, project the data to be registered of the M _i public feature marker points onto the plane corresponding to the target viewpoint, and obtain the two-dimensional coordinates of the M _i common feature marker points of the projected image, Let the two-dimensional coordinates of the common feature marker points of a projected image be

, the coordinate of the public feature mark point in the two-dimensional measurement data of the target viewpoint is x, and the coordinate transformation vector obtained in step 1 is used as the coordinate transformation vector p ₀ to be optimized; Step 3. According to the imaging principle, the two-dimensional coordinates of the public feature marker points of the projected image

The relationship with the coordinate transformation vector p ₀ to be optimized is expressed as Calculate the coordinates of the two-dimensional measurement data of the public feature marker point at the target viewpoint as x and the two-dimensional coordinates of the public feature marker point of the projection image as

The deviation between

Among them, f( ) is the projection function; Step 4, judging whether the deviation e described in step 3 is within the preset allowable deviation range, if the judgment result is yes, perform step 6, if the judgment result is no, perform step 5; Step 5: Use the beam adjustment method to calculate the variation δ _p of the coordinate transformation vector to be optimized, correct the coordinate transformation vector p ₀ to be optimized, the corrected coordinate transformation vector to be optimized p _k =p ₀ +δ _p , and Define the corrected coordinate transformation vector p _k to be optimized as the coordinate transformation vector p ₀ to be optimized, and return to step 3; Step 6, define the coordinate transformation vector _p0 to be optimized as the optimized coordinate transformation vector p, and perform step 7; Step 7. According to the optimized coordinate transformation vector p obtained in step 6, register the data to be registered to obtain the registration result data, In step five, the method for calculating the variation δ _p of the coordinate transformation vector to be optimized by using the beam adjustment method is: Step 51. At the coordinate transformation vector p ₀ to be optimized, for the variation δ _p of the coordinate transformation vector to be optimized, expand the projection function f(p ₀ +δ _p ) into a first-order Taylor polynomial: f(p ₀ +δ _p )≈f(p ₀ )+Jδ _p Among them, J is the Jacobian matrix

The projection function f(p ₀ +δ _p ) is the projection on the plane of the modified coordinate transformation vector p _k to be optimized; Step 52: Calculate the difference between the two-dimensional coordinate x of the target data of the public feature marker point and the projection f(p ₀ +δ _p ) of the revised coordinate transformation vector p _k to be optimized on the plane, ||xf( p ₀ +δ _p )||≈||xf(p ₀ )-Jδ _p ||=||e-Jδ _p ||; Step 53. In order to minimize ||xf(p ₀ +δ _p )|| mentioned in step 52, set e-Jδ _p to be orthogonal to J, that is, J ^T (e-Jδ _p )=0; Step five and four, arrange the equation J ^T (e-Jδ _p )=0 in step five and three, and obtain the following form: J ^T Jδ _p =J ^T e; Step five and five, converting the sorted equation obtained in step five and four into an enhanced equation, (J ^T J+μI)δ _p =J ^T e, Among them, I is the unit matrix, μ is the unit matrix coefficient, and μ>0; Step five and six, according to the enhanced equation obtained in step five and five, the variation δp of the coordinate transformation vector to be optimized is δ _p =-(J ^T J+μI) ^-1 J ^T e. 2. The global optimization registration method of multi-view point cloud data in optical three-dimensional measurement according to claim 1, characterized in that: the acquisition device is a CCD camera. 3. The global optimization registration method for multi-view point cloud data in optical three-dimensional measurement according to claim 1, characterized in that: in step 1, the rotation matrix is a 3×3 matrix, and the translation vector is a 3×1 vector, the unit quaternion is a 4×1 vector, and the coordinate transformation vector composed of the unit quaternion and the translation vector is a 7×1 vector.

Images