CN105678833A - Point cloud geometrical data automatic splicing algorithm based on multi-view image three-dimensional modeling - Google Patents

Abstract

The invention discloses an automatic splicing algorithm of point cloud geometric data based on three-dimensional modeling of multi-viewpoint images, including: Step 1, under different viewpoints, calculate the point cloud geometric data of the three-dimensional object to be modeled, and construct the point cloud The feature matching point pairs of geometric data; step 2, randomly select a viewpoint as the reference viewpoint, and use the feature matching point pairs of point cloud geometric data to calculate the relationship correspondence matrix representing the relative position relationship between each viewpoint and the reference viewpoint; step 3, pair the relationship Singular value decomposition is performed on the corresponding matrix, and the relationship translation vector and relationship rotation vector of the feature matching point pairs between each viewpoint and the reference viewpoint are obtained; step 4, according to the relationship translation vector and relationship rotation vector, the point cloud geometric data in each viewpoint It is represented in the reference viewpoint coordinate system to complete the automatic splicing of point cloud geometric data. The invention is simple and reliable, easy to realize, convenient to operate, and can achieve higher modeling precision.

Description

An automatic mosaic algorithm of point cloud geometric data based on multi-view image 3D modeling

technical field

The invention relates to the technical field of computer modeling, in particular to an automatic splicing algorithm of point cloud geometric data based on three-dimensional modeling of multi-viewpoint images.

Background technique

In the process of constructing the 3D shape in the multi-viewpoint image 3D geometric modeling system, the geometric modeling between adjacent viewpoint images can only obtain the geometric point cloud geometric data of the local area on the surface of the measured object. In order to obtain the entire 3D shape The data needs to acquire images from multiple viewpoints, and at the same time perform geometric modeling for every two adjacent viewpoints, which leads to different geometric coordinate systems of point cloud geometric data calculated under different viewpoints.

In order to obtain the entire three-dimensional shape geometric data of the surface of the measured object, it is necessary to transform the local geometric data in different coordinate systems into the same unified coordinate system. The automatic stitching and registration of 3D point cloud geometric data in different coordinate systems obtained by modeling multiple adjacent viewpoint images has always been a thorny problem. Existing methods mainly include the following:

1) By pasting auxiliary marking points on the surface of the object to be measured, searching for several different measured marking points to construct matching marking point pairs, while ensuring that there are at least three common marking points between the two viewpoints, and then passing These matched common marker points calculate the coordinate transformation relationship between the point cloud geometric data measured by multiple different viewpoints, so as to realize the automatic splicing of multi-view point cloud geometric data.

But pasting auxiliary markers on the surface of the measured object will not only destroy the texture information of the surface of the measured object, but also fail to model the surface geometry and texture data of the measured object where the markers are pasted and covered. In addition, this method is not suitable for pasting marking points on some special objects to be measured (such as the surface of historical relics), so its use has certain limitations.

2) Use some gimbals to determine the position change relationship between the measured object and multiple viewpoints, and directly calculate the coordinate change relationship between point cloud geometric data under multiple viewpoints through the motion parameters of the gimbal.

This method is stable and reliable, and has high precision, but requires additional high-precision mechanical equipment, which leads to complex structures of multi-viewpoint acquisition equipment and cannot measure larger objects.

3) Manually select the matching feature points for pre-matching, and then complete the splicing of point cloud geometric data through existing commercial software processing algorithms.

This type of method first needs to achieve data matching through manual pre-intervention, but the artificial matching error is too large to achieve the ideal stitching effect, and it is impossible to realize the automatic stitching of point cloud geometric data after 3D modeling of multi-viewpoint images.

Contents of the invention

The present invention provides an automatic splicing algorithm of point cloud geometric data based on multi-viewpoint image three-dimensional modeling, which can complete multi-viewpoint without resorting to mechanical hardware auxiliary equipment and pasting auxiliary mark points on the surface of the modeled three-dimensional object The automatic splicing and matching of point cloud geometric data for image three-dimensional modeling is simple, reliable, easy to implement, can provide high modeling accuracy, and has wide applicability and practicability.

An automatic splicing algorithm of point cloud geometric data based on multi-view image three-dimensional modeling, including:

Step 1, under different viewpoints, calculate the point cloud geometric data of the 3D object to be modeled, and construct the feature matching point pairs of the point cloud geometric data;

Step 2, randomly select a viewpoint as the reference viewpoint, use the feature matching point pairs of the point cloud geometric data, and calculate the relationship correspondence matrix representing the relative positional relationship between each viewpoint and the reference viewpoint;

Step 3, perform singular value decomposition on the relationship correspondence matrix, and obtain the relationship translation vector and relationship rotation vector of the feature matching point pairs between each viewpoint (excluding the reference viewpoint) and the reference viewpoint;

Step 4: According to the relationship translation vector and the relationship rotation vector, the point cloud geometric data in each viewpoint is expressed in the reference viewpoint coordinate system, and the automatic splicing of the point cloud geometric data is completed.

The more the number of different viewpoints, the more accurate the point cloud geometric data obtained after automatic splicing and matching, but the corresponding calculation amount will be greatly increased. Preferably, the number of different viewpoints is at least 6.

Randomly select a viewpoint from different viewpoints as a reference viewpoint, and convert the point cloud geometric data under other viewpoints to be expressed in the coordinate system of the reference viewpoint.

After performing singular value decomposition on the relationship correspondence matrix in step 3, the relationship translation vector and relationship rotation vector of feature matching point pairs can be obtained, and the relationship translation vector and relationship rotation vector of feature matching point pairs are also the relationship translation of all point cloud geometric data Vectors and relational rotation vectors.

The point cloud geometric data under different viewpoints can be transformed into the coordinate system of the reference viewpoint through the relationship translation vector and the relationship rotation vector.

Preferably, the number of feature matching point pairs is 100-120. Further preferably, the number of feature matching point pairs is 100.

When calculating the relationship correspondence matrix, 100 feature matching point pairs are randomly selected from the feature matching point pairs of the point cloud geometric data of other viewpoints except the reference viewpoint and the reference viewpoint for calculation. (Select 100 feature matching point pairs for each viewpoint)

Number n different viewpoints respectively, in order of 1, 2, 3...n, refer to the viewpoint, that is, n=1, and calculate the kth (k=2, 3...n) For the relative geometric relationship between the first viewpoint and the reference viewpoint, randomly select 100 feature matching point pairs from the feature matching point pairs of the point cloud geometric data of the k (k=2, 3...n) viewpoint and the reference viewpoint ,Calculation.

The relative geometric relationship between the kth (k=2, 3...n) viewpoint and the reference viewpoint constitutes a relationship correspondence matrix M.

Preferably, an optimization mechanism is used when calculating the relationship correspondence matrix in the step 2. Using the optimization mechanism can further ensure the robustness of the calculated relationship correspondence matrix, and increase the fault tolerance of the wrong feature matching point pairs.

Assuming that the optimization mechanism randomly selects the feature matching point pairs multiple times, each time a hundred pairs of feature matching point pairs are selected, for the 100 feature matching points in image 1 in a certain viewpoint, for these 100 feature matching points p _{( i=1...100)} , find its corresponding epipolar line L _(i=1...100) in the viewpoint image 2 through the epipolar geometric constraint relationship, and then calculate p _{(i=1... ... 20)} corresponds to the distance D (i=1...100) from the feature point in the viewpoint image 2 to L _{(i=1 ...100)} , and calculates the total distance D=D ₁ + D ₂ +D ₃ +...+D ₉₉ +D ₁₀₀ , finally select a group of matching point pairs with the smallest total distance D value as the final feature matching point pair. Using this optimization method can improve the fault tolerance of feature matching point pairs through extreme geometric constraints on the one hand, and on the other hand can ensure the robustness of the algorithm.

In the step 3, the singular value decomposition is performed on the relationship correspondence matrix M, and the singular value SVD decomposition (SingularValueDecomposition) can calculate the normalized relationship translation vector T and the relationship rotation matrix R between two different viewpoints, and then use the multi-viewpoint The relationship translation vector T and the relationship rotation matrix R of the feature matching point pairs between other viewpoints and the reference viewpoint, and the actual relationship translation vector T′ of the point cloud geometric data corresponding to each viewpoint relative to the reference viewpoint is obtained.

The automatic splicing algorithm of point cloud geometric data based on multi-viewpoint image three-dimensional modeling of the present invention only needs to use feature matching point pairs under different viewpoints to realize automatic splicing of point cloud geometric data based on multi-viewpoint image three-dimensional modeling, Simple and reliable, easy to implement, easy to operate, and can achieve high modeling accuracy.

Description of drawings

Fig. 1 is a flow chart of the automatic mosaic algorithm of point cloud geometric data based on multi-view image three-dimensional modeling in the present invention.

detailed description

The present invention will be described in detail below in conjunction with the accompanying drawings.

As shown in Figure 1, an automatic splicing algorithm for point cloud geometric data based on three-dimensional modeling of multi-viewpoint images includes the following steps:

(1) Calculate the relationship corresponding matrix M

Under different viewpoints, multiple images of the object to be modeled are captured, and the different viewpoints are marked as 1, 2, 3...n in sequence, and a viewpoint is randomly selected as the reference viewpoint, for example, n=1 is taken as the reference viewpoint.

Establish stable feature matching point pairs between random image I ^k and random image I ¹ between the kth (k=2, 3...n) viewpoint and the reference viewpoint, assuming random image I ^k and random image I ¹ The corresponding image coordinates of the feature points in the camera coordinate system of their respective viewpoints are I ^k and I ¹ , respectively expressed as (I ₁ ^k , I ₂ ^k , I ₃ ^k ), (I ₁ ¹ , I ₂ ¹ , I ₃ ¹ ).

According to the geometric constraint relationship, the limit constraint equation can be obtained

(I ¹ ) ^T FI ^k ＝0(1)

Among them: F is the basic matrix, which is an algebraic representation of epipolar geometry and a very important matrix in multi-view 3D modeling.

At the same time, F also satisfies the following relationship

F＝K ₂ ^-T EK ₁ ^-1 (2)

K ₁ and K ₂ are upper triangular matrices of 3×3, which respectively represent the internal parameters of the two cameras, and E is a matrix, which contains the structural parameters between two adjacent viewpoints.

Put formula (2) into formula (1) to get the following formula

(I ¹ ) ^T K ₁ ^-T FK ₁ ^-1 I ^k ＝0(3)

Assuming that the feature points in the random image I ^k and the random image I ¹ correspond to the normalized three-dimensional homogeneous image coordinates in the camera coordinate system of the respective measurement viewpoints are I _l ^k and I _l ¹ respectively, let

Q _l ^k ＝K ₁ ^-1 I ^k (4)

I _l ¹ = K ₁ ^-1 I ¹ (5)

Then the epipolar constraint equation can be simplified as

(I _l ¹ ) ^T EQ _l ^k ＝0(6)

The basic matrix F is generally a 3×3 non-zero matrix, and the value of its determinant is 0, that is

Det(F)=0(7)

According to the formula (2), the matrix E also satisfies the formula (7), and the matrix E has the following properties

$\mathrm{<span\; class="notranslate">\; E.</span>}\mathrm{<span\; class="notranslate">\; E.</span>}\mathrm{<span\; class="notranslate">\; T</span>}\mathrm{<span\; class="notranslate">\; E.</span>}<span\; class="notranslate">\; -</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; 2</span>}}\mathrm{<span\; class="notranslate">\; t</span>}\mathrm{<span\; class="notranslate">\; r</span>}\mathrm{<span\; class="notranslate">\; a</span>}\mathrm{<span\; class="notranslate">\; c</span>}\mathrm{<span\; class="notranslate">\; e</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; E.</span>}\mathrm{<span\; class="notranslate">\; E.</span>}\mathrm{<span\; class="notranslate">\; T</span>}<span\; class="notranslate">\; )</span>\mathrm{<span\; class="notranslate">\; E.</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 8</span>}<span\; class="notranslate">\; )</span>$

Using formulas (7) and (8), the matrix E is obtained by selecting 100 pairs of matching pixels between the random image I ^k and the random image I ¹ between the k-th viewpoint and the reference viewpoint through the hundred-point algorithm.

Use the hundred-point algorithm to calculate the relationship correspondence matrix M. The hundred-point algorithm is an iterative evaluation method for calculating the geometric coordinate transformation relationship between different viewpoints. The main steps are as follows:

Randomly select 100 groups from the set of stable feature matching point pairs established by random image I ^k and random image I ¹ , then these 100 groups of feature matching point pairs all satisfy the formula (6). Therefore, the epipolar constraint equation can be expressed as

in: ${\tilde{I}}^T={\&lsqb;I_1^1{I}_1^2{I}_2^1{I}_1^2{I}_3^1{I}_1^2{I}_1^1{I}_2^2{I}_1^1{I}_1^2{I}_2^1{I}_2^2{I}_3^1{I}_2^2......I_{98}^1{I}_{20}^2{I}_{99}^1{I}_{100}^2{I}_{100}^1{I}_{100}^2\&rsqb;}^T---\left(10\right)$

Accumulate a vector of hundreds of pairs of feature matching points A 100×9 relational correspondence matrix M can be obtained.

After obtaining the null space of the matrix M corresponding to the relationship, calculate the expansions of formula (7) and formula (8) respectively.

(2) Use the singular value SVD decomposition of the matrix (see Dai Hua. Matrix Theory. Beijing, Science Press, 2001) to perform singular value matrix decomposition on the relationship correspondence matrix M, and obtain the relationship rotation matrix R and the relationship translation vector T value.

Assume that the geometric data of the point cloud under the geometric coordinate system established by the reference viewpoint is X={X _i ,i=1,2,...}, the kth (k=2,3...n)th The geometric data of the point cloud under the geometric coordinate system established by the viewpoint is X′={X′ _i ,j=1,2,...}.

In order to obtain the overall point cloud geometric data, the point cloud geometric data of the kth viewpoint (k=2, 3...n) is transformed into a unified geometric coordinate system representation using the reference viewpoint through geometric transformation coordinates.

Assume that the point cloud geometric data of the kth viewpoint (k=2, 3...n) is transformed through geometric coordinates, and the point cloud geometric data obtained after using the coordinate system of the reference viewpoint is expressed as Then the coordinate transformation formula of any geometric data point _X′i in the point cloud geometric data set X′ is

Wherein: R represents the relational rotation matrix from the geometric coordinate system of the kth (k=2,3...n) viewpoint to the geometric coordinate system of the reference viewpoint;

T represents a relational translation vector from the geometric coordinate system of the kth (k=2, 3...n) viewpoint to the geometric coordinate system of the reference viewpoint.

To realize the splicing and matching of point cloud geometric data from different viewpoints, the relational rotation matrix R and relational translation vector T of the geometric coordinate systems of the two viewpoints must be calculated.

Using the relationship correspondence matrix M representing the relative geometric positions between two viewpoints and the relationship between the relationship correspondence matrix M, the relationship rotation matrix R and the relationship translation vector T, the relationship rotation matrix R and the relationship translation vector T can be obtained.

The relationship between the relationship correspondence matrix M, the relationship rotation matrix R and the relationship translation vector T is as follows

$\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; R</span>}\left[\begin{array}{ccc}\mathrm{<span\; class="notranslate">\; 0</span>}& <span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; t</span>}}^{\mathrm{<span\; class="notranslate">\; 3</span>}}& {\mathrm{<span\; class="notranslate">\; t</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}\\ {\mathrm{<span\; class="notranslate">\; t</span>}}^{\mathrm{<span\; class="notranslate">\; 3</span>}}& \mathrm{<span\; class="notranslate">\; 0</span>}& <span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; t</span>}}^{\mathrm{<span\; class="notranslate">\; 1</span>}}\\ <span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; t</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}& {\mathrm{<span\; class="notranslate">\; t</span>}}^{\mathrm{<span\; class="notranslate">\; 1</span>}}& \mathrm{<span\; class="notranslate">\; 0</span>}\end{array}\right]<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 13</span>}<span\; class="notranslate">\; )</span>$

where T = (t ¹ , t ² , t ³ ) (14)

In the case that the relationship correspondence matrix M has been calculated, the value of the relationship rotation matrix R and the relationship translation vector T can be obtained by performing matrix singular value decomposition on the relationship correspondence matrix M.

(3) By using the relationship translation vector T and the relationship rotation matrix R of the feature matching point pairs between each viewpoint and the reference viewpoint, the actual relationship translation vector T′ of the point cloud geometric data in each viewpoint relative to the reference viewpoint can be calculated, and the actual relationship The translation vector T' is the same as the relationship translation vector T.

(4) According to the relationship rotation matrix R and the relationship actual translation vector T′, the geometric coordinate transformation of the point cloud geometric data under each viewpoint is performed, and the formula (15) is used to uniformly express all the point cloud geometric data in the reference viewpoint coordinate system, Realize the automatic splicing and matching of point cloud geometric data under different viewpoints.

X=RX'+T'(15)

Among them, X is the point cloud geometric data under the geometric coordinate system established by the reference viewpoint;

X' is the point cloud geometric data under the geometric coordinate system established by the kth (k=2,3...n) viewpoint;

R is the relational rotation matrix;

T' is the actual relationship translation vector.

Claims (5)

1. the automatic Mosaic algorithm based on the some cloud geometric data of multi-view image three-dimensional modeling, it is characterised in that including:

Step 1, under different points of view, calculates the some cloud geometric data of the three-dimensional body needing modeling, and builds the characteristic matching point pair of a cloud geometric data;

Step 2, randomly selects a viewpoint as reference view, utilizes the relation homography of the characteristic matching point pair of some cloud geometric data, each viewpoint of computational representation and reference view relative position relation;

Step 3, carries out singular value decomposition to relation homography, asks for relation translation vector and the relation rotating vector of characteristic matching point pair between each viewpoint and reference view;

Step 4, according to relation translation vector and relation rotating vector, is indicated the some cloud geometric data in each viewpoint under reference view coordinate system, completes an automatic Mosaic for cloud geometric data.

2. the automatic Mosaic algorithm of the some cloud geometric data based on multi-view image three-dimensional modeling as claimed in claim 1, it is characterised in that the number of different points of view is at least 6.

3. the automatic Mosaic algorithm of the some cloud geometric data based on multi-view image three-dimensional modeling as claimed in claim 2, it is characterised in that the number of characteristic matching point pair is 100～120.

4. the automatic Mosaic algorithm of the some cloud geometric data based on multi-view image three-dimensional modeling as claimed in claim 3, it is characterised in that the number of characteristic matching point pair is 100.

5. the automatic Mosaic algorithm of the some cloud geometric data based on multi-view image three-dimensional modeling as claimed in claim 4, it is characterised in that use optimization mechanism in described step 2 during calculated relationship homography.