CN118913242A - Method and device for estimating relative pose and scale of multi-camera system - Google Patents

Abstract

The invention provides a method and a device for estimating relative pose and scale of a multi-camera system, comprising the following steps: performing time synchronization and space synchronization on the multi-camera system and the IMU to obtain a synchronous multi-camera inertial navigation system; acquiring a camera corresponding point pair in the synchronous multi-camera inertial navigation system, determining a generalized camera model by the camera corresponding point pair, substituting an IMU rotation matrix in the vertical direction into the generalized camera model to obtain a constrained generalized camera model; minimizing algebraic errors in the constrained generalized camera model by using a least square method, establishing a target optimization function, and converting the target optimization function into a polynomial equation; and solving the relative rotation angle of the polynomial equation by adopting a eigenvalue method to obtain a camera translation vector and a scale.

Description

A method and device for estimating relative pose and scale of a multi-camera system

Technical Field

The present invention relates to the field of camera positioning technology, and in particular to a method and device for estimating relative posture and scale of a multi-camera system.

Background Art

Estimating relative pose from two adjacent views is the basis of multi-view geometry problem, which has been applied in many fields, such as Visual Odometry (VO), Structure-from-Motion (SfM), Simultaneous Localization and Mapping (SLAM). A multi-camera system consists of multiple independent cameras fixed on a system. It has the advantages of large field of view, high accuracy of metric scale estimation, and high accuracy of relative pose estimation. Therefore, a large number of scholars have studied how to improve the accuracy, robustness and efficiency of relative pose estimation algorithms.

The biggest difference between a multi-camera system and a single-camera system is that the light rays of a multi-camera system cannot intersect at a single point, and therefore do not conform to the pinhole model. Most of the previous research on multi-camera systems is based on the fact that the internal scale of the multi-camera system is known, which to some extent limits the application of multi-camera systems.

Therefore, a method is needed to estimate the relative pose and scale of a multi-camera system when the internal scale information is unknown.

Summary of the invention

The present invention provides a method and device for estimating the relative pose and scale of a multi-camera system, so as to solve the defects existing in the prior art when the internal scale information is unknown during the pose estimation of the multi-camera system.

In a first aspect, the present invention provides a method for estimating relative pose and scale of a multi-camera system, comprising:

Perform time and space synchronization on the multi-camera system and IMU to obtain a synchronized multi-camera inertial navigation system;

Obtaining camera corresponding point pairs in the synchronous multi-camera inertial navigation system, determining a generalized camera model from the camera corresponding point pairs, substituting the IMU rotation matrix in the vertical direction into the generalized camera model to obtain a constrained generalized camera model;

Minimize the algebraic error in the constrained generalized camera model by using the least square method, establish a target optimization function, and convert the target optimization function into a polynomial equation;

The relative rotation angle of the polynomial equation is solved by the eigenvalue method to obtain the camera translation vector and scale.

According to a method for estimating relative position and scale of a multi-camera system provided by the present invention, the multi-camera system and the IMU are synchronized in time and space to obtain a synchronized multi-camera inertial navigation system, including:

The clock error is calculated based on the BIC observation value of BeiDou and modulated into a PPS signal, and the PPS provides time synchronization for sensors of multiple cameras;

Taking any camera in the multi-camera system as the reference frame, the camera reference coordinate system is determined. Based on the camera reference coordinate system, the visual-inertial calibration toolbox Kalibr is used to calibrate the remaining camera coordinate systems and the IMU coordinate system.

According to a method for estimating relative pose and scale of a multi-camera system provided by the present invention, corresponding point pairs of cameras in the synchronous multi-camera inertial navigation system are obtained, a generalized camera model is determined by the corresponding point pairs of cameras, and an IMU rotation matrix in the vertical direction is substituted into the generalized camera model to obtain a constrained generalized camera model, including:

Obtain the camera corresponding point pair (x _ki ,x _k'j ,R _k ,R _k' ,t _k ,t _k' ) based on the Plücker line, where x _ki represents the normalized coordinates of the image obtained by camera k at time i, x _k'j represents the normalized coordinates of the image obtained by camera k' at time j, the rotation matrix and translation vector of camera k relative to the reference frame are R _k and t _k respectively, and the rotation matrix and translation vector of camera k' relative to the reference frame are R _k' and t _k' respectively;

Assuming that the rotation matrix of the reference frame from time i to time j is R, the translation vector is t, and the scale is s, the generalized camera model with unknown scale is expressed as:

Formula (1) can be reformulated as:

The essential matrix in formula (2) is E = [t] _× R;

Assuming that the rotation matrix provided by the IMU at time i is R _imu and the rotation matrix provided by the IMU at time j is R' _imu , then formula (2) can be re-expressed as:

Where R _y can be expressed as:

I _ki and I _k′j in formula (3) can be expressed as:

f _ki and f _k′j in formula (3) can be expressed as:

Substituting formula (6) and formula (5) into formula (3) yields the constrained generalized camera model:

According to a method for estimating relative pose and scale of a multi-camera system provided by the present invention, the least squares method is used to minimize the algebraic error in the constrained generalized camera model, and a target optimization function is established, including:

Assuming there are n pairs of feature points, n>5, the algebraic error is minimized according to the least squares principle, and the objective function can be obtained:

In formula (8) At the same time, C＝M*M ^T

Formula (8) can be transformed into the problem of solving the minimum eigenvalue of matrix C:

Define the eigenvalue of matrix C as λ. According to the fifth-order expansion, we can get:

det(C-λI)＝f ₅ λ ⁵ +f ₄ λ ⁴ +f ₃ λ ³ +f ₂ λ ² +f ₁ λ+f ₀ (11)

According to the property of eigenvalue det(C-λI)=0, we can get:

-λ ⁵ +f ₄ λ ⁴ +f ₃ λ ³ +f ₂ λ ² +f ₁ λ+f ₀ =0 (12)

Assume that λ is the minimum eigenvalue of matrix C, and according to the property of the minimum value, we get Taking partial derivatives of formula (12), we get:

Let α＝1+y ² , multiply formula (12) by α ⁶ , multiply formula (12) by α ⁷ , and we get

Where β = α ² λ.

According to a method for estimating relative pose and scale of a multi-camera system provided by the present invention, the objective optimization function is converted into a polynomial equation, including:

By multiplying the first equation of formula (14) by β ³ , β ² and β, and multiplying the second equation of formula (14) by β ⁴ , β ³ , β ² and β, the following equations can be obtained:

According to a method for estimating relative position and scale of a multi-camera system provided by the present invention, the relative rotation angle of the polynomial equation is solved by an eigenvalue method to obtain a camera translation vector and scale, including:

Combining formula (14) and formula (15) we get 9 equations and 9 monomials, expressed as:

B _9×9 J _9×1 ＝0 (16)

The matrix B and matrix J in formula (16) can be expressed as:

J＝[β ⁸ β ⁷ β ⁶ β ⁵ β ⁴ β ³ β ² β 1] ^T (18)

The matrix B contains only the unknown number y. Let z = 1/y, and formula (16) can be re-expressed as:

(z ¹⁶ B ₀ +z ¹⁵ B ₁ +z ¹⁴ B ₂ +…+B ₁₆ )J＝0 (19)

The unknown number z is the eigenvalue of the matrix G, and the expression of the matrix G is:

Solving the eigenvalue of matrix G can get the unknown number z, whose reciprocal is the variable y. Substitute y into matrix C, and the eigenvalue vector corresponding to matrix C is Extract the corresponding translation vector and scale.

In a second aspect, the present invention further provides a device for estimating relative pose and scale of a multi-camera system, comprising:

A synchronization module is used to synchronize the time and space of the multi-camera system and the IMU to obtain a synchronized multi-camera inertial navigation system;

Establish a module for obtaining camera corresponding point pairs in the synchronous multi-camera inertial navigation system, determining a generalized camera model from the camera corresponding point pairs, substituting the IMU rotation matrix in the vertical direction into the generalized camera model, and obtaining a constrained generalized camera model;

An optimization module, configured to minimize the algebraic error in the constrained generalized camera model by using a least squares method, establish a target optimization function, and convert the target optimization function into a polynomial equation;

The estimation module is used to solve the relative rotation angle of the polynomial equation by using an eigenvalue method to obtain a camera translation vector and a scale.

In a third aspect, the present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a method for estimating the relative pose and scale of a multi-camera system as described in any one of the above-mentioned methods is implemented.

In a fourth aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a method for estimating relative pose and scale of a multi-camera system as described in any one of the above.

In a fifth aspect, the present invention also provides a computer program product, comprising a computer program, which, when executed by a processor, implements a method for estimating relative pose and scale of a multi-camera system as described in any one of the above.

The method and device for estimating the relative posture and scale of a multi-camera system provided by the present invention synchronize the IMU and the camera in time and space, reduce the degree of freedom of the relative rotation matrix at adjacent moments from 3 to 1, use least squares to minimize the algebraic error to establish an optimization function, convert the cost function into two polynomial equations containing only two unknowns, and finally use the eigenvalue method to solve the relative rotation angle, thereby effectively solving the problem of estimating the relative posture and scale of the multi-camera system when the internal scale information is unknown.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

FIG1 is a schematic flow chart of a method for estimating relative pose and scale of a multi-camera system provided by the present invention;

FIG2 is a diagram of an imaging model of a multi-camera system provided by the present invention;

FIG3 is a diagram of a multi-camera system imaging model provided by the present invention when the vertical line is known;

FIG4 is a schematic diagram of the structure of a device for estimating relative pose and scale of a multi-camera system provided by the present invention;

FIG. 5 is a schematic diagram of the structure of an electronic device provided by the present invention.

DETAILED DESCRIPTION

In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

In view of the limitations of the prior art, the present invention proposes a method for estimating the relative pose and scale of a multi-camera system from 2D-2D information.

FIG1 is a flow chart of a method for estimating relative pose and scale of a multi-camera system provided by an embodiment of the present invention. As shown in FIG1 , the method includes:

Step 100: Performing time synchronization and space synchronization on the multi-camera system and the IMU to obtain a synchronized multi-camera inertial navigation system;

Step 200: Obtain camera corresponding point pairs in the synchronous multi-camera inertial navigation system, determine a generalized camera model based on the camera corresponding point pairs, substitute the IMU rotation matrix in the vertical direction into the generalized camera model, and obtain a constrained generalized camera model;

Step 300: using the least square method to minimize the algebraic error in the constrained generalized camera model, establishing a target optimization function, and converting the target optimization function into a polynomial equation;

Step 400: using the eigenvalue method to solve the relative rotation angle of the polynomial equation to obtain the camera translation vector and scale.

It is understandable that with the popularity of inertial measurement units, more and more moving platforms are equipped with inertial measurement units (IMU) and cameras at the same time. First, the IMU and the camera are synchronized in time and space, mainly including time synchronization and space synchronization. The IMU can provide vertical direction information, so the coordinate system can be aligned in a common direction. The degrees of freedom (DOF) of the relative rotation matrix at adjacent moments is reduced from 3 to 1. First, the optimization function is established by using least squares to minimize the algebraic error. Then, the cost function is converted into two polynomial equations containing only two unknowns. Finally, the eigenvalue method is used to solve the relative rotation angle, and the corresponding eigenvector is the translation vector.

Specifically, the implementation process includes:

The first step is the spatiotemporal synchronization of the inertial navigation system and the camera.

(1) Time synchronization:

Time synchronization is to unify the time of the two sensors, the inertial navigation and the camera, into the same high-precision time system. The embodiment of the present invention modulates the clock difference calculated according to the B1C observation value of Beidou into a PPS signal to provide time synchronization for the sensor. The GNSS timestamp is entered into the IMU data. Although the inertial navigation time may not be completely synchronized with the GNSS time to correspond to the whole second, it can be calculated through interpolation. Camera sensor synchronization can control the triggering of the camera through the GNSS time, so that each frame of the picture has a corresponding timestamp.

(2) Spatial synchronization

For a multi-camera system, you need to select a reference frame. For convenience, you can establish the reference frame on a certain camera. Assume that it is established on camera No. 1 in the multi-camera system. For the conversion between the reference frame (camera No. 1 coordinate system) and the IMU coordinate system, you can directly use the open source Kalibr solution and software to calibrate the IMU and camera. Kalibr is a visual-inertial calibration toolbox.

The second step is inertial navigation/vision fusion positioning.

The biggest difference between a multi-camera system and a single-camera system is that all rays of a multi-camera system cannot intersect at the same point, which means that it does not conform to the pinhole camera model. When representing the generalized camera model, the Plücker line is usually used. The Plücker line is a 6×1 column vector, the first three rows of which represent the direction of the ray, and the last three rows represent the moment of the corresponding line.

Assume that there is a pair of corresponding points (x _ki ,x _k'j ,R _k ,R _k' ,t _k ,t _k' ) in a multi-camera system. x _ki represents the normalized coordinates of the image obtained by camera k at time i. x _k'j represents the normalized coordinates of the image obtained by camera k' at time j. The rotation matrix and translation vector of camera k relative to the reference frame are R _k and t _k respectively. The rotation matrix and translation vector of camera k' relative to the reference frame are R _k' and t _k' respectively.

As shown in Figure 2, assume that the rotation matrix of the reference frame from time i to time j is R, the translation vector is t, and the scale is s. When the scale is unknown, the generalized camera model can be expressed as:

Formula (1) can be reformulated as:

The essential matrix in formula (2) is E = [t] _× R;

The pitch and roll angles provided by the IMU are more accurate than the yaw angles, so the present invention uses the pitch angle (θ _x ) and roll angle (θ _z ) provided by the IMU to align the camera in the vertical direction. Assume that the rotation matrix provided by the IMU at time i is R _imu , and the rotation matrix provided by the IMU at time j is R' _imu . Formula (2) can be re-expressed as:

Where R _y can be expressed as:

I _ki and I _k′j in formula (3) can be expressed as:

f _ki and f _k′j in formula (3) can be expressed as:

Substituting formula (6) and formula (5) into formula (3) yields:

(1) Establish the objective function:

Assuming that there are n (n>5) pairs of feature points, the algebraic error is minimized according to the least squares principle, and the objective function can be obtained:

In formula (8) At the same time, C = M*M ^T ;

Formula (8) can be transformed into the problem of solving the minimum eigenvalue of matrix C:

Define the eigenvalue of matrix C as λ. According to the fifth-order expansion, we can get:

det(C-λI)＝f ₅ λ ⁵ +f ₄ λ ⁴ +f ₃ λ ³ +f ₂ λ ² +f ₁ λ+f ₀ (11)

According to the property of eigenvalue det(C-λI)=0, we can get:

-λ ⁵ +f ₄ λ ⁴ +f ₃ λ ³ +f ₂ λ ² +f ₁ λ+f ₀ =0 (12)

For the convenience of description, assume that λ is the minimum eigenvalue of the matrix C. According to the property of the minimum value, we can get Taking partial derivatives of formula (12), we get:

Let α＝1+y ² , multiply formula (12) by α ⁶ , multiply formula (12) by α ⁷ , and we get

Where β = α ² λ.

(2) Polynomial feature solver:

Formula (14) contains 2 equations and 6 monomials. In order to make the number of equations equal to the number of monomials, the first equation of formula (14) is multiplied by β ³ , β ² , β, and the second equation of formula (14) is multiplied by β ⁴ , β ³ , β ² , β, and the following equations can be obtained:

Combining formula (14) and formula (15) we get 9 equations and 9 monomials, expressed as:

B _9×9 J _9×1 ＝0 (16)

The matrix B and matrix J in formula (16) can be expressed as:

The matrix B contains only the unknown number y. Let z = 1/y, and formula (16) can be re-expressed as:

(z ¹⁶ B ₀ +z ¹⁵ B ₁ +z ¹⁴ B ₂ +…+B ₁₆ )J＝0 (19)

The unknown number z is the eigenvalue of the matrix G, and the expression of the matrix G is:

Solving the eigenvalue of matrix G can get the unknown number z, whose reciprocal is the variable y. Substitute y into matrix C, and the eigenvalue vector corresponding to matrix C is Finally, the corresponding translation vector and scale are extracted, and the adjusted system imaging model is shown in FIG3 .

The following is a description of the relative pose and scale estimation device for a multi-camera system provided by the present invention. The relative pose and scale estimation device for a multi-camera system described below and the relative pose and scale estimation method for a multi-camera system described above can refer to each other.

FIG4 is a schematic diagram of the structure of a device for estimating relative pose and scale of a multi-camera system provided by an embodiment of the present invention. As shown in FIG4 , the device comprises: a synchronization module 41, an establishment module 42, an optimization module 43 and an estimation module 44, wherein:

The synchronization module 41 is used to perform time synchronization and space synchronization on the multi-camera system and the IMU to obtain a synchronized multi-camera inertial navigation system; the establishment module 42 is used to obtain the corresponding point pairs of cameras in the synchronized multi-camera inertial navigation system, determine the generalized camera model based on the corresponding point pairs of cameras, substitute the IMU rotation matrix in the vertical direction into the generalized camera model to obtain a constrained generalized camera model; the optimization module 43 is used to minimize the algebraic error in the constrained generalized camera model by using the least squares method, establish a target optimization function, and convert the target optimization function into a polynomial equation; the estimation module 44 is used to solve the relative rotation angle of the polynomial equation by using the eigenvalue method to obtain the camera translation vector and scale.

FIG5 illustrates a schematic diagram of the physical structure of an electronic device. As shown in FIG5 , the electronic device may include: a processor 510, a communication interface 520, a memory 530 and a communication bus 540, wherein the processor 510, the communication interface 520 and the memory 530 communicate with each other through the communication bus 540. The processor 510 may call the logic instructions in the memory 530 to execute a method for estimating the relative pose and scale of a multi-camera system, the method comprising: performing time synchronization and space synchronization on the multi-camera system and the IMU to obtain a synchronized multi-camera inertial navigation system; obtaining a pair of camera corresponding points in the synchronized multi-camera inertial navigation system, determining a generalized camera model from the pair of camera corresponding points, substituting the IMU rotation matrix in the vertical direction into the generalized camera model to obtain a constrained generalized camera model; using the least squares method to minimize the algebraic error in the constrained generalized camera model, establishing a target optimization function, and converting the target optimization function into a polynomial equation; using the eigenvalue method to solve the relative rotation angle of the polynomial equation to obtain a camera translation vector and scale.

In addition, the logic instructions in the above-mentioned memory 530 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the method for estimating the relative posture and scale of a multi-camera system provided by the above methods. The method includes: performing time synchronization and spatial synchronization on the multi-camera system and the IMU to obtain a synchronized multi-camera inertial navigation system; obtaining camera corresponding point pairs in the synchronized multi-camera inertial navigation system, determining a generalized camera model from the camera corresponding point pairs, substituting the IMU rotation matrix in the vertical direction into the generalized camera model to obtain a constrained generalized camera model; using the least squares method to minimize the algebraic error in the constrained generalized camera model, establishing a target optimization function, and converting the target optimization function into a polynomial equation; using the eigenvalue method to solve the relative rotation angle of the polynomial equation to obtain the camera translation vector and scale.

On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the method for estimating the relative posture and scale of a multi-camera system provided by the above-mentioned methods, the method comprising: performing time and space synchronization on the multi-camera system and the IMU to obtain a synchronized multi-camera inertial navigation system; obtaining camera corresponding point pairs in the synchronized multi-camera inertial navigation system, determining a generalized camera model from the camera corresponding point pairs, substituting the IMU rotation matrix in the vertical direction into the generalized camera model to obtain a constrained generalized camera model; using the least squares method to minimize the algebraic error in the constrained generalized camera model, establishing a target optimization function, and converting the target optimization function into a polynomial equation; using the eigenvalue method to solve the relative rotation angle of the polynomial equation to obtain the camera translation vector and scale.

The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM/RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims (10)

1. A method for estimating relative pose and scale of a multi-camera system, comprising: Perform time and space synchronization on the multi-camera system and the inertial measurement unit IMU to obtain a synchronized multi-camera inertial navigation system; Obtaining camera corresponding point pairs in the synchronous multi-camera inertial navigation system, determining a generalized camera model from the camera corresponding point pairs, substituting the IMU rotation matrix in the vertical direction into the generalized camera model to obtain a constrained generalized camera model; Minimize the algebraic error in the constrained generalized camera model by using the least squares method, establish a target optimization function, and convert the target optimization function into a polynomial equation; The relative rotation angle of the polynomial equation is solved by the eigenvalue method to obtain the camera translation vector and scale. 2. The method for estimating relative pose and scale of a multi-camera system according to claim 1, characterized in that the multi-camera system and the IMU are synchronized in time and space to obtain a synchronized multi-camera inertial navigation system, comprising: The clock error is calculated based on the BIC observation value of BeiDou and modulated into a PPS signal, and the PPS provides time synchronization for sensors of multiple cameras; Taking any camera in the multi-camera system as the reference frame, the camera reference coordinate system is determined. Based on the camera reference coordinate system, the visual-inertial calibration toolbox Kalibr is used to calibrate the remaining camera coordinate systems and the IMU coordinate system. 3. The method for estimating relative pose and scale of a multi-camera system according to claim 1, characterized in that the corresponding point pairs of cameras in the synchronous multi-camera inertial navigation system are obtained, a generalized camera model is determined by the corresponding point pairs of cameras, and the IMU rotation matrix in the vertical direction is substituted into the generalized camera model to obtain a constrained generalized camera model, comprising: Obtain the camera corresponding point pair (x _ki ,x _k'j ,R _k ,R _k' ,t _k ,t _k' ) based on the Plücker line, where x _ki represents the normalized coordinates of the image obtained by camera k at time i, x _k'j represents the normalized coordinates of the image obtained by camera k' at time j, the rotation matrix and translation vector of camera k relative to the reference frame are R _k and t _k respectively, and the rotation matrix and translation vector of camera k' relative to the reference frame are R _k' and t _k' respectively; Assuming that the rotation matrix of the reference frame from time i to time j is R, the translation vector is t, and the scale is s, the generalized camera model with unknown scale is expressed as: Formula (1) can be reformulated as: The essential matrix in formula (2) is E = [t] _× R; Assuming that the rotation matrix provided by the IMU at time i is R _imu and the rotation matrix provided by the IMU at time j is R' _imu , then formula (2) can be re-expressed as: Where R _y can be expressed as: I _ki and I _k′j in formula (3) can be expressed as: f _ki and f _k′j in formula (3) can be expressed as: Substituting formula (6) and formula (5) into formula (3) yields the constrained generalized camera model: 4. The method for estimating relative pose and scale of a multi-camera system according to claim 1, characterized in that the algebraic error in the constrained generalized camera model is minimized by using the least squares method to establish a target optimization function, comprising: Assuming there are n pairs of feature points, n>5, the algebraic error is minimized according to the least squares principle, and the objective function can be obtained: In formula (8) At the same time, C = M*M ^T ; Formula (8) can be transformed into the problem of solving the minimum eigenvalue of matrix C: Define the eigenvalue of matrix C as λ. According to the fifth-order expansion, we can get: det(C-λI)＝f ₅ λ ⁵ +f ₄ λ ⁴ +f ₃ λ ³ +f ₂ λ ² +f ₁ λ+f ₀ (11) According to the property of eigenvalue det(C-λI)=0, we can get: -λ ⁵ +f ₄ λ ⁴ +f ₃ λ ³ +f ₂ λ ² +f ₁ λ+f ₀ =0 (12) Assume that λ is the minimum eigenvalue of matrix C, and according to the property of the minimum value, we get Taking partial derivatives of formula (12), we get: Let α＝1+y ² , multiply formula (12) by α ⁶ , multiply formula (12) by α ⁷ , and we get Where β = α ² λ. 5. The method for estimating relative pose and scale of a multi-camera system according to claim 4, wherein converting the objective optimization function into a polynomial equation comprises: By multiplying the first equation of formula (14) by β ³ , β ² and β, and multiplying the second equation of formula (14) by β ⁴ , β ³ , β ² and β, the following equations can be obtained: 6. The method for estimating relative position and scale of a multi-camera system according to claim 5, characterized in that the relative rotation angle of the polynomial equation is solved by an eigenvalue method to obtain a camera translation vector and scale, comprising: Combining formula (14) and formula (15) we get 9 equations and 9 monomials, expressed as: B _9×9 J _9×1 ＝0 (16) The matrix B and matrix J in formula (16) can be expressed as: J＝[β ⁸ β ⁷ β ⁶ β ⁵ β ⁴ β ³ β ² β 1] ^T (18) The matrix B contains only the unknown number y. Let z = 1/y, and formula (16) can be re-expressed as: (z ¹⁶ B ₀ +z ¹⁵ B ₁ +z ¹⁴ B ₂ +…+B ₁₆ )J＝0 (19) The unknown number z is the eigenvalue of the matrix G, and the expression of the matrix G is: Solving the eigenvalue of matrix G can get the unknown number z, whose reciprocal is the variable y. Substitute y into matrix C, and the eigenvalue vector corresponding to matrix C is Extract the corresponding translation vector and scale. 7. A device for estimating relative pose and scale of a multi-camera system, comprising: A synchronization module is used to synchronize the time and space of the multi-camera system and the IMU to obtain a synchronized multi-camera inertial navigation system; Establish a module for obtaining camera corresponding point pairs in the synchronous multi-camera inertial navigation system, determining a generalized camera model from the camera corresponding point pairs, substituting the IMU rotation matrix in the vertical direction into the generalized camera model, and obtaining a constrained generalized camera model; An optimization module, configured to minimize the algebraic error in the constrained generalized camera model by using a least squares method, establish a target optimization function, and convert the target optimization function into a polynomial equation; The estimation module is used to solve the relative rotation angle of the polynomial equation by using an eigenvalue method to obtain a camera translation vector and a scale. 8. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for estimating the relative pose and scale of a multi-camera system according to any one of claims 1 to 6 is implemented. 9. A non-transitory computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the method for estimating relative pose and scale of a multi-camera system as claimed in any one of claims 1 to 6 is implemented. 10. A computer program product, comprising a computer program, characterized in that when the computer program is executed by a processor, the method for estimating relative pose and scale of a multi-camera system according to any one of claims 1 to 6 is implemented.