CN114926538A - External parameter calibration method and device for monocular laser speckle projection system - Google Patents

Abstract

The application discloses an external parameter calibration method and device of a monocular laser speckle projection system. The external parameter calibration method of the monocular laser speckle projection system calculates a space plane equation by using a calibration plate and calculates the three-dimensional coordinates of the dotted speckle image points according to the space plane equation; then estimating the optical center and the optical axis position of the laser speckle projector according to the three-dimensional coordinates of the speckle image points with the same name; and finally, calculating the position and posture relation between the camera and the laser speckle projector according to a preset laser speckle projector coordinate system, and generating a plane homography matrix between the camera image and the virtual image of the laser speckle projector by utilizing the position and posture relation so as to realize the calibration of the external parameters of the monocular laser speckle projection system. The external parameter calibration method of the monocular laser speckle projection system can obviously improve the measurement efficiency of external parameter calibration and can obviously improve the measurement precision.

Description

External parameter calibration method and device for monocular laser speckle projection system

technical field

The present application relates to the field of computer vision, and in particular, to a method and device for calibrating external parameters of a monocular laser speckle projection system.

Background technique

High-precision depth measurement is one of the important research topics in the field of computer vision. Traditional depth measurement methods mainly include Time-of-Flight (ToF for short) and binocular stereo vision methods. ToF obtains the depth information of the target by measuring the time-of-flight or phase transformation of modulated light, and has the advantages of less influence by ambient light, fast measurement speed, and long-distance measurement. However, the measurement accuracy of ToF can only reach the centimeter level, which cannot meet the needs of some high-precision measurement tasks.

The binocular stereo vision method obtains the disparity map of the target area by matching two image pairs captured by cameras at different positions, thereby obtaining the depth information of the target to be measured. This method usually uses block matching or semi-global matching algorithms to search for similar regions of image pairs, which can achieve sub-pixel level matching accuracy. Since the binocular stereo vision method performs image matching based on visual features, it will be difficult to match scenes with obvious changes in ambient light or lack of texture features, resulting in large matching errors or even matching failures. In addition, the huge computational load caused by the image feature extraction and matching process limits its application in real-time measurement.

In order to overcome the shortcomings of the above-mentioned ToF and binocular stereo vision methods, researchers invented a projection system based on laser speckle images. The laser in the laser speckle projector emits infrared laser light, which passes through a diffraction grating (ground glass) to form a highly random speckle image. The system uses an infrared camera to capture speckle images on the surface of the target to be measured, which greatly reduces the influence of ambient light on the measurement. At the same time, based on the randomness of the speckle image, the measurement time of the laser speckle projection system can be shortened to a single exposure time, thereby realizing real-time dynamic measurement.

According to the number of cameras in the speckle projection system, it can be divided into two categories: binocular speckle system and monocular speckle system. The binocular laser speckle projection system is equivalent to the binocular stereo vision system with speckle images. The speckle image with high randomness endows the textureless area with rich feature information, which significantly improves the image matching accuracy and measurement accuracy of the binocular stereo vision system. However, the manufacturing cost of the binocular speckle system is relatively high, and the system calibration steps are relatively complicated. The monocular laser speckle projection system only consists of one infrared camera and one laser speckle projector, and its system is more compact and less expensive.

Since the laser speckle projector does not have a standard speckle image, and as the distance between the projector and the target to be measured increases, the speckle image will be deformed to varying degrees. Before leaving the factory or after returning to the factory for repair, the monocular laser The speckle projection system needs to use a high-precision ranging instrument to shoot the corresponding speckle images in advance at different standard distances, which makes the external parameter calibration of the monocular laser speckle projection system complicated and cumbersome.

SUMMARY OF THE INVENTION

In view of this, the embodiments of the present application provide a method and device for calibrating external parameters of a monocular laser speckle projection system, so as to solve the problem that the external parameter calibration of the monocular laser speckle projection system in the prior art is too complicated and cumbersome .

In a first aspect, an embodiment of the present application provides a method for calibrating external parameters of a monocular laser speckle projection system, the method comprising:

A camera calibration image is collected under a monocular laser speckle projection system, the monocular laser speckle projection system includes a camera and a laser speckle projector;

Calibrate camera parameters according to the camera calibration image;

A calibration plate is prepared, and speckle images are collected under the monocular laser speckle projection system, wherein the calibration plate is provided with at least three marking features;

Extract the sign feature of the speckle image according to the camera parameters, and calculate the space plane equation of the calibration plate;

obtaining speckle image points with the same name according to the speckle image;

Calculate the three-dimensional coordinates of the speckle image point with the same name according to the space plane equation of the calibration plate;

Estimating the position of the optical center and the optical axis of the laser speckle projector according to the three-dimensional coordinates of the speckle image point with the same name;

calculating the pose relationship between the camera and the laser speckle projector according to the preset laser speckle projector coordinate system;

The plane homography matrix between the camera image and the laser speckle projector virtual image is calculated according to the pose relationship, and the laser speckle projector virtual image is generated.

The above aspect and any possible implementation manner further provide an implementation manner, wherein the sign feature is a diagonal sign, and the calculation of the space plane equation of the calibration plate includes:

A corresponding world coordinate system is established on the calibration board, with the diagonal mark on the upper left as the origin of the coordinate system, the x-axis passes vertically downward through the diagonal mark on the lower left, and the y-axis passes horizontally to the right through the upper right The diagonal mark of the square, the z-axis is perpendicular to the calibration plate outward, wherein the coordinates of the diagonal mark in the world coordinate system are expressed as (X _w , Y _w , 1), the diagonal The relationship between the pixel coordinates (u, v) of the logo and the three-dimensional coordinates (X _w , Y _w , 1) of the world coordinate system is expressed as:

Wherein, K _c is the internal parameter matrix of the camera, s is the scale coefficient, R is the rotation matrix, and T is the translation vector; the coordinates of the diagonal mark on the surface of the calibration plate in the camera coordinate system (X _c ,Y _c ,Z _c ):

The space plane equation of the calibration plate in the camera coordinate system is obtained by least squares fitting.

The above aspect and any possible implementation manner further provide an implementation manner, wherein the calculating the three-dimensional coordinates of the speckle image point with the same name includes:

According to the speckle image point coordinates (u, v) and the camera parameters, the equivalent three-dimensional coordinates (X _c , Y _c , Z _c ) of the speckle image point in the camera coordinate system are calculated as:

where (C _x , C _y ) are the coordinates of the principal point of the camera, d _x and _dy are the physical dimensions of the pixel in the x-axis and y-axis directions, respectively, and f is the camera focal length;

The space straight line equation through the speckle image point and the origin of the camera coordinate system can be expressed as:

According to the space straight line equation and the space plane equation of the calibration plate, the three-dimensional coordinates of the speckle image points with the same name are calculated to obtain a three-dimensional coordinate set corresponding to the speckle image points with the same name.

According to the above aspect and any possible implementation manner, an implementation manner is further provided. The calculating the pose relationship between the camera and the laser speckle projector includes:

Wherein, the normalized direction vector of the optical axis of the laser speckle projector in the camera coordinate system is V _c =[v _x , _vy ,v _z ] ^T , which is represented as V in the laser speckle projector coordinate system _p = [0,0,1] ^T , the relationship between the two can be expressed as:

Calculate A _x and A _y :

A preset A _z value is selected, and a rotation matrix R is obtained by calculation, wherein A _x , A _y , and A _z are the Euler angles of the rotation matrix R;

The coordinates of the optical center of the laser speckle projector in the camera coordinate system are (x _p , y _p , z _p ), and the coordinates in the laser speckle projector coordinate system are (0,0,0) , the relationship between the two can be expressed as:

Get the translation vector T:

The above aspects and any possible implementation manners further provide an implementation manner in which the plane homography matrix between the camera image and the virtual image of the laser speckle projector is calculated according to the pose relationship, including:

It is assumed that the coordinates of the speckle image point in the camera coordinate system are (x ₁ , y ₁ , z ₁ ), and in the laser speckle projector coordinate system is (x ₂ , y ₂ , z ₂ ), and both The relationship can be expressed as:

According to the speckle image point on the surface of the calibration plate, the coordinates of the speckle image point in the camera coordinate system satisfy the following equation:

Wherein, n is the normalized normal vector of the calibration plate, and d is the distance from the origin of the camera coordinate system to the calibration plate;

Combining the above two formulas, we get:

Then the plane homography matrix between the camera image and the projector virtual image is expressed as:

Wherein, K _c and K _p are the internal parameter matrices of the camera and the laser speckle projector, and K _p =K _c .

According to the above aspect and any possible implementation manner, an implementation manner is further provided. The generating the virtual image of the laser speckle projector includes:

According to the plane homography matrix between the camera image and the laser speckle projector virtual image, the camera image is mapped to the virtual projector plane to obtain the laser speckle projector virtual image.

According to the above aspect and any possible implementation manner, an implementation manner is further provided, after the virtual image of the laser speckle projector is generated, the laser speckle projector is used as a second camera, A binocular camera system is formed with the camera in the monocular laser speckle projection system.

In a second aspect, an embodiment of the present application provides a device for calibrating external parameters of a monocular laser speckle projection system, the device comprising:

a camera calibration image acquisition module, used for acquiring a camera calibration image under a monocular laser speckle projection system, the monocular laser speckle projection system comprising a camera and a laser speckle projector;

a camera parameter calibration module for calibrating camera parameters according to the camera calibration image;

a speckle image acquisition module, used for making a calibration plate, and collecting speckle images under the monocular laser speckle projection system, wherein the calibration plate is provided with at least three marking features;

a space plane equation calculation module, configured to extract the sign feature of the speckle image according to the camera parameters, and calculate the space plane equation of the calibration plate;

a speckle image point acquisition module with the same name, used for acquiring the speckle image point with the same name according to the speckle image;

A speckle image point calculation module with the same name, for calculating the three-dimensional coordinates of the speckle image point with the same name according to the space plane equation of the calibration plate;

an optical center and optical axis estimation module, configured to estimate the optical center and optical axis positions of the laser speckle projector according to the three-dimensional coordinates of the speckle image point with the same name;

a pose relationship calculation module, configured to calculate the pose relationship between the camera and the laser speckle projector according to a preset coordinate system of the laser speckle projector;

The virtual speckle image generation module is configured to calculate the plane homography matrix between the camera image and the virtual image of the laser speckle projector according to the pose relationship, and generate the virtual image of the laser speckle projector.

In a third aspect, the present application provides a computer device comprising a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor, the processor executing the computer-readable instructions When performing the steps of the method for calibrating the external parameters of the monocular laser speckle projection system according to the first aspect.

In a fourth aspect, the present application provides a computer-readable storage medium, where computer-readable instructions are stored in the computer-readable storage medium, and when the computer-readable instructions are executed by a processor, any one of the first aspect is implemented The steps of the extrinsic parameter calibration method of monocular laser speckle projection system.

In the embodiment of the present application, the present application uses the calibration plate to calculate the space plane equation, and calculates the three-dimensional coordinates of the speckle image point with the same name according to the space plane equation; The position of the optical center and optical axis; finally, according to the preset laser speckle projector coordinate system, the pose relationship between the camera and the laser speckle projector is calculated, and the camera image and the laser speckle projector virtual camera can be generated by using the pose relationship. The plane homography matrix between the images can realize the calibration of the external parameters of the monocular laser speckle projection system. The present application does not need to use an accurate ranging device to shoot corresponding speckle images at different standard distances, which can significantly improve the measurement efficiency of external parameter calibration and the measurement accuracy.

Description of drawings

In order to illustrate the technical solutions of the embodiments of the present application more clearly, the following briefly introduces the accompanying drawings used in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained from these drawings without any creative effort.

1 is a flowchart of a method for calibrating external parameters of a monocular laser speckle projection system in an embodiment of the present application;

2 is a schematic diagram of a calibration plate in the embodiment of the present application;

FIG. 3 is a schematic diagram of an optical axis and an optical center of a laser speckle projector in an embodiment of the present application.

Detailed ways

In order to better understand the technical solutions of the present application, the embodiments of the present application are described in detail below with reference to the accompanying drawings.

It should be clear that the described embodiments are only a part of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the present application. As used in the embodiments of this application and the appended claims, the singular forms "a," "the," and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise.

It should be understood that the term "and/or" used in this document is only one of the same fields to describe the associated objects, indicating that three relationships can exist, for example, A and/or B, which can mean: A alone exists, and A exists at the same time and B, there are three cases of B alone. In addition, the character "/" in this document generally indicates that the related objects are an "or" relationship.

It should be understood that although the terms first, second, third, etc. may be used in the embodiments of the present application to describe the preset range and the like, these preset ranges should not be limited to these terms. These terms are only used to distinguish preset ranges from one another. For example, without departing from the scope of the embodiments of the present application, the first preset range may also be referred to as the second preset range, and similarly, the second preset range may also be referred to as the first preset range.

Depending on the context, the word "if" as used herein can be interpreted as "at" or "when" or "in response to determining" or "in response to detecting." Similarly, the phrases "if determined" or "if detected (the stated condition or event)" can be interpreted as "when determined" or "in response to determining" or "when detected (the stated condition or event)," depending on the context )" or "in response to detection (a stated condition or event)".

FIG. 1 is a flowchart of a method for calibrating external parameters of a monocular laser speckle projection system in an embodiment of the present application. As shown in Figure 1, the external parameter calibration method of the monocular laser speckle projection system specifically includes the following steps:

S10: The camera calibration image is collected under the monocular laser speckle projection system. The monocular laser speckle projection system includes a camera and a laser speckle projector.

In one embodiment, the infrared camera and the laser speckle projector are fixed on a tripod at an appropriate angle to form a monocular laser speckle projection system. Afterwards, select a checkerboard calibration board of suitable size and place it within the field of view of the camera, turn on the camera and adjust the focus to shoot the image of the checkerboard calibration board. During this period, the position and posture of the checkerboard calibration board need to be continuously adjusted.

S20: Calibrate camera parameters according to the camera calibration image.

In one embodiment, according to the checkerboard calibration plate image set captured by the camera, the Zhang Zhengyou calibration method is used to calculate the internal parameters of the camera, including focal length and principal point coordinates, as well as radial distortion and tangential distortion.

S30: A calibration plate is prepared, and a speckle image is collected under a monocular laser speckle projection system, wherein the calibration plate is provided with at least three marking features.

Wherein, the calibration plate includes at least three mark features, which are set on the plane of the calibration plate by means of marks or specific features. This application does not limit the signature features.

Specifically, the calibration plate may be provided with marker features that can be used to calculate the space plane equation of the calibration plate in the form of diagonal markers. It can be understood that three or more sign features on the calibration board can be set, and the more the set features (to help with calculation and verification), the more helpful it is to improve the accuracy of external parameter calibration of the monocular laser speckle projection system. In order to improve the measurement accuracy of the monocular laser speckle projection system.

In one embodiment, a diagonal mark calibration paper with a pair of corner marks on each of the four corners and blank positions is printed on a sheet, and the diagonal mark calibration paper is closely attached to the flat plate to obtain a calibration plate with diagonal marks. Place the calibration plate in the field of view of the monocular laser speckle system, turn on the laser speckle projector, project the speckle image to the surface of the calibration plate, and record the corresponding speckle image with the camera. During this period, it is necessary to continuously adjust the position and attitude of the calibration plate.

S40 : Extract the signature features of the speckle image according to the camera parameters, and calculate the space plane equation of the calibration plate.

In one embodiment, the marking feature in the calibration plate may specifically be a diagonal mark, that is, a mark arranged at a diagonal position of the calibration plate.

In one embodiment, the distortion of the speckle image can be corrected according to the camera distortion coefficient (including radial distortion and tangential distortion) obtained by solving in S20. The Harris corner detection algorithm is used to identify the diagonal signs in the speckle image, and extract the diagonal signs, so as to calculate the space plane equation of the calibration plate according to the diagonal signs.

Further, a corresponding world coordinate system can be established on the calibration board, with the upper left diagonal mark as the origin of the coordinate system, the x-axis vertically downwards passes through the lower left diagonal mark, and the y-axis passes horizontally to the right through the upper right corner. Diagonal sign, the z-axis is perpendicular to the calibration plate outwards. Fig. 2 is a schematic diagram of a calibration plate in the embodiment of the present application. As can be seen from Fig. 2, a pair of corner marks are provided on each of the four corners of the calibration plate, and the world coordinate system established according to the diagonal marks is shown in Fig. 2 shown on each axis.

Understandably, since the physical distance between the diagonal markers is known, the coordinates of the diagonal markers in the world coordinate system are known, and the coordinates of the diagonal markers in the world coordinate system are expressed as (X _w , Y _w , 1), the relationship between the pixel coordinates (u, v) of the diagonal sign and the three-dimensional coordinates (X _w , Y _w , 1) of the world coordinate system is expressed as:

Among them, K _c is the internal parameter matrix of the camera, s is the scale coefficient, R is the rotation matrix, T is the translation vector, and R and T describe the pose relationship between the camera coordinate system and the world coordinate system; The corner mark coordinate pair is enough to solve R and T, then the coordinates of the diagonal mark on the surface of the calibration plate in the camera coordinate system (X _c , Y _c , Z _c ) can also be obtained:

Finally, the least squares method is used to obtain the space plane equation of the calibration plate in the camera coordinate system.

S50: Acquire speckle image points with the same name according to the speckle image.

In one embodiment, the first speckle image can be used as a reference image for speckle matching, and the digital image correlation method can be used to determine the size of the speckle image point by solving the displacement shape function including the first-order and second-order displacement gradient parameters. The best matching position to obtain the speckle image point set of the same name for the speckle image.

S60: Calculate the three-dimensional coordinates of the speckle image point with the same name according to the space plane equation of the calibration plate.

Understandably, in the distortion-free camera imaging model, the light entering the camera must pass through the optical center of the camera, that is, the origin of the camera coordinate system. After obtaining the space plane equation of the calibration plate, the speckle image of the same name can be calculated according to the space plane equation. 3D coordinates of the point. It can be understood that the speckle image point with the same name is a three-dimensional coordinate on the plane of the calibration plate, and the three-dimensional coordinate is the intersection of the speckle ray emitted by the laser speckle projector and the plane of the calibration plate.

Specifically, according to the speckle image point coordinates (u, v) and camera parameters, the equivalent three-dimensional coordinates (X _c , Y _c , Z _c ) of the speckle image point in the camera coordinate system are calculated as:

Where (C _x ,C _y ) is the principal point coordinate of the camera, d _x and _dy are the physical dimensions of the pixel in the x-axis and y-axis directions, respectively, and f is the camera focal length; through the speckle image point and the camera coordinate system The space line equation at the origin can be expressed as:

According to the space straight line equation and the space plane equation of the calibration plate, the three-dimensional coordinates of the speckle image points with the same name are calculated, and the three-dimensional coordinate set corresponding to the speckle image points with the same name is obtained.

S70: Estimate the position of the optical center and the optical axis of the laser speckle projector according to the three-dimensional coordinates of the speckle image point of the same name.

In one embodiment, during the manufacture of the laser speckle projector, the optical axis of the laser speckle projector is designed to be strictly perpendicular to the diffraction grating and pass through its center position. Therefore, the center points of the speckle images on the calibration plate with different positions and attitudes are all located at the optical axis or the vicinity of the laser speckle projector. According to the three-dimensional coordinate set corresponding to the center point of the speckle image obtained in step S60, the position of the optical axis of the laser speckle projector can be determined by straight line fitting. In addition, the straight line fitted by the speckle with the same name corresponds to the light emitted by the laser speckle projector, wherein the light source point passing through the laser is the optical center of the laser speckle projector. It is understandable that under the influence of factors such as camera calibration error, image matching error, fitting error, etc., the fitted line set will not intersect at one point, but will have different degrees of offset. Therefore, the closest spatial point to the fitted line set is calculated as the optimal projector optical center.

FIG. 3 is a schematic diagram of an optical axis and an optical center of a laser speckle projector in an embodiment of the present application. From Figure 3, the physical space relationship between the speckle with the same name, the optical axis and the (laser speckle) projector on the calibration board can be seen.

S80: Calculate the pose relationship between the camera and the laser speckle projector according to the preset laser speckle projector coordinate system.

In one embodiment, a laser speckle projector coordinate system can be established according to the obtained optical center and optical axis positions of the laser speckle projector, and the laser speckle projector coordinate system is calculated between the camera and the laser speckle projector. pose relationship.

Specifically, the laser speckle projector coordinate system takes the optical center of the laser speckle projector as the origin, the z-axis coincides with the optical axis of the laser speckle projector, and the direction facing the target is the positive direction. The normalized direction vector of the optical axis of the laser speckle projector in the camera coordinate system is V _c =[v _x , _vy ,v _z ] ^T , which is expressed as V _p =[0 in the laser speckle projector coordinate system ,0,1] ^T , the relationship between the two can be expressed as:

Among them, A _x , A _y , and A _z are the Euler angles of the rotation matrix. It should be noted that the rotation order of the Euler angles can adopt the rotation order of xyz, and the Euler angles of other rotation orders are also possible.

The above formula can be simplified to:

A _x and A _y can be calculated:

A _z determines the orientation of the x and y axes of the laser speckle projector coordinate system, and the speckle coordinates of the speckle image. Since the position of the optical center and optical axis of the laser speckle projector has been determined, the absolute physical position of the speckle image in the laser speckle projector is fixed, independent of Euler angles. In the case of ensuring that most or the entire speckle image can be in the virtual image of the laser speckle projector, select an appropriate A _z value, and calculate the rotation matrix R.

The coordinates of the optical center of the laser speckle projector in the camera coordinate system are (x _p , y _p , z _p ), and the coordinates in the laser speckle projector coordinate system are (0, 0, 0). The relationship can be expressed as:

Then the translation vector can be obtained:

S90: Calculate a plane homography matrix between the camera image and the virtual image of the laser speckle projector according to the pose relationship, and generate a virtual image of the laser speckle projector.

Specifically, it is assumed that the coordinates of the speckle image point in the camera coordinate system are (x ₁ , y ₁ , z ₁ ), and in the laser speckle projector coordinate system are (x ₂ , y ₂ , z ₂ ), and the two The relationship can be expressed as:

where R and T are the rotation matrix and translation vector calculated by S80, respectively, which describe the pose relationship between the camera coordinate system and the laser speckle projector coordinate system. Since the speckle image points are located on the surface of the calibration plate, the coordinates of the speckle image points in the camera coordinate system satisfy the following equation:

Among them, n is the normalized normal vector of the calibration plate, and d is the distance from the origin of the camera coordinate system to the calibration plate;

Combining the above two formulas, we get:

Then the plane homography matrix between the camera image and the projector virtual image is expressed as:

where K _c and K _p are the intrinsic parameter matrices of the camera and the laser speckle projector, and K _p =K _c . Since the speckle image is generated by the laser passing through the diffraction grating without passing through the lens group, the distortion of the projector is not considered.

In specific implementation, a calibration plate is arranged on the target to be measured, a laser speckle projector projects a speckle image onto the surface of the calibration plate, and a camera shoots a corresponding speckle image. By calculating the space plane equation of the calibration plate, the plane homography matrix H can be obtained. Understandably, with the plane homography matrix H, the image captured by the camera can be converted to the laser speckle projector, so that the monocular laser speckle projection system of the present application has the same binocular stereo vision capability as the binocular camera system. , the calibration of external parameters can be completed conveniently and efficiently.

Further, in step S90, that is, the step of generating the virtual image of the laser speckle projector, it further includes: mapping the camera image to the virtual image according to the plane homography matrix between the camera image and the virtual image of the laser speckle projector. The projector plane gets the virtual image of the laser speckle projector.

In an embodiment, the plane homography matrix describes the mapping relationship between points on the same plane and different images, and the plane may be the plane where the calibration plate is located or the plane where the image corresponding to the target to be measured is located. Understandably, for example, when a binocular camera is used for shooting, the two cameras will obtain images at different shooting angles on the plane, and the plane homography matrix describes the mapping relationship between the images at different shooting angles. Combining with the solution of the present application, the present application establishes the connection between the camera and the laser speckle projector by means of a space plane, and restores the virtual speckle image corresponding to the laser speckle projector through the plane homography matrix. Further, after recovering the virtual speckle image, the monocular laser speckle projection system in this application is equivalent to the binocular camera system, and the camera and the laser speckle projector (which can be recovered to obtain Virtual speckle image, equivalent to the role of the camera) for stereo correction, online calibration and other operations.

It is understandable that the traditional monocular laser speckle projection system needs to use an accurate ranging device to capture corresponding speckle images at different standard distances, so as to further complete the external parameter calibration, while the monocular laser speckle in this application The projection system can calculate the plane homography matrix of the image conversion relationship between the camera and the laser speckle projector through the calibration plate with the sign feature, so that the laser speckle projector can restore the virtual speckle image, so that the The monocular laser speckle projection system has the ability equivalent to binocular stereo vision, and can improve the image matching speed by means of dual view constraints, thereby improving the measurement accuracy and reducing the complexity of calibration; further, if the equipment is used for a long time, If the measurement accuracy decreases, the user can re-calibrate the external parameters of the monocular laser speckle system quickly without returning to the factory for re-calibration.

In the embodiment of the present application, compared with the traditional monocular laser speckle projection system, the present application realizes the calibration of the external parameters of the monocular laser speckle projection system through the calibration plate, and does not need to use an accurate distance measuring device in different standards. The corresponding speckle image is taken at the distance. This significantly improves measurement efficiency and reduces measurement costs. The monocular laser speckle projection system in this application is equivalent to a binocular camera system with speckle images, which improves the measurement accuracy. In addition, the present application can also calibrate the position of the optical center and optical axis of the laser speckle projector, so that the user can online correct the optical axis deviation of the monocular laser speckle projection system during use.

It should be noted that, in addition to using a single plane calibration plate to realize the external parameter calibration of the monocular laser speckle projection system adopted in this application, the monocular laser speckle projection can also be realized by using a stereo calibration component composed of multiple planes. External parameter calibration of the system. Specifically, with reference to the present application, for a three-dimensional calibration piece including a plurality of planes, the spatial plane equation of each plane can be obtained by calculating the landmark features on the plane, and then by changing the position of the plane, the speckle image point of the same name can be found, and further Then calculate the three-dimensional coordinates of the speckle image point with the same name, so as to estimate the position of the optical center and optical axis of the laser speckle projector, determine the homography matrices of the different planes of the camera and the laser speckle projector, and finally generate the homography matrix according to these homography matrices. Laser speckle projector virtual image. The monocular laser speckle projection system using a stereo calibration piece composed of multiple planes for external parameter calibration has the binocular stereo vision capability equivalent to the binocular camera system, and can conveniently and efficiently complete the external parameter calibration. It should be understood that other calibration methods implemented by using multiple planes should also be included within the protection scope of the present application.

It should be understood that the size of the sequence numbers of the steps in the above embodiments does not mean the sequence of execution, and the execution sequence of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

The embodiment of the present application provides an external parameter calibration device of a monocular laser speckle projection system. The device includes:

The camera calibration image acquisition module is used to acquire the camera calibration image under the monocular laser speckle projection system, and the monocular laser speckle projection system includes a camera and a laser speckle projector;

The camera parameter calibration module is used to calibrate the camera parameters according to the camera calibration image;

The speckle image acquisition module is used to make a calibration plate and collect speckle images under the monocular laser speckle projection system, wherein the calibration plate is provided with at least three marking features;

The space plane equation calculation module is used to extract the signature features of the speckle image according to the camera parameters, and calculate the space plane equation of the calibration plate;

The speckle image point acquisition module of the same name is used to obtain the speckle image point of the same name according to the speckle image;

The speckle image point calculation module with the same name is used to calculate the three-dimensional coordinates of the speckle image point with the same name according to the space plane equation of the calibration plate;

The optical center and optical axis estimation module is used to estimate the optical center and optical axis position of the laser speckle projector according to the three-dimensional coordinates of the speckle image point of the same name;

The pose relationship calculation module is used to calculate the pose relationship between the camera and the laser speckle projector according to the preset laser speckle projector coordinate system;

The virtual speckle image generation module is used to calculate the plane homography matrix between the camera image and the virtual image of the laser speckle projector according to the pose relationship, and generate the virtual image of the laser speckle projector.

In the embodiment of the present application, compared with the traditional monocular laser speckle projection system, the present application realizes the calibration of the external parameters of the monocular laser speckle projection system through the calibration plate, and does not need to use an accurate distance measuring device in different standards. The corresponding speckle image is taken at the distance. This significantly improves measurement efficiency and reduces measurement costs. The monocular laser speckle projection system in this application is equivalent to a binocular camera system with speckle images, which improves the measurement accuracy. In addition, the present application can also calibrate the position of the optical center and optical axis of the laser speckle projector, so that the user can online correct the optical axis deviation of the monocular laser speckle projection system during use.

The present application provides a computer device including a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor, the processor executing the computer-readable instructions when executing the For example, the steps of the external parameter calibration method of the monocular laser speckle projection system are described.

The present application provides a computer-readable storage medium, where the computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, implements the monocular laser speckle projection system according to the embodiment. The steps of the extrinsic parameter calibration method.

Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. Module completion means dividing the internal structure of the device into different functional units or modules to complete all or part of the functions described above.

The above embodiments are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: The recorded technical solutions are modified, or some technical features thereof are equivalently replaced; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the application, and should be included in the application. within the scope of protection.

Claims (10)

1. an external parameter calibration method of a monocular laser speckle projection system, is characterized in that, comprises: A camera calibration image is collected under a monocular laser speckle projection system, the monocular laser speckle projection system includes a camera and a laser speckle projector; Calibrate camera parameters according to the camera calibration image; A calibration plate is prepared, and speckle images are collected under the monocular laser speckle projection system, wherein the calibration plate is provided with at least three marking features; Extract the sign feature of the speckle image according to the camera parameters, and calculate the space plane equation of the calibration plate; obtaining speckle image points with the same name according to the speckle image; Calculate the three-dimensional coordinates of the speckle image point with the same name according to the space plane equation of the calibration plate; Estimating the position of the optical center and the optical axis of the laser speckle projector according to the three-dimensional coordinates of the speckle image point with the same name; calculating the pose relationship between the camera and the laser speckle projector according to the preset laser speckle projector coordinate system; The plane homography matrix between the camera image and the laser speckle projector virtual image is calculated according to the pose relationship, and the laser speckle projector virtual image is generated. 2. The method according to claim 1, wherein the sign feature is a diagonal sign, and the calculation of the space plane equation of the calibration plate comprises: A corresponding world coordinate system is established on the calibration board, with the diagonal mark on the upper left as the origin of the coordinate system, the x-axis passes vertically downward through the diagonal mark on the lower left, and the y-axis passes horizontally to the right through the upper right The diagonal mark of the square, the z-axis is perpendicular to the calibration plate outward, wherein the coordinates of the diagonal mark in the world coordinate system are expressed as (X _w , Y _w , 1), the diagonal The relationship between the pixel coordinates (u, v) of the logo and the three-dimensional coordinates (X _w , Y _w , 1) of the world coordinate system is expressed as:

Wherein, K _c is the internal parameter matrix of the camera, s is the scale coefficient, R is the rotation matrix, and T is the translation vector; the coordinates of the diagonal mark on the surface of the calibration plate in the camera coordinate system (X _c ,Y _c ,Z _c ):

The space plane equation of the calibration plate in the camera coordinate system is obtained by least squares fitting. 3. The method according to claim 1, wherein the calculating the three-dimensional coordinates of the speckle image point with the same name comprises: According to the speckle image point coordinates (u, v) and the camera parameters, the equivalent three-dimensional coordinates (X _c , Y _c , Z _c ) of the speckle image point in the camera coordinate system are calculated as:

where (C _x , C _y ) are the coordinates of the principal point of the camera, d _x and _dy are the physical dimensions of the pixel in the x-axis and y-axis directions, respectively, and f is the camera focal length; The space straight line equation through the speckle image point and the origin of the camera coordinate system can be expressed as:

According to the space straight line equation and the space plane equation of the calibration plate, the three-dimensional coordinates of the speckle image points with the same name are calculated to obtain a three-dimensional coordinate set corresponding to the speckle image points with the same name. 4. The method according to claim 1, wherein the calculating the pose relationship between the camera and the laser speckle projector comprises: The normalized direction vector of the optical axis of the laser speckle projector in the camera coordinate system is V _c =[v _x , _vy ,v _z ] ^T , which is expressed as V _p = [0,0,1] ^T , the relationship between the two can be expressed as:

Calculate A _x and A _y :

A preset A _z value is selected, and a rotation matrix R is obtained by calculation, wherein A _x , A _y , and A _z are the Euler angles of the rotation matrix R; The coordinates of the optical center of the laser speckle projector in the camera coordinate system are (x _p , y _p , z _p ), and the coordinates in the laser speckle projector coordinate system are (0,0,0) , the relationship between the two can be expressed as:

Get the translation vector T:

5. The method according to claim 1, wherein the calculating a plane homography matrix between the camera image and the laser speckle projector virtual image according to the pose relationship comprises: It is assumed that the coordinates of the speckle image point in the camera coordinate system are (x ₁ , y ₁ , z ₁ ), and in the laser speckle projector coordinate system is (x ₂ , y ₂ , z ₂ ), and both The relationship can be expressed as:

According to the speckle image point on the surface of the calibration plate, the coordinates of the speckle image point in the camera coordinate system satisfy the following equation:

Wherein, n is the normalized normal vector of the calibration plate, and d is the distance from the origin of the camera coordinate system to the calibration plate; Combining the above two formulas, we get:

Then the plane homography matrix between the camera image and the projector virtual image is expressed as:

Wherein, K _c and K _p are the internal parameter matrices of the camera and the laser speckle projector, and K _p =K _c . 6. The method according to claim 1, wherein the generating the virtual image of the laser speckle projector comprises: According to the plane homography matrix between the camera image and the laser speckle projector virtual image, the camera image is mapped to the virtual projector plane to obtain the laser speckle projector virtual image. 7. The method according to any one of claims 1-6, wherein after the generating the virtual image of the laser speckle projector, the laser speckle projector is used as a second camera, A binocular camera system is formed with the camera in the monocular laser speckle projection system. 8. An external parameter calibration device of a monocular laser speckle projection system, characterized in that, comprising: a camera calibration image acquisition module, used for acquiring a camera calibration image under a monocular laser speckle projection system, the monocular laser speckle projection system comprising a camera and a laser speckle projector; a camera parameter calibration module for calibrating camera parameters according to the camera calibration image; a speckle image acquisition module, used for making a calibration plate, and collecting speckle images under the monocular laser speckle projection system, wherein the calibration plate is provided with at least three marking features; a space plane equation calculation module, configured to extract the sign feature of the speckle image according to the camera parameters, and calculate the space plane equation of the calibration plate; a speckle image point acquisition module with the same name, used for acquiring the speckle image point with the same name according to the speckle image; A speckle image point calculation module with the same name, for calculating the three-dimensional coordinates of the speckle image point with the same name according to the space plane equation of the calibration plate; an optical center and optical axis estimation module, configured to estimate the optical center and optical axis positions of the laser speckle projector according to the three-dimensional coordinates of the speckle image point with the same name; a pose relationship calculation module, configured to calculate the pose relationship between the camera and the laser speckle projector according to a preset coordinate system of the laser speckle projector; The virtual speckle image generation module is configured to calculate the plane homography matrix between the camera image and the virtual image of the laser speckle projector according to the pose relationship, and generate the virtual image of the laser speckle projector. 9. A computer device comprising a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor, wherein when the processor executes the computer-readable instructions The steps of executing the method for calibrating the external parameters of the monocular laser speckle projection system according to any one of claims 1-7. 10. A computer-readable storage medium storing computer-readable instructions, characterized in that, when the computer-readable instructions are executed by a processor, any one of claims 1 to 7 is implemented. The steps of the external parameter calibration method of the monocular laser speckle projection system are described.

Images