CN112465898B - Object 3D pose tag acquisition method based on checkerboard calibration plate - Google Patents

Abstract

The invention discloses a chessboard marking plate-based object 3D pose tag acquisition method, which relates to the technical field of object pose acquisition and comprises the following steps: establishing a position and pose template of the object and the chessboard pattern calibration plate by depending on the CAD model of the object; calibrating camera internal parameter, camera external parameter and radial distortion parameter; acquiring the pose of an object under a camera coordinate system; and verifying the correctness of pose acquisition. The invention provides a low-cost and high-precision method for acquiring the real pose of the object and a real verification label for an object pose estimation method by using a checkerboard calibration method on the basis of restraining the pose relationship between the object and the checkerboard.

Description

A Method for Obtaining 3D Pose Labels of Objects Based on Checkerboard Calibration Board

technical field

The present invention relates to the technical field of object pose acquisition, in particular to a method for acquiring object pose labels based on a checkerboard calibration board.

Background technique

With the development of deep learning, more and more networks focus on object space pose estimation. The spatial pose estimation network requires a large amount of training data with pose labels, and at the same time requires the pose labels of objects in the real scene as the basis for calculating the network regression accuracy. Therefore, the acquisition of the real pose of the object is of great significance in the construction of the dataset and the evaluation of the accuracy of the pose estimation.

The existing object pose information is mainly obtained in two ways, one is to obtain the simulation data set and its labels through the physical simulation engine, and the other is to obtain the pose information in the real scene indirectly. In the article "Large-scale 6D Object Pose Estimation Dataset for Industrial Bin-Picking", Kilian Kleeberger et al. use the closest point iteration algorithm to obtain object labels for real scenes. In the article "Symmetry Aware Evaluation of 3D Object Detection and Pose Estimation in Scenes of Many Parts in Bulk", Romain Bregier et al. use markers attached to the object to obtain the pose information of the object. When verifying the accuracy of pose estimation, Chien-Ming Lin et al. used a two-degree-of-freedom turntable to provide labels for the relative pose of objects in real scenes in the article "Visual Object Recognition and Pose Estimation Based on a Deep Semantic Segmentation Network".

The way indirect algorithms acquire poses relies on complete and reliable 3D information of the scene and a highly robust template matching algorithm. In some complex scenarios, it is necessary to manually remove the interference information. Using the data generated by the physics engine will cause an inevitable loss of precision in the process of generalization to the real scene. The estimation accuracy of the absolute pose of an object in the camera coordinate system is a direct factor affecting the success rate of object grasping, so the evaluation using relative pose cannot reflect the performance of pose estimation in object grasping tasks.

Therefore, those skilled in the art are devoting themselves to developing a checkerboard calibration board-based object pose label acquisition method that does not rely on 3D point cloud information and does not require a highly robust matching algorithm.

Contents of the invention

In view of the above-mentioned defects in the prior art, the technical problem to be solved by the present invention is to provide a low-cost and high-precision method for obtaining the pose of objects in real scenes, and provide a real verification label for object pose estimation methods.

In order to achieve the above object, the present invention provides a method for obtaining object 3D pose labels based on a checkerboard calibration board, which is characterized in that it includes the following steps:

Step 1, establishing a CAD model of the object;

Step 2, projecting the CAD model in equal proportions to obtain the object outline template;

Step 3, aligning the origin of the object outline template with the checkerboard calibration plate;

Step 4, calculating the transformation matrix from the object coordinate system to the checkerboard coordinate system to obtain the first transformation matrix;

Step 5. Place the object and the checkerboard on the template, and calibrate the camera extrinsic parameters, camera intrinsic parameters and radial distortion parameters;

Step 6. Move the camera so that the object and the checkerboard are within the camera's field of view at the same time, and obtain a single picture;

Step 7, using the radial distortion parameters to correct the single picture, calculating the transformation matrix from the checkerboard coordinate system to the camera coordinate system to obtain a second transformation matrix;

Step 8. According to the first transformation matrix and the second transformation matrix, calculate the transformation matrix from the object coordinate system to the camera coordinate system to obtain a third transformation matrix;

Step 9. Sampling the CAD model to obtain the coordinate values of the sampling points in the object coordinate system, which is the coordinate values of the sampling objects, using the third transformation matrix to calculate the coordinate values of the sampling points in the camera coordinate system to obtain the sampling camera coordinate value;

Step 10, using the object pose estimation method to obtain the estimated pose value of the object in the camera coordinate system, calculate the coordinate value of the sampling point in the camera coordinate system, obtain the sampled estimated coordinate value, and compare it with the sampled camera coordinate value , to get the pose estimation accuracy index.

Further, the step 3 includes: selecting template feature points in the object outline template, aligning the template feature points with the origin of the checkerboard calibration plate, and calculating the offset of the template feature points.

Further, the first transformation matrix is:

In the formula, x _f , y _f , and z _f are the coordinate values of the template feature points in the object coordinate system, Δx, Δy are the offsets from the template feature points to the origin of the checkerboard calibration plate, and Δh is the high-precision checkerboard The thickness of the grid calibration plate, its value is 0 when no high-precision calibration plate is used.

Furthermore, the camera extrinsic parameters are obtained by fitting the corner points of the checkerboard with their actual coordinates, and the camera intrinsic parameters are calculated by Zhang Zhengyou's calibration method.

Further, the internal parameters of the camera and the radial distortion parameters are respectively:

In the formula, f is the focal length of the industrial camera, dx is the horizontal ratio of the pixel, dy is the vertical ratio of the pixel, u ₀ and v ₀ are the principal point coordinates of the image.

Further, the second transformation matrix is:

In the formula, R is the rotation transformation from the checkerboard coordinate system to the camera coordinate system, and t is the translation vector from the checkerboard coordinate system to the camera coordinate system.

Further, the third transformation matrix is:

T _obj2camera ＝T _board2camera T _board2obj ^-1

Further, the object coordinate value of the sampling point is:

是第i个采样点在物体坐标系的坐标值。In the formula

is the coordinate value of the i-th sampling point in the object coordinate system.

Further, the sampling camera coordinate value is:

是使用所述第三变换矩阵得到的第i个采样点在相机坐标系的坐标值。In the formula

is the coordinate value of the i-th sampling point in the camera coordinate system obtained by using the third transformation matrix.

Further, the estimated sampling coordinate value is:

是使用位姿估计方法所得的变换矩阵计算的第i个采样点在相机坐标系的坐标值。In the formula

is the coordinate value of the i-th sampling point in the camera coordinate system calculated using the transformation matrix obtained by the pose estimation method.

Further, the calculation formula of the pose estimation accuracy index is:

P_cam ⁱ分别是第i个采样点的采样估计坐标值和采样相机坐标值。In the formula

P _cam ⁱ are the sampling estimated coordinate value and the sampling camera coordinate value of the i-th sampling point, respectively.

Compared with the prior art, the present invention has at least the following beneficial technical effects:

1. The attitude information of the object in the camera coordinate system can be obtained at low cost;

2. Directly use the coordinate transformation from the checkerboard to the camera in the process of calculating the pose of the object, avoiding robot motion errors and other errors in other existing methods;

3. The present invention provides tags in real scenes for object pose estimation, and provides a basis for the establishment of real data sets for object pose estimation and the evaluation of pose estimation.

The idea, specific structure and technical effects of the present invention will be further described below in conjunction with the accompanying drawings, so as to fully understand the purpose, features and effects of the present invention.

Description of drawings

Fig. 1 is a flow chart of a preferred embodiment of the present invention;

Fig. 2 is a CAD model schematic diagram of a preferred embodiment of the present invention;

Fig. 3 is a schematic diagram of an object outline template in a preferred embodiment of the present invention;

Fig. 4 is a schematic diagram of the alignment of an object outline template and a checkerboard calibration plate according to a preferred embodiment of the present invention;

Fig. 5 is a schematic diagram of the relationship between the object coordinate system and the checkerboard coordinate system in a preferred embodiment of the present invention;

Fig. 6 is a schematic diagram of an eye-in-hand calibration model according to a preferred embodiment of the present invention;

Fig. 7 is a CAD model sampling schematic diagram of a preferred embodiment of the present invention;

Fig. 8 is a distribution diagram of sampling point errors in a preferred embodiment of the present invention.

Among them, 1-connecting rod workpiece CAD model, 2-checkerboard calibration board, 3-industrial camera, 4-industrial robot.

detailed description

The following describes several preferred embodiments of the present invention with reference to the accompanying drawings, so as to make the technical content clearer and easier to understand. The present invention can be embodied in many different forms of embodiments, and the protection scope of the present invention is not limited to the embodiments mentioned herein.

In the drawings, components with the same structure are denoted by the same numerals, and components with similar structures or functions are denoted by similar numerals. The size and thickness of each component shown in the drawings are shown arbitrarily, and the present invention does not limit the size and thickness of each component. In order to make the illustration clearer, the thickness of parts is appropriately exaggerated in some places in the drawings.

The object of this embodiment is a common connecting rod workpiece in an industrial scene, as shown in Figure 1, which is a flow chart of this embodiment, including the following steps:

Step 1. Establish the CAD model 1 of the connecting rod workpiece. This model is obtained in the product design stage, or can be obtained by using a reconstruction system or surveying and mapping. The CAD model 1 is shown in FIG. 2 .

Step 2. Project the CAD model 1 of the connecting rod workpiece in equal proportions to obtain the contour template of the connecting rod workpiece. The contour template is shown in FIG. 3 .

Step 3. Align the outline template obtained in step 2 with the origin of the checkerboard calibration plate 2. In this embodiment, the center of the small shaft hole is selected as the template feature point of the outline template, and the distance between the feature point and the origin of the checkerboard is fixed Δx=0mm, Δy=-90mm, and the outline template and the checkerboard calibration plate 2 are aligned as shown in Figure 4. Show.

Step 4. Calculate the transformation matrix from the object coordinate system to the checkerboard coordinate system according to the coordinate values of the template feature points in the object coordinate system and the deviation from the template feature points to the checkerboard coordinate origin.

The relationship between the object coordinate system of the connecting rod workpiece and the checkerboard coordinate system is shown in Figure 5. The object coordinate system {O ₁ } of the connecting rod workpiece is established at the size center of the workpiece, that is, the coordinate origin is at the size center of the workpiece. The X-axis points from the center of the size to the center of the small axis hole, the Y-axis points from the center of the size to the top of the workpiece, and the coordinate system conforms to the right-hand rule. According to the definition in Zhang Zhengyou’s calibration method, the center of the coordinate system {O ₂ } for checkerboard calibration is located at the lower right corner point shown in Figure 5. The point points to the direction with fewer squares, and the coordinate system conforms to the right-hand rule.

The coordinates of the projection point of the center of the small axis obtained from the connecting rod workpiece CAD model 1 in the object coordinate system are x _f = 50.5 mm, y _f = -12.5 mm, z _f = 0 mm, then the coordinates of the object coordinate system to the checkerboard coordinate system The transformation matrix is:

Among them, Δx is positive when it is in the same direction as the X-axis of the object coordinate system, and positive when Δy is opposite to the Z-axis of the object coordinate system;

Align the outer contour of the connecting rod workpiece with its contour template. If the high-precision checkerboard calibration plate 2 is used, align the high-precision checkerboard calibration plate 2 with the checkerboard template. If the thickness of the high-precision checkerboard calibration plate 2 is Δh=3mm, the transformation matrix from the object coordinate system to the checkerboard coordinate system is:

Among them, Δh is positive when it is in the same direction as the Y axis of the object coordinate system.

Step 5. Place the connecting rod workpiece CAD model 1 and the calibration plate under the built eye-in-hand calibration system, as shown in Figure 6, follow the camera calibration process of Zhang Zhengyou, keep the checkerboard calibration plate 2 still, and change the industrial robot 4 10 to 20 checkerboard images are collected in a single shape.

According to Zhang Zhengyou's camera calibration method, the internal parameters of the industrial camera 3 are calculated:

Where f is the focal length of the industrial camera 3, dx is the horizontal ratio of the pixel, dy is the vertical ratio of the pixel, u ₀ and v ₀ are the principal point coordinates of the image;

Radial distortion parameters of industrial camera 3:

Step 6. After the internal parameters and distortion parameters of the industrial camera 3 are calibrated, move the industrial robot 4 so that the connecting rod and the calibration plate are in the field of view of the camera at the same time, and collect the image of the industrial camera 3 at this time. In order to obtain images and labels of the connecting rod at different distances and attitudes in the camera coordinate system, keep the workpiece and the checkerboard calibration plate 2 stationary. Change the 4-position shape of the industrial robot to ensure that the workpiece and the checkerboard calibration plate 2 are within the camera field of view, so that the posture of the workpiece in the camera coordinate system changes.

Step 7. Use the radial distortion parameter k obtained in step 5 to correct the image collected in step 6. The relationship between the pixel coordinates of the corrected image and the pixel coordinates of the original image is:

是工业相机3产生径向畸变后的图像像素坐标，u、v是工业相机3在理想情况下呈像所得的图像像素坐标，u₀、v₀是图像主点坐标值，k₁、k₂是径向畸变参数，r是像素点与主点的实际距离；in

are the image pixel coordinates after the radial distortion of the industrial camera 3, u and v are the image pixel coordinates obtained by the industrial camera 3 under ideal conditions, u ₀ and v ₀ are the principal point coordinates of the image, k ₁ , k ₂ is the radial distortion parameter, r is the actual distance between the pixel point and the main point;

Using the Zhang Zhengyou calibration method, use the de-distorted image to obtain the transformation matrix from the checkerboard coordinate system to the camera coordinate system:

Step 8, calculated according to step 4 and step 7, obtain the pose of the workpiece in the image under the image coordinate system, that is, the homogeneous transformation matrix from the object coordinate system of the connecting rod to the camera coordinate system is:

T _obj2camera ＝T _board2camera T _board2obj ^-1

Calculated to get:

Step 9. Randomly sample the CAD model 1 of the connecting rod workpiece to obtain the coordinate value P _obj of the sampling point in the object coordinate system. The sampling point is shown in FIG. 7 , and calculate the coordinate value P _cam of the sampling point in the camera coordinate system.

以

与P_cam的平均距离作为该网络位姿估计的精度指标。本实施例中平均距离为4.337mm，采样各点误差如图8所示。该平均距离描述了网络所得的物体位姿结果与本文给出的位姿标签的误差，平均距离越大说明该网络预测的物体位姿误差大，精度低。Step 10. In order to evaluate the accuracy of the existing object pose estimation network, use the object pose estimated by the network to calculate the coordinate values of the sampling points under the camera coordinates in step 9

by

The average distance from P _cam is used as the accuracy index for pose estimation of this network. In this embodiment, the average distance is 4.337 mm, and the errors of each sampling point are shown in FIG. 8 . The average distance describes the error between the object pose result obtained by the network and the pose label given in this paper. The larger the average distance, the greater the error and low accuracy of the object pose predicted by the network.

The object pose label acquisition method based on the checkerboard calibration board 2 in this embodiment provides a low-cost, high-precision acquisition method for object labels in real scenes. The average distance calculated by this label can provide a loss definition for network training, and at the same time can give an accuracy indicator of the network result.

The preferred specific embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make many modifications and changes according to the concept of the present invention without creative efforts. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning or limited experiments on the basis of the prior art shall be within the scope of protection defined by the claims.

Claims (10)

1. an object 3D pose label acquisition method based on a checkerboard calibration plate, is characterized in that, comprises the steps: Step 1, establishing a CAD model of the object; Step 2, projecting the CAD model in equal proportions to obtain the object outline template; Step 3, aligning the origin of the object outline template and the checkerboard calibration plate template; Step 4, aligning the object with the object outline template, aligning the checkerboard calibration plate with the checkerboard calibration plate template, calculating the transformation matrix from the object coordinate system to the checkerboard coordinate system, to obtain the first transformation matrix; Step 5, Calibrate camera extrinsic parameters, camera intrinsic parameters and radial distortion parameters; Step 6. Move the camera so that the object and the checkerboard are within the camera's field of view at the same time, and obtain a single picture; Step 7, using the radial distortion parameter to correct the single picture, calculating the transformation matrix from the checkerboard coordinate system to the camera coordinate system to obtain a second transformation matrix; Step 8. According to the first transformation matrix and the second transformation matrix, calculate the transformation matrix from the object coordinate system to the camera coordinate system to obtain the third transformation matrix, that is, under the current camera viewing angle, the object based on the checkerboard calibration board 3D pose label; Step 9. Sampling the CAD model to obtain the coordinate values of the sampling points in the object coordinate system, which is the coordinate values of the sampling objects, using the third transformation matrix to calculate the coordinate values of the sampling points in the camera coordinate system to obtain the sampling camera coordinate value; Step 10, using the object pose estimation method to obtain the estimated pose value of the object in the camera coordinate system, calculate the coordinate value of the sampling point in the camera coordinate system, obtain the sampled estimated coordinate value, and compare it with the sampled camera coordinate value , to get the pose estimation accuracy index. 2. The object 3D pose label acquisition method based on checkerboard calibration board as claimed in claim 1, is characterized in that, described step 3 comprises: select template feature point in described object outline template, described template feature Points are aligned with the origin of the checkerboard calibration plate, and the offset of the feature points of the template is calculated. 3. the object 3D pose label acquisition method based on the checkerboard calibration board as claimed in claim 2, is characterized in that, described first transformation matrix is:

In the formula, x _f , y _f , and z _f are the coordinate values of the template feature points in the object coordinate system, Δx, Δy are the offsets from the template feature points to the origin of the checkerboard calibration plate, and Δh is the high-precision checkerboard The thickness of the grid calibration plate, its value is 0 when no high-precision calibration plate is used. 4. the object 3D pose label acquisition method based on the checkerboard calibration board as claimed in claim 3, is characterized in that, described camera intrinsic parameter and described radial distortion parameter are respectively:

In the formula, f is the focal length of the industrial camera, dx is the horizontal ratio of the pixel, dy is the vertical ratio of the pixel, u ₀ and v ₀ are the principal point coordinates of the image; k ₁ and k ₂ are the radial distortion parameters. 5. the object 3D pose label acquisition method based on the checkerboard calibration board as claimed in claim 4, is characterized in that, described second transformation matrix is:

In the formula, R is the rotation transformation from the checkerboard coordinate system to the camera coordinate system, and t is the translation vector from the checkerboard coordinate system to the camera coordinate system. 6. the object 3D pose label acquisition method based on checkerboard calibration board as claimed in claim 5, is characterized in that, described the 3rd transformation matrix is: T _obj2camera =T _board2camera ·T _board2obj ^-1 . 7. The object 3D pose label acquisition method based on the checkerboard calibration plate as claimed in claim 6, wherein the object coordinate value of the sampling point is:

In the formula

is the coordinate value of the i-th sampling point in the object coordinate system. 8. The object 3D pose label acquisition method based on the checkerboard calibration board as claimed in claim 7, wherein the sampling camera coordinate value is:

In the formula

is the coordinate value of the i-th sampling point in the camera coordinate system obtained by using the third transformation matrix. 9. The object 3D pose label acquisition method based on the checkerboard calibration board as claimed in claim 8, wherein the sampling estimated coordinate value is:

In the formula

is the coordinate value of the i-th sampling point in the camera coordinate system calculated using the transformation matrix obtained by the pose estimation method. 10. The object 3D pose label acquisition method based on the checkerboard calibration board as claimed in claim 9, wherein the calculation formula of the pose estimation accuracy index is:

In the formula

P _cam ⁱ are the sampling estimated coordinate value and the sampling camera coordinate value of the i-th sampling point, respectively.

Images