CN116823963A - External parameter calibration method and device based on vehicle-mounted reversing image system - Google Patents
External parameter calibration method and device based on vehicle-mounted reversing image system Download PDFInfo
- Publication number
- CN116823963A CN116823963A CN202310721159.1A CN202310721159A CN116823963A CN 116823963 A CN116823963 A CN 116823963A CN 202310721159 A CN202310721159 A CN 202310721159A CN 116823963 A CN116823963 A CN 116823963A
- Authority
- CN
- China
- Prior art keywords
- calibration
- coordinate system
- intersection point
- coordinates
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Closed-Circuit Television Systems (AREA)
Abstract
The application provides an external parameter calibration method and device based on a vehicle-mounted reversing image system. The method comprises the following steps: determining a plurality of calibration intersection coordinates of intersection points of a plurality of preset calibration lines on the ground behind a vehicle under a world coordinate system, and obtaining a plurality of reference intersection coordinates of the plurality of reference lines under an image pixel coordinate system based on a rear view image in response to the acquisition of the rear view image based on a rear view camera of the vehicle; and obtaining the external parameters of the rearview camera based on a plurality of calibration intersection point coordinates in a world coordinate system and a plurality of reference intersection point coordinates in an image pixel coordinate system. The external parameters of the rearview camera are obtained through a plurality of calibration intersection point coordinates of intersection points of a plurality of preset calibration lines on the ground behind the vehicle under a world coordinate system and a plurality of reference intersection point coordinates of a plurality of reference lines in a rear view image under an image pixel coordinate system. The calibration process is simple, the requirements on the calibration site and the calibration tool are low, the operation is simple, the cost is low, and the application is easy.
Description
Technical Field
The application relates to the technical field of camera calibration, in particular to an external parameter calibration method, device, medium and electronic equipment based on a vehicle-mounted reversing image system.
Background
Under the vehicle-mounted reversing image system, the calibration of the internal parameters and the external parameters of the rearview camera is an important precondition for accurate distance measurement and positioning of computer vision. The internal parameters of the rearview camera are determined by the nature of the camera, and the external parameters of the camera are determined by the installation position and angle of the camera.
Typically, the camera's internal parameters are calibrated using a Zhang Zhengyou calibration method, while the external parameters are typically calibrated by a number of different calibration methods. However, most of the current external parameter calibration methods have strict requirements on calibration sites and calibration tools.
Therefore, the application provides an external parameter calibration method based on a vehicle-mounted reversing image system, so as to solve the technical problems.
Disclosure of Invention
The application aims to provide an external parameter calibration method, device, medium and electronic equipment based on a vehicle-mounted reversing image system, which can solve at least one technical problem. The specific scheme is as follows:
according to a first aspect of the present application, there is provided an external parameter calibration method based on a vehicle-mounted reverse image system, including:
determining a plurality of calibration intersection coordinates of intersection points of a plurality of calibration lines preset on the ground behind the vehicle under a world coordinate system,
and responding to a rear view image obtained by a rear view camera of the vehicle, and obtaining a plurality of reference intersection point coordinates of the plurality of reference lines under an image pixel coordinate system based on the rear view image, wherein the rear view image at least comprises the images of the plurality of calibration lines and the images of the plurality of reference lines generated by the rear view camera, and the images of the plurality of reference lines are overlapped with the images of the plurality of calibration lines;
and obtaining the external parameters of the rearview camera based on a plurality of calibration intersection point coordinates in a world coordinate system and a plurality of reference intersection point coordinates in an image pixel coordinate system.
Optionally, the plurality of calibration lines includes: the first calibration line, the second calibration line and the third calibration line are parallel to the Yw axis direction under a vehicle body coordinate system, the first calibration line and the second calibration line are respectively arranged on two sides of the vehicle body of the vehicle, and the distance between the first calibration line and the second calibration line is equal to the width of the vehicle body of the vehicle; a projection of a preset position of a rear bumper of the vehicle on the ground, wherein the distance from the third calibration line behind the vehicle body is 1 meter, and the third calibration line is perpendicular to the first calibration line and the second calibration line; the plurality of calibration intersection point coordinates comprise a first calibration intersection point coordinate and a second calibration intersection point coordinate; the images of the plurality of reference lines include: an image of a first reference line that coincides with the image of the first calibration line, an image of a second reference line that coincides with the image of the second calibration line, and an image of a third reference line that coincides with the image of the third calibration line; the plurality of reference intersection coordinates includes: the first reference intersection point coordinate at which the first reference line intersects the third reference line, the second reference intersection point coordinate at which the second reference line intersects the third reference line, and the third reference intersection point coordinate at which the first reference line intersects the second reference line.
Optionally, the obtaining the external parameters of the rearview camera based on the coordinates of a plurality of calibration intersection points in the world coordinate system and the coordinates of a plurality of reference intersection points in the image pixel coordinate system includes:
respectively normalizing a plurality of reference intersection point coordinates under an image pixel coordinate system into a preset normalization plane to obtain a plurality of intersection point projection coordinates, wherein the preset normalization plane is a plane in which Zc is equal to 1 under a camera coordinate system, the plurality of intersection point projection coordinates at least comprise vanishing point coordinates, and the vanishing point coordinates are projection coordinates of the third reference intersection point coordinates in the preset normalization plane;
and obtaining the external parameters of the rearview camera based on a plurality of intersection point projection coordinates in a preset normalized plane and a plurality of calibration intersection point coordinates in a world coordinate system.
Optionally, the obtaining the external parameters of the rearview camera based on the projection coordinates of the plurality of intersection points in the preset normalization plane and the coordinates of the plurality of calibration intersection points in the world coordinate system includes:
acquiring ZXZ Euler angles based on a plurality of intersection point projection coordinates in a preset normalization plane and a plurality of calibration intersection point coordinates in a world coordinate system, wherein ZXZ Euler angles represent a rotation matrix of a first external parameter of the rearview camera;
and bringing the rotation matrix of the rearview camera, a plurality of calibrated intersection point coordinates in a world coordinate system and a plurality of intersection point projection coordinates in a preset normalization plane into a preset linear equation to obtain a translation vector representing the second external parameter of the rearview camera.
Optionally, the obtaining ZXZ euler angles based on the projection coordinates of the plurality of intersection points in the preset normalization plane and the plurality of calibration intersection point coordinates in the world coordinate system includes:
obtaining direction vectors of all axial directions under a camera coordinate system based on a plurality of intersection point projection coordinates in a preset normalization plane, a plurality of calibration intersection point coordinates under a world coordinate system and a preset perspective projection relation between each calibration intersection point coordinate and a corresponding intersection point projection coordinate;
the ZXZ euler angles are obtained based on the direction vectors of the respective axis directions in the camera coordinate system.
Optionally, the preset linear equation is:
wherein u represents the coordinate of the Xc axis direction in the preset normalization plane, v represents the coordinate of the Yc axis direction in the preset normalization plane, x represents the coordinate of the Xw axis direction in the world coordinate system, y represents the coordinate of the Yw axis direction in the world coordinate system, and t represents the translation vector in the camera coordinate system.
Optionally, after the vehicle-based rearview camera acquires the rearview image, the method includes:
calibrating internal parameters and distortion parameters of the rearview camera based on a Zhang Dingyou calibration method before the multiple reference intersection point coordinates of the multiple reference lines under an image pixel coordinate system are obtained based on the rearview image;
correcting the rearview image based on internal parameters and distortion parameters of the rearview camera.
According to a second aspect of the present application, there is provided an external parameter calibration device based on a vehicle-mounted reverse image system, including:
a coordinate obtaining unit for determining a plurality of calibration intersection coordinates of intersection points of a plurality of calibration lines preset on the ground behind the vehicle in a world coordinate system,
and responding to a rear view image obtained by a rear view camera of the vehicle, and obtaining a plurality of reference intersection point coordinates of the plurality of reference lines under an image pixel coordinate system based on the rear view image, wherein the rear view image at least comprises the images of the plurality of calibration lines and the images of the plurality of reference lines generated by the rear view camera, and the images of the plurality of reference lines are overlapped with the images of the plurality of calibration lines;
and the external reference obtaining unit is used for obtaining the external reference of the rearview camera based on a plurality of calibration intersection point coordinates in a world coordinate system and a plurality of reference intersection point coordinates in an image pixel coordinate system.
Optionally, the plurality of calibration lines includes: the first calibration line, the second calibration line and the third calibration line are parallel to the Yw axis direction under a vehicle body coordinate system, the first calibration line and the second calibration line are respectively arranged on two sides of the vehicle body of the vehicle, and the distance between the first calibration line and the second calibration line is equal to the width of the vehicle body of the vehicle; a projection of a preset position of a rear bumper of the vehicle on the ground, wherein the distance from the third calibration line behind the vehicle body is 1 meter, and the third calibration line is perpendicular to the first calibration line and the second calibration line; the plurality of calibration intersection point coordinates comprise a first calibration intersection point coordinate and a second calibration intersection point coordinate; the images of the plurality of reference lines include: an image of a first reference line that coincides with the image of the first calibration line, an image of a second reference line that coincides with the image of the second calibration line, and an image of a third reference line that coincides with the image of the third calibration line; the plurality of reference intersection coordinates includes: the first reference intersection point coordinate at which the first reference line intersects the third reference line, the second reference intersection point coordinate at which the second reference line intersects the third reference line, and the third reference intersection point coordinate at which the first reference line intersects the second reference line.
Optionally, the obtaining the external parameters of the rearview camera based on the coordinates of a plurality of calibration intersection points in the world coordinate system and the coordinates of a plurality of reference intersection points in the image pixel coordinate system includes:
respectively normalizing a plurality of reference intersection point coordinates under an image pixel coordinate system into a preset normalization plane to obtain a plurality of intersection point projection coordinates, wherein the preset normalization plane is a plane in which Zc is equal to 1 under a camera coordinate system, the plurality of intersection point projection coordinates at least comprise vanishing point coordinates, and the vanishing point coordinates are projection coordinates of the third reference intersection point coordinates in the preset normalization plane;
and obtaining the external parameters of the rearview camera based on a plurality of intersection point projection coordinates in a preset normalized plane and a plurality of calibration intersection point coordinates in a world coordinate system.
Optionally, the obtaining the external parameters of the rearview camera based on the projection coordinates of the plurality of intersection points in the preset normalization plane and the coordinates of the plurality of calibration intersection points in the world coordinate system includes:
acquiring ZXZ Euler angles based on a plurality of intersection point projection coordinates in a preset normalization plane and a plurality of calibration intersection point coordinates in a world coordinate system, wherein ZXZ Euler angles represent a rotation matrix of a first external parameter of the rearview camera;
and bringing the rotation matrix of the rearview camera, a plurality of calibrated intersection point coordinates in a world coordinate system and a plurality of intersection point projection coordinates in a preset normalization plane into a preset linear equation to obtain a translation vector representing the second external parameter of the rearview camera.
Optionally, the obtaining ZXZ euler angles based on the projection coordinates of the plurality of intersection points in the preset normalization plane and the plurality of calibration intersection point coordinates in the world coordinate system includes:
obtaining direction vectors of all axial directions under a camera coordinate system based on a plurality of intersection point projection coordinates in a preset normalization plane, a plurality of calibration intersection point coordinates under a world coordinate system and a preset perspective projection relation between each calibration intersection point coordinate and a corresponding intersection point projection coordinate;
the ZXZ euler angles are obtained based on the direction vectors of the respective axis directions in the camera coordinate system.
Optionally, the preset linear equation is:
wherein u represents the coordinate of the Xc axis direction in the preset normalization plane, v represents the coordinate of the Yc axis direction in the preset normalization plane, x represents the coordinate of the Xw axis direction in the world coordinate system, y represents the coordinate of the Yw axis direction in the world coordinate system, and t represents the translation vector in the camera coordinate system.
Optionally, after the vehicle-based rearview camera acquires the rearview image, the method includes:
calibrating internal parameters and distortion parameters of the rearview camera based on a Zhang Dingyou calibration method before the multiple reference intersection point coordinates of the multiple reference lines under an image pixel coordinate system are obtained based on the rearview image;
correcting the rearview image based on internal parameters and distortion parameters of the rearview camera.
According to a third aspect of the present application, there is provided a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the method for calibrating an external parameter based on an on-board reverse image system as described in any one of the above.
According to a fourth aspect of the present application, there is provided an electronic device comprising: one or more processors; and the storage device is used for storing one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors are enabled to realize the external parameter calibration method based on the vehicle-mounted reversing image system.
Compared with the prior art, the scheme provided by the embodiment of the application has at least the following beneficial effects:
the application provides an external parameter calibration method, device, medium and electronic equipment based on a vehicle-mounted reversing image system. The method comprises the following steps: determining a plurality of calibration intersection coordinates of intersection points of a plurality of preset calibration lines on the ground behind a vehicle under a world coordinate system, and obtaining a plurality of reference intersection coordinates of the plurality of reference lines under an image pixel coordinate system based on a rear view image in response to the acquisition of the rear view image based on a rear view camera of the vehicle; and obtaining the external parameters of the rearview camera based on a plurality of calibration intersection point coordinates in a world coordinate system and a plurality of reference intersection point coordinates in an image pixel coordinate system. The external parameters of the rearview camera are obtained through a plurality of calibration intersection point coordinates of intersection points of a plurality of preset calibration lines on the ground behind the vehicle under a world coordinate system and a plurality of reference intersection point coordinates of a plurality of reference lines in a rear view image under an image pixel coordinate system. The calibration process is simple, the requirements on the calibration site and the calibration tool are low, the operation is simple, the cost is low, and the application is easy.
Drawings
FIG. 1 shows a flow chart of an extrinsic calibration method based on a vehicle-mounted reverse image system according to an embodiment of the application;
FIG. 2 shows a schematic diagram of a plurality of calibrated intersection coordinates in a world coordinate system according to an embodiment of the present application;
FIG. 3 shows a schematic diagram of a plurality of reference intersection coordinates in an image pixel coordinate system according to an embodiment of the application;
FIG. 4 is a schematic diagram showing the relationship of a plurality of calibrated intersection coordinates to a camera coordinate system in a world coordinate system according to an embodiment of the present application;
FIG. 5 shows a schematic diagram of a plurality of intersection point projection coordinates within a preset normalized plane according to an embodiment of the present application;
FIG. 6 shows a schematic diagram of the conversion of ZXZ Euler angles in accordance with an embodiment of the present application;
FIG. 7 shows a block diagram of a unit of an external parameter calibration device based on an on-board reverse imaging system according to an embodiment of the application.
Reference numerals illustrate:
CA-rearview camera;
a1-a first calibration line, B1-a second calibration line, C1-a third calibration line, PA 1-a first calibration intersection point coordinate, PB 1-a second calibration intersection point coordinate;
a2-a first reference line, B2-a second reference line, C2-a third reference line, PA 2-a first reference intersection point coordinate, PB 2-a second reference intersection point coordinate, P2-a third reference intersection point coordinate;
PA 3-first intersection point projection coordinates, PB 3-second intersection point projection coordinates and P3-vanishing point coordinates.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be described in further detail below with reference to the accompanying drawings, and it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
The terminology used in the embodiments of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in this application and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise, the "plurality" generally includes at least two.
It should be understood that the term "and/or" as used herein is merely one relationship describing the association of the associated objects, meaning that there may be three relationships, e.g., a and/or B, may represent: a exists alone, A and B exist together, and B exists alone. In addition, the character "/" herein generally indicates that the front and rear associated objects are an "or" relationship.
It should be understood that although the terms first, second, third, etc. may be used in embodiments of the present application, these descriptions should not be limited to these terms. These terms are only used to distinguish one from another. For example, a first may also be referred to as a second, and similarly, a second may also be referred to as a first, without departing from the scope of embodiments of the application.
The words "if", as used herein, may be interpreted as "at … …" or "at … …" or "in response to a determination" or "in response to a detection", depending on the context. Similarly, the phrase "if determined" or "if detected (stated condition or event)" may be interpreted as "when determined" or "in response to determination" or "when detected (stated condition or event)" or "in response to detection (stated condition or event), depending on the context.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a product or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such product or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a commodity or device comprising such element.
In particular, the symbols and/or numerals present in the description, if not marked in the description of the figures, are not numbered.
Alternative embodiments of the present application will be described in detail below with reference to the accompanying drawings.
The embodiment provided by the application is an embodiment of an external parameter calibration method based on a vehicle-mounted reversing image system.
An embodiment of the present application will be described in detail with reference to fig. 1.
Step S101, determining a plurality of calibration intersection coordinates of intersection points of a plurality of calibration lines preset on the ground behind the vehicle in a world coordinate system, and obtaining a plurality of reference intersection coordinates of the plurality of reference lines in an image pixel coordinate system based on a rear view image in response to obtaining the rear view image based on the rear view camera CA of the vehicle.
The rearview image at least comprises images of the plurality of calibration lines and images of a plurality of reference lines generated by the rearview camera CA, and the images of the plurality of reference lines are overlapped with the images of the plurality of calibration lines.
The ground refers to a flat ground. The embodiment of the application marks on a flat ground.
The rear view camera CA is provided at a center position of a rear bumper of the vehicle.
The key of the technical implementation of the embodiment of the application is to establish the projection mapping of the space scenery to the camera image. In the field of computer vision research, it is generally represented by a camera imaging model. The pinhole imaging model is a commonly used camera imaging model. Four coordinate systems are involved in the process of establishing the pinhole imaging model: a two-dimensional image pixel coordinate system, an image physical coordinate system, a camera coordinate system and a world coordinate system.
In the image pixel coordinate system, the image obtained through the optical system and the photoelectric conversion element (such as CCD, CMOS) is stored in the storage device of the computer. It is defined in pixels, the size of an image being the resolution of the image. The coordinate system is set with the upper left corner of the image as the origin, and the behavior u-axis direction and the v-axis direction.
The physical coordinate system of the image is established for describing the imaging relation of the small hole. The origin is the intersection point between the optical axis of the optical system and the imaging plane of the photoelectric conversion element (such as CCD, CMOS), and the x-axis and the y-axis are respectively parallel to the u-axis and the v-axis of the image pixel coordinate system. The image physical coordinate system and the image pixel coordinate system are referred to as an image coordinate system.
A camera coordinate system, which is built on the camera, which changes with the movement of the camera, so that it is a local coordinate system. The camera coordinate system consists of Xc-axis, yc-axis and Zc-axis. The camera coordinate system takes the optical center Oc of the optical system as the origin of coordinates, the Xc-Oc-Yc plane is parallel to the plane of the image coordinate system, and the Xc axis and the Yc axis are respectively parallel to the u axis and the v axis of the image coordinate system. The Zc axis of the camera coordinate system is the optical axis of the optical system, and forms a right-hand coordinate system with the Xc axis and the Yc axis.
The world coordinate system, the establishment of which is determined by the measuring environment and the measuring purpose. It does not change with the movement of the camera once established, so it is sometimes also called global coordinate system. The world coordinate system consists of an Xw axis, a Yw axis and a Zw axis.
In some embodiments, as shown in fig. 2, the plurality of calibration lines includes: a first calibration line A1, a second calibration line B1 and a third calibration line C1, wherein the first calibration line A1 and the second calibration line B1 are parallel to the Yw axis direction under a vehicle body coordinate system, the first calibration line A1 and the second calibration line B1 are arranged on two sides of the vehicle body of the vehicle, and the distance between the first calibration line A1 and the second calibration line B1 is equal to the vehicle body width of the vehicle; a projection of a preset position of a rear bumper of the vehicle on the ground, wherein the distance from the third calibration line C1 behind the vehicle body is 1 meter, and the third calibration line C1 is perpendicular to the first calibration line A1 and the second calibration line B1; the plurality of calibration intersection coordinates include a first calibration intersection coordinate PA1 and a second calibration intersection coordinate PB1.
A left straight line first calibration line A1 and a right straight line second calibration line B1 are attached to the ground position (namely, the Yw direction under a vehicle coordinate system) of an extension line of the external contour of the vehicle body; a third calibration line C1 is attached to the first calibration line A1 and the second calibration line B1 at a position 1 m behind the rear bumper position of the vehicle. The intersection point of the first calibration line A1 and the third calibration line C1 is a first calibration intersection point coordinate PA1 under the world coordinate system, and the intersection point of the second calibration line B1 and the third calibration line C1 is a second calibration intersection point coordinate PB1 under the world coordinate system.
As shown in fig. 3, the images of the plurality of reference lines include: an image of a first reference line A2 that coincides with the image of the first calibration line A1, an image of a second reference line B2 that coincides with the image of the second calibration line B1, and an image of a third reference line C2 that coincides with the image of the third calibration line C1; the plurality of reference intersection coordinates includes: a first reference intersection point coordinate PA2 where the first reference line A2 intersects the third reference line C2, a second reference intersection point coordinate PB2 where the second reference line B2 intersects the third reference line C2, and a third reference intersection point coordinate P2 where the first reference line A2 intersects the second reference line B2.
In some specific embodiments, after the vehicle-based rearview camera CA acquires a rear view image, the method includes:
step S101-1, before the obtaining, based on the rear view image, a plurality of reference intersection coordinates of the plurality of reference lines in an image pixel coordinate system, calibrating internal parameters and distortion parameters of the rear view camera CA based on a Zhang Dingyou calibration method.
Step S101-2, correcting the rearview image based on the internal parameters and distortion parameters of the rearview camera CA.
According to the embodiment, the internal parameters and distortion parameters of the rearview camera CA are calibrated through a Zhang Dingyou calibration method, so that the rearview camera CA can obtain a rearview image without distortion. The coordinates of a plurality of reference lines in the rear view image can accurately correspond to the coordinates of a plurality of calibration intersection points in the world coordinate system, so that accurate external parameters can be obtained.
Step S102, obtaining the external parameters of the rearview camera CA based on a plurality of calibration intersection point coordinates in a world coordinate system and a plurality of reference intersection point coordinates in an image pixel coordinate system.
According to the embodiment of the application, the external parameters of the rearview camera CA are obtained through a plurality of calibration intersection point coordinates of intersection points of a plurality of preset calibration lines on the ground behind the vehicle under a world coordinate system and a plurality of reference intersection point coordinates of a plurality of reference lines in a rear view image under an image pixel coordinate system. The calibration process is simple, the requirements on the calibration site and the calibration tool are low, the operation is simple, the cost is low, and the application is easy.
In some specific embodiments, the obtaining the external parameters of the rear-view camera CA based on the plurality of calibration intersection coordinates in the world coordinate system and the plurality of reference intersection coordinates in the image pixel coordinate system includes:
and step S102-1, respectively normalizing a plurality of reference intersection point coordinates in an image pixel coordinate system into a preset normalized plane to obtain a plurality of intersection point projection coordinates.
The preset normalization plane is a plane in which Zc is equal to 1 in the camera coordinate system, the plurality of intersection point projection coordinates at least comprise vanishing point coordinates P3, and the vanishing point coordinates P3 are projection coordinates of the third reference intersection point coordinates P2 in the preset normalization plane.
As shown in fig. 3, the projection coordinates of the third reference intersection point coordinate P2 in the preset normalization plane are vanishing point coordinates P3.
Step S102-2, obtaining the external parameters of the rearview camera CA based on a plurality of intersection point projection coordinates in a preset normalization plane and a plurality of calibration intersection point coordinates in a world coordinate system.
In the embodiment, the coordinates of a plurality of reference intersection points under the image pixel coordinate system are normalized to the preset normalization plane under the camera coordinate system respectively, so that the complexity of data processing is simplified, the complex data is simplified, and the efficiency of external parameter calibration is improved.
In some specific embodiments, the obtaining the external parameters of the rear-view camera CA based on the projection coordinates of the plurality of intersection points in the preset normalized plane and the plurality of calibration intersection point coordinates in the world coordinate system includes:
and step S102-2-1, obtaining ZXZ Euler angles based on a plurality of intersection point projection coordinates in a preset normalized plane and a plurality of calibration intersection point coordinates in a world coordinate system.
Wherein ZXZ euler angles characterize the rotation matrix of the first outlier of the rear-view camera CA.
In some specific embodiments, the obtaining ZXZ euler angles based on the plurality of intersection projection coordinates in the preset normalized plane and the plurality of calibration intersection coordinates in the world coordinate system includes:
step S102-2-1-1, obtaining direction vectors of all axial directions under a camera coordinate system based on a plurality of intersection point projection coordinates in a preset normalized plane, a plurality of calibration intersection point coordinates under a world coordinate system and a preset perspective projection relation between each calibration intersection point coordinate and a corresponding intersection point projection coordinate.
Step S102-2-1-2, obtaining ZXZ Euler angles based on the direction vectors of the directions of the axes in the camera coordinate system.
For example, as shown in fig. 2 to 5, since the first calibration line A1 and the second calibration line B1 are both parallel to the Yw axis in the world coordinate system, vectors are in the camera coordinate system according to the geometric meaning of vanishing pointsParallel to each otherThe Yw axis in the world coordinate system; in the image physical coordinate system, a first physical intersection point coordinate of the first physical straight line and the third physical straight line is (x 0, y 0), and a second physical intersection point coordinate of the second physical straight line and the third physical straight line is (x 1, y 1); combining a first calibration intersection point coordinate PA1 and a second calibration intersection point coordinate PB1 under the world coordinate system, and a first intersection point projection coordinate PA3 and a second intersection point projection coordinate PB3 of the first calibration intersection point coordinate PA1 and the second calibration intersection point coordinate PB 1; according to the perspective projection relation, the origin Oc, the first calibration intersection point coordinate PA1 and the first reference intersection point coordinate PA2 are collinear under the camera coordinate system, and the origin Oc, the second calibration intersection point coordinate PB1 and the second reference intersection point coordinate PB2 are also collinear, so thaty1,1}; the normal vector of the plane OcPA1PB1 can be calculatedSince the third calibration line C1 is in the plane OcPA1PB1, the straight line OcNab and the third calibration line C1 are also perpendicular; since the third calibration line C1 is parallel to the Xw axis, the straight line OcNab must be in the plane OwYwZw; the direction vector of the Xw axis can be expressed as +.>From the direction vectors of Xw and Yw, the direction vector of Zw can be calculated +.>In FIG. 6, the straight line Nab is the intersection of the plane of the Xw-Yw and the plane of the Xc-Yc of the two coordinate systems before and after rotation, and thus the vector +.>Necessarily perpendicular to the Zw and Zc axes, +.>The direction vector of each vector in fig. 6 can be obtained, and ZXZ euler angles can be obtained from the geometric meaning of the vector inner product:
thus, the rotation matrix R can be obtained.
Step S102-2-2, a rotation matrix of the rearview camera CA, a plurality of calibration intersection point coordinates under a world coordinate system and a plurality of intersection point projection coordinates in a preset normalization plane are brought into a preset linear equation to obtain a translation vector representing a second external parameter of the rearview camera CA.
In some embodiments, the predetermined linear equation is:
wherein u represents the coordinate of the Xc axis direction in the preset normalization plane, v represents the coordinate of the Yc axis direction in the preset normalization plane, x represents the coordinate of the Xw axis direction in the world coordinate system, y represents the coordinate of the Yw axis direction in the world coordinate system, and t represents the translation vector in the camera coordinate system.
Since the calibration process is performed on a plane of the world coordinate system zw=0, the above-described preset linear equation can be obtained in a preset normalization plane. And finally obtaining a translation vector of the second external parameter of the rearview camera CA.
The present application also provides an embodiment of a device for carrying out the method steps described in the above embodiment, and the explanation based on the meaning of the same names is the same as that of the above embodiment, which has the same technical effects as those of the above embodiment, and is not repeated here.
As shown in fig. 7, the present application provides an external parameter calibration device 700 based on a vehicle-mounted reverse image system, comprising:
a coordinate obtaining unit 701 for determining a plurality of calibration intersection coordinates of intersection points of a plurality of calibration lines preset on the ground behind the vehicle in a world coordinate system,
and responding to a rear view image obtained by a rear view camera of the vehicle, and obtaining a plurality of reference intersection point coordinates of the plurality of reference lines under an image pixel coordinate system based on the rear view image, wherein the rear view image at least comprises the images of the plurality of calibration lines and the images of the plurality of reference lines generated by the rear view camera, and the images of the plurality of reference lines are overlapped with the images of the plurality of calibration lines;
and an external reference obtaining unit 702, configured to obtain external references of the rearview camera based on a plurality of calibration intersection coordinates in a world coordinate system and a plurality of reference intersection coordinates in an image pixel coordinate system.
Optionally, the plurality of calibration lines includes: the first calibration line, the second calibration line and the third calibration line are parallel to the Yw axis direction under a vehicle body coordinate system, the first calibration line and the second calibration line are respectively arranged on two sides of the vehicle body of the vehicle, and the distance between the first calibration line and the second calibration line is equal to the width of the vehicle body of the vehicle; a projection of a preset position of a rear bumper of the vehicle on the ground, wherein the distance from the third calibration line behind the vehicle body is 1 meter, and the third calibration line is perpendicular to the first calibration line and the second calibration line; the plurality of calibration intersection point coordinates comprise a first calibration intersection point coordinate and a second calibration intersection point coordinate; the images of the plurality of reference lines include: an image of a first reference line that coincides with the image of the first calibration line, an image of a second reference line that coincides with the image of the second calibration line, and an image of a third reference line that coincides with the image of the third calibration line; the plurality of reference intersection coordinates includes: the first reference intersection point coordinate at which the first reference line intersects the third reference line, the second reference intersection point coordinate at which the second reference line intersects the third reference line, and the third reference intersection point coordinate at which the first reference line intersects the second reference line.
Optionally, the obtaining the external parameters of the rearview camera based on the coordinates of a plurality of calibration intersection points in the world coordinate system and the coordinates of a plurality of reference intersection points in the image pixel coordinate system includes:
respectively normalizing a plurality of reference intersection point coordinates under an image pixel coordinate system into a preset normalization plane to obtain a plurality of intersection point projection coordinates, wherein the preset normalization plane is a plane in which Zc is equal to 1 under a camera coordinate system, the plurality of intersection point projection coordinates at least comprise vanishing point coordinates, and the vanishing point coordinates are projection coordinates of the third reference intersection point coordinates in the preset normalization plane;
and obtaining the external parameters of the rearview camera based on a plurality of intersection point projection coordinates in a preset normalized plane and a plurality of calibration intersection point coordinates in a world coordinate system.
Optionally, the obtaining the external parameters of the rearview camera based on the projection coordinates of the plurality of intersection points in the preset normalization plane and the coordinates of the plurality of calibration intersection points in the world coordinate system includes:
acquiring ZXZ Euler angles based on a plurality of intersection point projection coordinates in a preset normalization plane and a plurality of calibration intersection point coordinates in a world coordinate system, wherein ZXZ Euler angles represent a rotation matrix of a first external parameter of the rearview camera;
and bringing the rotation matrix of the rearview camera, a plurality of calibrated intersection point coordinates in a world coordinate system and a plurality of intersection point projection coordinates in a preset normalization plane into a preset linear equation to obtain a translation vector representing the second external parameter of the rearview camera.
Optionally, the obtaining ZXZ euler angles based on the projection coordinates of the plurality of intersection points in the preset normalization plane and the plurality of calibration intersection point coordinates in the world coordinate system includes:
obtaining direction vectors of all axial directions under a camera coordinate system based on a plurality of intersection point projection coordinates in a preset normalization plane, a plurality of calibration intersection point coordinates under a world coordinate system and a preset perspective projection relation between each calibration intersection point coordinate and a corresponding intersection point projection coordinate;
the ZXZ euler angles are obtained based on the direction vectors of the respective axis directions in the camera coordinate system.
Optionally, the preset linear equation is:
wherein u represents the coordinate of the Xc axis direction in the preset normalization plane, v represents the coordinate of the Yc axis direction in the preset normalization plane, x represents the coordinate of the Xw axis direction in the world coordinate system, y represents the coordinate of the Yw axis direction in the world coordinate system, and t represents the translation vector in the camera coordinate system.
Optionally, after the vehicle-based rearview camera acquires the rearview image, the method includes:
calibrating internal parameters and distortion parameters of the rearview camera based on a Zhang Dingyou calibration method before the multiple reference intersection point coordinates of the multiple reference lines under an image pixel coordinate system are obtained based on the rearview image;
correcting the rearview image based on internal parameters and distortion parameters of the rearview camera.
According to the embodiment of the application, the external parameters of the rearview camera are obtained through a plurality of calibration intersection point coordinates of intersection points of a plurality of preset calibration lines on the ground behind the vehicle under a world coordinate system and a plurality of reference intersection point coordinates of a plurality of reference lines in a rear view image under an image pixel coordinate system. The calibration process is simple, the requirements on the calibration site and the calibration tool are low, the operation is simple, the cost is low, and the application is easy.
The present embodiment provides an electronic device, including: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the one processor to enable the at least one processor to perform the method steps described in the embodiments above.
Embodiments of the present application provide a non-transitory computer storage medium storing computer executable instructions that perform the method steps described in the embodiments above.
Finally, it should be noted that: in the present specification, each embodiment is described in a progressive manner, and each embodiment is mainly described in a different point from other embodiments, and identical and similar parts between the embodiments are all enough to refer to each other. The system or the device disclosed in the embodiments are relatively simple in description, and the relevant points refer to the description of the method section because the system or the device corresponds to the method disclosed in the embodiments.
The above embodiments are only for illustrating the technical solution of the present application, and are not limiting; although the application has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present application.
Claims (10)
1. An external parameter calibration method based on a vehicle-mounted reversing image system is characterized by comprising the following steps of:
determining a plurality of calibration intersection coordinates of intersection points of a plurality of calibration lines preset on the ground behind the vehicle under a world coordinate system,
and responding to a rear view image obtained by a rear view camera of the vehicle, and obtaining a plurality of reference intersection point coordinates of the plurality of reference lines under an image pixel coordinate system based on the rear view image, wherein the rear view image at least comprises the images of the plurality of calibration lines and the images of the plurality of reference lines generated by the rear view camera, and the images of the plurality of reference lines are overlapped with the images of the plurality of calibration lines;
and obtaining the external parameters of the rearview camera based on a plurality of calibration intersection point coordinates in a world coordinate system and a plurality of reference intersection point coordinates in an image pixel coordinate system.
2. The method of claim 1, wherein the plurality of calibration lines comprises: the first calibration line, the second calibration line and the third calibration line are parallel to the Yw axis direction under a vehicle body coordinate system, the first calibration line and the second calibration line are respectively arranged on two sides of the vehicle body of the vehicle, and the distance between the first calibration line and the second calibration line is equal to the width of the vehicle body of the vehicle; a projection of a preset position of a rear bumper of the vehicle on the ground, wherein the distance from the third calibration line behind the vehicle body is 1 meter, and the third calibration line is perpendicular to the first calibration line and the second calibration line; the plurality of calibration intersection point coordinates comprise a first calibration intersection point coordinate and a second calibration intersection point coordinate; the images of the plurality of reference lines include: an image of a first reference line that coincides with the image of the first calibration line, an image of a second reference line that coincides with the image of the second calibration line, and an image of a third reference line that coincides with the image of the third calibration line; the plurality of reference intersection coordinates includes: the first reference intersection point coordinate at which the first reference line intersects the third reference line, the second reference intersection point coordinate at which the second reference line intersects the third reference line, and the third reference intersection point coordinate at which the first reference line intersects the second reference line.
3. The method of claim 2, wherein the obtaining the external parameters of the rearview camera based on a plurality of calibrated intersection coordinates in a world coordinate system and a plurality of reference intersection coordinates in an image pixel coordinate system comprises:
respectively normalizing a plurality of reference intersection point coordinates under an image pixel coordinate system into a preset normalization plane to obtain a plurality of intersection point projection coordinates, wherein the preset normalization plane is a plane in which Zc is equal to 1 under a camera coordinate system, the plurality of intersection point projection coordinates at least comprise vanishing point coordinates, and the vanishing point coordinates are projection coordinates of the third reference intersection point coordinates in the preset normalization plane;
and obtaining the external parameters of the rearview camera based on a plurality of intersection point projection coordinates in a preset normalized plane and a plurality of calibration intersection point coordinates in a world coordinate system.
4. The method of claim 3, wherein the obtaining the external parameters of the rearview camera based on the projection coordinates of the plurality of intersection points in the preset normalized plane and the coordinates of the plurality of calibration intersection points in the world coordinate system comprises:
acquiring ZXZ Euler angles based on a plurality of intersection point projection coordinates in a preset normalization plane and a plurality of calibration intersection point coordinates in a world coordinate system, wherein ZXZ Euler angles represent a rotation matrix of a first external parameter of the rearview camera;
and bringing the rotation matrix of the rearview camera, a plurality of calibrated intersection point coordinates in a world coordinate system and a plurality of intersection point projection coordinates in a preset normalization plane into a preset linear equation to obtain a translation vector representing the second external parameter of the rearview camera.
5. The method of claim 4, wherein the obtaining ZXZ euler angles based on a plurality of intersection projection coordinates in a preset normalized plane and a plurality of calibrated intersection coordinates in a world coordinate system comprises:
obtaining direction vectors of all axial directions under a camera coordinate system based on a plurality of intersection point projection coordinates in a preset normalization plane, a plurality of calibration intersection point coordinates under a world coordinate system and a preset perspective projection relation between each calibration intersection point coordinate and a corresponding intersection point projection coordinate;
the ZXZ euler angles are obtained based on the direction vectors of the respective axis directions in the camera coordinate system.
6. The method of claim 5, wherein the predetermined linear equation is:
wherein u represents the coordinate of the Xc axis direction in the preset normalization plane, v represents the coordinate of the Yc axis direction in the preset normalization plane, x represents the coordinate of the Xw axis direction in the world coordinate system, y represents the coordinate of the Yw axis direction in the world coordinate system, and t represents the translation vector in the camera coordinate system.
7. The method of claim 1, wherein the vehicle-based rearview camera acquiring a rear view image comprises:
calibrating internal parameters and distortion parameters of the rearview camera based on a Zhang Dingyou calibration method before the multiple reference intersection point coordinates of the multiple reference lines under an image pixel coordinate system are obtained based on the rearview image;
correcting the rearview image based on internal parameters and distortion parameters of the rearview camera.
8. An external parameter calibration device based on a vehicle-mounted reversing image system is characterized by comprising:
a coordinate obtaining unit for determining a plurality of calibration intersection coordinates of intersection points of a plurality of calibration lines preset on the ground behind the vehicle in a world coordinate system,
and responding to a rear view image obtained by a rear view camera of the vehicle, and obtaining a plurality of reference intersection point coordinates of the plurality of reference lines under an image pixel coordinate system based on the rear view image, wherein the rear view image at least comprises the images of the plurality of calibration lines and the images of the plurality of reference lines generated by the rear view camera, and the images of the plurality of reference lines are overlapped with the images of the plurality of calibration lines;
and the external reference obtaining unit is used for obtaining the external reference of the rearview camera based on a plurality of calibration intersection point coordinates in a world coordinate system and a plurality of reference intersection point coordinates in an image pixel coordinate system.
9. A computer readable storage medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the method according to any one of claims 1 to 7.
10. An electronic device, comprising:
one or more processors;
storage means for storing one or more programs,
wherein the one or more processors implement the method of any of claims 1 to 7 when the one or more programs are executed by the one or more processors.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310721159.1A CN116823963A (en) | 2023-06-16 | 2023-06-16 | External parameter calibration method and device based on vehicle-mounted reversing image system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310721159.1A CN116823963A (en) | 2023-06-16 | 2023-06-16 | External parameter calibration method and device based on vehicle-mounted reversing image system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116823963A true CN116823963A (en) | 2023-09-29 |
Family
ID=88119676
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310721159.1A Pending CN116823963A (en) | 2023-06-16 | 2023-06-16 | External parameter calibration method and device based on vehicle-mounted reversing image system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116823963A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118397109A (en) * | 2024-06-27 | 2024-07-26 | 杭州海康威视数字技术股份有限公司 | Vehicle-mounted camera calibration method and device, electronic equipment and machine-readable storage medium |
-
2023
- 2023-06-16 CN CN202310721159.1A patent/CN116823963A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118397109A (en) * | 2024-06-27 | 2024-07-26 | 杭州海康威视数字技术股份有限公司 | Vehicle-mounted camera calibration method and device, electronic equipment and machine-readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9978146B2 (en) | Data-processing system and method for calibration of a vehicle surround view system | |
| JP4803449B2 (en) | On-vehicle camera calibration device, calibration method, and vehicle production method using this calibration method | |
| JP4803450B2 (en) | On-vehicle camera calibration device and vehicle production method using the device | |
| JP4751939B2 (en) | Car camera calibration system | |
| CN102356633B (en) | Calibration device and method for vehicle-mounted camera | |
| WO2017135081A1 (en) | Vehicle-mounted camera calibration system | |
| JP5124147B2 (en) | Camera calibration apparatus and method, and vehicle | |
| JP5455124B2 (en) | Camera posture parameter estimation device | |
| US20090299684A1 (en) | Method for calibrating cameras installed on vehicle | |
| JP5811327B2 (en) | Camera calibration device | |
| EP2530647A1 (en) | Method of calibrating a vehicle vision system and vehicle vision system | |
| CN102262780A (en) | Image correction method for camera system | |
| CN112802341B (en) | Method for measuring position and attitude of front vehicle | |
| CN113658262A (en) | Camera external parameter calibration method, device, system and storage medium | |
| CN107437264A (en) | In-vehicle camera external parameter automatic detection and bearing calibration | |
| KR20180127671A (en) | Method of automatic calibration of AVM system | |
| CN115797467B (en) | Method, device, equipment and storage medium for detecting calibration result of vehicle camera | |
| JP2013024712A (en) | Method and system for calibrating multiple camera | |
| JP2008522268A (en) | A method for identifying the position of an object from a digital image. | |
| CN116823963A (en) | External parameter calibration method and device based on vehicle-mounted reversing image system | |
| CN113888650A (en) | Camera calibration method and device | |
| JP4873272B2 (en) | Camera calibration device | |
| WO2018130605A1 (en) | Method for calibrating a camera for a motor vehicle considering a calibration error, camera as well as motor vehicle | |
| CN114299163A (en) | Camera calibration verification method, storage medium and electronic device | |
| CN117541659A (en) | Calibration method, device and storage medium of vehicle-mounted camera |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |