[go: up one dir, main page]

CN108909625B - Vehicle bottom ground display method based on panoramic all-round viewing system - Google Patents

Vehicle bottom ground display method based on panoramic all-round viewing system Download PDF

Info

Publication number
CN108909625B
CN108909625B CN201810659018.0A CN201810659018A CN108909625B CN 108909625 B CN108909625 B CN 108909625B CN 201810659018 A CN201810659018 A CN 201810659018A CN 108909625 B CN108909625 B CN 108909625B
Authority
CN
China
Prior art keywords
car
angle
image
front wheel
automobile
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
CN201810659018.0A
Other languages
Chinese (zh)
Other versions
CN108909625A (en
Inventor
张学武
卢鑫
李丽媛
徐晓龙
李敏
范新南
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hohai University HHU
Original Assignee
Hohai University HHU
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hohai University HHU filed Critical Hohai University HHU
Priority to CN201810659018.0A priority Critical patent/CN108909625B/en
Publication of CN108909625A publication Critical patent/CN108909625A/en
Application granted granted Critical
Publication of CN108909625B publication Critical patent/CN108909625B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R21/00—Arrangements or fittings on vehicles for protecting or preventing injuries to occupants or pedestrians in case of accidents or other traffic risks
    • B60R21/01—Electrical circuits for triggering passive safety arrangements, e.g. airbags, safety belt tighteners, in case of vehicle accidents or impending vehicle accidents
    • B60R21/013—Electrical circuits for triggering passive safety arrangements, e.g. airbags, safety belt tighteners, in case of vehicle accidents or impending vehicle accidents including means for detecting collisions, impending collisions or roll-over
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • H04N7/181—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources

Landscapes

  • Engineering & Computer Science (AREA)
  • Mechanical Engineering (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Closed-Circuit Television Systems (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

The invention discloses a vehicle bottom ground display method based on a panoramic looking-around system. And secondly, finding out the area to be covered by the automobile in the next frame in the current frame through the displacement information, and extracting the area to be covered by the small automobile in the next frame. Furthermore, the running track of the trolley in a short time can be obtained according to the deflection angle of the steering wheel of the trolley, whether the trolley runs forwards or backwards on the track is judged according to the gear of the trolley, and the running distance of the trolley on the track line after one frame of time can be roughly estimated according to the speed of the trolley. The invention aims to realize the real-time display of the ground image at the bottom of the automobile by matching with an ADAS panoramic all-around viewing system. According to the top view of the vehicle body, the information of the image is extracted from the previous frame, and the ground area covered by the vehicle in the current frame is displayed, so that the function of the ADAS system is further improved.

Description

Vehicle bottom ground display method based on panoramic all-round viewing system
Technical Field
The invention relates to a vehicle bottom ground display method based on a panoramic all-round viewing system, and belongs to the technical field of automatic driving of vehicles.
Background
At present, a panoramic looking-around system collects road conditions around a vehicle body through wide-angle cameras around the vehicle body and synthesizes a complete looking-around image. However, the image of the vehicle bottom part cannot be completely displayed due to the fact that the image is shielded by the automobile, at the moment, the part, which is shielded by the automobile, in the current frame can be extracted through the image information in the video frame collected before, and the part is spliced into the current frame, so that the real-time display of the vehicle bottom image is realized.
By the technology, the automobile can effectively detect the blind area of the automobile bottom in the driving process, avoid being trapped in a hollow or running over sharp objects, and provide useful automobile bottom information for the automobile under special driving conditions such as narrow bridge passing, backing and warehousing, and the like, thereby ensuring the driving safety.
Disclosure of Invention
The purpose is as follows: in order to overcome the defects in the prior art, the invention provides a vehicle bottom ground display method based on a panoramic all-round viewing system.
The technical scheme is as follows: in order to solve the technical problems, the technical scheme adopted by the invention is as follows:
a vehicle bottom ground display method based on a panoramic all-round viewing system specifically comprises the following steps:
step S101, reading a video frame of an automobile panoramic all-around system;
step S102, reading the running parameters of the automobile;
step S103, calculating a running track line of the automobile;
step S104, predicting the position to which the automobile of the next frame is going to move;
step S105, intercepting the image of the position predicted in step S104 from the current frame;
step S106, carrying out affine transformation on the intercepted image to obtain an underbody image;
and S107, reading the panoramic all-around view image of the next frame of automobile, splicing the estimated vehicle bottom image in the previous frame to the part covered by the automobile in the frame, and obtaining the panoramic all-around view image of the next frame of the displayed vehicle bottom image.
As a preferred scheme, the panoramic all-round looking system collects multiple paths of video images through a plurality of wide-angle lenses around a vehicle body, and the multiple paths of video images are spliced after distortion correction, so that each frame is synthesized into a complete overlook image; in the image, the automobile area is positioned in the middle, the surrounding images are the environment where the automobile is located, and the surrounding areas are environment top views which change in real time in the moving process of the automobile; in the panoramic all-round viewing system, at the next frame time, the covered image of the automobile after the movement is obtained from the current image.
Preferably, the step S102 includes: the ADAS system collects information of the driving speed, gear and steering wheel angle of the automobile at regular collection time intervals, and because the frequency of information collection is less than the frequency of displayed video frames, the driving speed, gear and steering wheel angle of the automobile when a frame of image of which the automobile information is not collected is displayed need to be estimated according to the automobile information read for the last few times.
Preferably, the step S103 includes:
step 1: obtaining the steering angle of an automobile tire according to the rotation angle of a steering wheel of the automobile;
step 2: the steering angle of the automobile tyre is mu, the intersection point of the outer circular rail and the front axle is alpha, and the included angle between the connecting line of the alpha and the circle center o of the outer circular rail and the rear axle of the automobile is
Figure BDA0001704210190000021
According to the estimation, the method can obtain,
Figure BDA0001704210190000022
Figure BDA0001704210190000023
and step 3: knowing the distance between the front and rear wheels as L, the large radius of the outer rail is given by:
Figure BDA0001704210190000031
and 4, step 4: according to the vehicle width W and a right triangle formed by the circle center o and the two wheels, the radius of the inner rail is obtained as follows:
Figure BDA0001704210190000032
and 5: since the inner track line is tangent to the inner steerable wheel in the front wheel of the vehicle and the outer track line is tangent to the outer steerable wheel in the rear wheel of the vehicle, the track line on which the vehicle travels in each frame time difference can be approximately regarded as a circle of fixed radius.
Preferably, the step S104 includes:
step 1: performing curve fitting on the change relation of the automobile speed and the steering wheel angle along with time by a least square method to obtain a driving speed and steering wheel angle prediction formula of the automobile;
step 2: assuming that the current motion state of the automobile is only related to the motion state of the first 1s, predicting the current running speed and the current steering wheel angle of the automobile through the first several groups of data;
and step 3: because the time interval between frames of the video image is short, i.e. assuming that the car is moving at a constant speed along the fixed track, the displacement of the car on the trajectory line is calculated to be L'.
Preferably, the step S105 includes:
step 1: because the image is composed of closely arranged pixel points, the image is regarded as a coordinate space according to the sequence of the pixel points from top to bottom and from left to right, the distance between adjacent pixel points in the horizontal direction and the vertical direction is the unit interval in the x-axis direction and the y-axis direction, and the displacement L' of each time of the automobile is reflected as S pixel points moved on the image according to the proportional relation between the image and a real object; establishing a rectangular coordinate system by taking the lower left corner of the panoramic view image as an origin, the horizontal direction as the x-axis direction and the vertical direction as the y-axis direction;
step 2: the coordinates of four vertexes and four tires of the automobile are known, the relative position of the area to which the automobile moves in the next frame in the current frame image is calculated according to the angle and distance information, and then the area covered by the automobile in the next frame is obtained from the current frame, and the specific steps are as follows:
when the automobile runs to the right front, the direction angle from the right front wheel to the circle center of the automobile is gamma, and the size is
Figure BDA0001704210190000041
When the vehicle moves S pixel points along the track in the image, the rotating angle lambda of the vehicle on the track line is
Figure BDA0001704210190000042
At the moment, the right front wheel and the x axis of the automobile are alignedThe included angle of the direction is gamma-lambda; according to the coordinates (x) of the right rear wheel of the automobile0，y0) Obtaining the coordinates (x) of the center of the circle0+R-W，y0) Further, the coordinate of the right front wheel after displacement is obtained as (x)0+R-W+R*cos(γ-λ)，y0+ R × sin (γ - λ)); the angle of the automobile body is gamma-lambda-mu-90 degrees, and the angle of the right front wheel perpendicular to the automobile body along the direction from the axle to the left front wheel is gamma-lambda-mu;
when the automobile runs to the left front, the direction angle from the initial left front wheel to the circle center of the automobile is gamma, and the magnitude of the direction angle is gamma
Figure BDA0001704210190000043
When the vehicle moves S pixel points along the track in the image, the rotating angle lambda of the vehicle on the track line is
Figure BDA0001704210190000044
At the moment, the positive direction included angle between the left front wheel of the automobile and the x axis is 90 degrees + gamma + lambda; according to the coordinates (x) of the left rear wheel of the automobile0，y0) The center coordinates (x) of the circle can be obtained0-R+W，y0) And the direction angle from the center of the circle to the left front wheel of the automobile after displacement is gamma + lambda, and the left front wheel coordinate of the automobile after displacement is further obtained as (x)0-R+W+R*cos(γ+λ)，y0+ R × sin (γ + λ)); the deflection angle of the automobile body is gamma + lambda + 90-mu, and the angle of the left front wheel perpendicular to the automobile body along the direction from the axle to the right front wheel is gamma + lambda-mu;
when the automobile runs to the right rear direction, the direction angle from the right front wheel to the circle center of the automobile is gamma, and the magnitude is
Figure BDA0001704210190000051
When the vehicle moves S pixel points along the track in the image, the rotating angle lambda of the vehicle on the track line is
Figure BDA0001704210190000052
At the moment, the positive direction included angle between the right front wheel of the automobile and the x axis is gamma + lambda; according to the coordinates (x) of the right rear wheel of the automobile0，y0) The center coordinates (x) of the circle can be obtained0+R-W，y0) Further can obtainThe coordinate of the right front wheel after displacement is (x)0+R-W+R*cos(γ+λ)，y0+ R × sin (γ + λ)); the angle of the automobile body is gamma + lambda + mu-90 degrees, and the angle of the right front wheel perpendicular to the automobile body along the direction from the axle to the left front wheel is gamma + lambda + mu;
when the automobile runs towards the left rear part, the direction angle from the initial left front wheel to the circle center of the automobile is gamma, and the magnitude of the direction angle is gamma
Figure BDA0001704210190000053
When the vehicle moves S pixel points along the track in the image, the rotating angle lambda of the vehicle on the track line is
Figure BDA0001704210190000054
At the moment, the positive direction included angle between the left front wheel of the automobile and the x axis is 90 degrees + gamma-lambda; according to the coordinates (x) of the left rear wheel of the automobile0，y0) The center coordinates (x) of the circle can be obtained0-R+W，y0) And the direction angle from the center of the circle to the left front wheel of the automobile after the displacement is gamma-lambda, and the left front wheel coordinate of the automobile after the displacement is further obtained as (x)0-R+W+R*cos(γ-λ)，y0+ R × sin (γ - λ)); the angle of the automobile body is gamma-lambda + 90-mu, and the angle of the left front wheel perpendicular to the automobile body along the direction from the axle to the right front wheel is gamma-lambda-mu;
and (3) estimating the position of the displaced automobile in an image coordinate system according to the estimated displacement information which mainly comprises the coordinates of the displaced wheels in the 360 images and the inclination angle of the automobile body, wherein the image in the area is the image area to be covered by the automobile in the next frame.
Preferably, the step S106 includes: and intercepting the image of the area where the automobile is located after the prediction displacement, and mapping the image to the vertical direction to be used as an automobile bottom image.
Has the advantages that: the vehicle bottom ground display method based on the panoramic all-around system provided by the invention displays the vehicle bottom ground in real time by means of image splicing by utilizing useful information in a video sequence of the panoramic all-around system, thereby further improving the function of the ADAS system.
Drawings
FIG. 1 is a top view image of a single frame of a panoramic look-around system;
FIG. 2 is a vehicle travel trajectory line;
FIG. 3 shows the process of displaying images of the vehicle bottom (taking a right turn as an example);
fig. 4 is a flowchart of a method for displaying vehicle bottom images of a look-around system.
Detailed Description
The present invention will be further described with reference to the accompanying drawings.
As shown in fig. 4, a vehicle bottom ground display method based on a panoramic all-around viewing system specifically comprises the following steps:
and step S101, reading a video frame of the panoramic all-around system.
As shown in fig. 1, in the panoramic all-round viewing system, multiple paths of video images are collected through multiple wide-angle lenses around a vehicle body, and are spliced after distortion correction, so that each frame is combined into a complete overhead view image. In the image, the automobile area is located in the middle, the surrounding images are the environment where the automobile is located, and the surrounding areas are the environment top view which changes in real time in the moving process of the automobile. In the panoramic all-round system, the distance from the automobile to the four sides of the image is about 1/4 automobile body lengths, so that the image covered by the automobile after the automobile moves can be obtained from the current image at the next frame time. In addition, the video frame frequency of the video image acquisition and display is about 25-30 frames/s, namely the interval time between frames is 30-40 ms.
And step S102, reading parameters such as the speed of the automobile and the like.
Specifically, the ADAS system (advanced driving assistance system) collects information about the driving speed, gear position, and steering wheel angle of the vehicle at regular collection intervals of about 100ms, so the frequency of information collection is less than the frequency of the displayed video frames, about 1: 3, that is, the driving information of the vehicle is collected once within the time of displaying three frames of images. According to the information read for the last time, the driving speed, the gear and the steering wheel angle of the automobile can be estimated when each frame of image is displayed, and the information can be considered to be approximately unchanged from the current frame to the next frame. Since the time interval of image display is short and the acceleration of the vehicle is limited, the error caused by the slight change of the driving state of the vehicle in the time interval of each frame is negligible in the case of normal driving.
Step S103, calculating the running track line of the automobile, wherein the track line is shown in FIG. 2.
Specifically, the tire steering angle of the automobile can be obtained according to the steering wheel angle of the automobile, and the tire is generally turned for 45 degrees by one and a half turns of the steering wheel, namely, the automobile tire rotates for 1 degree every 12 degrees of the steering wheel.
Referring to fig. 2, it can be known through analysis that the steering angle of the automobile tire is μ, the intersection point of the outer circular rail and the front axle is α, and the included angle between the connecting line of α and the circle center o of the outer circular rail and the rear axle of the automobile is α
Figure BDA0001704210190000071
According to the estimation, the method can obtain,
Figure BDA0001704210190000072
knowing the distance between the front and rear wheels as L, it can be obtained that the large radius of the outer rail is:
Figure BDA0001704210190000073
according to the vehicle width W and a right triangle formed by the circle center and the two wheels, the radius of the inner rail is obtained as follows:
Figure BDA0001704210190000081
since the inner track line is tangent to the inner steering wheel of the front wheel of the vehicle and the outer track line is tangent to the outer steering wheel of the rear wheel of the vehicle, the track line of the vehicle driving in each frame time difference can be approximately regarded as a circle with a fixed radius, and the radius of the track circle is related to the deflection angle of the vehicle tire.
Step S104, predicting the position to which the automobile of the next frame is going to move.
Specifically, the reading frequency of the driving state (speed, gear position, tire deflection angle) of the automobile and the display frequency of the video frames are 1: 3, so that the driving information of the automobile needs to be estimated from the automobile information read in the previous times when each frame of image is displayed. By the least square method, curve fitting can be carried out on the change relation of the information such as the automobile speed and the steering wheel angle along with time, and the driving speed and the steering wheel angle of the automobile in the current frame are obtained. Assuming that the current motion state of the vehicle is only related to the motion state of the first 1s, the current driving information of the vehicle can be predicted through 10 sets of data. Because the time interval between frames is short, the change of the speed and the steering wheel angle of the automobile in the time interval can be ignored, namely, the automobile is assumed to move at a constant speed along a fixed track, so that the displacement of the automobile on the track can be calculated to be L', and the area of the automobile after the displacement.
Step S105, the region is cut out from the current frame.
As shown in fig. 3, since the image is composed of closely arranged pixels, all the pixels can be regarded as a coordinate space according to the order of the pixels from top to bottom and from left to right, and the distance between adjacent pixels in the horizontal and vertical directions is the unit interval in the x and y axis directions. According to the proportional relation between the image and the real object, the displacement L' of the automobile each time is reflected on the image as the movement of S pixel points. Because the automobile in the panoramic all-around view image is vertically embedded in the middle of the image all the time, for the convenience of calculation, a rectangular coordinate system is established by taking the lower left corner of the panoramic all-around view image as the origin, the horizontal direction as the x-axis direction and the vertical direction as the y-axis direction. And because the structure of the panoramic all-round view image is fixed, the coordinates of four vertexes and four tires of the automobile are known, so that the relative position of the area to which the automobile moves in the next frame in the current frame image can be calculated according to the angle and distance information, and the area covered by the trolley in the next frame can be obtained from the current frame.
Specifically, when the automobile is driven to the right front, (along the direction represented by the angle and length in FIG. 2) the right front wheel of the automobile has a direction angle γ from the center of the circle, which is the magnitude of
Figure BDA0001704210190000091
When the vehicle moves S pixel points along the track in the image, the rotating angle lambda of the vehicle on the track line is
Figure BDA0001704210190000092
At the moment, the positive direction included angle between the right front wheel of the automobile and the x axis is gamma-lambda. According to the coordinates (x) of the right rear wheel of the automobile0，y0) The center coordinates (x) of the circle can be obtained0+R-W，y0) Further, the coordinates of the right front wheel after the displacement can be obtained as (x)0+R-W+R*cos(γ-λ)，y0+ R × sin (γ - λ)). The angle of the automobile body is gamma-lambda-mu-90 degrees, and the angle of the right front wheel perpendicular to the automobile body along the direction from the axle to the left front wheel is gamma-lambda-mu.
When the automobile runs to the left front, the direction angle from the initial left front wheel to the circle center of the automobile is gamma, and the magnitude of the direction angle is gamma
Figure BDA0001704210190000093
When the vehicle moves S pixel points along the track in the image, the rotating angle lambda of the vehicle on the track line is
Figure BDA0001704210190000094
At the moment, the included angle between the left front wheel of the automobile and the positive direction of the x axis is 90 degrees + gamma + lambda. According to the coordinates (x) of the left rear wheel of the automobile0，y0) The center coordinates (x) of the circle can be obtained0-R+W，y0) And the direction angle from the center of the circle to the left front wheel of the automobile after displacement is gamma + lambda, and the left front wheel coordinate of the automobile after displacement is further obtained as (x)0-R+W+R*cos(γ+λ)，y0+ R × sin (γ + λ)). The deflection angle of the automobile body is gamma + lambda + 90-mu, and the angle of the left front wheel perpendicular to the automobile body along the direction angle from the axle to the right front wheel is gamma + lambda-mu.
When the automobile runs to the right rear direction, the direction angle from the right front wheel to the circle center of the automobile is gamma, and the magnitude is
Figure BDA0001704210190000101
After the vehicle moves S pixel points along the track in the image, the vehicle rotates on the track lineAngle lambda is
Figure BDA0001704210190000102
At the moment, the positive direction included angle between the right front wheel of the automobile and the x axis is gamma + lambda. According to the coordinates (x) of the right rear wheel of the automobile0，y0) The center coordinates (x) of the circle can be obtained0+R-W，y0) Further, the coordinates of the right front wheel after the displacement can be obtained as (x)0+R-W+R*cos(γ+λ)，y0+ R × sin (γ + λ)). The angle of the automobile body is gamma + lambda + mu-90 degrees, and the angle of the right front wheel perpendicular to the automobile body along the direction angle from the axle to the left front wheel is gamma + lambda + mu.
When the automobile runs towards the left rear part, the direction angle from the initial left front wheel to the circle center of the automobile is gamma, and the magnitude of the direction angle is gamma
Figure BDA0001704210190000103
When the vehicle moves S pixel points along the track in the image, the rotating angle lambda of the vehicle on the track line is
Figure BDA0001704210190000104
At the moment, the included angle between the left front wheel of the automobile and the positive direction of the x axis is 90 degrees + gamma-lambda. According to the coordinates (x) of the left rear wheel of the automobile0，y0) The center coordinates (x) of the circle can be obtained0-R+W，y0) And the direction angle from the center of the circle to the left front wheel of the automobile after the displacement is gamma-lambda, and the left front wheel coordinate of the automobile after the displacement is further obtained as (x)0-R+W+R*cos(γ-λ)，y0+ R × sin (γ - λ)). The angle of the automobile body is gamma-lambda + 90-mu, and the angle of the left front wheel perpendicular to the automobile body along the direction from the axle to the right front wheel is gamma-lambda-mu.
And estimating the position of the displaced automobile in an image coordinate system according to the estimated displacement information which mainly comprises the coordinates of the displaced wheels in the 360 images and the inclination angle of the automobile body, wherein the image in the area is the image area to be covered by the automobile in the next frame.
Step S106, affine transformation.
Specifically, after the image of the area where the automobile is located after the displacement is captured, the image is a pair of inclined rectangular images, and the images need to be mapped to the vertical direction to be spliced to the automobile area of the next frame.
And S107, reading the panoramic all-around view image of the next frame of automobile, splicing the estimated vehicle bottom image in the previous frame to the part covered by the automobile in the frame, and obtaining the panoramic all-around view image of the next frame of the displayed vehicle bottom image.
The above description is only of the preferred embodiments of the present invention, and it should be noted that: it will be apparent to those skilled in the art that various modifications and adaptations can be made without departing from the principles of the invention and these are intended to be within the scope of the invention.

Claims (4)

1.一种基于全景环视系统的车底地面显示方法，其特征在于：具体包括步骤如下：1. a vehicle bottom ground display method based on panoramic surround view system, is characterized in that: specifically comprise the steps as follows: 步骤S101，读取汽车全景环视系统的视频帧；Step S101, read the video frame of the car panoramic surround view system; 步骤S102，读取汽车的运行参数；Step S102, reading the operating parameters of the vehicle; 步骤S103，计算汽车的行驶轨迹线；Step S103, calculating the driving trajectory of the vehicle; 所述步骤S103包括：The step S103 includes: 步骤1：根据汽车的方向盘转动角度得到汽车轮胎转向角；Step 1: Obtain the steering angle of the car tires according to the steering wheel rotation angle of the car; 步骤2：汽车轮胎转向角为μ，外圆轨与前桥的交点为α，α与外圆轨圆心o的连线和汽车后桥间的夹角为φ，根据推算可得，μ＝φ；Step 2: The steering angle of the car tire is μ, the intersection of the outer circular rail and the front axle is α, and the angle between the connection line between α and the center o of the outer circular rail and the rear axle of the vehicle is φ. According to calculation, μ=φ ; 步骤3：已知前后轮之间的距离为L，可得外轨的大圆半径为：Step 3: Knowing that the distance between the front and rear wheels is L, the radius of the great circle of the outer rail can be obtained as:
Figure FDA0003116602650000011
Figure FDA0003116602650000011
 步骤4：根据车宽W，以及圆心o与两个车轮形成的直角三角形，可得到内轨半径为：Step 4: According to the width W of the vehicle and the right triangle formed by the center o and the two wheels, the radius of the inner rail can be obtained as:
Figure FDA0003116602650000012
Figure FDA0003116602650000012
 步骤5：由于内轨迹线与汽车前轮中的内转向轮相切，外轨迹线与汽车后轮中的外转向轮相切，每一帧的时间差内汽车行驶的轨迹线可近似看作一个固定半径的圆；Step 5: Since the inner trajectory line is tangent to the inner steering wheel in the front wheel of the car, and the outer trajectory line is tangent to the outer steering wheel in the rear wheel of the car, the trajectory line of the car in the time difference of each frame can be approximately regarded as a A circle with a fixed radius; 步骤S104，预测下一帧汽车将要运动到的位置；Step S104, predicting the position where the car will move to in the next frame; 所述步骤S104包括：The step S104 includes: 步骤1：通过最小二乘法，对汽车速度、方向盘角度随时间的变化关系进行曲线拟合，得到汽车的行驶速度和方向盘角度预测公式；Step 1: Through the least square method, curve fitting is performed on the relationship between the vehicle speed and the steering wheel angle with time, and the prediction formula of the driving speed and steering wheel angle of the vehicle is obtained; 步骤2：假设汽车当前的运动状态只与前1s的运动状态有关，通过前几组数据来预测汽车当前的行驶速度和方向盘角度；Step 2: Assuming that the current motion state of the car is only related to the motion state of the previous 1s, the current driving speed and steering wheel angle of the car are predicted through the previous sets of data; 步骤3：因为视频图像帧与帧之间的时间间隔短，即假定汽车沿着固定轨道做匀速运动，计算出汽车在轨迹线上的位移为L′；Step 3: Because the time interval between video image frames is short, that is, assuming that the car moves at a constant speed along the fixed track, the displacement of the car on the trajectory line is calculated as L'; 步骤S105，在当前帧中截取出步骤S104预测的位置的图像；所述步骤S105包括：Step S105, the image of the position predicted in step S104 is cut out from the current frame; the step S105 includes: 步骤1：由于图像由紧密排列的像素点组成，根据像素点从上到下、从左到右的次序将图像看作一个坐标空间，水平和竖直方向上相邻像素点的距离即为x、y轴方向上的单位间隔，根据图像和实物的比例关系，汽车每一次的位移L′在图像上反映为移动了S个像素点；以全景环视图像左下角为原点，水平方向为x轴方向，竖直方向为y轴方向，建立直角坐标系；Step 1: Since the image is composed of closely arranged pixels, the image is regarded as a coordinate space according to the order of pixels from top to bottom and from left to right, and the distance between adjacent pixels in the horizontal and vertical directions is x , the unit interval in the y-axis direction, according to the proportional relationship between the image and the real object, each displacement L' of the car is reflected in the image as moving S pixels; the lower left corner of the panoramic image is the origin, and the horizontal direction is the x-axis The vertical direction is the y-axis direction, and a Cartesian coordinate system is established; 步骤2：汽车的四个顶点，四个轮胎的坐标都是已知的，根据角度和距离信息，计算出下一帧时汽车位移到的区域在当前帧图像中的相对位置，进而从当前帧中获得下一帧被汽车覆盖的区域，具体步骤如下：Step 2: The coordinates of the four vertices of the car and the four tires are known. According to the angle and distance information, the relative position of the area where the car is displaced in the next frame is calculated in the current frame image, and then from the current frame. Get the area covered by the car in the next frame, the specific steps are as follows: 当汽车向右前方行驶时，汽车右前轮到圆心的方向角为γ，大小为
Figure FDA0003116602650000021
当车子在图像中沿轨迹移动了S个像素点后，其在轨迹线上转过的角度λ为
Figure FDA0003116602650000022
此时汽车右前轮与x轴的正方向夹角为γ-λ；根据汽车右后轮的坐标(x0,y0),得到圆心坐标(x0+R-W,y0)，进一步得到位移后右前轮的坐标为(x0+R-W+R*cos(γ-λ),y0+R*sin(γ-λ))；汽车车身角度为γ-λ-μ-90°，右前轮沿着车桥到左前轮的方向角垂直于车身的角度，为γ-λ-μ；When the car is driving forward to the right, the direction angle from the right front wheel of the car to the center of the circle is γ, and its magnitude is
Figure FDA0003116602650000021
When the car moves S pixels along the trajectory in the image, the angle λ that the car turns on the trajectory is
Figure FDA0003116602650000022
At this time, the angle between the right front wheel of the car and the positive direction of the x-axis is γ-λ; according to the coordinates of the right rear wheel of the car (x 0 , y 0 ), the coordinates of the center of the circle (x 0 +RW, y 0 ) are obtained, and the displacement is further obtained. The coordinates of the rear right front wheel are (x 0 +R-W+R*cos(γ-λ),y 0 +R*sin(γ-λ)); the vehicle body angle is γ-λ-μ-90°, The direction angle of the right front wheel along the axle to the left front wheel is perpendicular to the angle of the body, which is γ-λ-μ; 当汽车向左前方行驶时，汽车初始左前轮到圆心的方向角为γ，其大小为
Figure FDA0003116602650000031
当车子在图像中沿轨迹移动了S个像素点后，其在轨迹线上转过的角度λ为
Figure FDA0003116602650000032
此时汽车左前轮与x轴的正方向夹角为90°+γ+λ；根据汽车左后轮的坐标(x0,y0),可以得到圆心坐标(x0-R+W,y0)，位移后圆心到汽车左前轮的方向角为γ+λ，进一步得到位移后汽车的左前轮坐标为(x0-R+W+R*cos(γ+λ),y0+R*sin(γ+λ))；汽车车身的偏转角度为γ+λ+90°-μ，左前轮沿着车桥到右前轮的方向角垂直于车身的角度，为γ+λ-μ；When the car is driving to the left front, the direction angle from the initial left front wheel of the car to the center of the circle is γ, and its magnitude is
Figure FDA0003116602650000031
When the car moves S pixels along the trajectory in the image, the angle λ that the car turns on the trajectory is
Figure FDA0003116602650000032
At this time, the angle between the left front wheel of the car and the positive direction of the x-axis is 90°+γ+λ; according to the coordinates of the left rear wheel of the car (x 0 , y 0 ), the coordinates of the center of the circle (x 0 -R+W, y 0 ), the direction angle from the center of the circle to the left front wheel of the car after displacement is γ+λ, and the coordinates of the left front wheel of the car after further displacement are (x 0 -R+W+R*cos(γ+λ),y 0 + R*sin(γ+λ)); the deflection angle of the car body is γ+λ+90°-μ, and the direction angle from the left front wheel along the axle to the right front wheel is perpendicular to the body, which is γ+λ- μ; 当汽车向右后方行驶时，汽车右前轮到圆心的方向角为γ，大小为
Figure FDA0003116602650000033
当车子在图像中沿轨迹移动了S个像素点后，其在轨迹线上转过的角度λ为
Figure FDA0003116602650000034
此时汽车右前轮与x轴的正方向夹角为γ+λ；根据汽车右后轮的坐标(x0,y0),可以得到圆心坐标(x0+R-W,y0)，进一步可以得到位移后右前轮的坐标为(x0+R-W+R*cos(γ+λ),y0+R*sin(γ+λ))；汽车车身角度为γ+λ+μ-90°，右前轮沿着车桥到左前轮的方向角垂直于车身的角度，为γ+λ+μ；When the car is driving to the right and rear, the direction angle from the right front wheel of the car to the center of the circle is γ, and its magnitude is
Figure FDA0003116602650000033
When the car moves S pixels along the trajectory in the image, the angle λ that the car turns on the trajectory is
Figure FDA0003116602650000034
At this time, the angle between the right front wheel of the car and the positive direction of the x-axis is γ+λ; according to the coordinates of the right rear wheel of the car (x 0 , y 0 ), the coordinates of the center of the circle (x 0 +RW, y 0 ) can be obtained, and further After the displacement is obtained, the coordinates of the right front wheel are (x 0 +R-W+R*cos(γ+λ),y 0 +R*sin(γ+λ)); the angle of the car body is γ+λ+μ-90 °, the direction angle of the right front wheel along the axle to the left front wheel is perpendicular to the angle of the vehicle body, which is γ+λ+μ; 当汽车向左后方行驶时，汽车初始左前轮到圆心的方向角为γ，其大小为
Figure FDA0003116602650000041
当车子在图像中沿轨迹移动了S个像素点后，其在轨迹线上转过的角度λ为
Figure FDA0003116602650000042
此时汽车左前轮与x轴的正方向夹角为90°+γ-λ；根据汽车左后轮的坐标(x0,y0),可以得到圆心坐标(x0-R+W,y0)，位移后圆心到汽车左前轮的方向角为γ-λ，进一步得到位移后汽车的左前轮坐标为(x0-R+W+R*cos(γ-λ),y0+R*sin(γ-λ))；汽车车身角度为γ-λ+90°-μ，左前轮沿着车桥到右前轮的方向角垂直于车身的角度，为γ-λ-μ；When the car is driving to the left and rear, the direction angle of the initial left front wheel of the car to the center of the circle is γ, and its magnitude is
Figure FDA0003116602650000041
When the car moves S pixels along the trajectory in the image, the angle λ that the car turns on the trajectory is
Figure FDA0003116602650000042
At this time, the angle between the left front wheel of the car and the positive direction of the x-axis is 90°+γ-λ; according to the coordinates of the left rear wheel of the car (x 0 , y 0 ), the coordinates of the center of the circle (x 0 -R+W, y 0 ), the direction angle from the center of the circle to the left front wheel of the car after displacement is γ-λ, and the coordinates of the left front wheel of the car after further displacement are (x 0 -R+W+R*cos(γ-λ),y 0 + R*sin(γ-λ)); the angle of the car body is γ-λ+90°-μ, and the angle from the left front wheel along the axle to the right front wheel is perpendicular to the body, which is γ-λ-μ; 根据估算得到的位移信息，主要包括位移后车轮在360图像中的坐标，以及车身的倾斜角，即可估算出位移后汽车在图像坐标系中的位置，这个区域下的图像即为下一帧汽车将要覆盖的图像区域；According to the estimated displacement information, which mainly includes the coordinates of the displaced wheels in the 360 image, and the inclination angle of the body, the position of the displaced car in the image coordinate system can be estimated, and the image in this area is the next frame. The image area that the car will cover; 步骤S106，将截取图像仿射变换，作为车底图像；Step S106, affine transformation of the intercepted image as a vehicle bottom image; 步骤S107，读取下一帧汽车的全景环视图像，将上一帧中预估的车底图像拼接到该帧当中被汽车盖住的部分，即可得到下一帧显示车底图像的一幅全景环视图。Step S107, read the panoramic surround view image of the car in the next frame, and splicing the estimated car bottom image in the previous frame to the part covered by the car in the frame, so as to obtain a next frame showing the car bottom image. Panoramic ring view. 2.根据权利要求1所述的基于全景环视系统的车底地面显示方法，其特征在于：所述全景环视系统通过车身四周的多个广角镜头采集多路视频影像，经过畸变校正后进行拼接，将每一帧合成一幅完整的俯视图像；在该图像中，汽车区域位于正中间，四周的图像即为汽车所处的环境，在汽车移动的过程中，四周区域为实时变化的环境俯视图；在该全景环视系统中，下一帧时刻，汽车运动后覆盖掉的图像从当前图像中获取到。2 . The vehicle bottom ground display method based on a panoramic surround view system according to claim 1 , wherein the panoramic surround view system collects multi-channel video images through a plurality of wide-angle lenses around the vehicle body, and performs splicing after distortion correction. Each frame synthesizes a complete overhead image; in this image, the car area is located in the middle, and the surrounding images are the environment where the car is located. During the process of the car moving, the surrounding area is a real-time changing environment overhead view; In the panoramic surround view system, at the next frame moment, the image covered by the motion of the car is obtained from the current image. 3.根据权利要求1所述的基于全景环视系统的车底地面显示方法，其特征在于：所述步骤S102包括：ADAS系统每隔一定的采集时间采集汽车行驶速度、档位、方向盘角度的信息，由于信息采集的频率比显示的视频帧频率小，就需要根据最近几次读取到的汽车信息，估算出未采集到汽车信息的一帧图像在显示时汽车的行驶速度、档位以及方向盘角度。3. the vehicle bottom ground display method based on the panoramic surround view system according to claim 1, is characterized in that: described step S102 comprises: ADAS system collects the information of vehicle traveling speed, gear position, steering wheel angle every certain collection time , since the frequency of information collection is smaller than the displayed video frame frequency, it is necessary to estimate the driving speed, gear position and steering wheel of the car when a frame of image without collected car information is displayed based on the car information read several times recently. angle. 4.根据权利要求1所述的基于全景环视系统的车底地面显示方法，其特征在于：所述步骤S106包括：截取出预测位移后汽车所在区域的图像，将其映射到竖直方向，作为汽车车底图像。4. The vehicle bottom ground display method based on the panoramic surround view system according to claim 1, wherein the step S106 comprises: cutting out the image of the area where the car is located after the predicted displacement, and mapping it to the vertical direction, as Car underbody image.
CN201810659018.0A 2018-06-22 2018-06-22 Vehicle bottom ground display method based on panoramic all-round viewing system Expired - Fee Related CN108909625B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201810659018.0A CN108909625B (en) 2018-06-22 2018-06-22 Vehicle bottom ground display method based on panoramic all-round viewing system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810659018.0A CN108909625B (en) 2018-06-22 2018-06-22 Vehicle bottom ground display method based on panoramic all-round viewing system

Publications (2)

Publication Number Publication Date
CN108909625A CN108909625A (en) 2018-11-30
CN108909625B true CN108909625B (en) 2021-09-17

Family

ID=64422337

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810659018.0A Expired - Fee Related CN108909625B (en) 2018-06-22 2018-06-22 Vehicle bottom ground display method based on panoramic all-round viewing system

Country Status (1)

Country Link
CN (1) CN108909625B (en)

Families Citing this family (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111836005A (en) * 2019-04-23 2020-10-27 东莞潜星电子科技有限公司 A vehicle-mounted 3D panoramic surround view driving route display system
CN111845713A (en) * 2019-04-26 2020-10-30 东莞潜星电子科技有限公司 A parking route planning method based on vehicle 3D panoramic view
CN111856871A (en) * 2019-04-26 2020-10-30 东莞潜星电子科技有限公司 A vehicle-mounted 3D panoramic surround view display method
CN112215747A (en) * 2019-07-12 2021-01-12 杭州海康威视数字技术股份有限公司 Method and device for generating vehicle-mounted panoramic picture without vehicle bottom blind area and storage medium
CN110458884A (en) * 2019-08-16 2019-11-15 北京茵沃汽车科技有限公司 Method, apparatus, the medium of vehicle operation state trajectory line are generated in panorama sketch
CN113228135B (en) * 2021-03-29 2022-08-26 华为技术有限公司 Blind area image acquisition method and related terminal device
CN113382294B (en) * 2021-06-04 2023-05-12 广州小鹏汽车科技有限公司 Picture display processing method and device of remote cockpit, cockpit and system
CN113935942B (en) * 2021-09-02 2024-07-23 同致电子科技（厦门）有限公司 Non-linear optimization-based vehicle bottom transparency method, device, equipment, storage medium and computer program product

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
ATE311725T1 (en) * 2001-09-07 2005-12-15 Matsushita Electric Industrial Co Ltd DEVICE FOR DISPLAYING THE SURROUNDINGS OF A VEHICLE AND SYSTEM FOR PROVIDING IMAGE
JP4254887B2 (en) * 2006-07-06 2009-04-15 日産自動車株式会社 Image display system for vehicles
KR100956858B1 (en) * 2009-05-19 2010-05-11 주식회사 이미지넥스트 Sensing method and apparatus of lane departure using vehicle around image
US9187091B2 (en) * 2012-07-30 2015-11-17 Ford Global Technologies, Llc Collision detection system with a plausibiity module
CN203228708U (en) * 2013-04-25 2013-10-09 郑州宇通客车股份有限公司 Vehicular panoramic display system and a bus with same
CN106608220B (en) * 2015-10-22 2019-06-25 比亚迪股份有限公司 Generation method, device and the vehicle of vehicle bottom image
CN108068696B (en) * 2016-11-17 2020-06-19 比亚迪股份有限公司 Panoramic image display method and display device
CN107066954B (en) * 2017-03-23 2019-07-23 浙江零跑科技有限公司 A kind of vehicle-mounted 360 degree are looked around display methods and system
CN107274342A (en) * 2017-05-22 2017-10-20 纵目科技（上海）股份有限公司 A kind of underbody blind area fill method and system, storage medium, terminal device
CN107745677A (en) * 2017-09-30 2018-03-02 东南（福建）汽车工业有限公司 A kind of method of the 4D underbody transparent systems based on 3D full-view image systems

Also Published As

Publication number Publication date
CN108909625A (en) 2018-11-30

Similar Documents

Publication Publication Date Title
CN108909625B (en) Vehicle bottom ground display method based on panoramic all-round viewing system
CN102745138B (en) Dual view-field dynamic-trajectory reverse image system
JP6350374B2 (en) Road surface detection device
US9792683B2 (en) System, vehicle and method for online calibration of a camera on a vehicle
CN103763517B (en) Vehicle-mounted around view display method and system
CN102745198B (en) Auxiliary forward track device for vehicle
JP5077606B2 (en) Method for calibrating a device comprising at least one omnidirectional camera and an optical display unit
CN107284455B (en) An ADAS system based on image processing
JP7107931B2 (en) Method and apparatus for estimating range of moving objects
EP3803790B1 (en) Motion segmentation in video from non-stationary cameras
CN103072528A (en) Vehicle and panoramic parking method and system thereof
CN109389060B (en) Vision-based vehicle surrounding collision early warning method
CN103692974B (en) A kind of vehicle driving safety method for early warning based on environmental monitoring and system
CN103204104B (en) Monitored control system and method are driven in a kind of full visual angle of vehicle
CN107220632B (en) A road image segmentation method based on normal feature
CN107878561A (en) A kind of dynamic reverse trajectory discharge drawing method and its device
CN202847561U (en) Car-backing image system
CN108198248A (en) A kind of vehicle bottom image 3D display method
JP2011220929A (en) Wheel behavior measurement device
JP2004147083A (en) Driving support device
JP2004252837A (en) Vehicle periphery display device and vehicle periphery display program
CN113978456A (en) Reversing image method with target display
JPH07250268A (en) Vehicle surroundings monitoring device
CN112308786B (en) Method for resolving target vehicle motion in vehicle-mounted video based on photogrammetry
CN107103578A (en) It is a kind of to utilize the method for carrying out Panoramagram montage before high-speed railway to Vehicular video

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20210917

CF01 Termination of patent right due to non-payment of annual fee