JP2003189291A - Image processor for area around vehicle and recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processor used for the area around a vehicle and capable of displaying a conspicuous image by converting a birds-eye view image to an image similar to that photographed with a camera or the like and a recording medium. <P>SOLUTION: Step 140 moves a first birds-eye view image stored in an image memory B based on the degree of the movement of a vehicle to obtain the birds-eye view image obtained after moving the vehicle. Step 160 composites the birds-eye view image obtained after moving the vehicle and a second birds- eye view image to prepare a composite birds-eye view image. Step 180 applies reverse birds-eye view conversion to the composite birds-eye view image to prepare a projected image. Step 190 stores the projected image in an image memory C. Step 200 adds an image displaying the vehicle to display the projected image stored in the image memory C on a monitor 3. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a vehicle periphery image processing apparatus and a recording medium capable of photographing the periphery of a vehicle and displaying it on a monitor.

[0002]

2. Description of the Related Art Conventionally, as an apparatus for displaying the state of the rear of a vehicle, an image of a camera attached to the rear of the vehicle is
2. Description of the Related Art A display device is known which outputs the information to a monitor as it is. With this display, you can see how far behind the vehicle is,
There is a problem that it is difficult to understand the relative positional relationship between the image of the subject vehicle (for example, a parking frame) displayed on the monitor.

Separately from this, Japanese Patent Laid-Open No. 10-111
Japanese Patent No. 849 proposes a technique of converting an image captured by a rear camera (rear camera image) into a bird's-eye view and displaying the own vehicle therein.

[0004]

In the case of this technique,
Since the own vehicle is shown in the bird's-eye view, the positional relationship between the target object and the own vehicle displayed in the image is easier to understand than the one in which the captured image is displayed on the monitor as it is, but outside the current visual field of the camera. There was another problem that I couldn't display what was in.

Therefore, for example, when the vehicle is retracted and put in the parking frame, the parking frame outside the field of view of the camera cannot be displayed, so that it is still not easy to put the vehicle in the parking frame. was there. On the other hand, the inventors of the present application have already taken images of the rear of the vehicle from a camera attached to the rear of the vehicle, converted these into bird's-eye-view images viewed from above by image processing, and temporal changes in the converted images. There is a technology to display a wider range than before by calculating the movement of the vehicle from, calculating the image area that disappears outside the field of view of the camera, and pasting the converted image at the current time (creating a composite image). Being developed.

However, the above-mentioned bird's-eye view image is an image viewed from the sky and is different from an image that a driver is usually used to (that is, an image as it is taken by a camera). There was a problem that The present invention has been made to solve the above problems, and by converting a bird's-eye view image into an image taken by a camera or the like, a vehicle peripheral image processing device capable of displaying an easily viewable image, The purpose is to provide a recording medium.

[0007]

[Means for Solving the Problems and Effects of the Invention] (1) The invention of claim 1 includes a photographing means (for example, a camera) for photographing an image of the surroundings of a vehicle and a display means (for example, a monitor) for displaying the image. The present invention relates to a vehicle peripheral image processing device that processes an image captured by an image capturing unit and displays the image on a display unit of a provided vehicle.

In the present invention, an image photographed by the photographing means is converted into, for example, data of the ground plane coordinate system projected from the photographing means as viewpoint, and the image data of the bird's-eye view image is sequentially created. As a result, the image is not a distorted image as it was captured by a camera, but a bird's-eye view image that is easy to see as seen from above the ground.

Then, the first bird's-eye view image (such as the bird's-eye view image obtained last time or the last combined bird's-eye view image) is moved in accordance with the moving state of the vehicle to create image data of the moved bird's-eye view image. The image data of the post-movement bird's-eye view image and the image data of the new second bird's-eye view image (from the first bird's-eye view image) are combined to create the image data of the combined bird's-eye view image.

In other words, as the vehicle moves, for example, the field of view of the camera changes. Therefore, naturally, there is a difference between the first bird's-eye view image and the second bird's-eye view image at different shooting times (difference between the shot parts). Occur. Therefore, the second bird's-eye view image disappears from the camera's field of view on the monitor or the like by appropriately adding the moved bird's-eye view image of the shifted portion (that is, the part that was in the field of view in the past but disappears from the current field of view). The part that is displayed is displayed.

In particular, in the present invention, the above-mentioned combined bird's-eye view image is not displayed as it is on a monitor or the like, but the combined bird's-eye view image is projected from a predetermined viewpoint (for example, a virtual camera position attached to a room mirror in the vehicle interior). And the projected image is displayed on a monitor or the like.

That is, for example, contrary to the method of converting an image captured by a camera into a bird's-eye view, the bird's-eye view image is converted into a projected image as if it was captured by a camera, and the image as if it was captured by the camera is displayed on a monitor or the like. To do. As a result, an image that is normally used by the driver is displayed on the monitor or the like, which has the effect of making it difficult to see.

Further, in the case of synthesizing the surrounding images that change with the movement of the vehicle using the image data as taken by the camera, each image may be distorted. It is necessary to measure the distance of each pixel, but there is a problem that there is no sensor that can measure this with high accuracy.

In the present invention, the distance is known by using the planar bird's-eye view image, and the bird's-eye view image synthesized from this is changed to the original projection image. Therefore, the distance can be easily synthesized without using a distance measuring sensor or the like. There is an advantage that a projected image can be created. (2) In the invention of claim 2, the viewpoint is moved to a virtual position different from the shooting position of the shooting means, and the projection image is obtained with the virtual position as the viewpoint.

Normally, the rear camera is arranged on the rear end side of the vehicle. Therefore, for example, when the projected image is obtained by using the actual position of the camera as a viewpoint, when the vehicle is backed and approaches the wall at the parking lot. , The image of the wall is almost displayed, which is not very useful.

In such a case, for example, the entire parking frame can be displayed by virtually moving the position of the camera to the front end of the vehicle or the like to create a projected image. By overlapping and displaying, the vehicle position in the parking frame can be displayed in an easy-to-understand manner.

(3) According to the invention of claim 3, the virtual position of the viewpoint can be changed by the user. Therefore, the user can easily operate the vehicle at the time of parking, for example, by moving the viewpoint so that the position of the vehicle in the parking frame can be easily understood.

(4) According to the invention of claim 4, the image displayed on the display means can be changed by the user to a composite bird's-eye view image and a projected image. Therefore, the user can select a display that is easy for him to see, which facilitates vehicle operation.

(5) In the fifth aspect of the invention, when the projection image is displayed on the display means, the image showing the own vehicle is additionally displayed. In the present invention, by displaying an image showing the own vehicle in addition to the projected image showing the situation around the vehicle, the positional relationship between the own vehicle and, for example, the parking frame becomes clear, so that the operation of the vehicle becomes easy. .

Incidentally, the own vehicle is an image such as a photograph of the vehicle,
It can be displayed with a symbol in the shape of a vehicle, such as a frame. (6) In the invention of claim 6, as the first bird's-eye view image, the combined bird's-eye view image created before the current combined bird's-eye view image is used.

As a result, the monitor can display a projected image obtained by converting a continuous synthetic bird's-eye view image in which bird's-eye view images (images moved according to the movement of the vehicle) of images taken in the past are sequentially added. . (7) In the invention of claim 7, the combined bird's-eye view image is an image to which the moved bird's-eye view image is added in addition to the display area for displaying the second bird's-eye view image.

The present invention exemplifies a method of forming a synthetic bird's-eye view image. Thereby, images around the vehicle can be continuously displayed. That is, for example, the image currently captured by the camera can be displayed on the monitor as a projection image of the second bird's-eye view image portion, and therefore an image that disappears from the monitor before that is used as the projection image of the moved bird's-eye view image portion. By adding to the area other than the display area of the projection image of the second bird's-eye view image portion, it is possible to display the past surrounding situation disappeared from the field of view of the camera on the monitor.

The latest bird's-eye view image is usually used as the second bird's-eye view image used for the composite bird's-eye view image. (8) The invention according to claim 8 is a recording medium storing a means for executing the processing by the vehicle peripheral image processing device described above.

That is, there is no particular limitation as long as it stores means such as a program that can execute the processing of the above-described vehicle peripheral image processing apparatus. Examples of the recording medium include various recording media such as an electronic control device configured as a microcomputer, a microchip, a floppy disk, a hard disk, and an optical disk.

[0025]

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, an example (embodiment) of an embodiment of a vehicle peripheral image processing apparatus and a recording medium of the present invention will be described. (Example) a) The basic system configuration of the present example will be described with reference to FIGS. 1 and 2.

As shown in FIG. 1, the vehicle peripheral image processing apparatus of the present embodiment has a camera (for example, a CCD camera) 1 arranged at the rear part of an automobile and an in-vehicle monitor (for example, a liquid crystal display) arranged on a dashboard. 3, a vehicle speed sensor 5 that detects a vehicle speed, a yaw rate sensor 7 that detects a yaw rate, and an image processing unit 9 that performs image processing.

As shown in FIG. 2, the image processing unit 9 is an electronic device mainly for processing image data of a microcomputer, and performs coordinate conversion (bird's eye view conversion) of image data taken by the camera 1. A bird's-eye view conversion circuit 11 for performing a bird's-eye view image by performing an image memory A for temporarily storing the image data of the created bird's-eye view image and a bird's-eye view image in the image memory A are moved (for example, rotationally moved) to be combined. A CPU 13 for performing calculations such as image processing, and a CPU
Image memory B for storing the image data processed in 13
And a reverse bird's-eye view conversion circuit 15 that performs coordinate conversion (reverse bird's-eye view conversion) of the image data of the bird's-eye view image stored in the image memory B to return the bird's-eye view image to a projection image, and image data of the projection image (displayed on the monitor C). And an image memory C for storing.

A vehicle speed signal input circuit 17 for inputting a vehicle speed signal and a yaw rate signal input circuit 19 for inputting a yaw rate signal are connected to the image processing unit 9, and each signal is input to the CPU 13. Here, the image data of the bird's-eye view image stored in the image memory B is, as described later,
A bird's-eye view image (post-movement bird's-eye view image) obtained by moving the bird's-eye view image (first bird's-eye view image) that was captured by the camera 1 last time (for example, at time T) and created based on the image data.
And a bird's-eye view image (second bird's-eye view image) newly captured on the camera 1 (for example, at time T + 1) and created based on the image data, are image data of a bird's-eye view image (composite bird's-eye view image). .

The image data of the projected image stored in the image memory C is, as will be described later, the synthetic bird's-eye view image stored in the image memory B at a predetermined viewpoint (in this embodiment, not at the position of the camera 1). It is a projected image converted into a camera image which is an image viewed from a virtual position moved to the front end side of the vehicle).

The image processing unit 9 may have a structure integrated with the camera 1, and each of the components in the image processing unit 9 may be partially or wholly integrated into an LSI. Yes. b) Next, referring to FIG.
~ It demonstrates based on FIG.

Here, the case where the vehicle is moved backward (back) and put in the parking frame will be described as an example. First, a procedure for converting an image input from the camera 1 into a bird's-eye view image viewed from the sky by the bird's-eye view conversion circuit 11 and storing the image in the image memory A will be described. First, the original image (output image) output from the camera 1 is shown in FIG.
This original image is taken by the camera 1 placed at the upper rear part of the vehicle.
Is an image of the parking frame drawn on the ground and its surroundings, the originally rectangular parking frame is distorted depending on the distance between the vehicle (and therefore camera 1) and the position of each line of the parking frame. Is displayed.

Therefore, as will be described later in detail, the image data of the original image is subjected to coordinate conversion to create a bird's-eye view image without distortion as shown in FIG. 3B, and the image data of this bird's-eye view image is converted into image data. It is stored in the image memory A. Next, a procedure for converting an original image captured by the camera 1 into a bird's-eye view image will be described. It should be noted that, as will be described below, this procedure is performed by a conventional technique (for example, Japanese Unexamined Patent Publication No. 10-21184).
No. 9) can be used.

Here, an image on the ground plane (for example, a parking frame) is obtained by performing the reverse processing of the normal perspective transformation.
Position is obtained as a bird's-eye view. Specifically, as shown in FIG. 4, perspective transformation is performed when the position data of the image on the ground is projected on the screen plane T at the focal length f from the camera position R.

Specifically, the camera 1 has a point R on the Z axis.
It is located at (0,0, H) (camera height: H), the camera focal length f, and the view angle (depression angle of the axial center of the camera lens) τ, monitor the image on the ground plane (xy coordinate plane). If so, the relationship between the coordinates (X, Y) on the image of the screen plane T and the coordinates (x, y) on the real space (on the bird's eye view) is given by the following equation (1). X = f · x / (y · SINπ + H · COSπ) (1) Y = f · (−y · COSπ + H · SINπ) / (y · SINπ + H · COSπ) That is, by using the above formula (1) The projected image data can be converted into image data corresponding to the screen of the monitor 3 (showing a bird's eye view image) and displayed on the monitor.

Next, a procedure for moving (for example, rotating) the bird's-eye view image stored in the image memory A according to the amount of movement (movement amount) of the vehicle using the vehicle speed signal and the yaw rate signal will be described. Here, the vehicle position at time T is shown in FIG. 5, but the travel distance of the vehicle up to time T + 1 is L, and the angle the vehicle has moved up to time T + 1 is θ1.
(= Rotation angle θC at the center of rotation).

First, the sampling time of the camera 1 is 1
In the case of 00 [ms], the vehicle speed S [km obtained from the vehicle speed signal
/ H] can be used to calculate the distance L [m] traveled by the vehicle from the following equation (2). L [m] = S [km / h] × 1000 [m / km] / 3600 [S / h] × 100 [ms] ÷ 1000 [ms / s] (2) Also, the figure shown in FIG. Therefore, using the yaw rate θ0 [degrees / s] obtained from the yaw rate signal, the angle θ1 at which the vehicle has moved can be calculated from the following equation (3). θ1 [degree] = θ0 [degree / s] × 100 [ms] ÷ 1000 [ms / s] (3) Further, here, since the center of rotation of the vehicle is on the extension line of the rear wheel axle, , The rotation center is calculated with the camera position as the origin, the traveling direction of the vehicle is Y, and the direction perpendicular to this is X, the rotation center position XC [m] in the X direction is
It can be calculated from the following formula (4). XC [m] = L [m] ÷ TAN (θ1) (4) On the other hand, since the center position in the Y direction is on the rear wheel axle, the distance between the camera 1 and the rear wheel axle is YC [cm]. ].

The rotation angle θC at the center of rotation is
From the relationship of the figures shown in FIG. 5, it matches the angle θ1 at which the vehicle has moved. -Next, the rotation center position (XC,
YC) and the rotation angle θC are used to move the bird's-eye view image as follows.

The image data of the bird's-eye view image stored in the image memory A is once stored in the image memory B as well. still,
The monitor 3 using the image data stored in the image memory A
When it is displayed at, it becomes as shown in FIG. Therefore, here, the image data of the bird's-eye view image stored in the image memory B is transferred to the internal memory of the CPU 13, and the bird's-eye view image is rotationally moved using the rotation center position (XC, YC) and the rotation angle θC. .

The conversion formula for this rotational movement is shown in the following formula (5). The internal processing of the CPU 13 using this conversion formula causes the coordinates (XB, XB, X) after the movement at the rotation angle θ. YB), the corresponding point (XA,
YA) and calculate the pixel value of (XA, YA) as (XB, Y
Save in B) address memory. XA = XB / COSθ + YB / SINθ (5) YA = YB / COSθ-XB / SINθ However, (XA, YB): coordinates before movement (XB, YB): coordinates after movement θ: rotation angle, and , The bird's-eye view image stored in the image memory A (at time T
From the first bird's-eye view image in (1) to a post-movement bird's-eye view image (post-movement bird's-eye view image at time T).

At the same time, the bird's-eye view image created based on the image newly captured by the camera 1, that is, the current bird's-eye view image (the second bird's-eye view image at time T + 1),
It is newly transferred from the image memory A to the CPU 13. Therefore, the CPU 13 writes the second bird's-eye view image at a position having a positional relationship with the time T (a position where the image does not deviate from the post-movement bird's-eye view image), whereby the post-movement bird's-eye view image and the second bird's-eye view image are written. The image data of the synthetic bird's-eye view image is synthesized, and the image data is overwritten and stored in the image memory B.

At this stage, when the image data of the combined bird's-eye view image stored in the image memory B is used for display on the monitor 3, for example, after FIG. 6 (b), it becomes as shown in FIG. 6 (c). Next, a procedure of reverse bird's-eye view conversion for converting a synthetic bird's-eye view image into a projection image will be described.

This reverse bird's-eye view conversion is basically the reverse of the above-mentioned processing for converting the image photographed by the camera 1 into a bird's-eye view image. Here, as shown in FIG. 7, a virtual camera (virtual camera) 1 in which a composite bird's-eye view image can be seen.
The position of k is set, and an image as taken by the virtual camera 1k is displayed on the monitor 3.

For example, assuming that the height Hk of the virtual camera 1k, the camera depression angle τk, and the focal length fk, and the position of the virtual camera 1k in the front-back direction is Zk ahead of the actual position of the camera 1 (above the front bumper). The relationship between the real space coordinates (bird's eye view coordinates) (x, y) and the virtual camera image coordinates (Xk, Yk) is expressed by the following expression (6) which is a modification of the above expression (1). x = Xk · (y · SINπk + Hk · COSπk) / fk (6) y = Hk · (fk · SINπk−y · SINπk) / (fk · COSπk + Y · SINπk) As a result, the virtual camera image coordinates (Xk, The image data of Yk) (that is, the image data of the projected image) is obtained by substituting the image data of the real space coordinates (bird's-eye view coordinates) (x, y) (that is, the image data of the synthetic bird's-eye view image) into the above formula (6). Will be obtained by.

Therefore, the composite bird's-eye view image is converted into a projection image in this way, and the image data is stored in the image memory C. Then, the projection image based on the image data stored in the image memory C is displayed on the monitor 3.

For example, as shown in the parking start drawing of FIG. 7, the actual image of the camera 1 is only the image within the upper triangle of the same figure, but like the virtual camera 1k, the view point is on the front side of the vehicle. An image around the vehicle can also be displayed by moving the. When the projection image is created, the position of the vehicle and the image of the viewing angle of the camera 1 may be added and stored in the image memory C, and the projection image may be displayed on the screen of the monitor 3. When the projection image stored in the image memory C is drawn on the monitor 3 without storing the image of the position of the own vehicle or the viewing angle of the camera 1 in the image memory C, the position of the own vehicle and the field of view of the camera 1 are drawn. It may be drawn in addition to a corner image.

C) Next, the processing performed by the image processing unit 9 and the like will be described with reference to the flowchart of FIG. As shown in FIG. 8, first, at step 100, it is determined whether or not the shift position is the back position. If an affirmative judgment is made here, the routine proceeds to step 110, while if a negative judgment is made, the routine proceeds to step 210.

The shift position can be detected by a signal from a shift position detection sensor (not shown) or a signal from another electronic control unit. In step 210,
Erase the image of the area outside the field of view of the camera drawn previously,
That is, only the current camera image (bird's eye view image) is displayed, and the process returns to step 100.

On the other hand, in step 110, coordinate conversion of the image photographed by the camera 1 (the bird's-eye view conversion) is performed to create a bird's-eye view image. In the following step 120, the bird's-eye view image is stored in the image memory A. At this time, a similar bird's-eye view image is also stored in the image memory B.

In the following step 130, the moving distance L and the rotation angle θC are calculated based on the vehicle speed signal and the yaw rate signal.
The movement amount (movement amount) of the vehicle indicated by is calculated. In the following step 140, the bird's-eye view image (first bird's-eye view image at time T) stored in the image memory B is moved based on the amount of movement of the vehicle, and the bird's-eye view image after movement (the bird's-eye view after movement at time T) is moved. Image).

In the following step 150, the second bird's-eye view image (at time T + 1) newly captured and stored in the image memory A is read from the image memory A. In the following step 160, the moved bird's-eye view image and the second bird's-eye view image are combined to create a combined bird's-eye view image.

Specifically, all the images in the image memory A are overwritten at the corresponding coordinate positions on the moved bird's-eye view image. In the following step 170, the composite bird's-eye view image is stored in the image memory B. In the following step 180, a reverse bird's-eye view conversion is performed on the synthetic bird's-eye view image to create a projection image.

In the following step 190, the projected image is stored in the image memory C. In the following step 200, the projection image stored in the image memory C is displayed on the monitor 3 in consideration of the image showing the vehicle, the image showing the viewing angle, and the like, and the present processing is temporarily ended.

Thereafter, similar processing is performed every time an image newly captured by the camera 1 is captured. That is, next time, the combined bird's-eye view image is moved according to the movement of the vehicle to create the moved bird's-eye view image, the moved bird's-eye view image and the new second bird's-eye view image are combined, and the combined bird's-eye view image is projected. An image is created and displayed on the monitor 3, and thereafter, similar processing is repeated.

D) As described above, in this embodiment, the image photographed by the camera 1 is converted into a bird's-eye view image, the moving amount of the vehicle is detected based on the vehicle signal, and the bird's-eye view image is detected according to the moving amount of the vehicle. Is moved to create a bird's-eye view image after movement. Then, the moved bird's-eye view image and the newly read bird's-eye view image are combined to create a combined bird's-eye view image, and the combined bird's-eye view image is converted into a projection image and displayed on the screen of the monitor 3.

That is, in this embodiment, the synthetic bird's-eye view image is not displayed on the monitor 3 as it is, but the synthetic bird's-eye view image is converted into a reverse bird's-eye view image into a projected image as if it was taken by the camera 1, and the image is taken by the camera 1. Monitor 3
It is displayed in. As a result, the image that is normally used by the driver is displayed on the monitor 3, which is advantageous in that it is not difficult to see.

Further, in this embodiment, since the images are combined using the planar bird's-eye view image and the combined bird's-eye view image is changed to the original projection image, the load of image processing and the load of memory are reduced. The advantage is that Further, in the present embodiment, the distance is known by using the planar bird's-eye view image, and the bird's-eye view image synthesized from this is changed to the original projection image, so that the stitching can be easily performed without using a distance measuring sensor or the like. There is an advantage that a projected image can be created.

Moreover, in the present embodiment, the viewpoint at the time of creating the projected image is moved not to the position of the camera 1 but to the virtual position above the front end side of the vehicle, so that, for example, when the vehicle is parked. Operation becomes easy. For example, when the camera 1 is arranged at the rear end of the vehicle, when the vehicle is backed up in the parking lot and approaches the wall, the image captured by the camera 1 is displayed. As shown in, monitor 3
Most of the images of the walls are displayed on the screen, and even when this composite bird's-eye view image is displayed, the walls and the ground are displayed superficially as shown in FIG. is not.

Therefore, when the viewpoint is shifted to a virtual position with respect to this composite bird's-eye view image to perform a reverse bird's-eye view conversion to obtain a projection image, and the projection image is displayed on the monitor 3, FIG.
As shown in, it is extremely easy to see and has no discomfort. Needless to say, the present invention is not limited to the above-mentioned embodiments, and can be carried out in various modes without departing from the present invention.

(1) For example, the vehicle peripheral image processing apparatus has been described in the above embodiments, but a recording medium storing means for executing processing by this apparatus is also within the scope of the present invention. Examples of this recording medium include various recording media such as an electronic control unit configured as a microcomputer, a microchip, a floppy disk, a hard disk, and an optical disk.

That is, there is no particular limitation as long as it stores means such as a program that can execute the processing of the above-described vehicle peripheral image processing apparatus. (2) Also, the virtual position of the viewpoint (the position of the virtual camera)
May be a fixed position set in advance, but may be arbitrarily changed by the user. As a result, it is possible to display the image that is most easy for each user to see.

(3) Further, as the image displayed on the monitor, the composite bird's-eye view image and the projection image may be arbitrarily changed by the user. As a result, it is possible to display the image that is most easy for each user to see. (4) In addition, in the above-described embodiment, the moving state of the vehicle is detected by using the vehicle speed sensor and the yaw rate sensor, and the post-movement bird's-eye view image is created based on the information. The motion vector of (bird's eye view image) may be obtained, the moving state of the vehicle may be estimated from this motion vector, and the bird's eye view image after movement may be obtained.

For example, the moving state of the first bird's-eye view image and the second bird's-eye view image after that may be detected by so-called image matching processing, and the moving state of the vehicle may be obtained from the moving state of the image. .

[Brief description of drawings]

FIG. 1 is an explanatory diagram illustrating a main configuration of a vehicle peripheral image processing device according to an embodiment.

FIG. 2 is an explanatory diagram showing an electrical configuration of a vehicle peripheral image processing device of the embodiment.

3A and 3B show images used by the vehicle peripheral image processing apparatus of the embodiment, FIG. 3A is an explanatory view of an image captured by a camera, and FIG. 3B is an explanatory view showing a bird's-eye view image.

FIG. 4 is an explanatory diagram showing a positional relationship at the time of coordinate conversion by the vehicle peripheral image processing device of the embodiment.

FIG. 5 is an explanatory diagram showing a method for obtaining the movement amount of the vehicle by the vehicle peripheral image processing device of the embodiment.

FIG. 6 is an explanatory diagram showing a procedure of image creation by the vehicle periphery image processing apparatus of the embodiment.

7A and 7B show a method of creating an image by the vehicle peripheral image processing apparatus of the embodiment, FIG. 7A is an explanatory diagram showing a position of a camera, FIG. 7B is an explanatory diagram showing an image at the start of parking, and FIG. ) Is an explanatory view showing an image during parking, and (d) is an explanatory view showing an image after parking.

FIG. 8 is a flowchart showing the processing of the vehicle peripheral image processing device of the embodiment.

FIG. 9 is an explanatory diagram showing an image creating procedure performed by the vehicle periphery image processing apparatus according to the embodiment.

[Explanation of symbols]

1 ... Camera 3 ... In-vehicle monitor (monitor) 5 ... Vehicle speed sensor 7 ... Yaw rate sensor 9 ... Image processing unit

Front page continuation (51) Int.Cl. ⁷ Identification code FI theme code (reference) G06T 1/00 330 G06T 1/00 330A 3/00 100 3/00 100 (72) Inventor Hirohiko Yanagawa Showa Kariya, Aichi Prefecture 1-Chome 1-Densha Stock Company Denso (72) Inventor Tetsuri Ishikawa 1-1-Showa-cho Kariya City, Aichi F-Term inside DENSO Stock Company 5B057 AA16 BA02 CA12 CA16 CC01 CD20 CE08 5C054 AA01 FD01 FE01 FE12 GB12 GB15 HA30

Claims (8)

[Claims] 1. A photographing means for photographing an image of a periphery of a vehicle,
A vehicle peripheral image processing device for processing an image captured by the image capturing unit and displaying the image on the display unit for a vehicle including a display unit displaying an image, based on the image captured by the image capturing unit. A bird's-eye view image storage unit for storing the first bird's-eye view image created by the above; and a moved bird's-eye view image creation unit for moving the first bird's-eye view image according to the moving state of the vehicle to create a moved bird's-eye view image. , A second image created after the first bird's-eye view image
A bird's-eye view image and the moved bird's-eye view image are combined to create a composite bird's-eye view image creating means; and an image converting means for converting the composite bird's-eye view image into a projection image from a predetermined viewpoint. A vehicle peripheral image processing apparatus, wherein the projection image is displayed on the display means. 2. The projection image is obtained by moving the viewpoint to a virtual position different from a photographing position of the photographing means, and using the virtual position as the viewpoint.
The image processing apparatus for a vehicle periphery according to. 3. The vehicle peripheral image processing apparatus according to claim 1, wherein the virtual position of the viewpoint can be changed by a user. 4. The vehicle periphery according to claim 1, wherein the image displayed on the display means can be changed to the composite bird's-eye view image and the projection image by a user. Image processing device. 5. The vehicle peripheral image processing according to claim 1, wherein when the projection image is displayed on the display means, an image showing the own vehicle is additionally displayed. apparatus. 6. The vehicle peripheral image processing device according to claim 1, wherein a synthetic bird's-eye view image created prior to the present synthetic bird's-eye view image is used as the first bird's-eye view image. 7. The composite bird's-eye view image is an image to which the post-movement bird's-eye view image is added, in addition to the display area for displaying the second bird's-eye view image.
6. The vehicle peripheral image processing device according to any one of 6. 8. A recording medium storing means for executing processing by the vehicle peripheral image processing device according to claim 1.