WO2018048167A1 - Method for displaying image in multi-view mode - Google Patents

Abstract

Disclosed is a method for displaying an image in a multi-view mode. A method according to the present embodiment comprises the steps of: adjusting a light exposure time of an image acquisition unit by using a target image converted to be output in a desired output mode among a plurality of output modes; and displaying target images acquired according to the adjustment. Therefore, according to the present invention, an image corrected to be suitable for a specific view mode can be provided to a user.

Description

Image display method in multi view mode

The present invention relates to an image display method in a multi-view mode, and more particularly, to an image obtained by at least one image acquisition unit for around view monitoring (AVM) and an around view generated using the same. view) relates to a method of displaying an image in a multi-view mode.

As the increase of automobiles and traffic volume has caused a sudden increase in traffic accidents, researches on driver assistance systems that can prevent accidents are being actively conducted. The driver assistance system is a system that assists the driver while driving or parking for the driver's safe driving.

The driver assistance system essentially includes a device that provides image information so that the driver can grasp the situation outside the vehicle while sitting in the driver's seat. The apparatus for providing image information includes a camera.

In order to provide the image information to the driver, a plurality of cameras facing in various directions may be installed outside the vehicle. In addition, since the image acquired from each camera is provided to the driver, the acquired images may be differently converted according to various view modes and provided to the driver.

However, according to the existing technology, the provided images tend to be corrected based on the image according to the view mode displayed on the display screen which the driver can visually confirm. This trend can also be seen in images for around view monitoring (AVM) systems.

The around view monitoring system is being actively researched recently to provide the driver with a video of the situation around the vehicle equipped with multiple cameras. Several automotive companies in Germany and Japan have already developed and released products. In recent years, systems that have multiple cameras to give the driver a bird's eye view like the view from the sky have become mainstream.

The around view monitoring system may generate an image representing an object, for example, a 360-degree field of view around an object, for example, a vehicle, by using images acquired from a limited number of cameras through a device for providing image information. In this case, the lens mounted on the camera may be a fisheye lens or a similar type of wide-angle lens to obtain a wide viewing angle. However, the images acquired by the lens are different from those based on human vision, and the images to be finally output compared with the direction of the camera lens installed in the vehicle are top view images. Images acquired from the cameras must go through various image signal processing.

This series of processes is necessary because cameras cannot be installed in the car so that the roof of the car is photographed in front.

1 and 2 show an image according to a conventional technology.

1 and 2, the image of FIG. 1 represents an image acquired by a camera installed on a right side of a vehicle, and the image of FIG. 2 represents an around view image including the left image. In the image of FIG. 1, the ratio of the dark area due to the shadow of the vehicle and the bright area due to the lighting is different from the around view image, so that the brightness of the bright area due to the illumination is saturated in the image of FIG. This is a result of adjusting light exposure or adjusting white balance based on the image of FIG. 1.

An object of the present invention for solving the above problems is to provide a method for displaying a corrected image in the corresponding view mode by properly correcting the image displayed in the specific view mode in the corresponding view mode.

In an image processing method according to an embodiment of the present invention, a method of displaying a target image according to a plurality of modes of an image acquired from at least one camera, wherein the output image is changed from the first mode to the second mode. Changing and outputting from the first target image of the first mode to the second target image of the second mode; And adjusting the brightness of the second target image based on the second target image.

According to another aspect of the present invention, an image processing method supporting multiple modes determines whether to convert or match at least one received image in response to a user input, and receives the at least one received according to the conversion or matching. Outputting a target image by converting or matching an image; Extracting brightness information from the target image; And applying an adjustment value corresponding to the brightness information to convert or match the at least one image.

The image processing method supporting multiple modes may further include adjusting an exposure time of at least one camera device based on the target image, wherein the adjustment value may vary according to the brightness information and the exposure time. .

In addition, the image processing method supporting the multi mode may further include receiving the at least one image data obtained from each of the at least one camera device.

In addition, when the at least one image data is received in a Bayer pattern, an image processing method supporting multiple modes may further include outputting an image image by performing color interpolation and first image processing on the Bayer pattern. have.

In addition, the first image processing includes calibration, lens distortion correction, color correction, gamma correction, color space conversion, and edge enhancement. It may include performing at least one of (Enhancement).

The outputting of the target image may include selecting at least one of the at least one image corresponding to whether to convert or match the target image; Generating a transformed image obtained by removing perspective from the selected image; Extracting from the converted image corresponding to an area to be inserted into the target image; Placing the extracted data in the target image; And transferring the target image to a display device.

In addition, the transform image may be obtained by performing an inverse perspective mapping transform.

In addition, the reverse projection image conversion and placement on the target image may be performed together through a lookup table.

The applying of the at least one image to conversion or matching may include batch converting at least some data arranged in the lookup table.

The at least one image may be transmitted from at least one camera device mounted in a vehicle, and the user input may be input through an interface mounted in the vehicle.

The at least one image may be image information of at least one of the front, rear, left, and right sides of the vehicle, and the user input may include a top view, a front view, a rear view, a left side view, and a right side view. And selecting at least one of the combination thereof.

The apparatus may include at least one lookup table that is distinguished in correspondence with each of the image information, and the at least one lookup table may include a weight corresponding to the image information.

The computer readable recording medium according to another embodiment of the present invention may be recorded by an application program, which is implemented by a processor to realize the image processing method supporting the above-described multiple modes.

An image conversion or registration device according to another embodiment of the present invention includes a processing system including at least one processor and at least one memory device in which a computer program is stored, wherein the image conversion or registration device is a user. Determining whether to convert or match at least one image in response to an input of a, and outputting a target image by converting or matching the at least one image according to the conversion or matching; Extracting brightness information from the target image; And applying an adjustment value corresponding to the brightness information to convert or match the at least one image.

Further, the processing system further causes the image conversion or matching device to adjust the exposure time of the at least one camera device based on the target image, wherein the adjustment value is in accordance with the brightness information and the exposure time. It can support changing, multiple modes.

In addition, the processing system may cause the image conversion or matching device to perform the step of receiving the at least one image data obtained from each of the at least one camera device.

In addition, when the at least one image data is received in a Bayer pattern, the processing system further performs the step of outputting the image image by performing color interpolation and first image processing on the Bayer pattern by the image conversion or matching device. You can do that.

In addition, the first image processing includes calibration, lens distortion correction, color correction, gamma correction, color space conversion, and edge enhancement. Edge Enhancement) may be performed.

The outputting of the target image may include selecting at least one of the at least one image corresponding to whether to convert or match the target image; Generating a transformed image obtained by removing perspective from the selected image; Extracting from the converted image corresponding to an area to be inserted into the target image; Placing the extracted data in the target image; And transferring the target image to a display device.

In addition, the transform image may be obtained by performing an inverse perspective mapping transform.

In addition, the reverse projection image conversion and placement on the target image may be performed together through a lookup table.

The applying of the at least one image to conversion or matching may include batch converting at least some data arranged in the lookup table.

The at least one image may be transmitted from at least one camera device mounted in a vehicle, and the user input may be input through an interface mounted in the vehicle.

The at least one image may be image information of at least one of the front, rear, left, and right sides of the vehicle, and the user input may include a top view, a front view, a rear view, a left side view, and a right side view. And selecting at least one of the combination thereof.

The apparatus may include at least one lookup table that is distinguished in correspondence with each of the image information, and the at least one lookup table may include a weight corresponding to the image information.

In accordance with still another aspect of the present invention, an image processing apparatus includes: a conversion or matching unit configured to output a target image by converting or matching at least one image in response to a user input; A brightness controller which receives brightness information of the target image and outputs an adjustment value for updating a look-up table in a conversion or matching unit; And an adjusting unit configured to receive the brightness information and output a control signal for adjusting an exposure time of at least one camera device.

In addition, the image processing apparatus may include color interpolation, demosaicing, calibration, lens distortion correction, color correction, and gamma correction in a Bayer pattern transmitted from at least one camera device. The apparatus may further include an image processor configured to selectively output the at least one image by performing operations such as gamma correction, color space conversion, and edge enhancement.

The control unit may transmit a change amount or a change rate generated in adjusting the exposure time of the at least one camera device to the brightness controller, and the brightness controller determines the adjustment value based on the change rate and the brightness information. Can be.

The at least one image may be image information of at least one of the front, rear, left, and right sides of the vehicle, and the user input may include a top view, a front view, a rear view, a left side view, and a right side view. And selecting at least one of the combination thereof.

According to still another aspect of the present invention, there is provided a method of displaying an image in a multiple view mode, wherein the plurality of outputs display target images acquired by at least one image acquisition unit in a plurality of output modes. Adjusting brightness of the target image converted to be output in a desired output mode among the modes; And displaying the target image in which the white balance is adjusted.

According to still another aspect of the present invention, there is provided a method of displaying an image in a multiple view mode, wherein the plurality of outputs display target images acquired by at least one image acquisition unit in a plurality of output modes. Adjusting the light exposure time of the image acquisition unit by using the target image converted to be output in a desired output mode among modes; And displaying the target images acquired according to the adjustment.

According to still another aspect of the present invention, there is provided a method of displaying an image in a multiple view mode, wherein the plurality of outputs display target images acquired by at least one image acquisition unit in a plurality of output modes. Adjusting a white balance of the target image converted to be output in a desired output mode among the modes; And displaying the target image in which the white balance is adjusted.

The above aspects of the present invention are only some of the preferred embodiments of the present invention, and various embodiments in which the technical features of the present invention are reflected will be described in detail below by those skilled in the art. Can be derived and understood.

The effects on the apparatus according to the present invention are described as follows.

According to the present invention, an image corrected to be suitable for a specific view mode can be provided to a user.

In addition, the present invention can reduce the computation, time required to convert, process the image transmitted from the at least one camera device in response to the user input, and can reduce the degradation of the image image.

Effects obtained in the present invention are not limited to the above-mentioned effects, and other effects not mentioned above may be clearly understood by those skilled in the art from the following description.

1 and 2 show an image according to a conventional technology.

3 illustrates an image processing method in a multiple view mode.

4 illustrates a first example of an image processing system in a multiple view mode.

5 illustrates a second example of an image processing system in a multiple view mode.

6 is a schematic block diagram of an image display apparatus 100 that executes an image display method in a multiple view mode according to an embodiment of the present invention.

7 is a front view (a), a right side view (b), a rear view (c), and a left side view (d) in the actual view mode as images displayed by the image display apparatus 100.

FIG. 8 is a front view a converted from an image acquired by the image acquisition unit 110 to an image to be displayed in a top view mode.

FIG. 9 is an around view image generated by using the four images of FIG. 8.

10 is a front view of a vehicle for explaining the difference between the actual view mode and the top view mode.

11 is a flowchart illustrating an image display method in a multiple view mode according to another exemplary embodiment of the present invention.

12 is a flowchart illustrating an image display method in a multiple view mode according to another exemplary embodiment of the present invention.

13 is a flowchart illustrating a method of displaying an image in a multiple view mode according to another embodiment of the present invention.

14 is a flowchart illustrating a method of displaying an image in a multiple view mode according to another embodiment of the present invention.

As the present invention allows for various changes and numerous embodiments, particular embodiments will be illustrated in the drawings and described in detail in the written description. However, this is not intended to limit the present invention to specific embodiments, it should be understood to include all modifications, equivalents, and substitutes included in the spirit and scope of the present invention.

Terms such as first and second may be used to describe various components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be referred to as the second component, and similarly, the second component may also be referred to as the first component. The term and / or includes a combination of a plurality of related items or any item of a plurality of related items.

When a component is referred to as being "connected" or "connected" to another component, it may be directly connected to or connected to that other component, but it may be understood that other components may be present in between. Should be. On the other hand, when a component is said to be "directly connected" or "directly connected" to another component, it should be understood that there is no other component in between.

The terminology used herein is for the purpose of describing particular example embodiments only and is not intended to be limiting of the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this application, the terms "comprise" or "have" are intended to indicate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, and one or more other features. It is to be understood that the present invention does not exclude the possibility of the presence or the addition of numbers, steps, operations, components, components, or a combination thereof.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art. Terms such as those defined in the commonly used dictionaries should be construed as having meanings consistent with the meanings in the context of the related art and shall not be construed in ideal or excessively formal meanings unless expressly defined in this application. Do not.

Hereinafter, with reference to the accompanying drawings, it will be described in detail a preferred embodiment of the present invention. In the following description of the present invention, the same reference numerals are used for the same elements in the drawings and redundant descriptions of the same elements will be omitted.

3 illustrates an image processing method in a multiple view mode.

Referring to FIG. 3, the image processing method in the multi-view mode may include determining whether to convert or match at least one image in response to a user input, and determine brightness information in an extraction region of at least one image. Collecting and determining a correction criterion 14, and converting or matching at least one image according to the correction criterion 15.

Also, the image processing method in the multi-view mode includes adjusting the exposure time of the at least one camera device based on the target image (16) and images corresponding to the adjusted exposure time and the extracted brightness information. May further comprise applying 18 to transform or match. In this case, the correction criteria may change in correspondence with the brightness information and the exposure time. The at least one camera whose exposure time is adjusted may be a corresponding camera that captures an image disposed on the target image. For example, when the target image is an image captured by one first camera, the exposure time of the first camera may be adjusted. Alternatively, when the target image is made of at least two cameras including the first camera and the second camera, the exposure time of at least two cameras including the first camera and the second camera may be adjusted. The exposure time of each camera of a single camera or at least one camera may be changed in consideration of an area output from the target image in an image photographed by each camera of the single camera or at least one camera in the target image. In particular, when there is a change in the target image output according to the mode change, the part constituting the target image among the images photographed by each camera may be changed, and the exposure time may be adjusted in consideration of this.

In addition, the image processing method in the multi-view mode may further include the step 10 of receiving at least one image data obtained from each of the at least one camera device. For example, at least one camera device may include a camera mounted on a vehicle. Imaging equipment capable of capturing at least one of the front, rear, right, and left sides of the vehicle, respectively, may acquire surrounding information of the vehicle. According to an embodiment, the at least one camera device may output image data composed of a Bayer pattern.

In general, a camera apparatus includes an image sensor for converting an optical signal incident through a path formed through at least one lens into an electrical signal. The image sensor is composed of pixels arranged in a pattern for each color. For example, in an image sensor, R, G, and B color filters are arranged in a specific pattern on monochrome pixel cells arranged by the number of pixels. The R, G, and B color patterns intersect and are arranged according to the visual characteristics of the user (ie, human), which is called a Bayer pattern. Each pattern is a monochrome pixel that only detects the brightness of black and white, not the color. When outputting data having such a pattern, it is composed of several colors through color interpolation or demosaicing process. The image may be changed into the form of the image.

The Bayer pattern has a much smaller amount of data than the image data. Therefore, the size of the data transmitted by the in-vehicle communication network can be reduced, so that even when applied to an autonomous vehicle, the peripheral information obtained from at least one camera device disposed in the vehicle is analyzed. Can be removed.

If at least one image data is received in a Bayer pattern, the image processing method in the multi-view mode may further include outputting an image image by performing color interpolation and first image processing on the Bayer pattern. Here, the first image processing may include performing at least one of color correction, gamma correction, color space conversion, and edge enhancement.

According to an exemplary embodiment, each camera device mounted on the vehicle may output data in the form of an image after performing color interpolation and first image processing on the Bayer pattern output by the image sensor. In this case, the image processing apparatus does not need to perform operations for color interpolation and first image processing.

Although not shown, the step of outputting the target image by converting or matching at least one image 14 may include selecting at least one of the at least one image according to whether the image is converted or matched, calibration, and lens distortion correction. (lens distortion correction), or generating a transformed image from which the perspective image is removed from the selected image, extracting corresponding to the area to be inserted into the target image from the converted image, placing the extracted data in the target image, and Delivering the image to a display device.

In order to convert an image acquired from a camera module mounted on a vehicle into a top view image, it is necessary to remove perspective effects on objects and objects in the image. In the process of converting or matching at least one image, a converted image may be generated by removing perspective from an image obtained through a camera device mounted on a vehicle. For example, such a transform image can be obtained by performing an inverse perspective mapping transform.

The process of converting the images and the process of converting or matching at least two images, that is, reverse projection image conversion and placement on the target image, can be done together through a lookup table. If the vehicle has height and angle with the camera installed, and horizontal and vertical angle of view information of the camera, the relationship between the image plane acquired through the camera and the actual plane (top view target image plane) to be shown to the driver or user can be known. In addition, since the camera device mounted on the vehicle is fixed, when converting or matching the image acquired by the at least one camera device, an area to be converted or matched may be set in advance. Therefore, by arranging such information in the form of a lookup table, it is possible to shorten the calculation process and time for conversion and conversion or matching.

Meanwhile, the step of applying 18 to converting or matching at least one image may include batch converting at least some data arranged in the lookup table. Each image to be converted or matched is obtained from a camera disposed at different positions, and photographs of objects in different directions. Therefore, the amount of light is different for each image. As described above, when each camera individually adjusts brightness to adjust brightness, image information that the image processing apparatus may receive may inevitably vary, and computation amount for correcting the brightness may not decrease. Therefore, by gathering brightness information from the converted or matched image and adjusting the exposure time of the camera device based on it, the data in the lookup table used when converting or matching at least one image can be adjusted to provide better quality. The resulting image can be obtained.

As described above, at least one image may be transmitted from at least one camera device mounted in a vehicle, and an input of a user or a driver may be input through an interface mounted in the vehicle. For example, a user or driver, such as a head unit of a vehicle or a multimedia device (Audio-video-navigation), may select information of a direction or space desired by the user or the driver through an interface that can be manipulated while driving.

For example, the at least one image is image information of at least one of the front, rear, left and right sides of the vehicle, and the user input is a top view, front view, rear view, left side view, and right side view. And a combination thereof. For example, when a user wants a top view and a front view, the image processing apparatus may output image information of at least one of front, rear, left, and right rooms to output an image image of the top view. May be converted or matched, and the front view may output information of the front camera. In this case, the top view and the front view may have different brightness because the information shown to the user is different, and the image constituting the information is different.

Also, an area extracted for the target image output from the image information acquired by each camera in response to a user input may vary. For example, when the user wants the front view, a large portion of the image information obtained from the camera collecting the front information can be shown to the user. However, when the user wants the top view, a relatively small portion of the image information obtained from the camera collecting the front information may be shown to the user. That is, according to which image the user wants to check, it is different from which camera the image obtained is used, or among the images acquired by a specific camera, the area extracted for display to the user is different. Therefore, when the image is converted or matched in consideration of the image brightness of the region actually provided to the user, that is, the extraction region extracted for conversion or matching in each of the at least one image, a higher quality image may be provided to the user. .

Meanwhile, when the exposure time of the at least one camera is controlled using brightness information such as a top view image in which at least one image is converted or matched, the brightness difference according to the multiple views may be reduced.

The image processing apparatus may include at least one lookup table that is distinguished corresponding to each of the image information, and the at least one lookup table may include a weight corresponding to the image information. This may be necessary since the amount of light may be different for each image since the camera apparatus photographs an object in different directions.

4 illustrates a first example of an image processing system in a multiple view mode.

Referring to FIG. 4, the image processing system may include a camera device 30, an image processing device 40, and a vehicle multimedia device 20.

The camera device 30 outputs a Bayer pattern BP by converting an optical signal collected through the lens assembly 32 and the lens assembly 32 including at least one lens to collect the incoming optical signal into an electrical signal. It may include an image sensor 34 to. The camera device 30 may transfer the Bayer pattern BP output from the image sensor 34 to the image processing device 40 without performing color interpolation, image correction, or the like.

The image processing apparatus 40 may include color interpolation, demosaicing, color correction, gamma correction, and color space conversion into a Bayer pattern BP transmitted from the camera apparatus 30. a target image by converting or matching the image processor 42 and the at least one image CI output from the image processor 42, which may selectively perform operations such as space conversion, edge enhancement, and the like. A transform or matching unit 44 that outputs an OI, and receives image information (eg, brightness information BI) of the target image OI output from the transforming or matching unit 44. ) Image information (eg, brightness information BI) of the target image OI output from the variable for updating the table in the table, the brightness control unit 48 outputting the adjustment value LC0, and the conversion or matching unit 44. To adjust the exposure time of the at least one camera device 30 It may include an adjusting unit (46) for outputting a control signal (ETC).

The adjuster 46 may transmit the change amount or the change rate ETR generated in adjusting the exposure time of the at least one camera device 30 to the brightness controller 48. The brightness controller 148 may determine the adjustment value LC0 based on the change rate ETR and the image information (eg, the brightness information BI) of the target image OI.

On the other hand, the conversion or matching unit 44 through the user interface 22 of the in-vehicle multimedia device 20, the desired view mode (eg, top view, front view, rear view, left room view, right room view and The mode control signal VC for at least one of the combinations thereof may be received. The conversion or matching unit 44 may select and convert or match one or more image images that need to be converted or matched according to a user's input.

The conversion or matching unit 44 may perform at least one of calibration, lens distortion correction, and at least one operation for generating a converted image from which perspective is removed from the image. For example, in order to convert an image obtained from a camera module mounted on a vehicle into a top view image, it is necessary to remove perspective effects on objects and objects in the image. If the vehicle has height and angle with the camera installed, and horizontal and vertical angle of view information of the camera, the relationship between the image plane acquired through the camera and the actual plane (top view target image plane) to be shown to the driver or user can be known. Using this relationship, the image plane obtained from the camera can be converted into a plane to be shown to the user.

In addition, the vehicle multimedia apparatus 20 may include a display device 24 capable of displaying a target image OI transmitted from the conversion or matching unit 44 to a user or a driver.

5 illustrates a second example of an image processing system in a multiple view mode.

Referring to FIG. 5, the image processing system may include a camera apparatus 50, an image processing apparatus 60, and a vehicle multimedia apparatus 20. The image processing system described with reference to FIG. 5 may be similar to the image processing system described with reference to FIG. 4. The following description will focus on the differences.

The camera device 50 converts the optical signal collected through the lens assembly 32 and the lens assembly 32 including at least one lens to collect the incoming optical signal into an electrical signal, and outputs a Bayer pattern BP. The Bayer pattern BP output from the image sensor 34 and the image sensor 34 may output the image image II through color interpolation and image correction.

The image processing apparatus 60 that receives at least one video image I-I may include a conversion or matching unit 44, a brightness controller 48, and an adjusting unit 46. Here, the conversion or matching unit 44, the brightness control unit 48, and the adjustment unit 46 may operate similarly to that described with reference to FIG. 3.

In addition, the in-vehicle multimedia device 20 may include a user interface 24 and a display 22 and operate similarly to that described in FIG. 3.

6 is a schematic block diagram of an image display apparatus 100 that executes an image display method in a multiple view mode according to an embodiment of the present invention.

Referring to FIG. 6, the image display apparatus includes an image acquisition unit 110, an image conversion unit 120, a control unit 130, an image correction unit 140, and an output unit 150, and further includes a memory 160. It may further include.

The image acquisition unit 110 acquires a digital image using an image sensor. Here, the digital image obtained for display through the output unit 150 will be referred to as a target image. The image acquisition unit 110 may be implemented in the form of a camera module that is installed in the form of a component in a camera or an arbitrary device that is installed independently to photograph the subject. In particular, the image acquisition unit 110 is preferably installed on the outer surface of the vehicle to capture a view including the ground on which the vehicle is located in the direction of the vehicle, the right direction, the reverse direction, and the left direction in the form of a camera module. Do.

The image converting unit 120 receives the target image acquired by the image obtaining unit 110 and converts the image into a form suitable for the view mode, that is, the view mode. Here, the view mode refers to a mode according to a view point (viewpoint) that changes according to the change of the output mode.

Not all target images acquired by the image acquirer 110 may be converted by the image converter 120. Conversion of the target image refers to conversion of the target image according to the change of the view mode. For example, the image conversion refers to an image acquired by the virtual image acquisition unit 110 installed at a predetermined height on the vehicle with the target image acquired by the image acquisition unit 110 installed at the rear of the vehicle, that is, the top view mode. It means to convert the image from. Therefore, when outputting the target image acquired by the image acquisition unit 110 in the actual view mode, it is not necessary to convert the target image separately. In this case, the actual view mode means that there is no change in the viewpoint. That is, the image acquisition unit 110 at a specific position outputs the acquired target image as it is without conversion.

The controller 130 may control the image acquirer 110 to adjust a parameter indicating the attribute based on the target image acquired by the image acquirer 110 or the target image converted by the image converter 120. Can be controlled. For example, the controller 130 may adjust the degree of light exposure of the camera included in the image converter 110 based on a preset reference average brightness TH using the average brightness AVG of the converted target image. . Here, the reference average brightness refers to the average brightness of the reference image compared with the average brightness of the target image. In addition, the adjustment of the light exposure is to adjust the exposure time to adjust the shutter speed or the aperture of the camera of the image acquisition unit 110. As a result, the charge accumulation time and the gain of the photodiode can be adjusted by the above adjustment.

The image corrector 140 may correct the image acquired by the image acquirer 110 or the target image converted by the image converter 120 according to the corresponding view mode. For example, the image corrector 140 may adjust the white balance of the target image based on a preset average brightness TH using the converted average brightness AVG.

The output unit 150 may output the target image corrected by the image corrector 140 in a corresponding view mode. The output unit 150 may be implemented as an LCD display device. Prior to the output, the controller 130 may change the output mode, that is, the view mode.

The memory 160 may store the average brightness and the reference average brightness of the target image.

The image converter 120, the controller 130, and the image corrector 140 may execute a program command stored in the memory 160. In addition, the image converter 120, the controller 130, and the image corrector 140 may include a central processing unit (CPU), a graphics processing unit (GPU), or methods according to the present invention. It may mean a dedicated processor.

Memory 160 may also be comprised of volatile storage media and / or non-volatile storage media. For example, the memory 160 may be configured as read only memory (ROM) and / or random access memory (RAM).

7 is a front view (a), a right side view (b), a rear view (c), and a left side view (d) in the actual view mode as images displayed by the image display apparatus 100.

Referring to FIG. 7, the front view (a), the right side view (b), the rear view (c), and the left side view of the image displayed by the image display apparatus 100, more specifically, the image acquisition unit 110 are displayed. Images corresponding to (d) are shown respectively. Camera modules corresponding to the image acquisition unit 110 may be installed at a predetermined height on the front, right side, rear side, and right side of the outside of the vehicle. The location where the camera modules are installed may be a radiator grille, right and left indicator lights and vehicle trunk cover of the vehicle. The camera module may be installed such that the lens surface faces the direction of the ground at an angle to the installation position. Therefore, the images shown in FIG. 7 may appear to be longer in the direction than the actual image.

FIG. 8 is a front view a converted from an image acquired by the image acquisition unit 110 to an image to be displayed in a top view mode. Referring to FIG. 8, it can be seen that the image of the front view (a) converted by the image converter 120 is different from the image of the front view (a) before the conversion displayed in the actual view mode of FIG. 7. have. In other words, the image in the top view mode, which is directly viewed from a certain height on the vehicle, is a corrected image close to the actual appearance.

FIG. 9 is an around view image generated by using the four images of FIG. 8.

Referring to FIG. 9, an around view image displayed in the top view mode is shown. As described above, the around view image refers to an image that is captured by a virtual camera installed on a vehicle. The around view image is mainly used to display the surrounding ground, including the roof of the vehicle, marked with parking lines. As shown in FIG. 4, the four target images actually photographed and the around view image generated based on the same are different. In other words, the ratio of the total area of the entire area to the actual captured image may change in the around view mode. In particular, when a target image is acquired using a fisheye lens, the ratio may change even larger. For example, in the drawings of (a) to (d) shown in FIG. 7, the area under the A-2 may appear to occupy a high ratio in the entire area as the area closest to the camera, but in FIG. The ratio is reduced.

This means that the attributes of the actually captured target image may be changed even in the image in the around view mode.

10 is a front view of a vehicle for explaining the difference between the actual view mode and the top view mode.

Referring to FIG. 10, a camera module installed in a vehicle as the image acquisition unit 100 is illustrated. The actual camera modules L-1 and R-1 are camera modules installed in the left or right indicator light. The camera modules L-2 and R-2 are virtual camera modules. Referring to FIG. 10, a difference may occur in the appearance of images acquired by the two camera modules due to the difference in the angle at which the actual camera modules and the virtual camera modules photograph the subject. In order to reduce the difference to a minimum, the target images acquired by the actual camera modules may be converted to match the view mode so that the output image according to the view mode is output close to the actual state.

Then, an image display method in a multiple view mode according to another embodiment of the present invention will be described.

11 is a flowchart illustrating an image display method in a multiple view mode according to another exemplary embodiment of the present invention.

First, a reference value for adjustment is set, and this reference value may be stored in a memory or the like (S110).

Next, the image acquisition unit 110 obtains a target image (S120).

Next, the image converter 120 converts the obtained target image to fit the desired output mode, that is, the view mode (S130).

Next, the target image is corrected (S140). Correction of the target image may be performed in two ways. As a result, the correction of the target image may result in adjusting the brightness of the target image. As one of them, the controller 130 may calculate the average brightness of the target image and adjust the light exposure of the camera of the image acquisition unit 110 based on the reference average brightness using the calculated average brightness of the target image. . As another one, the image corrector 140 may adjust the white balance of the target image based on the reference average brightness using the average brightness of the target image.

Next, the output unit 150 displays the corrected target image (S150).

Prior to S150, the controller 130 may convert the output mode of the output unit 150. Here, the view mode may be changed by changing the view point at which the target image is output according to the change of the output mode. The adjustment of the camera light amount or the white balance of the target image may be performed by the number of different view modes in which the obtained target image is to be displayed.

12 is a flowchart illustrating an image display method in a multiple view mode according to another exemplary embodiment of the present invention.

First, a reference value for adjustment is set, and this reference value may be stored in a memory or the like (S210).

Next, the image acquisition unit 110 obtains a target image (S220).

Alternatively, the image converter 120 converts the obtained target image to fit the desired output mode, that is, the view mode, and in particular, converts the obtained target image into an around view image (S230).

Next, since S240 and S250 are common to the S140 and S150, description thereof will be omitted.

13 is a flowchart illustrating a method of displaying an image in a multiple view mode according to another embodiment of the present invention.

First, the controller 130 converts an output mode, that is, a view mode, for displaying a target image (S310).

Next, the image corrector 140 corrects or reverse corrects the target image to fit the converted view mode (S320). The reverse correction here means returning to the state without the said correction. The image corrector 140 corrects the target image according to the converted view mode according to the conversion of the view mode, and returns the corrected target image to the uncorrected target image when there is no conversion of the view mode. have.

Finally, the output unit 150 displays the corrected or de-corrected target image.

14 is a flowchart illustrating a method of displaying an image in a multiple view mode according to another embodiment of the present invention.

In particular, FIG. 14 illustrates a flowchart of adjusting the light exposure time of the image acquisition unit 110, which may be implemented as a camera module. Referring to FIG. 14, a threshold (TH) value may be set in advance, and the reference value may be stored in advance in a memory or the like (S410).

Next, the target image acquired by the image acquisition unit 110 is input to the control unit 130 (S420).

Next, the controller 130 calculates an average brightness (AVG) of the target image (S430).

When the average brightness and the reference average brightness of the target image are compared, if the average brightness and the reference value are the same (S440), the current exposure time is maintained (S441). In addition, when the average brightness and the reference value is different (S445) and the average brightness and the reference value are compared (S445), if the average brightness is larger than the reference value, the current exposure time can be reduced by adjustment (S442), and the average brightness is If it is not larger than the reference value, the current exposure time may be increased by adjustment (S443).

According to another embodiment of the present invention, an image display method in a multi-view mode may adjust brightness of each pixel constituting the target image by using a weight before converting the acquired target image into an image suitable for the corresponding view mode. have. Here, each pixel has a weight having a fixed weight according to the position of each pixel.

In FIG. 6 of the present invention, for convenience of description, each component is illustrated in another block and described as an example, but each component may be configured as one block. For example, each block may be configured in a controller or a processor to perform the above-described series of operations.

According to the present invention as described above can solve the problems according to the existing technology. That is, by adjusting the brightness of the target image based on the image output in the converted view mode, it is possible to prevent the area in which the brightness shown in FIG. 2 is saturated.

Although it has been described above with reference to the preferred embodiment of the present invention, those skilled in the art will be able to variously modify and change the present invention without departing from the spirit and scope of the invention described in the claims below. It will be appreciated.

Claims (10)

In the method for displaying a target image according to a plurality of modes of the image obtained from at least one camera, Changing the output image from the first mode to the second mode from the first target image of the first mode to the second target image of the second mode; And And adjusting the brightness of the second target image based on the second target image. The method of claim 1, Outputting the second target image whose brightness is adjusted; Further comprising, the image processing method. The method of claim 1, The brightness of the second target image is adjusted by adjusting the brightness of the image obtained by the at least one camera for constituting the second target image. The method of claim 1, The first target image and the second target image includes an image of the first camera, The exposure time of the first camera in the first mode and the exposure time of the first camera in the second mode are different. The method of claim 1, The first target image and the second target image includes at least two images of the camera, And an exposure time of the camera in the first mode and an exposure time of the camera in the second mode are different. The method of claim 1, The first target image in the first mode is an image converted or matched based on an image obtained from at least two or more cameras of the at least one camera, And the second target image in the second mode is an image based on an image obtained from one of the at least one cameras. The method of claim 1, The first target image in the first mode is an image based on an image obtained from one of the at least one camera, And the second target image in the second mode is an image converted or matched based on an image obtained from at least two cameras of the at least one camera. The method of claim 1, The first target image in the first mode is an image converted or matched based on an image obtained from at least two or more cameras of the at least one camera, And the second target image in the second mode is an image converted or matched based on an image obtained from at least two cameras of the at least one camera. The method of claim 1, And at least a portion of the first target image in the first mode overlaps with a portion of the second target image in the second mode. The method of claim 1, And the first mode and the second mode output different images composed of an image obtained from one of the at least one camera or at least one image obtained from each of the at least one camera.

Images