KR101830028B1 - Apparatus and method for detecting of low-light area for image dynamic range expansion - Google Patents

Abstract

The low-light region detection apparatus for extending the dynamic range of an image according to an embodiment of the present invention uses a low-light (LF) ) Image and a row light pixel detector for detecting pixels constituting the row light region by comparing a value of a pixel constituting the generated LF image with a predetermined threshold value .

Description

[0001] APPARATUS AND METHOD FOR DETECTING OF LOW-LIGHT AREA FOR IMAGE DYNAMIC RANGE EXPANSION [0002]

The present invention relates to a low light area detection apparatus and method for expanding a dynamic range of an image. More particularly, the present invention relates to a low light area detecting apparatus for detecting a low light In order to acquire high dynamic range (HDR) images by extending the dynamic range, we use the RGB values of each pixel in the original LDR image to increase the effect of the light source, And more particularly, to a low light area detection apparatus and method for expanding a dynamic range of an image for detecting an area.

Currently, when a camera is used in various fields such as a black box, a CCTV, and an autonomous vehicle, when an HDR image having a wider dynamic range (HDR) is required for acquiring more information from a photographed image . However, there is a problem that an expensive HDR camera needs to be used to acquire an HDR image, and there are many difficulties in acquiring an HDR image. Therefore, an LDR image having a narrow dynamic range (LDR) Many researches have been made on a method of acquiring an HDR image by image processing.

In recent years, it has been proposed to decompose an LDR image of a single frame into LDR images having different brightness values (exposure values) A method of generating an HDR image by synthesizing a plurality of LDR images by synthesizing an HDR image while expanding a dynamic range while synthesizing is mainly studied. However, a method of generating an HDR image by expanding a dynamic range while synthesizing a plurality of LDR images photographed at different exposure times may cause a ghost phenomenon caused by motion of the imaging device, A method of generating an HDR image by generating an exposure image of an image, normalizing the brightness, and estimating and synthesizing detail has a complicated method of decomposing and synthesizing an image, and it is difficult to generate a high quality HDR image .

In a single-frame LDR image, a high light area, which is a light source generated due to a long exposure time or a light source, corresponding to an illumination or exposure time, and a low light area (Low -light region may be generated. In the LDR image with a low dynamic range (LDR), the highlight area and the low light area may not be a big problem because of the small amount of time information, but the HDR image with a wide dynamic range (HDR) When the dynamic range is expanded, the highlight area and the low light area may look unnatural due to the wide dynamic range of the HDR image.

However, in the HDR image generation method using the LDR image, only the method of removing the highlight area, which is a reflection area of the light source or the light source, is mainly studied. However, the visual information due to the low light area caused by insufficient illumination or short exposure time In order to extend the dynamic range of the image, it is necessary to study the apparatus and method for detecting the low light region in the original LDR image in view of the low loss.

Korean Patent Laid-Open No. 10-2013-0040321 entitled " Single-frame-based high dynamic range image generation apparatus and method "(published on April 24, 2014) Korean Patent Laid-Open No. 10-2014-0072980 "Apparatus and method for generating HDR image using single image " (published on June 16, 2014)

SUMMARY OF THE INVENTION The present invention has been devised to solve the above problems, and it is an object of the present invention to provide a low light region detection method and a low light region detection method for removing a low light portion using RGB values of respective pixels in a source LDR image, And an object of the present invention is to provide an apparatus and method.

The low-light region detection apparatus for extending the dynamic range of an image according to an embodiment of the present invention uses a low-light (LF) ) Image and a row light pixel detector for detecting pixels constituting the row light region by comparing a value of a pixel constituting the generated LF image with a predetermined threshold value .

In the low light region detecting apparatus for expanding the dynamic range of an image, the LF image generating unit calculates the largest value among R (red), G (green) and B (Blue) values for each pixel of the original LDR image Is extracted as a maximum value, and the extracted maximum values of the R, G, and B channel values of each pixel are summed to generate an LF image.

In the low light area detection apparatus for expanding the dynamic range of an image, a predetermined threshold value is set such that the maximum value of R (red), G (green), and B (Blue) values of each pixel in each pixel of the original LDR image The smallest value is extracted as the minimum value, and the average value of the extracted minimum values is calculated. The maximum value among the R (red), G (green) and B And the average value of the minimum values may be set to a value obtained by summing the average value of the maximum values and dividing by two.

The low light region detection method for extending the dynamic range of an image according to an exemplary embodiment of the present invention is a method for detecting a low light region using an RGB value of each pixel in an original LDR (Low Dynamic Range) Free) image, and comparing the pixel value of the generated LF image with a predetermined threshold value to detect pixels constituting the rowlight area.

In the low light region detection method for expanding the dynamic range of an image, the step of generating an LF image may include calculating R (red), G (green), and B (Blue) values for each pixel of the original LDR image Extracting the largest value among the maximum values among the R, G, and B channel values of each pixel, and summing the extracted maximum values of the R, G, and B channel values of each pixel to generate an LF image.

Also, in the low light area detection method for expanding the dynamic range of an image, a predetermined threshold value is set to a value of R (red), G (green), and B (Blue) of each pixel in each pixel of the original LDR image Extracts the smallest value as the minimum value, calculates the average value of the extracted minimum values, extracts the largest value among R (red), G (green), and B The average value may be calculated, and the average value of the minimum values may be set to a value obtained by summing the average value of the maximum values and dividing by two.

Meanwhile, as an embodiment of the present invention, a computer-readable recording medium on which a program for causing the computer to execute the above-described method may be provided.

According to an embodiment of the present invention, there is provided a method and apparatus for detecting a low-light region in which information is lost due to low-light or low-light exposure, , It is possible to obtain a more natural HDR image by removing the low light component in the low light area from the original LDR image.

According to an apparatus and method for detecting a low-light region for dynamic range expansion of an image according to an exemplary embodiment of the present invention, in order to generate an HDR image, a detection speed and a processing speed of a low- Can be improved.

FIG. 1 is a block diagram showing a configuration of an apparatus for generating an HDR image from an LDR image including a rowlight region detecting apparatus for expanding a dynamic range of an image according to an embodiment of the present invention.
FIG. 2 is a diagram illustrating images generated in a row light region detection process according to an exemplary embodiment of the present invention. Referring to FIG. FIG. 2A is a diagram showing a source LDR image, FIG. 2B is a low-light free (LF) image, and FIG.
3 is a diagram illustrating an algorithm for generating an HDR image from an LDR image according to an embodiment of the present invention.
4 is a flowchart illustrating a method of detecting a row area for dynamic range extension of an image according to an exemplary embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, which will be readily apparent to those skilled in the art. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. In order to clearly illustrate the present invention, parts not related to the description are omitted, and similar parts are denoted by like reference characters throughout the specification.

The terms used in this specification will be briefly described, and the present invention will be described in detail.

While the present invention has been described in connection with what is presently considered to be the most practical and preferred embodiment, it is to be understood that the invention is not limited to the disclosed embodiments. Also, in certain cases, there may be a term selected arbitrarily by the applicant, in which case the meaning thereof will be described in detail in the description of the corresponding invention. Therefore, the term used in the present invention should be defined based on the meaning of the term, not on the name of a simple term, but on the entire contents of the present invention.

When an element is referred to as "including" an element throughout the specification, it is to be understood that the element may include other elements as well, without departing from the spirit or scope of the present invention. Also, the terms "part," " module, "and the like described in the specification mean units for processing at least one function or operation, which may be implemented in hardware or software or a combination of hardware and software . In addition, when a part is referred to as being "connected" to another part throughout the specification, it includes not only "directly connected" but also "connected with other part in between".

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram illustrating a configuration of an apparatus 100 for generating an HDR image from an LDR image including a rowlight region detecting apparatus 30 for expanding a dynamic range of an image according to an embodiment of the present invention.

1, an apparatus 100 for generating an HDR (High Dynamic Range) image from an LDR (Low Dynamic Range) image using image processing includes a highlight area detecting device 10, a row light area detecting device 30, A highlight component and a low light component removal unit 50, and a dynamic range extension unit 70. [

That is, the apparatus 100 for generating an HDR (High Dynamic Range) image from an LDR (Low Dynamic Range) image using image processing removes or corrects a low light area caused by insufficient light source or exposure time in the original LDR image A low light area detection device 30 may be included.

1, the low light region detecting apparatus 30 includes an LF image generating unit 34 for generating a low-light free (LF) image in which an effect of a light source is enhanced to detect a low light region in an original LDR image And a row light pixel detection unit 38 for detecting pixels constituting the row light region.

The LF image generating unit 34 according to an embodiment of the present invention detects a dark row area caused by insufficient exposure of the light source and the light source in the original LDR image, LF (low-light free) image can be generated by increasing the effect of the LF. Here, the LF (low-light free) image is obtained by extracting the largest value among the three channels of each pixel, that is, R (Red), G (Green), and B Quot; and " sum " That is, a low-light free (LF) image can mean an image in which the intensity of each of the RGB channel regions of each pixel is increased to enhance the effect of the light source. The method of generating the LF image using the original LDR image by the LF image generating unit 34 can be expressed as Equation (1) below.

는 원본 LDR 영상의 밝기값을 의미하고, i는 영상의 픽셀 번호를, x, y는 영상의 좌표를 의미한다. 예를 들어, i는 영상에 존재하는 픽셀의 번호로서, 영상의 가로의 크기가 N이고, 세로의 크기가 M이라고 가정하면, i는 NM 개까지 존재할 수 있다. 즉, <수학식 1>을 참조하면, LF 영상은 원본 LDR 영상에서 각 픽셀의 R(Red), G(Green), B(Blue) 값 중 가장 큰 값을 최대값으로 추출하고, 본래의 R(Red), G(Green), B(Blue) 채널값에서 추출된 최대값을 각각 합산하여 생성된다. 예를 들어, 원본 LDR 영상의 한 픽셀에서 R(Red), G(Green), B(Blue) 값이 각각, 100, 0, 50 일 경우에, 가장 큰 값은 100이 되므로, R(Red), G(Green), B(Blue) 값에서 가장 큰 값인 100을 최대값으로 추출하고, 추출된 최대값 100을 해당 픽셀의 RGB 값에 각각 합산하여 (R, G, B) = (200, 100, 150)인 픽셀이 생성되며, 상술한 과정을 원본 LDR 영상의 모든 픽셀에 대해 수행함으로써 LF 영상이 생성되게 된다. In Equation (1)

Denotes the brightness value of the original LDR image, i denotes the pixel number of the image, and x and y denote the coordinates of the image. For example, i is the number of pixels existing in the image. Assuming that the horizontal size of the image is N and the vertical size is M, i can exist up to NM. That is, referring to Equation (1), the LF image extracts the largest value among R (Red), G (Green) and B (Blue) values of each pixel in the original LDR image as a maximum value, (Red), G (Green), and B (Blue) channel values. For example, when the values of R (Red), G (Green), and B (Blue) are 100, 0, and 50 in one pixel of the original LDR image, the largest value is 100, (R, G, B) = (200, 100) is obtained by summing the extracted maximum value 100 to the RGB value of the corresponding pixel, , 150) is generated, and the LF image is generated by performing the above-described process on all the pixels of the original LDR image.

The low light pixel detector 38 according to an embodiment of the present invention compares a value of pixels constituting the LF image generated by the LF image generator 34 with a predetermined threshold value, It is possible to detect the pixels constituting the area. That is, when the pixel value of the LF image generated from the original LDR image is less than the predetermined threshold value, the low light pixel detector 38 can detect the region detected as the low light region and detect it.

The method of detecting the rowlight region by comparing the value of a pixel constituting the LF image generated from the original LDR image with a predetermined threshold value by the rowlight pixel detector 38 is expressed as Equation (2) .

Here, Th, which is a predetermined threshold value, can be set as the following Equation (3).

That is, referring to Equation (3), a predetermined threshold is obtained by extracting the smallest value among R (red), G (green) and B (Blue) values of each pixel in each pixel of the original LDR image as a minimum value And calculates the average value of the extracted maximum values by extracting the maximum value among the R (red), G (green), and B (Blue) values of each pixel, The average value may be set to a value obtained by summing the average value of the maximum values and dividing by two.

는 (10+10+0+0)/4 로 계산되므로, 추출된 최소값들의 평균값

는 5 가 된다. 또한, 각 픽셀의 R(red), G(green), B(Blue) 값 중 가장 큰 값들은 100, 100, 150, 150 이 되므로, 100, 100, 150, 150 이 최대값으로 추출되고, 추출된 최대값들의 평균값인

는 (100+100+150+150)/4 로 계산되므로, 추출된 최대값들의 평균값

는 125 가 된다. 여기에서, 소정의 임계값은 최소값들의 평균값에 최대값들의 평균값을 합산하고 2로 나눈 값으로 설정되므로, (5+125)/2 = 65 가 소정의 임계값으로 설정될 수 있다. 즉, 예를 든 상기 4개의 픽셀로 구성된 LDR 영상에서, 로우라이트 영역을 검출하기 위한 소정의 임계값으로 65 가 설정될 수 있다.For example, assuming an LDR image composed of four pixels, R (Red), G (Green) and B (Blue) values of four pixels are respectively (100, 10, 50) ), (150, 0, 50), (0, 150, 50). Here, since the smallest values among R (red), G (green) and B (Blue) values of each pixel are 10, 10, 0 and 0, 10, 10, 0 and 0 are extracted as the minimum values, The average value of the minimum values

Is calculated as (10 + 10 + 0 + 0) / 4, so the average value of the extracted minimum values

Becomes 5. 100, 100, 150, and 150 are extracted as the maximum values among the R (red), G (green), and B Which is the average value of the maximum values

Is calculated as (100 + 100 + 150 + 150) / 4, so that the average value of the extracted maximum values

Lt; / RTI &gt; Here, the predetermined threshold is set to a value obtained by adding the average value of the minimum values to the average value of the maximum values and dividing by 2, so (5 + 125) / 2 = 65 may be set to a predetermined threshold value. That is, in the LDR image composed of the above-mentioned four pixels, for example, 65 can be set to a predetermined threshold value for detecting the rowlight region.

That is, an LF image is generated from the original LDR image using the LF image generating unit 34, a predetermined threshold value is calculated from the pixels of the original LDR image using the low light pixel detecting unit 38, And a predetermined threshold value is compared with each other.

FIG. 2 is a diagram illustrating images generated in a row light region detection process according to an exemplary embodiment of the present invention. Referring to FIG. FIG. 2A is a diagram showing a source LDR image, FIG. 2B is a low-light free (LF) image, and FIG.

That is, the LF image of FIG. 2 (b) represents a low-light free (LF) image generated by increasing the effect of the light source on the original LDR image by the LF image generating unit 34, The detected low light area of the low light pixel indicates the detected low light area by comparing the pixel value of the LF image with a predetermined threshold value by the low light pixel detector 38. [

1, the highlight area detecting apparatus 10 includes a specular free (SF) image generating unit 14, a specular free (SF) image generating unit 16, and a highlighting unit 16 for detecting a highlight area in the original LDR image. And a pixel detection unit 18.

The SF image generating unit 14 according to an embodiment of the present invention can generate a SF (Specular Free) image using the original LDR image in order to remove the region reflected by the light source and the light source in the original LDR image . Here, the SF (Specular Free) image is an image generated by subtracting the smallest value among the three channels of each pixel, that is, R (Red), G (Green), and B (Blue) Which means that the intensity of the light source is lowered or decreased by lowering the intensity of the region. The method of generating the SF image using the original LDR image by the SF image generating unit 14 can be expressed as Equation (4) below.

는 원본 LDR 영상의 밝기값을 의미하고, i는 영상의 픽셀 번호를, x, y는 영상의 좌표를 의미한다. <수학식 4>를 참조하면, SF 영상은 원본 LDR 영상에서 각 픽셀의 R(Red), G(Green), B(Blue) 값 중 가장 작은 값을 최소값으로 추출하고, 본래의 R(Red), G(Green), B(Blue) 채널값에서 추출된 최소값을 각각 차감하여 생성된다. 예를 들어, 원본 LDR 영상의 한 픽셀에서 R(Red), G(Green), B(Blue) 값이 각각, 150, 50, 100 일 경우에, 가장 작은 값은 50이 되므로, R(Red), G(Green), B(Blue) 값에서 가장 작은 값인 50을 최소값으로 추출하고, 추출된 최소값 50을 각각 차감하여 100, 0, 50 이 SF 영상으로 생성되게 된다. In Equation (4)

Denotes the brightness value of the original LDR image, i denotes the pixel number of the image, and x and y denote the coordinates of the image. Referring to Equation (4), the SF image extracts the minimum value among R (Red), G (Green) and B (Blue) values of each pixel in the original LDR image as a minimum value, , G (Green), and B (Blue) channel values, respectively. For example, when the values of R (Red), G (Green), and B (Blue) are 150, 50, and 100 in one pixel of the original LDR image, the smallest value is 50, , 50 (G (Green), and B (Blue) are extracted as minimum values, and the extracted minimum values 50 are subtracted from each other to generate 100, 0, and 50 SF images.

The MSF image generating unit 16 according to an embodiment of the present invention generates an MSF (Modified Specular Free) image using the generated SF image to detect a region reflected by the light source and the light source in the original LDR image . That is, the MSF image generating unit 16 can generate an MSF image, which is a corrected SF image, by correcting or correcting the generated SF image in order to remove the region reflected by the light source and the light source in the original LDR image.

A method of generating the MSF image using the generated SF image by the MSF image generating unit 16 can be expressed as Equation (5).

은

의 평균을 나타낸다. 즉,

는 모든 픽셀들의 R(Red), G(Green), B(Blue) 값 중 최소값들의 평균값을 나타낸다. 즉, MSF 영상 생성부(16)는 생성된 SF 영상의 각 픽셀의 R(Red), G(Green), B(Blue) 값에

값을 가산함으로써 MSF 영상을 생성할 수 있다.In Equation (5)

silver

Lt; / RTI &gt; In other words,

Represents the average value of the minimum values among R (Red), G (Green) and B (Blue) values of all the pixels. That is, the MSF image generating unit 16 generates the MSF image by using the values of R (Red), G (Green), and B (Blue) of each pixel of the generated SF image

The MSF image can be generated.

The highlight pixel detector 18 according to an embodiment of the present invention calculates the difference value between the original LDR image and the generated MSF image, compares the calculated difference value with the Otsu's threshold value, Can be detected. That is, the highlight pixel detector 18 calculates the difference value between the original LDR image and the generated MSF image, and when the calculated difference value exceeds a predetermined threshold value, detects the highlighted area as a highlight area have.

The method of detecting the highlight area using the difference value and the Otsu's threshold value between the original LDR image and the generated MSF image by the highlight pixel detection unit 18 can be expressed as Equation (6) below .

이고, Th 는 원본 LDR 영상의 각 채널에 대한 오츠(Otsu's) 임계값을 의미한다. 즉, 하이라이트 픽셀 검출부(18)는 원본 LDR 영상과 생성된 MSF 영상과의 차이값이 오츠 임계값을 초과하면, 초과한 픽셀들을 하이라이트 영역으로 검출할 수 있다. In Equation (6)

, And Th denotes an Otsu's threshold value for each channel of the original LDR image. That is, if the difference value between the original LDR image and the generated MSF image exceeds the Ots threshold value, the highlight pixel detector 18 can detect the exceeding pixels as a highlight area.

Here, the Oats threshold value means a threshold value determined by the Otsu threshold method among the binarization methods used to separate the background and foreground of the image. For example, a binarization method divides an image into a background and a foreground based on a specific value, and a specific value is determined as a threshold value or a threshold value, thereby separating the image into a background and a foreground. Here foreground means an important object in vision, background can mean less important object.

That is, the Otsu threshold method is a method proposed by Nobuyuki Otsu as an image binarization method. The Otsu threshold value is determined by considering the variance (In Class Variance) and class Can be determined by calculating the Between Class Variance.

A highlight component and a row light component removal part 50 according to an embodiment of the present invention detect the red component, the red component, the green component, and the blue component, respectively, of the pixels constituting the detected highlight area and the detected row area The highlight component and the low light component can be respectively removed.

For example, the highlight component and the low light component removal unit 50 may use Principal Component Analysis (PCA) to remove the highlight component and the low light component from the original LDR image, respectively. That is, since the color image has three color channels of R (Red), G (Green), and B (Blue), it is assumed that principal components weighted by three eigenvectors of the original LDR image are combined . That is, the original LDR image is decomposed using the principal component vector, and the highlight component and the low light component are respectively removed by a method of removing the main component containing a lot of highlight components or low light components and reconstructing the image again.

In other words, Principal Component Analysis (PCA) method can be understood as a method of narrowing down the dimension while minimizing information loss by associating the data distributed in the image with the eigenvector combined with the principal component.

The original LDR image can be expressed by the principal component analysis method as shown in Equation (7) below.

In Equation (7), I denotes an RGB value of an image, and V _i and P _i denote a principal component vector and a corresponding principal component, respectively.

According to the principal component analysis method, the second principal component corresponding to the second largest eigenvalue includes all or most of the highlight component (or low light component). Whether or not exactly the second major component includes all or most of the highlight component (or the low light component) is determined by the fidelity ratio FR _i and the FR ₂ And a predetermined threshold value.

는 i번째 고유값을 의미하고, <수학식 9>에서 Second PC(Principal Component)는 두 번째로 큰 주성분을 의미한다. 즉, <수학식 9>에서 충실도 FR₂ 가 소정의 임계값보다 작으면, 두 번째 주성분이 하이라이트 성분(또는 로우라이트 성분)을 포함하고 있는 것으로 판단할 수 있다. 여기에서, 원본 영상을 두 번째 주성분을 제외하고, 첫 번째와 세 번째 주성분만을 이용하여 영상을 재구성(reconstruct) 하면, 원본 영상에서 하이라이트 성분(또는 로우라이트 성분)을 제거한 영상으로 재구성할 수 있다. 또한, 충실도 FR₂ 와 비교할 소정의 임계값은 실험에 의해서 0.02 로 미리 정의할 수 있다. In Equation (8)

Denotes an i-th eigenvalue, and a second PC (Principal Component) in Equation (9) denotes the second largest principal component. That is, when the fidelity FR ₂ is smaller than a predetermined threshold value in Equation (9), it can be determined that the second principal component includes a highlight component (or a row light component). Here, if the original image is reconstructed using only the first and third principal components except for the second principal component, reconstruction can be performed by removing the highlight component (or low light component) from the original image. In addition, the predetermined threshold value to be compared with the fidelity FR ₂ can be previously defined as 0.02 by an experiment.

However, the first principal component obtained from the principal component analysis (PCA) may still include a highlight component (or a low light component). As a method for removing a highlight component (or a low light component) from the first principal component, a histogram equalization histogram equalization can be applied. The reconstructed image using the histogram equalization can be expressed as Equation (10) below.

In Equation (10), P _h is the histogram equalization principal component, and I _R is the reconstructed image. V ₂ P ₂ in the right part of the equation may or may not be included in the reconstructed image, depending on the fidelity FR ₂ of the second largest eigenvector.

As a result, the reconstructed image I _R is shifted and reconstructed in the color domain, and the original color (color) without the highlight component (or low light component) is transformed using a second order polynomial transformation. Can be obtained. The important problem here is to find a weight function for polynomial transformation, which uses a part (diffuse part) that is not the highlight area (or low light area) detected in the original image and the corresponding reconstructed image I _R To obtain a weight function.

That is, the method of obtaining the weight function can be expressed as Equation (11) and Equation (12).

<Equation 11> at I _d stands for part (diffuse part) other than the highlight areas (or the low light region) is detected from the original image, and W is a weight function, M _d is the highlight area in the reconstructed image I _R ( Or a second order polynomial extension obtained in a diffuse part that is not a low light area. That is, I _d means an area excluding the highlight area (or row light area) detected by the highlight area detection device 10 (or the row light area detection device 30) in the original image.

That is, the weight function W can be obtained by a least square pseudo inverse matrix calculation using I _d and M _d , as shown in Equation (12).

The second-order polynomial expansion obtained in the diffuse part (not the highlight area) or the diffuse part in the reconstructed image I _R from the highlight area (or row light area) in the original image is compared , And the weight function is obtained, the image I _Diff with the highlight area removed from the entire image can be expressed as Equation (13).

In Equation (13), I _Diff , W denotes a weight function, and M denotes a second order polynomial extension of the reconstructed image I _R.

That is, I _Diff can be obtained by applying a weighting function W to the second-order polynomial extension M of the reconstructed image I _R.

I _Diff , which is a video whose highlight components have been removed from the original image and the entire image, The original image I and I _Diff can be expressed by a combination of the coefficients in order to remove the color difference and finally obtain the image in which the effect of the light source is removed from the original image. That is, the image I _D in which only the highlight effect (or the row light effect) is finally removed can be expressed as Equation (14).

In Equation (14), I _D denotes an image in which only the highlight effect (or row light effect) is finally removed from the original image, k _xy denotes a coefficient for a pixel at (x, y) 1 < / RTI > If the pixel at the (x, y) position is a highlight pixel, k _xy is 0, and if it is not a highlight pixel, k _xy is 1.

The dynamic range extending unit 70 according to an embodiment of the present invention can generate the HDR image by extending the dynamic range of the image from which the highlight component and the low light component are removed from the original LDR image.

For example, the dynamic range extending unit 70 linearly amplifies the image from which the highlight component and the low light component are removed from the original LDR image using Equation (15) Images can be generated.

는 동적 범위 확장 과정(scaling)에서 비선형성을 결정할 수 있다. <수학식 15>의 동적 범위 확장은 모든 픽셀에 개별적으로 수행된다. 지수

는 영상의 평균 밝기가 다른 픽셀들에 비해서 어떻게 변화할 것인지에 따라서 결정될 수 있다. 예를 들어,

가 1이면 모든 픽셀들이 동등하게 변화되고,

가 1을 초과하면 평균 밝기가 상대적으로 어둡게 변화되고,

가 1 미만이면 평균적으로 더 밝게 변화된다. 일반적으로 모든 픽셀들이 동등하게 변화되도록

는 1로 미리 결정될 수 있다.In Equation (15), L denotes the brightness (luminance) of the original pixel to be scaled by dynamic range, L _max and L _min denote the brightness of the image from which the highlight component and the low light component are removed from the original LDR image ) &Lt; / RTI > I _h means the maximum input brightness of the HDR display,

Can determine the non-linearity in the dynamic range scaling process. The dynamic range extension of Equation (15) is performed individually for all pixels. Indices

Can be determined according to how the average brightness of the image will change relative to other pixels. E.g,

Is 1, all of the pixels are changed equally,

Exceeds 1, the average brightness is changed to be relatively dark,

Is less than 1, the brightness changes to be brighter on the average. In general, all pixels should be changed equally

Can be determined in advance as 1.

3 is a diagram illustrating an algorithm for generating an HDR image from an LDR image according to an embodiment of the present invention.

Referring to FIG. 3, an algorithm for generating an HDR image in the LDR image detects a highlight area and a row light area in the original LDR image, and detects R (Red), G (Green ) And B (Blue) components, respectively, and an HDR image is generated by expanding the dynamic range of the image from which the highlight component and the low light component are removed.

That is, in order to generate the HDR image from the LDR image, the dark or bright part should be removed or corrected by the light source. Therefore, the highlight area and the low light area must be detected from the original LDR image.

4 is a flowchart illustrating a method of detecting a row area for dynamic range extension of an image according to an exemplary embodiment of the present invention.

Referring to FIG. 4, in step S10, the LF image generating unit 34 generates an LF (low-light free) image that enhances the effect of the light source by using the RGB values of the pixels in the original Low Dynamic Range Can be generated.

For example, in order to detect the low-illuminance region in the original LDR image, the LF image generating unit 34 generates R (red), G (red) image data for each pixel of the original LDR image using Equation the LF image can be generated by extracting the largest value among the green (B) and green (B) values as the maximum value, and summing the extracted maximum values for the R, G, and B channel values of each pixel.

In step S20, the row light pixel detector 38 may detect the pixels constituting the row light region by comparing a value of a pixel constituting the generated LF image with a predetermined threshold value.

For example, the low light pixel detector 38 detects the region detected when the pixel value of the LF image generated from the original LDR image is less than the predetermined threshold, using Equation (2) described above as a low light region Can be detected separately. Here, the predetermined threshold value may be set to the above-described Equation (3). That is, the predetermined threshold value is obtained by extracting the minimum value among R (red), G (green), and B (Blue) values of each pixel in each pixel of the original LDR image as a minimum value and calculating an average value of extracted minimum values , The maximum value among the R (red), G (green) and B (Blue) values of each pixel is extracted as a maximum value, the average value of the extracted maximum values is calculated, and the average value of the maximum values is added to the average value of the minimum values 2 < / RTI >

The contents of the rowlight region detecting device 30 for extending the dynamic range of the image described above may be applied to the rowlight region detecting method for extending the dynamic range of an image according to an embodiment of the present invention. Accordingly, the same contents as those of the row light region detection device 30 for expanding the dynamic range of the image are omitted from the description of the row light region detection method for expanding the dynamic range of the image.

One embodiment of the present invention may also be embodied in the form of a recording medium including instructions executable by a computer, such as program modules, being executed by a computer. Computer readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. In addition, the computer readable medium may include both computer storage media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data.

It will be understood by those skilled in the art that the foregoing description of the present invention is for illustrative purposes only and that those of ordinary skill in the art can readily understand that various changes and modifications may be made without departing from the spirit or essential characteristics of the present invention. will be. It is therefore to be understood that the above-described embodiments are illustrative in all aspects and not restrictive. For example, each component described as a single entity may be distributed and implemented, and components described as being distributed may also be implemented in a combined form.

The scope of the present invention is defined by the appended claims rather than the detailed description and all changes or modifications derived from the meaning and scope of the claims and their equivalents are to be construed as being included within the scope of the present invention do.

10: highlight area detecting device 14: SF image generating part
16: MSF image generating unit 18: highlight pixel detecting unit
30: Low light area detecting device 34: LF image generating part
38: Low light pixel detection section
50: Highlight component and low light component removal
70: dynamic range expanding section
100: Device for generating HDR image from LDR image using image processing

Claims (7)

A low light area detection apparatus for expanding a dynamic range of an image,
An LF image generating unit for generating an LF (low-light free) image that enhances the effect of the light source by using RGB values of each pixel to detect the row light region generated from the original LDR (Low Dynamic Range) image; And
And a low light pixel detection unit for detecting pixels constituting the low light region in the original LDR image by comparing a value of a pixel constituting the generated LF image with a predetermined threshold value, A low light area detection device for detecting a low light area.
The method according to claim 1,
Wherein the LF image generating unit comprises:
Extracting a maximum value among R (red), G (green), and B (Blue) values for each pixel of the original LDR image as a maximum value, And the LF image is generated by summing the maximum values of the LF images.
The method according to claim 1,
Wherein the predetermined threshold value is a predetermined threshold value,
A minimum value of R (red), G (green) and B (Blue) values of each pixel in each pixel of the original LDR image is extracted as a minimum value to calculate an average value of the extracted minimum values, The average value of the extracted maximum values is calculated by extracting the largest value among R (red), G (green) and B (Blue) values as a maximum value, and the average value of the maximum values is added to the average value of the minimum values. And a low light area detection device for expanding the dynamic range of the image.
A low light area detection method for extending a dynamic range of an image,
Generating a low-light free (LF) image that enhances the effect of the light source by using RGB values of each pixel to detect the low light region generated in the original LDR (Low Dynamic Range) image; And
Comparing the pixel value of the generated LF image with a predetermined threshold value to detect pixels constituting the rowlight region in the original LDR image, Detection method.
5. The method of claim 4,
Wherein the generating the LF image comprises:
Extracting a maximum value among R (red), G (green) and B (Blue) values for each pixel of the original LDR image as a maximum value; And
And summing the extracted maximum values to R, G, and B channel values of each pixel to generate an LF image.
5. The method of claim 4,
Wherein the predetermined threshold value is a predetermined threshold value,
A minimum value of R (red), G (green) and B (Blue) values of each pixel in each pixel of the original LDR image is extracted as a minimum value to calculate an average value of the extracted minimum values, The average value of the extracted maximum values is calculated by extracting the largest value among R (red), G (green) and B (Blue) values as a maximum value, and the average value of the maximum values is added to the average value of the minimum values. A method of detecting a low light region for expanding a dynamic range of an image,
A computer-readable recording medium on which a program for implementing the method of any one of claims 4 to 6 is recorded.

Images