KR102846634B1 - Image processing method and display device using the same - Google Patents

Abstract

An image processing method and a display device using the same are provided. The image processing method includes a step of extracting a motion vector corresponding to each of a plurality of blocks constituting a reference image based on a reference image and a comparison image, a step of generating an intermediate motion vector field inserted between the reference image and the comparison image based on the motion vector, and a step of matching a supplementary motion vector to each of at least one hole block existing in the intermediate motion vector field based on a previous image corresponding to a previous frame of the reference image, the reference image, and the comparison image. The image processing method according to one embodiment of the present invention can reduce a lattice effect existing between a foreground and a background and a dragging phenomenon of the foreground by filling an area where a hole occurs in the intermediate motion vector field with a foreground motion vector.

Description

Image processing method and display device using the same {IMAGE PROCESSING METHOD AND DISPLAY DEVICE USING THE SAME}

The present invention relates to an image processing method and a display device using the same, and more specifically, to an image processing method capable of improving the image quality of a display device and a display device using the same.

Flat panel displays (FPDs) are being adopted in various electronic devices such as mobile phones, tablets, laptop computers, televisions, and monitors. Recently, liquid crystal display devices (LCDs) and organic light emitting diode displays (OLEDs) have been used as FPDs. Such display devices display images and include a pixel array composed of a plurality of pixels and a driving circuit that controls light transmission or emission from each of the pixels. The driving circuit of the display device includes a data driving circuit that supplies data signals to data lines of the pixel array, a gate driving circuit (or scan driving circuit) that sequentially supplies gate signals (or scan signals) synchronized with the data signals to the gate lines (or scan lines) of the pixel array, and a timing controller that controls the data driving circuit and the gate driving circuit.

Recent display devices are using Frame Rate Up Conversion (FRUC) to improve compatibility between video media and to output images more naturally. FRUC is a technology that increases the frame rate of an input image by inserting an interpolation image between the input image frames.

Conventional FRUC methods simply repeat input image frames or insert the average of two adjacent frames between them. These methods often result in interrupted or blurred output images, degrading image quality.

Recent FRUC methods estimate inter-frame motion in an input image and insert interpolated images based on the estimated motion between frames. Specifically, in recent FRUC, motion is estimated from adjacent frames, motion vectors are extracted, and interpolated images are inserted between frames using these motion vectors.

FRUC, which utilizes motion vectors, can be used to estimate motion on a block-by-block basis. Specifically, each video frame is divided into blocks of a certain size, and the most similar blocks among adjacent frames are identified based on the divided blocks and connected as vectors. This allows motion to be estimated between frames and motion vectors to be determined.

Furthermore, an Intermediate Motion Vector Field (IMVF) is generated by projecting the motion vector extracted from each frame's video image onto the interpolated image domain. In this way, the final interpolated image is generated by filling each block with input image data in the interpolated image domain according to the intermediate motion vector field.

However, if the background image and the foreground image in each frame move in different directions, blocks without motion vectors may occur among the blocks that make up the intermediate motion vector field. In this way, blocks without motion vectors in the intermediate motion vector field or blocks without image data among the blocks that make up the interpolated image are called holes, and since there is no image data in blocks with holes, the problem of deteriorating image quality occurs.

[Related technical literature]

Device and method for increasing the frame rate of a video signal (Korean Patent Publication No. 10-2013-0031132)

Accordingly, the problem to be solved by the present invention is to provide an image processing method capable of filling a hole in an image data area with background image data by filling a region where a hole occurs in an intermediate motion vector field with a foreground motion vector, and a display device using the same.

In addition, another problem to be solved by the present invention is to provide an image processing method and a display device using the same that can reduce the grid effect occurring around the foreground and the grid effect occurring in the background around the area where a hole occurs by filling the area where a hole occurs with background image data.

In addition, another problem that the present invention seeks to solve is to provide an image processing method and a display device using the same that can reduce afterimages and improve image quality at low cost and high efficiency by correcting inaccurate motion vectors at the boundary between the foreground and the background.

The tasks of the present invention are not limited to the tasks mentioned above, and other tasks not mentioned will be clearly understood by those skilled in the art from the description below.

In order to solve the above-described problem, an image processing method according to an embodiment of the present invention includes the steps of extracting a motion vector corresponding to each of a plurality of blocks constituting a reference image based on a reference image and a comparison image, the step of generating an intermediate motion vector field inserted between the reference image and the comparison image based on the motion vector, and the step of matching a supplementary motion vector to each of at least one hole block existing in the intermediate motion vector field based on a previous image corresponding to a previous frame of the reference image, the reference image, and the comparison image. The image processing method according to an embodiment of the present invention can reduce a lattice effect existing between the foreground and the background and a foreground dragging phenomenon by filling a region where a hole occurs in the intermediate motion vector field with a foreground motion vector.

In order to solve the above-described problem, a display device according to another embodiment of the present invention includes a display panel and an image processing unit that processes an image input to the display panel. The image processing unit includes a motion vector extraction unit that extracts a motion vector corresponding to each of a plurality of blocks constituting a reference image based on a reference image and a comparison image, an intermediate motion vector field generation unit that generates an intermediate motion vector field to be inserted between the reference image and the comparison image based on the motion vector, and a hole filling unit that matches a supplementary motion vector to each of at least one hole block existing in the intermediate motion vector field based on a previous image corresponding to a previous frame of the reference image, the reference image, and the comparison image. The display device according to another embodiment of the present invention can reduce afterimages of an image and improve image quality at low cost and high efficiency by correcting a motion vector filled in a hole in the intermediate motion vector field.

Specific details of other embodiments are included in the detailed description and drawings.

The present invention provides an image processing method capable of reducing the lattice effect and foreground dragging phenomenon existing between the foreground and background by filling in a region where a hole has occurred in an intermediate motion vector field with a foreground motion vector, and a display device using the same.

In addition, the present invention can produce an image processing method and a display device using the same that can reduce afterimages and improve image quality at low cost and high efficiency by correcting motion vectors filled in holes in an intermediate motion vector field.

The effects according to the present invention are not limited to those exemplified above, and more diverse effects are included in this specification.

FIG. 1 is a block diagram schematically showing a display device according to one embodiment of the present invention.
FIG. 2 is a block diagram schematically showing an image processing unit according to one embodiment of the present invention.
FIG. 3 is a flowchart illustrating a procedure for processing an image according to an image processing method according to one embodiment of the present invention.
FIG. 4 is a flowchart illustrating a procedure for matching a supplementary motion vector to a hole block in an image processing method according to one embodiment of the present invention.
FIGS. 5A to 5E are exemplary diagrams for explaining a method of matching a supplementary motion vector to a hole block in an intermediate motion vector field according to an image processing method according to one embodiment of the present invention.
FIG. 6 is an exemplary diagram comparing before and after filling a hole block in an intermediate motion vector field according to an image processing method according to one embodiment of the present invention.
Figure 7 is a block diagram schematically showing an image processing unit according to another embodiment of the present invention.
FIG. 8 is a flowchart illustrating a procedure for processing an image according to an image processing method according to another embodiment of the present invention.
FIG. 9 is a flowchart illustrating a procedure for correcting a supplementary motion vector among image processing methods according to another embodiment of the present invention.
FIG. 10 is an exemplary diagram illustrating a method for determining a compensation supplementary motion vector according to an image processing method according to another embodiment of the present invention.

The advantages and features of the present invention, and the methods for achieving them, will become clearer with reference to the embodiments described in detail below together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms. These embodiments are provided solely to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the invention, and the present invention is defined solely by the scope of the claims.

The shapes, sizes, ratios, angles, numbers, etc. disclosed in the drawings for explaining embodiments of the present invention are exemplary, and therefore the present invention is not limited to the matters illustrated. In addition, in describing the present invention, if it is determined that a detailed description of a related known technology may unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted. When the terms “includes,” “has,” and “consists of” are used in this specification, other parts may be added unless “only” is used. When a component is expressed in the singular, it includes a case where the plural is included unless there is a specifically explicit description.

When interpreting components, it is interpreted as including the error range even if there is no separate explicit description.

When describing a positional relationship, for example, when the positional relationship between two parts is described as ‘on top of’, ‘upper part of’, ‘lower part of’, ‘next to’, etc., one or more other parts may be located between the two parts, unless ‘right away’ or ‘directly’ is used.

When an element or layer is referred to as being on another element or layer, this includes both cases where the other element is directly on top of the other element or layer or where another layer or other element is interposed therebetween.

Although terms like "first" and "second" are used to describe various components, these components are not limited by these terms. These terms are used merely to distinguish one component from another. Therefore, a "first" component referred to below may also be a "second" component within the technical scope of the present invention.

Identical reference numerals throughout the specification refer to identical components.

The size and thickness of each component shown in the drawing are shown for convenience of explanation, and the present invention is not necessarily limited to the size and thickness of the component shown.

The individual features of the various embodiments of the present invention can be partially or wholly combined or combined with each other, and as can be fully understood by those skilled in the art, various technical connections and operations are possible, and each embodiment can be implemented independently of each other or can be implemented together in a related relationship.

Hereinafter, various embodiments of the present invention will be described in detail with reference to the attached drawings.

Fig. 1 is a block diagram schematically illustrating a display device according to one embodiment of the present invention. Referring to Fig. 1, the display device (100) includes a display panel (110), a gate driving circuit (120), a data driving circuit (130), a timing control unit (140), and an image processing unit (150).

Referring to FIG. 1, a display device (100) includes a display panel (110) including a plurality of pixels (P), a gate driving circuit (120) that supplies a gate signal to each of the plurality of pixels (P), a data driving circuit (130) that supplies a data signal to each of the plurality of pixels (P), and a timing control unit (140) that controls the gate driving circuit (120) and the data driving circuit (130). In addition, the display device (100) includes an image processing unit (150) that receives data for an image input to the display panel (110), converts it into a digital signal, and supplies the digital signal and a control signal to the timing control unit (140).

In the display panel (110), a plurality of gate lines (GL) and a plurality of data lines (DL) intersect each other, and each of a plurality of pixels (P) is connected to the gate lines (GL) and the data lines (DL). Specifically, one pixel (P) receives a gate signal from a gate driving circuit (120) through a gate line (GL), receives a data signal from a data driving circuit (130) through a data line (DL), and receives various powers through a power supply line.

The gate driving circuit (120) supplies a gate signal to the gate line (GL) according to a gate control signal (GCS) supplied from the timing control unit (140). Here, the gate signal includes at least one scan signal and a light emission control signal. In FIG. 1, the gate driving circuit (120) is illustrated as being spaced apart from one side of the display panel (110), but the number and arrangement positions of the gate driving circuits (120) are not limited thereto. That is, the gate driving circuit (120) may be arranged on one side or both sides of the display panel (110) in a GIP (Gate In Panel) manner.

The data driving circuit (130) converts image data (RGB) into data voltage according to a data control signal (DCS) supplied from a timing control unit (140), and supplies the converted data voltage to a pixel (P) through a data line (DL).

The timing control unit (140) controls the timing of driving signals input to the display panel (110). Specifically, the timing control unit (140) processes image data (RGB) input from the image processing unit (150) to be appropriate for the size and resolution of the display panel (110) and supplies the processed image data to the data driving circuit (130). In addition, the timing control unit (140) generates a plurality of gate and data control signals (GCS, DCS) using synchronization signals (SYNC), which are timing control signals input from the image processing unit (150), for example, a dot clock signal (DCLK), a data enable signal (DE), a horizontal synchronization signal (Hsync), and a vertical synchronization signal (Vsync). By supplying the generated plurality of gate and data control signals (GCS, DCS) to the gate driving circuit (120) and the data driving circuit (130), the gate driving circuit (120) and the data driving circuit (130) are controlled.

The image processing unit (150) is connected to a timing control unit (140) that controls the timing of driving signals input to the display panel (110). The image processing unit (150) supplies image data (RGB) and a timing control signal to the timing control unit (140).

Additionally, the image processing unit (150) includes a FRUC-capable module for improving image quality based on information about the input image. Specifically, the image processing unit (150) can generate an interpolated image using FRUC to improve the image quality of the input image, and can extract a motion vector for generating the interpolated image.

Here, the image processing unit (150) can process the image input to the display panel (110) in units of frames. That is, the image processing unit (150) can divide the input image into images in units of frames and generate an interpolated image through FRUC based on the images for each frame.

In particular, the image processing unit (150) can perform FRUC by comparing images of adjacent frames. In addition, the image processing unit (150) can extract motion vectors by comparing images of adjacent frames to perform FRUC, and can create and insert an intermediate motion vector field between images of adjacent frames based on these motion vectors. Furthermore, the image processing unit (150) can fill a hole area with a motion vector by matching a supplementary motion vector to at least one hole block present in the intermediate motion vector field. Here, matching a supplementary motion vector to a hole block present in the intermediate motion vector field can be used in a substantially identical sense to filling a hole block present in the intermediate motion vector field with a supplementary motion vector. Accordingly, in the present specification, matching a motion vector to a hole block and filling a motion vector to a hole block can be used interchangeably with the same meaning.

Hereinafter, one of the images of adjacent frames processed by the image processing unit (150) is defined as a reference image, an image corresponding to the next frame of the reference image is defined as a comparison image, and an image corresponding to the previous frame of the reference image is defined as a previous image. The specific configuration and function of the image processing unit (150) will be described later with reference to FIG. 2.

A display device (100) according to one embodiment of the present invention includes an image processing unit (150) capable of generating an interpolated image and extracting a motion vector, thereby generating an intermediate motion vector field between adjacent frames and generating an interpolated image through the intermediate motion vector field, thereby improving the image quality of an image output through a display panel (110).

FIG. 2 is a block diagram schematically illustrating an image processing unit according to an embodiment of the present invention. FIG. 3 is a flowchart illustrating a procedure for processing an image according to an image processing method according to an embodiment of the present invention. FIG. 4 is a flowchart illustrating a procedure for matching a supplementary motion vector to a hole block in an image processing method according to an embodiment of the present invention. FIGS. 5A to 5E are exemplary diagrams for explaining a method for matching a supplementary motion vector to a hole block in an intermediate motion vector field according to an image processing method according to an embodiment of the present invention. FIG. 6 is an exemplary diagram comparing before and after filling a hole block in an intermediate motion vector field according to an image processing method according to an embodiment of the present invention. For convenience of explanation, reference will be made to FIG. 1 for description below.

Referring to FIG. 2, the image processing unit (150) includes a motion vector extraction unit (151), an intermediate motion vector field generation unit (152), a hole filling unit (153), and a motion compensation unit (155). The image processing unit (150) calculates a motion vector based on a reference image and a comparison image, generates an intermediate motion vector field based on the motion vector, fills hole blocks existing in the intermediate motion vector field with supplementary motion vectors, and generates an interpolated image by filling in image data in the reference image pointed to by the motion vector in each block of the intermediate motion vector field.

Referring to FIGS. 2 and 3, the motion vector extraction unit (151) extracts a motion vector corresponding to each of a plurality of blocks constituting the reference image based on the reference image and the comparison image (S310). Next, the intermediate motion vector field generation unit (152) generates an intermediate motion vector field to be inserted between the reference image and the comparison image based on the motion vector (S320).

In Fig. 5a, the previous image (img0), the reference image (img1), and the comparison image (img2) are images corresponding to frames, respectively, in that order. Specifically, with respect to one reference image (img1) in adjacent frames, the previous image (img0) is an image corresponding to a frame earlier than the frame of the reference image (img1), and the comparison image (img2) is an image corresponding to a frame later than the frame of the reference image (img1). That is, in Fig. 5a, images exist in the order of the previous image (img0), the reference image (img1), and the comparison image (img2), and an intermediate motion vector field (IMVF1) is inserted between the reference image (img1) and the comparison image (img2).

Referring to Fig. 5a, the image corresponding to each frame is composed of multiple blocks, and each of the multiple blocks contains image data. Here, the ‘F’ indicated in each block of the image indicates foreground image data, and ‘B’ indicates background image data.

Referring to FIGS. 5A and 5B, the motion vector extraction unit (151) compares the image data of a block in the reference image (img1) with the image data of a block in the comparison image (img2) to extract a motion vector (MV12). Referring to the motion vector (MV12) in FIG. 5B, it can be seen that the foreground moves toward the upper left and the background moves toward the right. That is, the embodiments illustrated in FIGS. 5A to 5E represent images in which the foreground moves toward the upper left and the background moves toward the right, and the following description will be made based on such images.

Referring to FIGS. 5A and 5B , a hole block may be generated in the intermediate motion vector field (IMVF1) depending on the direction in which the foreground and background move in the reference image (img1) and the comparison image (img2). Specifically, as the foreground moves to the upper left and the background moves to the right between the reference image (img1) and the comparison image (img2), new image data exists in a block between the foreground and the background of the comparison image (img2). Accordingly, a hole block may be generated in the intermediate motion vector field (IMVF1) corresponding to a block in which the new image data exists. The intermediate motion vector field (IMVF1) may include a block in which a hole exists depending on the movement of the foreground and background. In this specification, a block in which a hole exists is referred to as a hole block, and an area including a plurality of hole blocks is referred to as a hole area (HR). That is, the intermediate motion vector field generation unit (152) generates an intermediate motion vector field (IMVF1) based on the reference image (img1) and the comparison image (img2), and while generating the intermediate motion vector field (IMVF1), a hole block or hole area (HR) may be generated in the intermediate motion vector field (IMVF1) depending on the movement of the foreground and background.

Next, referring to FIGS. 2 and 3, the hole filling unit (153) matches a supplementary motion vector to each of at least one hole block existing in the intermediate motion vector field based on the previous image, the reference image, and the comparison image corresponding to the previous frame of the reference image (S330).

Specifically, referring to FIG. 4, the hole filling unit (153) can extract a backward motion vector corresponding to each of a plurality of blocks constituting the reference image based on the reference image and the previous image (S331).

Referring to FIGS. 5a and 5b, the hole filling unit (153) compares the image data of a block in the reference image (img1) with the image data of a block in the previous image (img0) to extract a backward motion vector (MV10). That is, the hole filling unit (153) calculates a backward motion vector (MV10), which is a motion vector from the reference image (img1) toward the previous image (img0), for each block of the reference image (img1).

Next, referring to FIG. 4, the hole filling unit (153) can calculate a binary consistency corresponding to each of the plurality of blocks in the reference image based on the sum of the motion vector and the backward motion vector in each of the plurality of blocks (S332).

Referring to FIG. 5b, the hole filling unit (153) calculates the reciprocal of the absolute value of the vector sum in each of the motion vector (MV12) blocks and each of the backward motion vector (MV10) blocks. If the reciprocal of the absolute value of the vector sum in each block is greater than a threshold value, the hole filling unit (153) determines the binary consistency corresponding to the corresponding block as ‘1’, and if the reciprocal of the absolute value of the vector sum is less than the threshold value, the binary consistency corresponding to the corresponding block is determined as ‘0’. Accordingly, each block of the reference image (img1) has a corresponding binary consistency by the hole filling unit (153), and the binary consistency (BC1) of the reference image (img1) can be exemplarily calculated as shown in FIG. 5b.

Next, referring to FIG. 4, the hole filling unit (153) can add up the binary consistency of each adjacent block within a predetermined range centered on a block in the reference image corresponding to the hole block (S333).

Referring to FIG. 5c, the hole filling unit (153) uses windows (W1, W2) having a predetermined range centered on a block corresponding to a hole block in a reference image to sum the binary consistency of each block within the windows (W1, W2). For example, in the first window (W1) and the second window (W2) including 3x3 blocks, the center block is included in the hole area (HR). The value of summing the binary consistency of each block within the first window (W1) based on the center block in the first window (W1) is ‘0’. Similarly, the value of summing the binary consistency of each block within the second window (W2) based on the center block in the second window (W2) is ‘6’.

Here, the hole filling unit (153) can only add up the binary consistency of the remaining blocks excluding the hole block among the adjacent blocks within a predetermined range.

Referring to FIG. 5c, since all blocks included in the first window (W1) are hole blocks included in the hole area (HR), the hole filling unit (153) does not add up the binary consistency of the blocks included in the first window (W1). Meanwhile, the hole filling unit (153) can only add up the binary consistency of the blocks not included in the hole area (HR) among the blocks included in the second window (W2), and the result of adding up only the binary consistency of the blocks that do not correspond to the hole blocks within the range of the second window (W2) is ‘4’. In other words, the result of adding up the binary consistency within the range of the window means the number of blocks with high consistency between the reference image (img1) and the comparison image (img2) among the surrounding blocks of the hole. Furthermore, the motion vector in the block with high consistency is likely to be a foreground motion vector, and the foreground motion vector in the intermediate motion vector field (IMVF1) points to background image data.

Meanwhile, the hole filling unit (153) can expand a predetermined range when the sum of the binary consistency of each adjacent block is less than the threshold consistency.

Referring to FIGS. 5c and 5d, since the first window (W1) includes only blocks included in the hole area (HR), the result of summing the binary consistency within the range of the first window (W1) is ‘0’. The hole filling unit (153) may have a predetermined threshold consistency. For example, the hole filling unit (153) may determine an integer whose ratio of the sum of the binary consistency to the total number of blocks within the predetermined range is close to 25% as the threshold consistency. Here, the standard for determining the threshold consistency may be variously changed. In FIG. 5c, the total number of blocks within the ranges of the first window (W1) and the second window (W2) may be ‘9’, and the threshold consistency may be ‘2’.

Accordingly, since the sum of the binary consistency of each block within the range of the first window (W1) is less than the critical consistency ‘2’, the hole filling unit (153) can recalculate the sum of the binary consistency of the adjacent blocks of the hole block based on the extended window (WE) that extends the first window (W1).

Referring to FIG. 5d, the extended window (WE) includes 5x5 blocks, there are 9 adjacent blocks that are not included in the hole region (HR), and the threshold consistency for the extended window (WE) may be ‘6’. In addition, the sum of the binary consistency of the adjacent blocks that are not included in the hole region (HR) within the extended window (WE) range is ‘7’. Accordingly, the sum of the binary consistency of each block within the extended window (WE) range is greater than the threshold consistency of ‘6’ for the extended window (WE). Since the sum of the binary consistency of each block within the extended window (WE) range is greater than the threshold consistency, the hole filling unit (153) may not expand the extended window (WE) any further.

Next, referring to FIG. 4, the hole filling unit (153) can calculate the average value of the motion vector corresponding to a block in the reference image whose binary consistency of the adjacent block is 1 when the sum of the binary consistency of each adjacent block is greater than the threshold consistency (S334). Next, the hole filling unit (153) can determine the average value of the motion vector as a supplementary motion vector (S335).

Referring to FIGS. 5C and 5E, the sum result of the binary consistency within the range of the second window (W2) is ‘4’, and the sum result of the binary consistency within the range of the second window (W2) is greater than ‘2’, which is the threshold consistency for the second window (W2). In this case, the hole filling unit (153) can calculate the average value of the motion vectors of the reference image (img1) corresponding to blocks whose binary consistency is ‘1’ and is not included in the hole area (HR) within the range of the second window (W2) among the binary consistency (BC1) of the reference image (img1). Accordingly, the hole filling unit (153) can determine the average value of the motion vectors of the reference image (img1) corresponding to blocks whose binary consistency is ‘1’ within the range of the second window (W2) as a supplementary motion vector (CV2) to supplement the hole block corresponding to the center block of the second window (W2).

Referring to FIGS. 5d and 5e, the sum result of the binary consistency within the extended window (WE) range is ‘7’, and the sum result of the binary consistency within the extended window (WE) range is greater than ‘6’, which is the threshold consistency for the extended window (WE). In this case, the hole filling unit (153) can calculate the average value of the motion vectors of the reference image (img1) corresponding to blocks whose binary consistency is ‘1’ and is not included in the hole area (HR) within the extended window (WE) range among the binary consistency (BC1) of the reference image (img1). Accordingly, the hole filling unit (153) can determine the average value of the motion vectors of the reference image (img1) corresponding to blocks whose binary consistency is ‘1’ within the extended window (WE) range as a supplementary motion vector (CVE) to supplement the hole block corresponding to the center block of the extended window (WE).

In Fig. 5e, the intermediate motion vector field (IMVF2) filled with a supplementary motion vector in the hole block is a motion vector field inserted between the reference image (img1) and the comparison image (img2), and includes a motion vector pointing to a block containing data to be obtained from the reference image (img1). Accordingly, the motion vector in each block constituting the intermediate motion vector field (IMVF2) can point in the opposite direction to the motion vector (MV12) of the reference image (img1).

Furthermore, the supplementary motion vector (CV2, CVE) filling the hole block can correspond to the foreground motion vector among the motion vectors (MV12) of the reference image (img1). That is, the supplementary motion vector (CV2, CVE) can be a motion vector that is the opposite direction and has the same size as one of the foreground motion vectors in the reference image (img1). Accordingly, the supplementary motion vector (CV2, CVE) filling the hole block of the intermediate motion vector field (IMVF2) can point to the background image data in the reference image (img1).

Referring to FIGS. 5E and 6, the hole filling unit (153) can fill the hole blocks with the supplementary motion vectors (CV2, CVE) by matching the supplementary motion vectors (CV2, CVE) to the hole blocks in the intermediate motion vector field (IMVF2). In the same manner as described above, the hole filling unit (153) calculates and sums binary consistencies within a window range, which is a predetermined range, centered on each hole block in the hole area (HR), determines a supplementary motion vector, and matches the determined supplementary motion vector to the hole block, thereby filling the hole blocks with the supplementary motion vectors. Accordingly, the hole filling unit (153) can fill all the supplementary motion vectors into each of the hole blocks included in the hole area (HR), as illustrated in FIG. 6.

Next, referring to FIGS. 2 and 3, the motion compensation unit (155) matches the image data in the reference image pointed to by the supplementary motion vector to the hole block of the intermediate motion vector field (S340).

Referring to FIG. 6, the hole filling unit (153) fills the hole area (HR) in the intermediate motion vector field (IMVF) with a supplementary motion vector, and the motion compensation unit (155) fills the image data in the reference image (img1) pointed to by each supplementary motion vector after the hole area (HR) is filled with the supplementary motion vector into the blocks of the intermediate motion vector field (IMVF). Here, the motion compensation unit (155) can fill the image data in the blocks of the intermediate motion vector field (IMVF) by matching the image data pointed to by the supplementary motion vectors matched to each hole block in the hole area (HR) to the blocks of the intermediate motion vector field (IMVF). Accordingly, the motion compensation unit (155) can generate an interpolated image in which no hole area (HR) exists and all blocks are filled with image data.

An image processing method and a display device using the same according to one embodiment of the present invention calculate a binary consistency (BC) based on a motion vector (MV12) for a reference image (img1) and a backward motion vector (MV10) indicating a relationship with a previous image (img0), add up the binary consistency (BC) in adjacent blocks of a hole block existing in an intermediate motion vector field (IMVF), and, when the sum of the binary consistency (BC) is greater than a threshold consistency, determine an average value of motion vectors corresponding to blocks having a binary consistency of ‘1’ as a supplementary motion vector (CV) for the hole block.

Accordingly, the probability that the hole block of the intermediate motion vector field (IMVF) will be filled with the foreground motion vector from the reference image (img1) increases. Furthermore, by filling the hole block in the intermediate motion vector field (IMVF) with the foreground motion vector from the reference image (img1), the probability that the hole block will be filled with the background image data from the reference image (img1) increases. That is, according to the embodiment of the present invention, by filling the hole of the intermediate motion vector field (IMVF) with the background image data indicated by the foreground motion vector from the reference image (img1), the foreground image data can be preserved, the dragging phenomenon of the foreground image can be reduced, and the gridding sensation between the foreground and the background can be reduced. Furthermore, by reducing the dragging phenomenon and the gridding sensation in the image, the image quality can be improved at low cost and with high efficiency.

Fig. 7 is a block diagram schematically showing an image processing unit according to another embodiment of the present invention. Fig. 8 is a flowchart illustrating a procedure for processing an image according to an image processing method according to another embodiment of the present invention. Fig. 9 is a flowchart illustrating a procedure for correcting a supplementary motion vector in an image processing method according to another embodiment of the present invention. Fig. 10 is an exemplary diagram for explaining a method for determining a corrected supplementary motion vector according to an image processing method according to another embodiment of the present invention. The image processing unit (750) of Fig. 7 differs only in a configuration that further includes a supplementary motion vector correction unit (754) in the image processing unit (150) of Fig. 1, and therefore, a duplicate description of the substantially same configurations as the motion vector extraction unit (151), the intermediate motion vector field generation unit (152), the hole filling unit (153), and the motion compensation unit (155) will be omitted. For convenience of explanation, the following description will be made with reference to Fig. 1.

Referring to FIGS. 7 and 8, the image processing unit (750) further includes a supplementary motion vector correction unit (754). After the hole filling unit (753) matches the supplementary motion vector to the hole block existing in the intermediate motion vector field (IMVF) (S830), the supplementary motion vector correction unit (754) corrects the supplementary motion vector in the intermediate motion vector field (S840). Next, the motion compensation unit (755) fills the intermediate motion vector field (IMVF) including the corrected supplementary motion vector with image data in the reference image (img1) pointed to by each motion vector. In particular, the motion compensation unit (755) matches the image data in the reference image (img1) pointed to by the supplementary motion vector to the hole block of the intermediate motion vector field (IMVF) (S850). Accordingly, the motion compensation unit (755) can generate an interpolated image from the intermediate motion vector field (IMVF) in which all holes are filled and no longer exist.

Referring to FIG. 9, the supplementary motion vector correction unit (754) can calculate the characteristic formula of the supplementary motion vector based on the block corresponding to the start point and the block corresponding to the end point of the supplementary motion vector in the reference image (S841).

Referring to FIG. 10, a reference image (1000) includes a foreground area (1001) and a background area (1002), and is composed of a plurality of blocks. Here, a supplementary motion vector (1010) has a block (1003) corresponding to a start point in the background area (1002) and a block (1004) corresponding to an end point in the foreground area (1001). Accordingly, the supplementary motion vector (1010) can instruct to obtain foreground image data by pointing from a block included in the background area (1002) to a block included in the foreground area (1001). By correcting supplementary motion vectors pointing to image data of different areas in this way, an afterimage problem can be reduced.

Referring to FIG. 10, if the coordinates of a block (1003) corresponding to the starting point of a supplementary motion vector are (x1, y1) and the coordinates of a block (1004) corresponding to the end point of the supplementary motion vector are (x2, y2), the supplementary motion vector correction unit (754) can produce a characteristic equation of a supplementary motion vector (1010) such as [Mathematical Expression 1].

Referring to FIG. 9, the supplementary motion vector correction unit (754) can extract a set of blocks through which the supplementary motion vector passes from the reference image (S842).

Referring to FIG. 10, a set of blocks through which a supplementary motion vector (1010) passes can be calculated and extracted according to a characteristic formula. That is, the supplementary motion vector correction unit (754) can extract a set of blocks through which a supplementary motion vector (1010) passes in a background area (1002) (1011) and a set of blocks through which a supplementary motion vector (1010) passes in a foreground area (1001) (1012).

Referring to FIG. 9, the supplementary motion vector correction unit (754) can extract a set of candidate blocks included in an area where a block corresponding to the point in the background area and the foreground area is located, from among a set of blocks, when the start point and the end point are each located in different areas among the background area and the foreground area in the reference image (S843).

Referring to FIG. 10, since the block (1003) corresponding to the point of the supplementary motion vector is located in the background area (1002), only the set of blocks (1011) through which the supplementary motion vector (1010) passes in the background area (1002) is extracted from among the set of blocks extracted by the supplementary motion vector correction unit (754).

Referring to FIG. 9, the supplementary motion vector correction unit (754) can extract a candidate block that is farthest from a block corresponding to a viewpoint among a set of candidate blocks in a reference image (S844).

Referring to FIG. 10, the set of blocks (1011) through which the supplementary motion vector (1010) passes in the background area (1002) extracted by the supplementary motion vector correction unit (754) includes a first background candidate block (1011a), a second background candidate block (1011b), a third background candidate block (1011c), and a fourth background candidate block (1011d). Among the set of blocks (1011) through which the supplementary motion vector (1010) passes in the background area (1002), the candidate block that is farthest from the block (1003) corresponding to the start point of the supplementary motion vector (1010) is the fourth background candidate block (1011d). Accordingly, the supplementary motion vector correction unit (754) can determine the fourth background candidate block (1011d) as the block corresponding to the corrected end point of the supplementary motion vector (1010).

Referring to FIG. 9, the supplementary motion vector correction unit (754) can determine a motion vector that uses a block corresponding to a point in the reference image as a starting point and a candidate block spaced farthest from the starting point as an end point as a corrected supplementary motion vector (S845).

Referring to FIG. 10, if the coordinates of the fourth background candidate block (1011d) determined as the block corresponding to the corrected end point of the supplementary motion vector (1010) are (x3, y3), the supplementary motion vector correction unit (754) can calculate the corrected supplementary motion vector (1020) through the difference between the coordinates (x1, y1) of the block (1003) corresponding to the start point of the supplementary motion vector and the coordinates (x3, y3) of the fourth background candidate block (1011d). That is, the supplementary motion vector correction unit (754) can determine the corrected supplementary motion vector (1020) by [Mathematical Formula 2].

Accordingly, through another embodiment of the present invention, by compensating motion vectors already filled in a hole area in an intermediate motion vector field (IMVF), the motion vectors supplemented in the hole area can be compensated to point to image data capable of reducing afterimages.

In particular, if a motion vector already filled in a hole area in an intermediate motion vector field (IMVF) points to incorrect image data in the reference image, afterimages may be aggravated or image quality may deteriorate due to the image data pointed to by the supplemented motion vector. Therefore, according to another embodiment of the present invention, a motion vector already filled in a hole area in an intermediate motion vector field (IMVF) can be corrected so that it can point to data of the same type of image area at the start and end points.

Accordingly, according to an embodiment of the present invention, by correcting the supplementary motion vector to a motion vector similar to that of the surrounding blocks, the afterimage problem can be reduced and the image quality can be improved at low cost and high efficiency.

An organic light emitting display device according to embodiments of the present invention can be described as follows.

An image processing method according to one embodiment of the present invention includes the steps of extracting a motion vector corresponding to each of a plurality of blocks constituting a reference image based on a reference image and a comparison image, generating an intermediate motion vector field inserted between the reference image and the comparison image based on the motion vector, and matching a supplementary motion vector to each of at least one hole block existing in the intermediate motion vector field based on a previous image corresponding to a previous frame of the reference image, the reference image, and the comparison image. The image processing method according to one embodiment of the present invention can reduce a lattice effect existing between a foreground and a background and a dragging phenomenon of the foreground by filling a region where a hole occurs in the intermediate motion vector field with a foreground motion vector.

According to another feature of the present invention, the supplementary motion vector can correspond to a foreground motion vector among the motion vectors.

According to another feature of the present invention, the step of correcting a supplementary motion vector in an intermediate motion vector field may be further included.

According to another feature of the present invention, the step of matching image data in a reference image pointed to by a supplementary motion vector to a hole block of an intermediate motion vector field may be further included.

According to another feature of the present invention, the image data in the reference image may be background image data in the reference image.

According to another feature of the present invention, the step of correcting the supplementary motion vector may include the steps of: calculating a characteristic formula of the supplementary motion vector based on a block corresponding to a start point and a block corresponding to an end point of the supplementary motion vector in a reference image; extracting a set of blocks through which the supplementary motion vector passes in the reference image; extracting a set of candidate blocks included in an area where a block corresponding to the start point is located among the background area and the foreground area, when the start point and the end point are each located in different areas among the background area and the foreground area in the reference image, from the set of blocks; extracting a candidate block that is spaced farthest from the block corresponding to the start point among the set of candidate blocks in the reference image; and determining a motion vector that uses the block corresponding to the start point in the reference image as a start point and the candidate block spaced farthest as an end point, as a corrected supplementary motion vector.

According to another feature of the present invention, an area where a block corresponding to a viewpoint is located may be a background area, and an area where a block corresponding to an end point is located may be a foreground area.

According to another feature of the present invention, the step of matching supplementary motion vectors may include the steps of extracting a backward motion vector corresponding to each of a plurality of blocks constituting the reference image based on the reference image and the previous image, calculating a binary consistency corresponding to each of the plurality of blocks in the reference image based on the sum of the motion vectors and backward motion vectors in each of the plurality of blocks, the step of summing the binary consistency of each of the adjacent blocks within a predetermined range centered on a block in the reference image corresponding to a hole block, the step of calculating an average value of motion vectors corresponding to blocks in the reference image whose binary consistency of the adjacent blocks is 1 when the sum of the binary consistency of each of the adjacent blocks is greater than a threshold consistency, and the step of determining the average value of the motion vectors as the supplementary motion vector.

According to another feature of the present invention, the step of matching supplementary motion vectors may further include the step of extending a predetermined range when the sum of binary consistencies of each adjacent block is less than a threshold consistency.

According to another feature of the present invention, the step of summing the binary consistency may be a step of summing only the binary consistency of the remaining blocks excluding the hole blocks among the adjacent blocks within a predetermined range.

A display device according to another embodiment of the present invention includes a display panel and an image processing unit that processes an image input to the display panel. The image processing unit includes a motion vector extraction unit that extracts a motion vector corresponding to each of a plurality of blocks constituting a reference image based on a reference image and a comparison image, an intermediate motion vector field generation unit that generates an intermediate motion vector field to be inserted between the reference image and the comparison image based on the motion vector, and a hole filling unit that matches a supplementary motion vector to each of at least one hole block existing in the intermediate motion vector field based on a previous image corresponding to a previous frame of the reference image, the reference image, and the comparison image. The display device according to another embodiment of the present invention can reduce afterimages of an image and improve image quality at low cost and high efficiency by correcting a motion vector filled in a hole in the intermediate motion vector field.

According to another feature of the present invention, the supplementary motion vector can correspond to a foreground motion vector among the motion vectors.

According to another feature of the present invention, the present invention may further include a supplementary motion vector correction unit for correcting a supplementary motion vector in an intermediate motion vector field.

According to another feature of the present invention, the present invention may further include a motion compensation unit that matches image data in a reference image pointed to by a supplementary motion vector to a hole block of an intermediate motion vector field.

Although the embodiments of the present invention have been described in more detail with reference to the attached drawings, the present invention is not necessarily limited to these embodiments, and various modifications may be implemented without departing from the technical spirit of the present invention. Therefore, the embodiments disclosed in the present invention are not intended to limit the technical spirit of the present invention, but to explain it, and the scope of the technical spirit of the present invention is not limited by these embodiments. The protection scope of the present invention should be interpreted by the following claims, and all technical ideas within a scope equivalent thereto should be interpreted as being included in the scope of the rights of the present invention.

100: Display device
110: Display panel
120: Gate drive circuit
130: Data drive circuit
140: Timing control unit
150: Image processing unit
151, 751: Motion vector extraction unit
152, 752: Intermediate motion vector field generation unit
153, 753: Hole filling section
155, 755: Motion compensation unit
754: Supplementary motion vector compensation unit
1000: Reference image
1001: Foreground area
1002: Background area
1003: Block corresponding to the point of the supplementary motion vector
1004: Block corresponding to the end point of the supplementary motion vector
1010: Supplementary motion vector
1011: A set of blocks through which the supplementary motion vector passes in the background area.
1011: A set of blocks through which the supplementary motion vector passes in the foreground area.
1020: Compensation supplementary motion vector

Claims (14)

A step of extracting a motion vector corresponding to each of a plurality of blocks constituting the reference image based on a reference image and a comparison image corresponding to a next frame of the reference image;
A step of generating an intermediate motion vector field inserted between the reference image and the comparison image based on the motion vector; and
A step of matching a supplementary motion vector to each of at least one hole block existing in the intermediate motion vector field based on a previous image corresponding to a previous frame of the reference image, the reference image, and the comparison image,
The step of matching the supplementary motion vector to each of the at least one hole blocks is:
A step of extracting a retrograde motion vector corresponding to each of a plurality of blocks constituting the reference image based on the reference image and the previous image;
A step of calculating a binary consistency corresponding to each of the plurality of blocks in the reference image based on the sum of the motion vector and the retrograde motion vector in each of the plurality of blocks;
A step of summing the binary consistency of each adjacent block within a predetermined range centered on a block in the reference image corresponding to the hole block;
When the sum of the binary consistency of each of the adjacent blocks is greater than the threshold consistency, a step of calculating the average value of the motion vector corresponding to a block in the reference image in which the binary consistency of the adjacent block is 1; and
A step of determining the average value of the above motion vector as the supplementary motion vector is included.
The step of summing the above binary consistency is an image processing method in which the binary consistency of the remaining blocks excluding the hole block among the adjacent blocks within the predetermined range is summed. In the first paragraph,
An image processing method wherein the above supplementary motion vector corresponds to a foreground motion vector among the above motion vectors. In the first paragraph,
An image processing method further comprising a step of correcting the supplementary motion vector in the intermediate motion vector field. In paragraph 1 or 3,
An image processing method further comprising a step of matching image data in the reference image pointed to by the supplementary motion vector to the hole block of the intermediate motion vector field. In paragraph 4,
An image processing method wherein the image data in the above reference image is background image data in the above reference image. In the third paragraph,
The step of correcting the above supplementary motion vector is:
A step of calculating a characteristic formula of the supplementary motion vector based on a block corresponding to a start point and a block corresponding to an end point of the supplementary motion vector in the reference image;
A step of extracting a set of blocks through which the supplementary motion vector passes from the reference image;
A step of extracting a set of candidate blocks included in an area where a block corresponding to the point in the background area and the foreground area is located, from among the set of blocks, when each of the start point and the end point in the reference image is located in a different area among the background area and the foreground area;
A step of extracting a candidate block that is farthest from the block corresponding to the point in time among the set of candidate blocks in the reference image; and
An image processing method comprising a step of determining a motion vector as a compensation supplementary motion vector, with the block corresponding to the point in the reference image as the starting point and the candidate block spaced farthest apart as the ending point. In paragraph 6,
An image processing method, wherein the area where the block corresponding to the above point is located is the background area, and the area where the block corresponding to the above end point is located is the foreground area. In the first paragraph,
The step of calculating the above binary consistency is:
A step of calculating the reciprocal of the absolute value of the sum of the motion vector and the retrograde motion vector in each of the plurality of blocks; and
An image processing method, comprising the step of determining the binary consistency corresponding to the block as 1 if the reciprocal of the absolute value is greater than a threshold value, and determining the binary consistency for the block as 0 if the reciprocal of the absolute value is less than the threshold value. In the first paragraph,
An image processing method, wherein the step of matching the supplementary motion vector further includes the step of expanding the predetermined range when the sum of the binary consistencies of each of the adjacent blocks is less than a threshold consistency. delete display panel; and
Includes an image processing unit that processes an image input to the display panel,
The above image processing unit,
A motion vector extraction unit that extracts a motion vector corresponding to each of a plurality of blocks constituting the reference image based on a reference image and a comparison image corresponding to a next frame of the reference image;
An intermediate motion vector field generation unit that generates an intermediate motion vector field to be inserted between the reference image and the comparison image based on the motion vector; and
A hole filling unit is included that matches a supplementary motion vector to each of at least one hole block existing in the intermediate motion vector field based on a previous image corresponding to a previous frame of the reference image, the reference image, and the comparison image,
The above hole filling part,
Extracting a retrograde motion vector corresponding to each of the plurality of blocks based on the reference image and the previous image,
Based on the sum of the motion vector and the backward motion vector in each of the plurality of blocks, a binary consistency corresponding to each of the plurality of blocks in the reference image is calculated,
Summing the binary consistency of the remaining blocks excluding the hole block among the adjacent blocks within a predetermined range centered on the block in the reference image corresponding to the hole block,
A display device that determines the average value of the motion vectors corresponding to blocks in which the binary consistency of each of the adjacent blocks is 1 in the reference image as the supplementary motion vector when the sum of the binary consistency of each of the adjacent blocks is greater than the threshold consistency. In Article 11,
A display device in which the above supplementary motion vector corresponds to a foreground motion vector among the above motion vectors. In Article 11,
A display device further comprising a supplementary motion vector correction unit for correcting the supplementary motion vector in the intermediate motion vector field. In claim 11 or 13,
A display device further comprising a motion compensation unit that matches image data in the reference image pointed to by the supplementary motion vector to the hole block of the intermediate motion vector field.