CN101459761B - Image processing method and related device - Google Patents

Abstract

The invention provides an image processing method. The image processing method comprises the steps of completing a block comparison operation only according to a plurality of image blocks in the horizontal direction without referring to image blocks in the vertical direction or other directions and executing an image processing operation according to a result of the block comparison operation.

Description

Image processing method and related device

technical field

The present invention relates to an image processing mechanism, in particular to an image processing method and a related device capable of reducing circuit cost and computational complexity.

Background technique

The known image processing mechanism considers a plurality of image blocks in a two-dimensional predetermined range in a previous image when performing a block matching operation, and finds out from the plurality of image blocks that are consistent with a certain image in the current image. The image block most similar to the image block is used to determine a motion vector. However, in practice, a large number of pixels need to be calculated when performing the block comparison operation, and the calculation program is quite complicated, and the line buffer (line buffer) for temporarily storing pixel data is limited due to the need to temporarily store a large amount of pixel data. As a result, more line buffers need to be used. Generally speaking, about 10-20 line buffers are actually needed in this case. Therefore, the overall circuit cost of the known image processing mechanism will be increased due to the use of more line buffers, and a longer computing time is required before the block comparison operation is completed.

Contents of the invention

Therefore, one object of the present invention is to provide an image processing method and a related device capable of reducing the overall circuit cost and computational complexity at the same time, so as to solve the above-mentioned problems.

According to an embodiment of the present invention, it discloses an image processing method. The image processing method includes only a previous image frame/field and two adjacent image frames/fields of a primary image frame/field that are horizontally identical in position corresponding to a current image block of a current image frame/field performing a block comparison operation on image blocks in the image block without referring to vertical or other direction image blocks; and performing an image processing operation in the current image frame/field according to a result of the block comparison operation; wherein The image processing operation is an image interpolation operation, and the step of performing the image interpolation operation according to the result of the block comparison operation includes: when the result indicates that the first group of image blocks and the second group of images When the block contrast difference between the image blocks is greater than a predetermined threshold, directly repeating an image block corresponding to the position to be interpolated in the previous image frame/field; and when the result indicates the first group of image areas When a minimum block contrast difference among the contrast differences between the block and the second group of image blocks is not greater than the predetermined threshold, the to-be-interpolated image is executed according to a motion vector corresponding to the minimum block contrast difference position for image interpolation.

According to an embodiment of the present invention, it further discloses an image processing device. The image processing device includes a calculation circuit, which is used only according to a previous image frame/field and two adjacent image frames/fields corresponding to the current image block of a current image frame/field in the horizontal direction. image blocks in the image frame/field without referring to image blocks in vertical or other directions to complete a block comparison operation; and an image processing circuit, coupled to the calculation circuit, according to a block comparison operation As a result, an image processing operation is performed in the current image frame/field; wherein the image processing operation is an image interpolation operation; when the result indicates the difference between the first group of image blocks and the second group of image blocks When the block contrast difference is greater than a predetermined threshold, the image processing circuit will directly repeat an image block corresponding to the position to be interpolated in the previous image frame/field; and when the result indicates the first group of image areas When a minimum block contrast difference value among the contrast difference values between the block and the second group of images is not greater than the predetermined threshold, the image processing circuit will process the image according to a motion vector corresponding to the minimum block contrast difference value. The image interpolation operation is performed on the position to be interpolated.

According to an embodiment of the present invention, it further discloses an image processing method. The image processing method includes only according to the position corresponding to the current image block of a current image frame/field which is the same in the horizontal direction in a previous image frame/field and a primary image frame/field in two adjacent image frames/fields complete a block comparison operation without referring to image blocks in vertical or other directions, so as to generate a block comparison result corresponding to a position to be interpolated; and according to the block comparison result in the current image An image interpolation operation is performed in a frame/field; wherein the step of performing image interpolation operation according to the block comparison result includes: When the contrast difference values of multiple blocks calculated by one group of image blocks and the second group of image blocks are greater than a predetermined critical value, directly repeat a step corresponding to the position to be interpolated in the previous image frame/field. an image block; and when the block comparison result indicates that a minimum block contrast difference among the contrast differences between the first group of image blocks and the second group of image blocks is not greater than the predetermined critical value , performing an image interpolation operation on the position to be interpolated according to a motion vector corresponding to the minimum block contrast difference.

According to an embodiment of the present invention, it further discloses an image processing device. The image processing device includes a calculation circuit, which is used only according to a previous image frame/field and two adjacent image frames/fields corresponding to the current image block of a current image frame/field in the horizontal direction. performing a block comparison operation on image blocks in the image frame/field without referring to image blocks in vertical or other directions, so as to generate a block comparison result corresponding to a position to be interpolated; and an image processing circuit, Coupled to the calculation circuit, an image interpolation operation is performed in the current image frame/field according to the block comparison result; wherein when the block comparison result indicates that according to the first horizontal position having the same horizontal position as the position to be interpolated When a plurality of block contrast differences calculated by a group of image blocks and a second group of image blocks are greater than a predetermined critical value, directly repeat a step corresponding to the position to be interpolated in the previous image frame/field. Image block; and when the block comparison result indicates that a minimum block contrast difference among the plurality of block contrast differences is not greater than the predetermined critical value, according to a movement corresponding to the minimum block contrast difference Vector to perform image interpolation operation on the position to be interpolated.

Description of drawings

FIG. 1 is a schematic diagram of an image processing device according to an embodiment of the present invention.

FIG. 2 is a schematic diagram of the calculation circuit shown in FIG. 1 performing block comparison on two frames.

FIG. 3 is a schematic diagram of the calculation circuit shown in FIG. 1 performing block comparison on two fields.

Description of main component symbols

100 image processing device 105 memory 110 computing circuit 115 image processing circuit

Detailed ways

Please refer to FIG. 1 , which is a schematic diagram of an image processing device 100 according to an embodiment of the present invention. The image processing device 100 includes a memory 105 , a computing circuit 110 and an image processing circuit 115 , wherein the computing circuit 110 also includes a plurality of line buffers (not shown in FIG. 1 ). The memory 105 is used to store the input data DATA _in received from the outside (the pixel data of multiple frames (frame) or field (field)), and the calculation circuit 110 is coupled to the memory 105 and only based on the level of the input data DATA _in The image processing circuit 115 is coupled to the calculation circuit 110 and executes an image processing circuit 115 according to the result of the block comparison operation. Processing operations (such as image interpolation); the calculation circuit 110 and the image processing circuit 115 can actually jointly form a motion estimation/compensation circuit unit. As mentioned above, since the calculation circuit 110 only completes the block comparison operation based on multiple image blocks in the horizontal direction without referring to image blocks in the vertical or other directions, the image processing device 100 is similar to the above-mentioned known image processing mechanism. Compared, a large number of pixel calculations can be avoided and the complexity of calculations can be reduced. At the same time, the number of line buffers can be reduced to reduce the overall circuit cost. The reason for this is that, statistically speaking, there are more objects that move horizontally in the image than vertically. For example, when interpolating image blocks in a frame/field at a certain point in time , perform a block comparison operation to find out which image block in the frame/field at the previous time point is most similar to which image block in the frame/field at the next time point to find out the interpolated image The motion vector required for the block, and there is a high probability that the found motion vector is a horizontal vector, that is, the object corresponding to the image block moves horizontally at this time. Therefore, this embodiment only refers to the image blocks in the horizontal direction to perform the block comparison operation to save the overall circuit cost, and at the same time avoid wasting computing time and reduce computing complexity; in fact, the computing circuit 110 in this embodiment only uses Only 2-3 line buffers can achieve the purpose of performing block comparison operation with reference to image blocks in the horizontal direction, which is considerably reduced compared with the number of line buffers generally required by conventional image processing mechanisms.

In this embodiment, an example of an image interpolation operation is used for illustration, but of course the present invention is not limited thereto. The calculation circuit 110 temporarily stores the pixel data required in the input data DATA _in of the memory 105 in its internal line buffer one by one when performing block comparison, and accumulates the calculated pixel data of each corresponding position. Pixel difference to obtain a block matching difference. Please refer to FIG. 2 . FIG. 2 is a schematic diagram illustrating the operation of the calculation circuit 110 shown in FIG. 1 performing a block comparison operation on two frames. As shown in Figure 2, the input data DATA _in is formed by a series of frames at this time, and in this case, if the number of line buffers is to be saved as much as possible, the calculation circuit 110 still needs to include at least two line buffers, and the calculation The operation of the circuit 110 is described in FIG. 2 using a two-line buffer as an example.

As mentioned above, computing circuit 110 in this example includes wired buffers L ₁ and L ₂ . For example, when the calculation _circuit 110 intends to _calculate a block contrast _{difference (} such as the sum of absolute differences of pixels ( Sum of Absolute Differences, SAD)), since the line buffers _L1 and _L2 can at most simultaneously store the pixel data corresponding to the same horizontal position in the frame Fn _-1 and _Fn , for example, the line buffer _L1 can Simultaneously store pixel data in multiple image blocks at the same horizontal position as image block B _n-1 , while the line buffer _L2 can simultaneously store pixel data in multiple image blocks at the same horizontal position as image block B _n Pixel data (that is, a plurality of image blocks referred to by the calculation circuit 110 are generated in the line buffers _L1 and _L2 ), therefore, the calculation circuit 110 can calculate the corresponding image blocks B _n-1 and B _{n one} by one The pixel difference of the position is accumulated, and finally the block comparison difference between the image blocks B _n-1 and B _n can be calculated. Please note that the horizontal positions of the image blocks B _n-1 and B _n are the same. Continuing to perform block comparison for image blocks at the same horizontal position in this way, a minimum block contrast difference must be calculated, and the motion vector corresponding to the minimum block contrast difference helps to determine the difference between the two frames F _{n -1} and F _n perform the image interpolation operation on an image block required by the position to be interpolated, wherein the position to be interpolated is at the same horizontal position as the image block B _n−1 /B _n . Of course, the line buffers _L1 and _L2 can also be designed to store at most the pixel data corresponding to the same horizontal scanning line in the frames Fn _-1 and _Fn at the same time. In other words, in order to save circuit cost, Then the line buffers _L1 and _L2 can be designed to only store pixel data on the same scanning line at most, and because the calculation circuit 110 can calculate the block contrast difference by accumulating pixel differences, the calculation circuit 110 operation will not be affected. In addition, the present invention does not limit the number of pixels and their area in the above-mentioned image blocks B _n-1 and B _n , for example, the image blocks B _n-1 and B _n may be blocks with 1*m pixels (that is, the image area formed by the intersection of 1 horizontal line and m vertical lines), rather than being limited to image blocks with n*m pixels as shown in Figure 2 (n and m are integers greater than 1, They respectively represent the number of pixels in the vertical direction and the horizontal direction); for an image block with 1*m pixels, only two lines of buffers are needed to complete the block comparison operation, and at this time, two lines of buffers At most, the device will simultaneously store the pixel data corresponding to the position of the same horizontal scanning line in two frames, and for the sake of brevity, the rest of the operations will not be described in detail.

Furthermore, if the result of the block comparison operation of the calculation circuit 110 indicates that the multiple block contrast differences calculated based on the image blocks in the horizontal direction are all greater than a predetermined threshold V _th , then the image processing circuit 115 determines the above-mentioned When the image of the position to be interpolated will directly repeat the image block at the same position as the position to be interpolated in a previous image (ie frame); When the small block contrast difference is not greater than the predetermined threshold V _th , the image processing circuit 115 will perform image interpolation on the position to be interpolated according to the motion vector corresponding to the smallest block contrast difference, that is, refer to the preceding description Move vectors for image interpolation. If the comparison difference values of the above multiple blocks are all greater than the predetermined threshold V _th , it may indicate that the object corresponding to the position to be interpolated should move vertically or in other directions instead of horizontally in the previous/next image. Because in this case, directly using the motion vector to perform image interpolation will cause the interpolated image frame to be greatly distorted, therefore, the image processing circuit 115 chooses to directly repeat a previous image that is the same as the position to be interpolated. position of the image block without reference to the motion vector.

In another example, if the input data DATA _in is composed of a series of image fields, besides the line buffers L ₁ and L ₂ , the calculation circuit 110 also includes a line buffer L ₃ . Wherein the line buffer _L1 stores at most the pixel data at the first horizontal line position in a first image (which is field data) at the same time, and the line buffer _L2 stores at most the pixel data at the first horizontal line position in a second image (also field data) at the same time. The pixel data at the previous adjacent horizontal line position of a horizontal line position, and the line buffer L ₃ at most simultaneously stores the pixel data at the next adjacent horizontal line position of the first horizontal line position in the second image. The calculation circuit 110 will first calculate the pixel data in the line buffers _L2 and _L3 to generate at least one temporary pixel data, and the plurality of image blocks in the horizontal direction referenced by the calculation circuit 110 will be based on the first image Pixel data at the position of the first horizontal line and this temporary pixel data are generated.

For example, please refer to FIG. 3 . FIG. 3 is a schematic diagram of the calculation circuit 110 shown in FIG. 1 performing block comparison for two fields. In the following, a line buffer that can only store pixel data on the same horizontal scanning line at most is described below, however, this is not a limitation of the present invention. Of course, a line buffer capable of simultaneously storing pixel data on multiple scan lines at the same horizontal position can also be implemented, and details are not repeated here to avoid excessive length. When the calculation circuit 110 calculates a block contrast difference for the image block B _{n- 1 ′ of the field F n} -1 ′ and the image block B _n ′ of the field F _{n ′} , the line buffer L ₁ at most Simultaneously store the pixel data at the position of the horizontal scanning line SL ₁ in the image block Bn _{-1 '} in the field Fn-1', and the line buffers _L2 and _L3 at most store the image in the field _Fn ' at the same time The pixel data on _the horizontal scanning line _SL2 adjacent to the horizontal scanning line SL1 in the block _Bn ' and the pixel data on the horizontal scanning line _SL3 next adjacent to the horizontal scanning line SL1. Since the fields Fn _-1 ' and _Fn ' belong to the even field and the odd field respectively, in this embodiment, in order to be able to calculate the contrast difference of the block, the calculation circuit 110 will firstly calculate the line buffer L ₂ and _L3 pixel data are averaged to generate at least _one temporary pixel data, and then the calculation circuit ₁₁₀ generates multiple Of course, the temporary pixel data can also be generated by weighted average or other calculation methods. Then, in order to calculate other pixel difference values, the line buffer _L2 will be changed to store the pixel data at the position of the horizontal scanning line SL ₄ in the image block Bn _-1 ' in the field Fn _-1 ', and the calculation circuit will then At least one temporary pixel data generated by averaging _{the pixel data at the position of the horizontal scanning line SL1 and the pixel data at the position of the horizontal scanning line SL4 and stored in the pixel at the position of the horizontal scanning line SL3 in the line} buffer L3 Data to calculate the pixel difference, such alternate operation will be able to calculate all the pixel difference. Similarly, by accumulating pixel difference values, the final calculation circuit 110 can also calculate the block comparison difference between the image blocks B _n-1 ′ and B _{n ′} . In addition, the present invention does not limit the number of pixels and their areas in the image blocks B _{n-1 ′} and B _{n ′} . For example, the image blocks Bn _-1 ' and _Bn ' may be an image area composed of 1 horizontal line and m vertical lines, instead of being limited to n horizontal lines and m vertical lines as shown in FIG. 3 composed image area. For an image area composed of 1 horizontal line and m vertical lines, only three line buffers are needed to complete the block comparison operation, and the rest of the operation process is similar to the operation process in the above example, in order to avoid The instructions are too lengthy and will not be repeated here.

In addition, the above-mentioned calculation of a pixel difference for odd and even image fields can also be implemented in different ways, for example, when a pixel data P ₁ at the position of the horizontal scanning line SL ₁ in the image block Bn _-1 ' falls into the image Between the two pixel data P ₂ and P ₃ corresponding to the same vertical position on the horizontal scanning lines SL ₂ and SL ₃ in the block B _n ', the pixel difference will be zero, because the two pixels composed of one odd and one even In a continuous image field, if a pixel value in the odd/even image field falls between two pixel values corresponding to the vertical up and down positions in another image field, it may indicate that the image corresponding to the pixel value is still, so The above-mentioned pixel difference value should be regarded as zero; otherwise, when the pixel data P ₁ does not fall between the pixel data P ₂ and P ₃ and the pixel difference value of the pixel data P ₁ and P ₂ is smaller than the pixel data P ₁ and P ₃ The pixel difference value is the pixel difference value of the pixel data P ₁ and P ₂ .

In addition, it is also possible to sequentially store frames/fields in the input data DATA _in by adding a plurality of additional memories instead of only using a single memory 105 . The advantage is that when the calculation circuit 110 occupies the bandwidth of a certain memory to read the pixel data stored in the memory to its internal line buffer, another memory can be used because its bandwidth is idle. Receive the frame/field of the input data DATA _in . In this way, the operation of receiving the frame/field of the input data DATA _in and the calculation circuit 110 reading the required pixel data from the memory will not simultaneously occupy the same memory, so the operation speed of the image processing device 100 can be accelerated.

The above descriptions are only preferred embodiments of the present invention, and all equivalent changes and modifications made according to the claims of the present invention shall fall within the scope of the present invention.

Claims (11)

1. An image processing method comprising: Only based on the image blocks in a previous image frame/field and two adjacent image frames/fields of the primary image frame/field whose position corresponding to the current image block of a current image frame/field is the same in the horizontal direction perform a block comparison operation without reference to vertical or other image blocks; and performing an image processing operation in the current image frame/field according to a result of the block comparison operation; Wherein the image processing operation is an image interpolation operation, and the step of performing the image interpolation operation according to the result of the block comparison operation includes: When the results indicate that the block contrast differences between the first group of image blocks and the second group of image blocks are greater than a predetermined threshold, directly repeat a step corresponding to the position to be interpolated in the previous image frame/field. image blocks; and When the result indicates that a minimum block contrast difference among the contrast differences between the first group of image blocks and the second group of image blocks is not greater than the predetermined threshold, according to the minimum block contrast difference A corresponding motion vector is used to perform image interpolation operation on the position to be interpolated. 2. An image processing device, comprising: A calculation circuit, used only for a previous image frame/field and two adjacent image frames/fields of the primary image frame/field based on the position corresponding to the current image block of a current image frame/field in the same horizontal direction performing a block comparison operation on image blocks in the image block without reference to vertical or other image blocks; and an image processing circuit, coupled to the calculation circuit, to perform an image processing operation in the current image frame/field according to a result of the block comparison operation; Wherein the image processing operation is an image interpolation operation; when the result indicates that the block contrast difference between the first group of image blocks and the second group of image blocks is greater than a predetermined threshold, the image processing circuit will directly repeat an image block corresponding to the position to be interpolated in the previous image frame/field; When the contrast difference of the small block is not greater than the predetermined threshold, the image processing circuit performs an image interpolation operation on the position to be interpolated according to a motion vector corresponding to the minimum block contrast difference. 3. The image processing device according to claim 2, further comprising: A memory, coupled to the calculation circuit, is used to store a plurality of input frame/field data. 4. The image processing device as claimed in claim 2, wherein the calculation circuit comprises a first line buffer and a second line buffer, and the first and second line buffers store the previous image frame/field and The pixel data corresponding to the position of the same horizontal line in the sub-image frame/field, wherein the previous image frame/field and the sub-image frame/field are frame data, and the first group of image blocks and the second group of image blocks generated in the first line buffer and the second line buffer. 5. The image processing device as claimed in claim 2, wherein the calculation circuit comprises: A first line buffer, used to simultaneously store at most pixel data on a first horizontal line position of the previous image frame/field; A second line buffer, used to simultaneously store at most the pixel data on the position of the first horizontal line adjacent to the position of the first horizontal line of the sub-image frame/field; and A third line buffer, used to simultaneously store at most the pixel data on the next adjacent horizontal line position of the first horizontal line position of the sub-image frame/field; Wherein the calculation circuit first calculates the pixel data in the second line buffer and the third line buffer to generate at least one temporary pixel data, the first group of image blocks and the second group of image blocks according to the previous image frame/ The pixel data at the position of the first horizontal line of the field and the temporary pixel data are generated, and the previous image frame/field and the secondary image frame/field are field data. 6. An image processing method, comprising: Only based on the image blocks in a previous image frame/field and two adjacent image frames/fields of the primary image frame/field whose position corresponding to the current image block of a current image frame/field is the same in the horizontal direction performing a block comparison operation without reference to image blocks in vertical or other directions to generate a block comparison result corresponding to a position to be interpolated; and performing an image interpolation operation in the current image frame/field according to the block comparison result; The steps of performing image interpolation operation according to the block comparison result include: When the block comparison result indicates that the multiple block contrast differences calculated according to the first group of image blocks and the second group of image blocks having the same horizontal position as the position to be interpolated are all greater than a predetermined When the critical value is reached, an image block corresponding to the position to be interpolated in the previous image frame/field is directly repeated; and When the block comparison result indicates that a minimum block contrast difference among the contrast differences between the first group of image blocks and the second group of image blocks is not greater than the predetermined critical value, according to the minimum area An image interpolation operation is performed on the position to be interpolated with a motion vector corresponding to the block comparison difference. 7. The image processing method as claimed in claim 6, wherein the block comparison operation refers to the first group of image blocks in the horizontal direction and the second group of image blocks and does not refer to the image blocks in the vertical or other directions. 8. An image processing device, comprising: A calculation circuit, used only for a previous image frame/field and two adjacent image frames/fields of the primary image frame/field based on the position corresponding to the current image block of a current image frame/field in the same horizontal direction performing a block comparison operation on the image blocks in the vertical or other directions without referring to image blocks in vertical or other directions, so as to generate a block comparison result corresponding to a position to be interpolated; and An image processing circuit, coupled to the calculation circuit, performs an image interpolation operation in the current image frame/field according to the block comparison result; Wherein when the block comparison result indicates that the multiple block contrast differences calculated according to the first group of image blocks and the second group of image blocks having the same horizontal position as the position to be interpolated are all greater than a predetermined When the critical value is reached, directly repeat an image block corresponding to the position to be interpolated in the previous image frame/field; and when the block comparison result indicates a minimum block contrast difference among the plurality of block contrast differences When it is not greater than the predetermined critical value, the image interpolation operation is performed on the position to be interpolated according to a motion vector corresponding to the minimum block contrast difference. 9. The image processing device as claimed in claim 8, wherein the calculation circuit completes the region with reference to the first group of image blocks in the horizontal direction and the second group of image blocks without reference to the image blocks in the vertical or other directions Block comparison operation. 10. The image processing device as claimed in claim 9, wherein the computing circuit comprises a first line buffer and a second line buffer, and the first and second line buffers store the first group of image regions at most simultaneously The block and the second group of image blocks correspond to pixel data at the same horizontal position, wherein the first group of image blocks and the second group of image blocks are frame data. 11. The image processing device as claimed in claim 9, wherein the computing circuit comprises: A first line buffer, used to simultaneously store at most the pixel data on a first horizontal position of the previous image frame/field; A second line buffer, used to simultaneously store at most the pixel data on the previous adjacent horizontal position of the first horizontal position of the sub-image frame/field; and A third line buffer, used to simultaneously store at most the pixel data on the next adjacent horizontal position of the first horizontal position of the sub-image frame/field; Wherein the calculation circuit first calculates the pixel data in the second line buffer and the third line buffer to generate at least one temporary pixel data, and a plurality of image blocks are based on the pixels on the first horizontal line position of the previous image frame/field data and the temporal pixel data are generated, and the previous image frame/field and the secondary image frame/field are field data.