CN102238380B - Method and system for layered motion estimation - Google Patents

Abstract

A method and system for hierarchical motion estimation. The reference frame and the current frame are down-sampled and the down-sampled reference frame is stored. A coarse motion vector map (MV map) is generated based on the downsampled reference frame and the downsampled current frame. Scan lines adjacent to a central scan line corresponding to a down-sampled scan line of the down-sampled reference frame are captured and stored. A fine Motion Vector (MV) map is generated based on the coarse MV map, the current frame and the stored scan line adjacent to the center scan line.

Description

Method and system for layered motion estimation

technical field

The present invention relates to image processing, in particular to a kind of hierarchical motion estimation (hierarchical motion estimation).

Background technique

When performing motion estimation (ME) to generate a motion vector (MV), pixel data of a reference frame (such as a previous frame) needs to be retrieved from an external memory device. However, due to the limitation of the transmission bandwidth of the memory device, the pixel data cannot be directly retrieved by the memory device (such as double data rate synchronous dynamic random access memory (DDR SDRAM)) in real time.

In order to solve the above problems, an internal memory (such as cache memory) of the integrated circuit can be used to temporarily store a part of the reference frame (such as the search range). However, for high-definition (hereinafter referred to as HD) images (the resolution of which can be 1920×1080), the capacity of the internal memory is insufficient. For example, when the search range is 1/10 of an HD image, a memory capacity of 108 (ie, 1080*(1/10)) scanning lines is required, or equivalent to 1658880 (ie, 108*1920*8) bit capacity.

Since traditional motion estimation systems or methods cannot be effectively applied to images with higher resolution, it is urgent to propose a novel mechanism applicable to images with higher resolution, such as HD images.

Contents of the invention

In view of the above, one of the objectives of the embodiments of the present invention is to provide a system and method for hierarchical motion estimation, which can reduce the demand for internal memory without affecting the precision of motion estimation under the limited bandwidth of external memory.

According to an embodiment of the present invention, the first downsampling unit downsamples the reference frame, and the second downsampling unit downsamples the current frame, wherein the downsampling reference frame is stored in the coarse line buffer. A coarse motion vector (MV) estimator generates a coarse MV map based on the downsampled reference frame and the downsampled current frame. The fine line buffer fetches and stores the scanlines adjacent to the central scanline corresponding to the downsampled scanlines of the downsampled reference frame. The fine motion vector estimator generates a fine motion estimation (MV) map based on the coarse motion vector (MV) map, the current frame and the stored scanlines adjacent to the central scanline.

Description of drawings

FIG. 1 is a block diagram showing a hierarchical motion estimation (ME) system according to an embodiment of the present invention.

FIG. 2 is a flowchart showing a hierarchical motion estimation method according to an embodiment of the present invention.

Figure 3 shows a portion of a downsampled frame.

FIG. 4 illustrates group motion estimation according to an embodiment of the present invention.

[Description of main component symbols]

10 first downsampling unit

11 Second downsampling unit

12 coarse line buffers

13 Coarse motion vector (MV) estimator

14 fine line buffers

15 Fine motion vector (MV) estimator

21-25 steps

Detailed ways

FIG. 1 is a block diagram showing a hierarchical motion estimation (ME) system according to an embodiment of the present invention. FIG. 2 is a flowchart showing a hierarchical motion estimation method according to an embodiment of the present invention. The embodiments of the present invention are applicable to encoding of high-resolution images (for example, the resolution is 1920×1080), but are not limited thereto. Although the present embodiment shows a two-stage hierarchical motion estimation (ME) method, the embodiments of the present invention are also applicable to more than two-stage hierarchical motion estimation (ME).

In the first stage of hierarchical motion estimation (ME) in this embodiment, in step 21 , the previous frame (generally a reference frame) is down-sampled (or sub-sampled) by the first down-sampling unit 10 . In general, the height of the frame is downsampled using a downsampling factor N, and the width of the frame is downsampled using a downsampling factor M. In this embodiment, the same downsampling factor N is used to sample the height and width of the frame. In a specific embodiment, the search range of the previous frame is selected for downsampling. The search range is a part of the original frame (for example, 1/10 of the original frame). Figure 3 shows downsampling of a portion of a frame using a downsampling factor of 4. In this example, one pixel is selected for every 4 pixels in the horizontal and vertical directions of the search range of the frame. Therefore, the size of the data will be reduced to 1/16 of the original search range (ie, (1/4)*(1/4), or generally (1/N)*(1/M)). In the same step, the second down-sampling unit 11 is used to down-sample the current frame to be encoded. Next, at step 22 , the downsampled previous frame is stored in a coarse line buffer 12 . For example, if the search range of a high-resolution frame includes 108 scan lines, the downsampled previous frame can be stored in a coarse line buffer 12 with a capacity of 108*(1/4)*(1/4) . It should be noted that although the forward motion estimation adopted in this embodiment uses the previous frame as a reference frame, this embodiment can also be applied to backward motion estimation, which uses the next frame as a reference frame.

Next, in step 23, the coarse motion vector (MV) estimator 13 generates a coarse motion vector map (MV map) according to the down-sampled previous frame and the down-sampled current frame. The resulting coarse motion vector map shows the motion or displacement of the current frame relative to the previous frame (or reference frame). Among them, for block-based motion estimation, each macroblock in the motion vector map contains a motion vector (MV) (including the horizontal component of the motion vector and the vertical component of the motion vector), to represent The macroblocks in the current frame correspond to the motion or displacement of the macroblocks in the previous frame. Conventional metrics such as (but not limited to) sum of absolute differences (SAD) may be used to generate coarse motion vectors.

Regarding the second stage of layered motion estimation (ME) in this embodiment, at step 24, the scanlines adjacent to the downsampled scanlines in the previous frame are retrieved (can be retrieved from external memory, such as double data rate synchronous dynamic random access memory (DDR SDRAM)), and store the captured scan lines in a refine line buffer (refine line buffer) 14. In this embodiment, if the downsampling factor of the height is N, N scanlines above and below the downsampled scanline (also referred to as the central scanline) are captured together with the central scanline. In other words, a total of (2*N+1) scan lines are stored in the fine line buffer 14 . FIG. 3 shows (2*4+1) adjacent scan lines when N=4.

Next, in step 25, the fine motion vector (MV) estimator 15 generates a fine motion vector (MV) according to the rough motion vector (MV) map, the current frame and the scan lines stored in the fine line buffer 14 picture. Thereby, the precision of the motion vector (MV) generated by the rough motion vector (MV) estimator 13 can be refined from N pixels to 1 pixel. Conventional metrics such as (but not limited to) sum of absolute differences (SAD) can be used to generate fine motion vectors (MV).

In view of the fact that in the current frame, the vertical components of the motion vectors (MV) of adjacent macroblocks are usually different, so that each group of adjacent scan lines corresponding to different vertical components must be reloaded into the fine line buffer 14, thus This creates a burden on the bandwidth of the external memory device. Therefore, this embodiment does not reload each group of adjacent scan lines for adjacent macroblocks, but uses a group motion estimation method, so that in the current frame, the same central scan line corresponding to the fine line buffer 14 A group of macroblocks of a line (ie, corresponding to the same vertical position of the previous frame) are processed simultaneously. In other words, the determination of each macroblock in the group is related to the vertical MV component of the macroblock. Figure 4 illustrates motion estimation for groups. As shown in FIG. 4, the three macroblocks of the current frame are related to the same vertical component of the previous frame (as indicated by the individual arrows), so these three macroblocks are grouped into the same group, according to the fine line buffer 14 The same scanline group for simultaneous motion estimation. When all the macroblocks of the same group have been processed, another scan line group is captured and stored in the fine line buffer 14 . In a specific embodiment, only macroblocks within the search range of the current frame (such as the search range shown in FIG. 4 ) are processed, thereby speeding up motion estimation. It is worth noting that the central position of the search area is determined by the central scan line. In other words, different central scan lines correspond to central positions of different search ranges.

According to the above embodiment, the capacity of the line buffers (12 and 14) will be reduced to SR*(1/N)*(1/M)+(2*N+1), where SR is the search range and N is the height The downsampling factor, where M is the downsampling factor for the width. The above-described embodiments can be implemented using hardware, software, or a combination thereof. Furthermore, embodiments may also be implemented using pipelining. For example, the second stage of hierarchical motion estimation for the nth frame can be performed concurrently with the first stage of the hierarchical motion estimation for the (n+1)th frame.

The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the claims of the present invention; all other equivalent changes or modifications that do not deviate from the disclosed spirit of the invention should be included in the appended claims .

Claims (16)

1. the method for a hierarchical motion estimation comprises;

Reduce sampling one reference frame and a present frame;

Store this reduction sampling reference frame;

Reduce sampling reference frame and this reduction sampling present frame according to this, to produce a coarse movement vectogram;

Acquisition and storage are adjacent to the scan line of a central scan line, and this central scan line reduces one of sampling reference frame corresponding to this and reduces the sampling scan line; And

According to this coarse movement vectogram, this present frame with store scan line adjacent to this of this central scan line, to produce a fine movement vectogram.

2. the method for hierarchical motion estimation as claimed in claim 1, wherein a search area of this reference frame and this present frame is subjected to reducing sampling.

3. the method for hierarchical motion estimation as claimed in claim 1, wherein this reference frame is the former frame of leading this present frame.

4. the method for hierarchical motion estimation as claimed in claim 1, use one to reduce sampling factor N to the height do reduction sampling of this reference/present frame, use one to reduce sampling factor M to the width do reduction sampling of this reference/present frame, by this, every N pixel is chosen a pixel in vertical direction, every M pixel is chosen a pixel in the horizontal direction, and therefore, the size of this reference frame is reduced to (1/N) * (1/M).

5. the method for hierarchical motion estimation as claimed in claim 4, wherein the storage scan line adjacent to this central scan line comprises:

Be positioned at the N bar scan line on this central scan line; And

Be positioned at the N bar scan line under this central scan line;

By this, store (2*N+1) bar scan line altogether.

6. the method for hierarchical motion estimation as claimed in claim 1 in the step that produces this fine movement vectogram, in the present frame, corresponds to the macro zone block of this central scan line with processing simultaneously according to this coarse movement vectogram.

7. the method for hierarchical motion estimation as claimed in claim 6 wherein in this present frame, is handled the macro zone block that is positioned at a default search area.

8. the coarse movement vectogram that the method for hierarchical motion estimation as claimed in claim 1, the fine movement vectogram of n present frame produce step and (n+1) individual present frame produces step and carries out simultaneously.

9. the system of a hierarchical motion estimation comprises:

One first reduces sampling unit, in order to reduce sampling one reference frame;

One second reduces sampling unit, in order to reduce sampling one present frame;

One rough line buffer is used for storing this and reduces the sampling reference frame;

One coarse movement vector estimator is taken a sample present frame to produce a coarse movement vectogram according to this reduction sampling reference frame and this reduction;

One fine lines buffer, acquisition and storage are adjacent to the scan line of a central scan line, and this central scan line reduces one of sampling reference frame corresponding to this and reduces the sampling scan line; And

One fine movement vector estimator, according to this coarse movement vectogram, this present frame with store scan line adjacent to this of this central scan line, to produce a fine movement drawing for estimate.

10. the system of hierarchical motion estimation as claimed in claim 9, wherein a search area of this reference frame is subjected to this first reduction sampling that reduces sampling unit, and a search area of this present frame is subjected to this second reduction sampling that reduces sampling unit.

11. the system of hierarchical motion estimation as claimed in claim 9, wherein this reference frame is the former frame of leading this present frame.

12. the system of hierarchical motion estimation as claimed in claim 9, use one to reduce sampling factor N to the height do reduction sampling of this reference/present frame, use one to reduce sampling factor M to the width do reduction sampling of this reference/present frame, by this, every N pixel is chosen a pixel in vertical direction, every M pixel is chosen a pixel in the horizontal direction, and therefore, the size of this reference frame is reduced to (1/N) * (1/M).

13. the system of hierarchical motion estimation as claimed in claim 12, wherein the storage scan line adjacent to this central scan line comprises;

Be positioned at the N bar scan line on this central scan line; And

Be positioned at the N bar scan line under this central scan line;

By this, store (2*N+1) bar scan line altogether.

14. the system of hierarchical motion estimation as claimed in claim 9, wherein this fine movement vector estimator corresponds to the macro zone block of this central scan line with processing simultaneously according to this coarse movement vectogram in present frame.

15. the system of hierarchical motion estimation as claimed in claim 14, wherein this fine movement vector estimator is handled the macro zone block that is positioned at a default search area in present frame.

16. the system of hierarchical motion estimation as claimed in claim 9, wherein the fine movement vectogram of n present frame producing of this fine movement vector estimator and the coarse movement vectogram of (n+1) individual present frame that the vectorial estimator of this coarse movement produces are carried out simultaneously.

Images