JP3639640B2 - Motion vector detection device - Google Patents

Description

[0001]
[Industrial application fields]
The present invention relates to a motion vector detection device for detecting a motion vector of an image in an imaging device such as a video camera or a camera-integrated VTR or an image output device.
[0002]
[Prior art]
2. Description of the Related Art Conventionally, a motion vector detection device for detecting a motion of a captured image or an image obtained from an image output device is known. This is to detect a motion vector representing how much the image has moved in which direction by performing pattern matching between an image and the image one field before or two arbitrary images. The pattern matching is, for example, taking the sum of EXOR for each pixel of the previous field image and the current field image, or two images, and determining that the pattern is most matched when the value is the smallest. This value is called a correlation value. The shift when the correlation value is the smallest, that is, the image shift is the motion vector.
If pattern matching is performed on the entire image, a large amount of calculation is required. For example, a moving image is difficult to process in real time during imaging. For this purpose, an image is divided into several regions, that is, several motion vector detection blocks, and a motion vector is calculated in each region.
[0003]
In each motion vector detection block, the detection accuracy of the detected motion vector is one pixel unit because pattern matching is performed for each pixel of each image. However, by performing interpolation using data at the time of pattern matching of pixels around the pixel, it is actually possible to calculate up to one pixel or less.
However, calculation of the motion vector by interpolation includes some errors. In order to calculate a motion vector value of one pixel or less from pattern matching in units of one pixel, when calculating from pixel data around the pixel, the pixel data is not necessarily the same motion vector value as the pixel. Because it does not always have. In addition, image noise is one of the major causes of errors.
[0004]
The error is not noticeable when an image with a large motion, that is, a large vector is detected, but the error becomes relatively conspicuous when the motion vector detected with a small motion becomes small.
In order to prevent this error from being output, there is known a processing method in which the motion vector to be output is 0 when the motion vector calculated by interpolation is near 0, which is called coring processing. It is known that this coring process can prevent an error from being output when there is a small movement of an image in which the error is relatively conspicuous (Japanese Patent Laid-Open No. 5-91392, 1992 TV). John Society Annual Conference Improvement of electronic image stabilization control system).
[0005]
[Problems to be solved by the invention]
However, when the coring process is performed, a motion vector is not output unless a motion vector of a certain level or more is generated, so that the detection characteristics are deteriorated. Therefore, in terms of motion vector detection characteristics, it is desirable that the threshold value for coring processing be as small as possible. However, if the coring process threshold is too small, an interpolation error is output when there is little motion of the image. Therefore, the threshold value for the coring process is almost determined by the system configuration. That is, the threshold value is set to a level slightly higher than the error level including the interpolation error of the system.
[0006]
Pattern matching is performed for several motion vector detection blocks in the captured image. Thereby, a plurality of motion vectors are detected in one field image. Then, in order to finally make one motion vector in the field, an average value and a median value of these motion vectors are taken. As a result, the interpolation error is reduced, so that the threshold value of the coring process can be reduced. However, depending on the captured image, motion vectors detected by some motion vector detection blocks may not be used. The reason is that an accurate motion vector may not be detected because, for example, the subject has low contrast or low illuminance. In such a case, the final motion vector in the field is calculated from the effective motion vector remaining after deleting the invalid motion vector, resulting in an increase in error.
[0007]
When the threshold value for coring processing is set based on the error level when valid vectors can be detected by all motion vector detection blocks, the valid motion vectors are reduced when invalid motion vectors are detected as described above. Since the interpolation error becomes large, the error included in the calculated final motion vector exceeds the set coring threshold. Therefore, the error becomes conspicuous when the motion vector is small. Conversely, if the threshold value for coring processing is set based on when the number of effective motion vectors is small, errors will not be noticeable even if the number of effective motion vectors is small, but the motion vector detection characteristics will deteriorate.
[0008]
Therefore, an object of the present invention is to obtain a motion vector detection apparatus that can obtain good detection characteristics while reducing errors.
[0009]
[Means for Solving the Problems]
In the present invention, an input image is divided into a plurality of detection blocks, a motion vector detection means for detecting a motion vector for each detection block, and an effective motion vector for determining the validity of the motion vector detected by the motion vector detection means. A motion vector determination means for determining one representative motion vector based on the effective motion vector determined by the effective motion vector determination means from the motion vector detected by the motion vector detection means; and the motion vector determination When the representative motion vector determined by the means exceeds the threshold value, the representative motion vector is output, and the threshold value is changed according to the number of effective motion vectors determined by the effective motion vector determination means. And an arithmetic means configured as described above.
[0010]
[Action]
The motion vector detected by a plurality of motion vector detection blocks in the image of one field by the motion vector detection means is judged as valid or invalid by the valid motion vector discrimination means, and the motion determined as valid When the number of vectors is large, the threshold value in the calculation means is reduced to lower the effective detection level and effectively detect and output even a small motion vector. When the number of motion vectors determined to be valid is small An error is not output by increasing the threshold value in the computing means.
[0011]
【Example】
Hereinafter, an embodiment of the present invention will be described.
FIG. 1 is a block diagram showing a configuration of an imaging apparatus to which the present invention is applied. In the figure, reference numeral 1 denotes an image pickup means, and 2 denotes a motion vector detection means, which detects a motion vector from an image including a blur imaged by the image pickup means 1. Reference numeral 3 denotes effective motion vector discrimination means for discriminating the validity of the motion vector detected by the motion vector detection means 2. Reference numeral 4 denotes a motion vector determining means for discriminating an effective motion vector based on a plurality of motion vectors detected in one field by the motion vector detecting means 2 and an effective motion vector discriminating signal from the effective motion vector discriminating means 3. This is means for determining one motion vector representing the field, that is, a representative motion vector, based on the average value or median value of the remaining effective motion vectors. Reference numeral 5 denotes coring means for performing coring processing of the representative motion vector determined by the motion vector determining means 4. Reference numeral 6 denotes a shake correction unit, which performs shake correction of an image including a shake imaged by the imaging unit 1 in accordance with a motion vector subjected to coring processing by the coring unit 5 as a calculation unit.
[0012]
FIG. 2 is a diagram showing a motion vector detection block, where 7 is an entire captured image area, 8 is a motion vector detection block, and 50 are set here. Reference numeral 9 denotes a search area, and reference numeral 10 denotes a search area for searching for the motion vector detection block at the top left of the motion vector detection block 8.
[0013]
FIG. 3 is a diagram showing how a template shifts while matching a search area. 11 is a search area of an arbitrary motion vector detection block 8, and 12 is a template.
[0014]
FIG. 4 is a graph showing correlation values output when pattern matching is performed while the template 12 is shifted in the search area 11.
FIG. 5 is a graph showing the coring process.
[0015]
When the image pickup means 1 is picked up by hand, the image pickup means 1 may be shaken by an external force due to technical immaturity of the photographer or wind. As a result, the image captured at this time includes blurring. In order to correct the blur of the image, the motion vector detecting means 2 first detects a motion vector included in the image. The motion vector detection means 2 has a motion vector detection block 8 as shown in FIG. Then, as shown in FIG. 3, the correlation value is obtained while moving the template 12 on the search area 11. At this time, each data of the template 12 and the search area 11 is around one field. For example, an image in a certain field is a search area 11, and an image after one field is a template 12. The reverse is also possible.
[0016]
Then, EXOR is performed for each pixel of the template 12 and the search area 11 at a certain position on the search area 11, and the outputs of all the EXOR are added to obtain a correlation value. The correlation values are calculated while moving the template 12 pixel by pixel on the search area 11 to obtain the characteristics shown in FIG. EXOR is 0 when they match, and 1 when they do not match, so the correlation value is minimum when the image patterns of the template 12 and the search area 11 most closely match. In FIG. 4, the shift when the correlation value is minimum becomes the motion vector in the motion vector detection block 8. Here, when the motion vector is 0, that is, if the image in a certain field and the image after one field are the same, the minimum correlation value is obtained at the center of the search area 11. The accuracy of the obtained motion vector is only one pixel unit in order to obtain a correlation value for each pixel. Therefore, interpolation is performed using data around the minimum correlation value, and a motion vector is calculated with an accuracy of one pixel or less. In this way, the motion vector detection means 2 detects several motion vectors of the motion vector detection block 8 for each field.
[0017]
However, the detected motion vectors may include invalid motion vectors. For example, if there is a low-contrast part of the captured image, the motion vector cannot be accurately detected, and therefore an invalid motion vector may be output. For this purpose, the effective motion vector discriminating means 3 discriminates an effective motion vector from a plurality of motion vectors. If the effective motion vector is determined by the correlation value, for example, when the average value of the correlation value is low, it can be determined that the contrast is low. It can also be determined that the effectiveness is low when the inclination around the minimum correlation value is small.
[0018]
Thus, the motion vector determining means 4 determines the final motion vector in the field from the plurality of motion vectors determined to be effective by the effective motion vector determining means 3. Here, the representative motion vector is determined by taking the average value or the median value of the effective motion vectors.
Although the blur correction is performed using the representative motion vector, since the motion vector detecting means 2 performs interpolation to calculate the accuracy of one pixel or less, the calculation error at this time is included in the motion vector. An error also occurs due to image noise or the like. These errors can be reduced by taking an average value of effective motion vectors in the motion vector determining means 4 or deleting invalid motion vectors in the effective motion vector determining means 3. However, some error will remain. This error is not a significant problem because the effect is relatively small when the final motion vector is large. However, when the motion vector is small, the influence of the error is relatively large. Therefore, even if the motion vector is actually 0, the shake correction means 6 corrects the error due to an error, so that the finally output corrected image will be finely blurred. In order to avoid the influence of this error, the coring means 5 performs a coring process.
[0019]
In the example of FIG. 2, since there are 50 motion vector detection blocks 8, 50 motion vectors are detected per field. Of the detected 50 motion vectors, the valid motion vector discriminating means 3 discriminates whether the motion vector is valid or invalid. Next, according to the valid discrimination signal output from the valid motion vector discriminating unit 3, the motion vector determining unit 4 calculates the average value or median value using only the valid motion vectors, thereby obtaining the final value in the field. One representative motion vector is calculated.
[0020]
The calculated representative motion vector is subjected to coring processing in the coring means 5. At this time, the coring means 5 is inputted with the representative motion vector and the effective motion vector discrimination signal. The coring means 5 correlates the representative motion vector by adaptively changing the coring threshold in FIG. 5 in accordance with the effective motion vector discrimination signal. Specifically, when the effective motion vector discrimination signal indicates that there are many effective motion vectors, the threshold value of the coring process is reduced to prioritize the motion vector detection characteristics. Further, when the effective motion vector discrimination signal indicates that there are few effective motion vectors, the coring processing threshold value is increased so as not to output an error.
[0021]
The blur correction unit 6 receives a video signal including the blur imaged by the imaging unit 1 and a representative motion vector subjected to coring processing. This blur correction unit 6 outputs a stable video without blur by cutting out and outputting the video signal in a direction to cancel the representative motion vector.
[0022]
In this embodiment, the motion vector detected from the image picked up by the image pickup means 1 is used for blur correction, but it can also be applied to an image from an image reproducing apparatus such as a VTR. It can also be applied to a TV conference system and other image processing.
[0023]
Further, as shown in FIG. 6, a variable apex angle prism 7 is provided in the imaging optical system as a blur correction unit, and the variable apex angle prism 7 is adjusted by the prism angle control unit 8 according to the motion vector from the coring unit 5. The angle may be controlled in the direction to cancel out the shake.
[0024]
【The invention's effect】
As described above, according to the present invention, in an apparatus for detecting a motion vector from an image, an error output from the apparatus is reduced by adaptively changing the threshold value of the coring process according to the number of effective motion vectors. In addition, good motion vector detection characteristics can be obtained.
[Brief description of the drawings]
FIG. 1 is a block diagram showing an embodiment of an imaging apparatus to which the present invention is applied.
FIG. 2 is a block diagram showing a motion vector detection block.
FIG. 3 is a configuration diagram showing how a search area is searched while a template performs matching.
FIG. 4 is a graph showing a correlation value output when pattern matching is performed while a template moves in a search area.
FIG. 5 is a graph showing a coring process.
FIG. 6 is a block diagram illustrating another embodiment of the imaging apparatus.
[Explanation of symbols]
2 motion vector detection means 3 effective motion vector discrimination means 4 motion vector determination means 5 coring means 7 total captured image area 8 motion vector detection block

Claims (1)

Motion vector detection means for dividing an input image into a plurality of detection blocks and detecting a motion vector for each detection block;
Effective motion vector discriminating means for discriminating the validity of the motion vector detected by the motion vector detecting means;
Motion vector determining means for determining one representative motion vector based on the effective motion vector determined by the effective motion vector determining means from the motion vector detected by the motion vector detecting means;
When the representative motion vector determined by the motion vector determination means exceeds a threshold value, the representative motion vector is output, and the above-mentioned value is determined according to the number of effective motion vectors determined by the effective motion vector determination means. A motion vector detection device comprising: an arithmetic means configured to change a threshold value.

Images