JPH02205810A - Automatic focus adjusting device - Google Patents

Abstract

PURPOSE:To enable stable, speedy, and high-accuracy automatic focus adjusting operation with simple constitution by detecting the direction of relative movement between a subject and a camera from motion vector information on an image and restarting the automatic focus adjusting device. CONSTITUTION:A motion vector detecting circuit 9 detects motion vectors in plural blocks on a photographic image plane to find the optical flow of the image plane and detects whether or not the subject 1 moves from its displacement to decide whether the subject 1 approaches or leaves a lens 2. A restart decision circuit 14 decides whether the automatic focus adjusting operation is started or not according to the decision information. Consequently, the need to used a complex mechanism which requires high accuracy is eliminated, the restart of the focus adjusting device can accurately be decided with high precision from the image signal, and the automatic focus adjusting device which operates stably and securely without any unnecessary operation is obtained.

Description

DETAILED DESCRIPTION OF THE INVENTION [Field of Industrial Application] The present invention relates to a restart device for an automatic focus adjustment device that is suitable for use in an automatic focus adjustment device that adjusts the focus by detecting the sharpness of an image from an image signal. It is something.

[Conventional technology]

2. Description of the Related Art Automatic focus adjustment devices have been widely used in the field of cameras, including video cameras, and their operability has been greatly improved by automating focus adjustment, which is the most troublesome task.

Now, there are various focus detection methods for automatic focus adjustment devices, but in video cameras, automatic focus adjustment devices that use the video signal detect image information within the screen regardless of the distance to the subject, etc. Since focus detection can be carried out using the method, it is seen as a promising alternative to focus detection methods that project infrared rays, ultrasonic waves, etc. and detect the reflected waves.

Focus detection methods using this type of video signal can be broadly classified into two methods.

One is a method that uses actuators such as bimorphs and voice coils to slightly vibrate the lens or image sensor that makes up the photographic optical system in the optical axis direction, forcibly creating an out-of-focus state and modulating the video signal ( (hereinafter referred to as the modulation method).

Another method is a method (hereinafter referred to as a trial method) in which the photographing optical system is driven so that, for example, the high frequency component in the video signal is maximized.

According to the former modulation method, when in focus, even if the photographic optical system is vibrated, the change in focus degree is small and the modulated signal is approximately O, whereas when out of focus, a modulated signal is generated. Moreover, the phase of the modulated signal with respect to the modulated signal is reversed depending on the front pin and rear pin. Therefore, the focus adjustment actuator (generally using a DC motor) can be restarted depending on the presence or absence of this modulated signal, and the direction of restart can be determined based on the phase of the modulated signal, allowing accurate restart. It has the feature of being able to start up.

According to the latter trial method, the lens or image sensor is moved so that the signal component corresponding to the degree of focus is maximized, and after focusing, the focus signal at the time of focus is memorized, and the current focus It restarts the focus adjustment actuator when there is a change compared to the signal, and has the advantage of not requiring a complicated focus adjustment drive system, and having a simple configuration and miniaturization.

[Problem to be solved by the invention]

However, although the former modulation method provides information on the lens drive direction, it requires a precise modulation actuator that can operate with extremely high precision, making the mechanism complicated and large, and making adjustments difficult. This method has drawbacks such as being troublesome and disadvantageous in terms of power consumption.

In addition, according to the latter trial method, if the camera stops after focusing, it is often impossible to obtain information about the driving direction of the lens or image sensor when restarting. If the focus signal increases (towards the in-focus point), the lens is driven as is, and if it decreases, it is determined that the direction of lens movement is in the opposite direction, and control is performed to reverse the drive direction of the actuator. Since the automatic focus operation in principle includes automatic focus operation, it has the disadvantage of lacking stability in autofocus operation.

Therefore, in order to realize an automatic focusing device using a video signal, it is necessary to solve these problems.

[Means for solving problems]

The present invention has been made with the aim of solving the above-mentioned problems, and its features include a focus detection means for detecting the degree of focus on a subject within the screen and adjusting the focus; Motion vector detection means calculates the motion vector of the image in each region divided into a plurality of regions, and calculates the amount of motion of the image based on the motion vector information output from the motion vector detection means, and calculates the amount of motion of the image based on the motion vector information output from the motion vector detection means. and control means for activating the focus detection means when the focus is out of range.

[Effect]

This allows the direction of relative movement between the subject and the camera to be detected from the image motion vector information, and restarts the automatic focus adjustment device. Basically, a stable, rapid, and highly accurate automatic focus adjustment operation can be performed using a trial method configuration without using a mechanism that requires the following steps.

〔Example〕

DESCRIPTION OF THE PREFERRED EMBODIMENTS Practical examples of the automatic focusing device according to the present invention will be described in detail below with reference to the figures.

FIG. 1 is a block diagram showing a case where the automatic focus adjustment device according to the present invention is applied to a video camera.

In the figure, 1 is an object, 2 is a photographing lens system, and 3 is an image sensor, which is composed of, for example, a two-dimensional CCD. A signal processing circuit 4 performs processing such as gamma correction, blanking processing, and addition of a synchronization signal to the image signal output from the image sensor 3, and outputs a standard television signal conforming to, for example, the NTSC standard. Y is the luminance signal, H
-8YNC is a horizontal synchronization signal, and V-3YNC is a vertical synchronization signal. Reference numeral 5 denotes a delay circuit that delays the luminance signal Y by a predetermined period of time, such as a FIFO (first-in-flight circuit).
It is composed of a (st-out) type field memory, etc. 6 is a block division pulse signal generation circuit for dividing the signal scanning the screen into a plurality of predetermined blocks; 7, 8;
is a division circuit that divides the luminance signal Y according to the output pulse of the block division pulse generation circuit 6. Split circuit 7.8
is composed of a gate circuit whose opening and closing are controlled by the output pulses of the block division pulse generation circuit 6, and a memory, and is configured to capture the input luminance signal for each block set on the photographing screen and store it in the memory. The screen 9 compares the signal of the current screen outputted from the dividing circuit 8 with the signal of the screen a predetermined time ago outputted from the dividing circuit 7 via the delay circuit 5, and calculates the result for each divided block. 10 is a memory for storing motion vector information in each block on the shooting screen outputted from the motion vector detection circuit 9; 11 is a memory 10;
A statistical calculation circuit 12 creates a histogram in the x and Y directions based on motion vector information stored in the statistical calculation circuit 11, and a threshold determination circuit 12 recognizes the shape of the histogram processed by the statistical calculation circuit 11 and determines a threshold value to be described later. , 13 is an area determination circuit that searches for blocks within a predetermined threshold range from a histogram to be described later and determines a distance measurement area, and 14 is an output of the motion vector detection circuit 9.

This is a restart determination circuit that determines whether or not to restart the automatic focus adjustment device based on the output of a distance measurement area setting circuit 15 (described later) and the output of a comparison circuit 20 (described later).

The above is the configuration of the motion detection means that detects the motion of an image from the motion vector of the image on the photographic screen.

Next, the automatic focus adjustment system will be explained. Reference numeral 15 denotes a distance measurement area setting circuit consisting of a gate circuit that allows only the brightness signal corresponding to the focus detection area (range measurement area) set on the photographing screen to pass from the brightness signal Y output from the signal processing circuit 4. Based on the output of the area determining circuit 13, a ranging area is set on the screen. Reference numeral 16 denotes a bypass filter that extracts high frequency components from the luminance signal corresponding to the range measurement area set by the range measurement area setting circuit 15, and 17 a detection unit that converts the high frequency components extracted by the bypass filter 16 into a DC level. A circuit 18 is an integrating circuit that integrates the DC level signal output from the detection circuit 17 at a predetermined period. Further, the integral sensitivity of the integrating circuit is controlled by a control signal 15a outputted from the distance measurement area setting circuit 15 according to changes in the distance measurement area, and area correction is performed. 1
Reference numeral 9 denotes a delay circuit 20 for delaying the output signal of the integrating circuit 18 (hereinafter referred to as the focus signal) by a predetermined time, for example, l field.
21 is a comparison circuit that compares a current focus signal with a delayed focus signal from, for example, one field before, and 21 is a comparison result of the comparison circuit 20 outputted via a restart determination circuit, and determines the focus state of the photographic lens 2. This is a drive control circuit that drives and controls an actuator (for example, a DC servo motor) for controlling the motor.

The automatic focus adjustment device according to the present invention has the above-mentioned configuration, and its operation will be explained in detail below.

An image of the subject 1 formed on the imaging surface of the image sensor 3 through the photographic lens 2 is photoelectrically converted and output as an image signal. This image signal is sent to the signal processing circuit 4.

Therefore, the gamma correction and blanking processing described above.

After processing such as adding a synchronization signal, the signal is output as a standardized television signal. The luminance signal Y output from the signal processing circuit 4 is divided into two systems; one goes directly to the division circuit 8, and the other goes to the division circuit 7 after being delayed by the delay circuit 5 for an l field period (approximately 16.7 m5ec). is supplied. The division circuit 7.8 divides the photographic screen into mXn blocks in accordance with the screen division pulse outputted from the block division pulse generation circuit 6. Here, an example will be explained in which m=20°n=14 and the block is divided into a total of 280 blocks. The motion vector detection circuit 9 uses, for example, brightness level information of the current photographic screen supplied from the division circuit 8 and a predetermined information supplied via the division circuit 7 after being delayed by a predetermined period (l field) by the delay circuit 5. A comparison operation is performed with information on the screen before the period, and a motion vector for each of mXn blocks is determined.

As a calculation method for this motion vector, for example, a spatio-temporal gradient method can be used. This method uses B, to,P,
Artificial Intelligence (Artificial Intelligence)
e) 17. P2S5~203 (1981
), and real-time processing is possible by using dedicated hardware. The vector distribution obtained by determining the motion vector for each block in the entire photographic screen in this way is called an optical flow.

The operations from the motion vector detection circuit 9 to the area determination circuit 13 will be described below with reference to FIG.

Figure 2 (a) is an example of the image of the current field screen;
b) is the optical flow obtained by each motion vector distribution obtained by accumulating the optical flow obtained based on the difference with the image of the previous field screen for a certain period of time, and (C) of the same figure is the optical flow obtained by calculating the optical flow in the X direction. , a histogram created based on the size of each component in the Y direction, and (d) in the same figure shows the area division between the subject and the background recognized by the apparatus of this embodiment.

When shooting with a video camera as in this example, the subject is a moving video and the photographer is trying to prevent camera shake, so the movement of the background is less than the movement of the subject. The motion vector detected by the motion vector detection circuit 9 is stored in the memory lO for a certain period (for example, one second) and then input to the statistical processing calculation circuit 11. The statistical processing calculation circuit 11 ranks each vector according to the size of the XI and Y components, and creates a histogram (Fig. 2 (C)).
Create. The upper part of FIG. 2(C) is a histogram of motion vectors in the X direction, and the lower part is a histogram of motion vectors in the Y direction. Each histogram has the zero point on the horizontal axis as its origin, and represents the positive and negative directions, respectively, the distance from the origin represents the magnitude of the vector, and the height of the histogram represents the frequency of the vector. The threshold value determination circuit 12 determines the threshold value from the shapes of these two histograms. Here, the nearest minimum value of the histogram distribution having a peak closest to 0 in each of the X and Y directions is found, and the position of this minimum value is set as the threshold. This threshold value is determined on both the positive and negative sides. In FIG. 2(C), these are Thxl, Thx2. It was written as Thyl and Thy2.

Thxl and Thx2 are thresholds for the magnitude of the vector in the direction opposite to the origin, respectively, and T
A vector exceeding hxl and Thx2 is determined as a displacement of the image, and a vector whose size is less than or equal to Thxl and Thx2 can be determined to be a portion of the image that does not move substantially (such as a background).

The threshold determined in this way is input to the region determining circuit 13, and the region determining circuit 13 sequentially searches for blocks within the dark value range among the motion vectors in the memory 10. For example, if the amount of movement in the X direction is u + the amount of movement in the ISY direction is Vii in m = i-th and n = j-th block Bli, then blocks that satisfy Thxl<u+1<Thx2 and Thyl<v+H<Thy2 are placed in the screen. ′ and the area where the motion vector is small is turned on, and the other blocks are turned off. The on/off state obtained in this way is as shown in FIG. 2(d). The shaded area is "on" and corresponds to the area of the background background.

The area other than this shaded area is the subject area A,
B and C, which are the main subjects. The distance measurement area setting circuit 15 is configured such that the luminance signal is passed through only the main subject area other than the diagonal line in cD as shown in FIG. The bypass filter 16 extracts the high frequency component in the input luminance signal, converts it into a DC level signal in the detection circuit 17, integrates it over a predetermined period in the integration circuit 18, and converts it into a DC level signal according to the degree of focus. A focus signal is output.The focus signal output from the integration circuit 18 is supplied to the comparison circuit 20 via the delay circuit I9 and compared with the focus signal on the previous screen that is delayed by one field.

This comparison result is supplied to the drive control circuit 21 via the restart determination circuit 14, and the drive control circuit 21 controls the actuator 22 to drive the photographing lens 2 in a direction in which the focus signal level increases. Then, when the focus signal level takes the maximum value and decreases, the position where the maximum value is taken is determined to be the in-focus position, and the photographing lens is returned to that position to complete the automatic focus adjustment operation.

In the above automatic focus adjustment operation, when the integrated output of the integrating circuit 18 is compared by the comparator circuit 20, if the size of the distance measurement area set before l field is different from the size of the distance measurement area of the current field, Even if the focus state is the same, since the integrated signal level is different, the magnitude determination may not correspond to the degree of focus, resulting in an erroneous determination. Therefore, a control signal 15a representing the size of the ranging area is supplied to the integrating circuit 18.
The difference in area between the distance measurement areas on both field screens is corrected.

In other words, the integrated signal output level of the integrating circuit can be normalized by the area of the ranging area (As a specific means for this, in this embodiment, the above-mentioned correction is performed by switching the integration sensitivity of the integrating circuit. .

The specific configuration of the ranging area setting means has been described above. In the statistical processing by the above-mentioned memory lO9 statistical processing circuit 11゜threshold determination circuit 12, etc., a method is used to create histograms in each of the x and Y directions of the photographic screen, but the histogram is created between the X-Y two-dimensional dimension. But that's fine. Furthermore, when the motion vector detection circuit 9 calculates the optical flow using images accumulated for a certain period of time, the operation cycles of the memory IO to the area determination circuit 13 must also be changed according to the accumulation time.

Furthermore, according to the above explanation, an area according to the output of the area determination circuit 13 is set as the distance measurement area, but considering that in normal shooting, the main subject is located in the center of the screen, and In order to avoid conflict between distance and distance, it is also possible to select only the area near the center of the photographic screen and use it as the distance measurement area. In this case, in each area of FIG. 2(d), areas A and C may be completely cut, or the center of the screen may be set to 100%, the four diagonal corners of the screen may be set to 0%, and in between, A window function that continuously changes the weighting may be provided.

The actual restart of the automatic focus adjustment device from the motion vector of an image will be explained below. To simplify the diagram,
We will explain the case where the distance measurement area is set at the center of the shooting screen and its vicinity, for example, the case where it is set at area B in Figure 2 ((1)). Figure 3 shows area B under this setting condition.
This shows the optical flow in The state in the figure shows the optical flow when the subject corresponding to area B approaches the camera and moves slightly to the right on the screen. The motion vector of each block has a direction and magnitude that diverge toward the right as a whole and toward the periphery, which indicates that the subject is gradually getting thicker (i.e., approaching the camera).
It can also be seen that it is moving to the right relative to the camera.

FIG. 4 is a diagram for explaining the nature of such an optical flow. In the diagram, 2 is the same as the photographing lens system 2 in FIG. 1. 51' is a subject corresponding to area B in FIG. 2(d), and 53 is an image of the subject formed on the photographic screen. 52 indicates a state in which the subject has moved upward as seen in the figure while approaching the photographic lens;
4 is an image on the photographing screen at that time. The magnitude of the optical flow vector caused by the subject moving upward in the figure is constant regardless of the position within the screen. On the other hand, the optical flow that occurs when the subject approaches the lens is caused by the enlargement of the subject image, and changes depending on the position within the screen. The magnitude of the vector is O, and it becomes thicker in proportion to the distance away from the center of the screen.The optical flow in Figure 3 is a composite of vectors caused by these two causes. Although the images 53 and 54 on the photographic screen are drawn at different positions on the optical axis for the sake of explanation, it is assumed that they are actually formed on the same plane.

Next, the calculations performed within the restart determination circuit 14 shown in FIG. 1 will be explained. In Figure 5, the area B in Figure 2(d) is extracted, the center of the screen is taken as the origin (0,0), the X and Y coordinates are taken, and the coordinates of each block in the distance measurement area B are (
i, D, and the optical flow vector in each block is A(i, j). Furthermore, let the unit vector in the X direction be 11 and the unit vector in the y direction be j, and rewrite it as A (i, j)=iAx+jAy.

The displacement of the optical flow vector due to the enlargement or reduction of the image means the divergence of the vector on the screen.
nce) divA can be expressed as 1, and this divergence represents the displacement of the distance between the subject and the camera.

The restart determination circuit 14 takes in the control information 15b indicating the distance measurement area from the distance measurement area setting circuit 15, and also takes in the motion vector information in each block on the shooting screen from the motion vector detection circuit 9. The sum d−Σ divA is calculated. The sum of the divergence divA within this area B changes according to the enlargement or reduction of the photographic screen of the subject image.

In the case of FIG. 3, the image is enlarged, so d>O.

On the other hand, if the subject is moving away from the photographic lens, the image will be reduced, so d<O.

Therefore, if d=o continues after focusing, there is no change in the distance to the subject and the lens remains stopped, and if d≠0 after focusing, the automatic focus adjustment device assumes that there has been a change in the distance between the subject and the camera. can be restarted and its direction can be determined by the positive or negative value of d.

That is, if d>O, the subject is closer to the photographic lens than before the movement, and the focal position of the photographic lens is behind the subject, resulting in a so-called rear bin state.

Further, if d<O, the subject is farther away from the photographic lens than before the movement, and the photographic lens is in the front bin state.

In this way, by examining the divergence divA of the optical flow vector within the distance measurement area, it is possible to detect the displacement of the subject distance and its direction with high precision, which allows the automatic focusing device to restart after focusing. Even if an automatic focus adjustment device is used in the trial method, the front focus can be achieved without performing any trial movements.

Rear focus information can be obtained, and the automatic focus adjustment device can be restarted quickly, stably, and accurately.

Furthermore, according to the present invention, even if there are multiple subjects in the shooting screen, the background is complex, or the image has small brightness differences and it is difficult to extract the feature points of the subjects, motion vectors can be detected. Statistical processing is applied to the optical
By calculating the divergence of the flow vector, it is possible to reliably detect the displacement of the distance to the subject, so both sensitivity and accuracy are significantly improved compared to conventional devices.

〔Effect of the invention〕

As described above, the automatic focus adjustment device of the present invention detects motion vectors in multiple blocks on the photographic screen to determine the optical flow of the screen, and detects whether or not the subject has moved from the displacement thereof. In addition, it is possible to accurately determine whether the subject is approaching or moving away from the lens, and the automatic focus adjustment operation can be restarted based on this determination information. Accurately and precisely focus from image signals without using complex mechanisms that require high precision to create in-focus and out-of-focus states and modulate the focus signal to obtain front focus and rear focus information. It is possible to determine the restart of the adjustment device,
It is possible to realize an automatic focus adjustment device that is stable and operates reliably without unnecessary movement.

[Brief explanation of the drawing]

FIG. 1 is a block diagram showing an embodiment in which the automatic focus adjustment device of the present invention is applied to a video camera, and FIG. 2(a)
- Figures 2(d) and 3 show the optical image on the shooting screen.
A diagram to explain the operation of determining the flow and setting the distance measurement area. Figure 4 is a diagram to explain the properties of optical flow. Figure 5 is a diagram to explain the operation of determining the flow and setting the distance measurement area. FIG. 6 is a diagram for explaining the operation of determining activation. Patent applicant Canon Co., Ltd.

Claims (1)

[Claims] A focus detection means for detecting the degree of focus on a subject within a screen and adjusting the focus; a motion vector detection means for calculating a motion vector of an image in each region of the screen divided into a plurality of regions; and the motion vector detection means and a control means for calculating the amount of movement of an image based on the motion vector information output from the apparatus, and activating the focus detection means when the amount of movement falls outside a predetermined range. Regulator.