JP2006058431A - Autofocusing system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To track an object, even when image of the subject changes by updating a reference pattern, whenever the subject image is detected by a pattern matching processing. <P>SOLUTION: An autofocusing system is disclosed, in which the image of the subject to be focused is stored as the reference pattern, and an image matching with the reference pattern is detected from among photographed images by the pattern matching processing; the AF area is moved to the detection position to thereby track the desired subject, wherein the tracking apparatus 16 stores the image of the subject to be focused as the reference pattern; the image matching with the reference pattern is detected out of the photographed images; the position of the subject is detected; and then the position of the AF area as the range to be AF is moved to the position of the subject. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an autofocus system, and in particular, an auto that can automatically focus a position (range) targeted for autofocus (AF) within a shooting range of a camera to track a desired target subject. Concerning the focus system.

  In an imaging system that converts an image of a subject into an electrical signal (image signal) using a photographing element (CCD or the like) like a television camera, a contrast method is generally employed as an autofocus (hereinafter referred to as AF) method. . The contrast method is a method of automatically focusing to the best focus state by detecting the contrast of the subject image from the image signal captured by the image sensor and controlling the focus of the photographic lens so that the contrast becomes the highest. is there.

  In AF such as a contrast method, the entire subject within the shooting range of the camera is not targeted for AF, but only the subject within a part of the shooting range is often targeted. In contrast-based AF, an image signal in a range targeted for AF is extracted from image signals captured by the entire imaging area of the image sensor, and focus is performed so that the contrast of the image signal in the extracted range is the highest. By controlling this, the target of AF is limited to only a certain range of subjects. In this specification, the range of the subject to be AF in the shooting range of the camera, or the range on the screen of the subject to be AF in the screen of the shot image when the shot image of the camera is reproduced. It is referred to as an AF area, and a frame indicating the outline of the AF area is referred to as an AF frame.

When a single AF area is fixedly set at a predetermined position within the shooting range, a rectangular AF area is normally set at the center of the shooting range. However, the AF area is specified by the operator. There is also known one that can change the position of. In this specification, the AF point is a point that determines the position (for example, the center point of the rectangular AF area or the position of any of the four corners) on the shooting range (or the screen of the shot image) of the AF area. That's it. When an AF point is specified using an operation device in a television camera or the like, an operation device such as a trackball or a joystick that can instruct movement of the AF point vertically and horizontally is used (for example, Patent Document 1). reference).
JP 2002-365519 A

  By the way, when the AF point can be changed and the subject to be focused is an object that moves on the screen, an operation device such as a trackball or a joystick is operated to perform AF according to the movement of the target subject. Since it is necessary to move the point, there is a problem that it requires time and effort for the operation. Therefore, it is effective to automatically track the AF point with respect to the moving target subject. As a method for automatically tracking an AF point to a desired target subject, it is conceivable to apply a pattern matching technique used in product inspection or the like. For example, an image of a subject to be tracked is stored as a reference pattern, and an image that matches the reference pattern is detected from captured images that are sequentially captured by a camera. Then, the AF point can be tracked to a desired target subject by moving the position of the AF point in accordance with the position of the image that matches the reference pattern.

  However, even if the target subject is the same, if the target subject moves, the image may change greatly and may not match the image of the reference pattern stored in the initial stage. Further, even when the target subject itself does not move, if the size of the target subject image changes due to zooming or the like, there is a possibility that the target pattern image does not match the same. In such a case, there arises a problem that the target subject cannot be tracked.

  The present invention has been made in view of such circumstances, and even if the image changes with respect to a desired target subject that moves within the shooting range, the AF area (AF point) is accurately tracked to perform the operation. An object of the present invention is to provide an autofocus system capable of continuously focusing on a desired target subject by AF without the effort of a person.

  In order to achieve the above object, the autofocus system according to claim 1 controls the focus of the photographic lens with the subject in the AF area within the photographing range of the camera as the subject of autofocus, and automatically controls the subject. In an autofocus system that performs focusing, an imaging unit that sequentially acquires an image of a subject within the shooting range, a target subject specifying unit that specifies a target subject to be focused by the autofocus, and the target subject specification A storage unit that acquires and stores the image of the target subject specified by the unit as a reference image from the imaging unit, and an image that matches the reference image from the images acquired by the imaging unit. The target subject is selected from the images sequentially acquired by the imaging means. Target subject detection means for sequentially detecting images, and the position of the AF area so as to follow the target subject based on the position of the target subject image detected by the target subject detection means within the shooting range. An AF area changing unit for changing, and an updating unit for updating a reference image stored in the storage unit with an image of the target subject detected by the target subject detecting unit are provided. According to the present invention, since the reference image (reference pattern) stored in order to detect the image of the target subject from the images picked up by the imaging unit is sequentially updated, the image of the target subject changes. However, the target subject can be detected, and the AF area can be accurately tracked by the target subject.

  According to a second aspect of the present invention, in the first aspect of the invention, the target subject detection unit is configured to detect an image of the target subject detected last time in the range of images acquired by the imaging unit. The present invention is characterized in that an image that matches the reference image is detected from images in the range by limiting to a partial range including the range. According to the present invention, since the image of the target subject is not detected within the entire range of the image acquired by the imaging means, but limited to a part of the range, the processing time for the detection is detected. And the processing burden can be reduced.

  The autofocus system according to a third aspect of the present invention is the autofocus system according to the first aspect, wherein the target subject detection means is sequentially acquired by the imaging means when the image that matches the reference image is not detected. It is characterized in that an image of the target subject is detected by obtaining a difference between two images. According to the present invention, for example, even when the change of the image of the target subject is extremely fast and the image of the target subject cannot be detected from the reference image, the AF area can be tracked by the target subject.

  According to the autofocus system of the present invention, the AF area (AF point) can be accurately tracked even if the image of the desired subject moving within the shooting range changes, and the operator's effort is reduced. The desired object can be continuously focused by AF.

  Hereinafter, preferred embodiments of an autofocus system according to the present invention will be described in detail with reference to the accompanying drawings.

  FIG. 1 is a block diagram showing the overall configuration of an imaging system to which an autofocus system according to the present invention is applied. The image pickup system shown in FIG. 1 is an image pickup system used for shooting with, for example, a broadcast television camera. In the figure, a camera head 10 with a replaceable lens, and a shooting lens (optical) mounted on the lens mount of the camera head 10 are shown. A lens device 12, a frame operation unit 14, a tracking device 16 and the like provided with a system) are shown.

  The camera head 10 is equipped with an image sensor (for example, a CCD), a required signal processing circuit, and the like, and an image formed by the photographing lens of the lens device 12 is photoelectrically converted by the image sensor and then subjected to signal processing. Necessary signal processing is performed by the circuit. A video signal in a predetermined format generated by the signal processing circuit is output from the video signal output terminal of the camera head 10 to an external device. Further, the camera head 10 is provided with a view finder (monitor) 18 for composition confirmation, etc., and a video signal from the camera head 10 is given to the view finder 18 and is photographed by the camera head 10. A real-time image (video) is displayed on the screen of the viewfinder 18. In addition, information such as an AF frame indicating the range of the AF area that is the target of autofocus (AF) is also displayed on the screen of the viewfinder 18.

  The lens device 12 includes a photographing lens (optical system) (not shown) attached to a lens mount of the camera head 10, and an image of a subject is formed on the imaging surface of the imaging element of the camera head 10 by the photographing lens. . The photographing lens is provided with movable parts for adjusting photographing conditions such as a focus lens group, a zoom lens group, and a diaphragm as its constituent parts. These movable parts are electrically driven by a motor (servo mechanism) (not shown). It is designed to be driven. For example, the focus lens group and zoom lens group move in the optical axis direction, and the focus (subject distance) adjustment is performed by moving the focus lens group, and the focal length (zoom magnification) adjustment is performed by moving the zoom lens group. Is done. In the system relating to autofocus as in the present embodiment, it is sufficient that at least the focus lens group can be driven electrically, and other movable parts may be driven only manually. Further, when the predetermined movable part is electrically driven in accordance with the operation of the operator, it is based on a control signal output in accordance with the operation of the operator from an operation unit (not shown) (such as an operation unit of a controller connected to the lens device 12). Thus, the operation of the movable part is controlled, but details are omitted.

  As shown in the figure, the lens device 12 is equipped with a lens CPU 20 that performs overall control of the lens device 12, an AF processing unit 22 that executes autofocus (AF) processing, an AF imaging circuit 24, and the like. Yes. The AF image pickup circuit 24 is disposed in the lens device 12 to acquire a video signal for AF processing, and outputs an image pickup device (CCD or the like) or an output signal of the image pickup device as a video signal of a predetermined format. Etc. The image pickup element of the AF image pickup circuit 24 is referred to as an AF image pickup element. The video signal output from the AF imaging circuit 24 is a luminance signal.

  On the imaging surface of the AF imaging element, subject light branched from the subject light incident on the imaging element of the camera head 10 is imaged by a half mirror or the like disposed on the optical path of the photographing lens. The shooting range and subject distance (distance of the subject in focus) with respect to the imaging area of the AF imaging element are configured to match the shooting range and subject distance with respect to the imaging area of the imaging element of the camera head 10 and are used for AF. The subject image captured by the image sensor matches the subject image captured by the image sensor of the camera head 10. Note that it is not necessary for the shooting ranges of the two to be completely the same. For example, the shooting range of the AF imaging device may be a larger range including the shooting range of the imaging device of the camera head 10. In addition, a video signal from the camera 10 may be provided to the AF processing unit 22 without providing an AF image sensor.

  The AF processing unit 22 acquires a video signal from the AF imaging circuit 24, and calculates a focus evaluation value indicating the level of contrast of the subject image based on the video signal. For example, after extracting a high-frequency component signal of a video signal obtained from an AF image sensor with a high-pass filter, a signal in a range corresponding to the AF area targeted for AF is extracted from the high-frequency component signal. Accumulate one screen at a time. The integrated value obtained for each screen in this manner indicates the level of contrast of the subject image, and is given to the lens CPU 20 as a focus evaluation value. The AF area range is designated by the lens CPU 20 as will be described later.

  As will be described in detail later, the lens CPU 20 acquires AF frame information (AF frame information) indicating the range (contour) of the AF area from the frame operation unit 14, and the range within the AF frame designated by the AF frame information. Is designated to the AF processing unit 22 as an AF area. Then, the focus evaluation value obtained from the image (video signal) in the AF area is acquired from the AF processing unit 22, and the acquired focus evaluation value is maximum (maximum), that is, the contrast of the subject image in the AF area is maximum. The focus lens group is controlled so that For example, a hill-climbing method is generally known as a control method for a focus lens group based on a focus evaluation value, and the focus evaluation value starts to decrease by moving the focus lens group in a direction in which the focus evaluation value increases. Is detected, the focus lens group is set at that position. Thereby, the subject in the AF frame is automatically focused.

  The frame operation unit 14 includes an operation member for the operator to specify the setting contents of the AF frame such as the position, shape, and size of the AF frame. Note that the AF frame indicates the outline of the AF area indicating the AF target range with respect to the shooting range or the shot image screen of the imaging element of the camera head 10 as shown in FIG. When the operation member is operated by the operator, the frame operation unit 14 sets and changes setting contents such as the position of the AF frame in accordance with the operation. In this embodiment, it is assumed that the shape of the AF frame is limited to a rectangle. In addition, the setting contents such as the position and size of the AF frame by the operation member of the frame operation unit 14 are changed by a change amount corresponding to the operation amount of the subsequent operation member based on the setting contents of the AF frame before the operation. It is only done by making changes to the settings. For example, as for the position of the AF frame, the AF frame is displaced in the vertical and horizontal directions by a direction and a movement amount corresponding to the rotation direction and rotation amount of the trackball. Also, if the point that determines the position of the AF frame (the center position of the rectangular AF frame in the present embodiment) within the shooting range (screen of the captured image) is referred to as an AF point, the position of the AF frame is the position of the AF point. Is determined by setting

  On the other hand, when the operation member of the frame operation unit 14 is not operated and the operator is not instructed to change the setting contents of the AF frame, the setting contents of the AF frame are set by the AF frame information given from the tracking device 16 described later. Set and change. The frame operation unit 14 transmits AF frame information indicating the setting contents of the AF frame set or changed by the operation of the operator or the AF frame information from the tracking device 16 to the lens CPU 20 in accordance with a request from the lens CPU 20. As a result, the AF target range is set to the range of the AF frame set and changed in the frame operation unit 14.

  The tracking device 16 is a device that automatically changes the position of the AF frame by moving the AF point according to the movement of the target while tracking the target (target subject) designated by the operator on the screen. Yes, the AF frame is automatically changed when the operation of changing the AF frame by the operator is not performed. When the operation of changing the AF frame by the operator is performed, the operation is performed by the tracking device 16. And the AF frame is changed according to the operation of the operator.

  The frame operation unit 14 also transmits AF frame information indicating the setting contents of the AF frame set and changed as described above to the camera head 10, and places the AF frame at a corresponding position on the screen of the viewfinder 18. Display. As a result, the operator can recognize the position, shape, size, etc. of the AF frame while viewing the viewfinder 18.

  The tracking device 16 includes an image processing unit 26, an image input unit 28, a serial communication interface 30, and the like. The image input unit 28 acquires the video signal (luminance signal) obtained by the AF imaging circuit 24 via the AF processing unit 22 and obtains the image obtained from the video signal in response to a request from the image processing unit 26. The image data of the image is given to the image processing unit 26.

  Based on the image data acquired from the image input unit 28, the image processing unit 26 tracks the target specified by the operator on the screen by the pattern matching method or the inter-frame difference extraction method, and moves the target. Follow and move the AF point. Then, AF frame information indicating the position of the AF point is transmitted to the frame operation unit 14 via the serial communication interface 30, and the AF point for automatically changing the AF frame is designated. Note that information other than AF frame information is also exchanged between the frame operation unit 14 and the image processing unit 26 via the serial communication interface 30.

  Next, the tracking process in the image processing unit 26 in the tracking device 16 will be described with reference to the flowchart of FIG. First, the operator operates the operation member of the frame operation unit 14 while viewing the image displayed on the screen of the viewfinder 18 and the AF frame, and performs AF on all or a part of the object to be tracked (focused). Fit in the frame. As a result, the object is brought into focus by AF processing in the lens CPU 20. Subsequently, the operator presses the determination button of the frame operation unit 14. As a result, an instruction to set a reference pattern is given from the frame operation unit 14 to the image processing unit 26. The image processing unit 26 reads AF frame information indicating the setting contents of the AF frame at that time from the frame operation unit 14 via the serial communication interface 30, and is given to the image input unit 28 from the AF processing unit 22 of the lens device 12. The image (image data) in the range of the AF frame acquired by the AF frame information is captured from the image input unit 28 among the video signals (images for one screen of the entire shooting range) (step S10). Then, the image is set and registered (stored) as a reference pattern (step S12). If the video signal given from the AF processing unit 22 to the image input unit 28 is, for example, an interlaced video signal in which one screen (screen for one frame) is composed of screens for two fields, image input is performed. An image (image data) captured as one screen from the unit 28 may be an image (image data) obtained from a video signal for one field or an image obtained from a video signal for two fields. Good.

  In the setting of the reference pattern, the size of the reference pattern may be automatically set according to the type of the object, not the size of the AF frame. For example, the type of the object can be selected by a predetermined switch of the frame operation unit 14, and when a person is selected as the object, for example, the average face size of the person is automatically set as the reference pattern size. If set, a reference pattern having an appropriate size according to the type of the object may be set.

  Subsequently, the image processing unit 26 repeatedly executes the following steps S14 to S38. First, a new image (image data) for one screen is fetched from the image input unit 28 (step S14). The image acquired in step S14 is referred to as a B image. Then, in a predetermined pattern detection range set within the screen range of the B image, the position of the image that matches the reference pattern image is detected by a known pattern matching process (step S16). Here, the pattern detection range is, for example, a range slightly larger than the reference pattern with the currently set AF frame as the vertical and horizontal centers. As a result, the time and processing load required for the pattern matching process can be reduced.

  Next, the image processing unit 26 determines whether or not a reference pattern is detected within the pattern detection range in step S16 (step S18). That is, it is determined whether or not there is an image that matches the image of the reference pattern within the pattern detection range.

  If the determination is YES, then, based on the position of the detected reference pattern image, that is, the position on the screen of the target set as the reference pattern, the target is compared with the position at the previous detection. It is determined whether or not it has moved on the screen (step S20). If YES is determined, the AF point is displaced in the same direction as the moving direction by the amount of movement of the object, and AF frame information indicating the AF point is transmitted to the frame operation unit 14. Thereby, the setting content of the AF frame, that is, the position of the AF frame is updated (step S22), and the AF frame is moved to the position of the object. If it is determined NO in step S20, the process of step S20 is not executed. The AF point (the position of the AF frame) may be set based on the position of the image of the reference pattern detected without performing the determination in step S20.

  Next, the image processing unit 26 updates the image of the reference pattern with the image detected from the B image as being matched with the image of the reference pattern by the pattern matching process in step S16 (step S24). That is, an image detected from the B image as being matched with the image of the reference pattern in step S16 is set and registered as a new reference pattern. As a result, even if the image of the object to be tracked changes, there is little change from the image of the object stored as the reference pattern, and it becomes easy to detect the image of the object by pattern matching processing. When the process of step S24 ends, the process returns to step S14.

  If it is determined as NO in step S18, that is, if the reference pattern image is not detected within the pattern detection range, the image processing unit 26 sets the reference pattern image for the entire screen range of the B image. The position of the image that matches is detected by pattern matching processing (step S26). Then, it is determined whether or not a reference pattern image is detected in step S26 (step S28). When it determines with YES, it transfers to said step S20, and the process to step S20-step S24 is performed similarly to the case where it determines with YES at step S18.

  On the other hand, if NO is determined in step S28, that is, the image of the object cannot be detected by the pattern matching process because the image of the object is rapidly changed even if the image of the reference pattern is updated in step S24. For example, the object is tracked by the inter-frame difference extraction method from step S30 to step S40. First, the image processing unit 26 captures image data for one screen from the image input unit 28 (step S30). This image is referred to as A image. Subsequently, an absolute value of a difference between pixel values of corresponding pixels of the A image captured in step S30 and the B image captured in step S14 is obtained, and image data of the difference image C between the A image and the B image is obtained. Obtained (step S32). Then, the image data of the difference image C is binarized, and the center of gravity and area of the pixel having the pixel value of 1 are obtained (step S34). Subsequently, it is determined whether or not the object has moved according to whether or not the area is larger than a predetermined threshold and the center of gravity has been displaced with respect to the center of gravity detected last time (step S36). When it is determined YES, the AF point is displaced in the same direction as the moving direction by the amount of movement of the center of gravity, and AF frame information indicating the AF point is transmitted to the frame operation unit 14. Thereby, the setting content of the AF frame, that is, the position of the AF frame is updated (step S38). When the above processing ends, the process proceeds to step S24, and the reference pattern image is updated. That is, even when the tracking process is performed by the inter-frame difference extraction method, the position of the object can be detected. Based on the detected position, an image of the object is extracted from, for example, the A image, and the image is used as a reference. Set and register as a pattern.

  When the process in step S24 is completed, the process returns to the process from step S14 and the process in step S16 is performed. The pattern detection range at this time is, for example, the difference image C between the A image and the B image in step S32. It is preferable that the range is a little larger than the minimum rectangular range including the pixel having a pixel value of 1 when the image data is binarized.

  According to the above tracking process, the image of the reference pattern is periodically updated by the process of step S24, so that the object moves or the imaging condition of the camera changes (zoom magnification, imaging position, etc.). Thus, even when the image of the object changes, the position of the object can be detected by the pattern matching process, and the AF frame can be accurately tracked on the object.

  As described above, in the above embodiment, the frame frequency of the video signal acquired by the AF imaging circuit 24 (AF imaging device) of the lens device 12 is the frame frequency of the video signal obtained by the imaging device of the camera head 10. However, by increasing the frame frequency of the video signal in the AF imaging circuit 24, adaptability to an object that moves at high speed or an object that changes rapidly is improved.

  In the above embodiment, only one reference pattern (object) is registered and set. However, a plurality of reference patterns (objects) can be registered and set, and a plurality of set reference patterns can be set. Of these, a reference pattern to be actually used may be selected with a switch or the like. This is effective when, for example, a person who wants to focus is switched by AF when, for example, a plurality of persons are photographed simultaneously or alternately in a dialogue program or the like.

  Moreover, in the said embodiment, although the lens apparatus 12, the frame operation part 14, and the tracking apparatus 16 were illustrated as a separate apparatus, any two or all may be an integral apparatus. In the above embodiment, an operation member (for example, a determination switch for determining an object) related to the processing of the tracking device 16 is also provided in the frame operation unit 14, but may be provided in the tracking device 16.

  In the above-described embodiment, the image signal is acquired for the AF by using an image sensor dedicated for AF that is different from the image sensor for the camera head 10, but the image obtained by the image sensor for the camera head 10 for AF is used. A signal may be used. However, if the AF video signal is obtained from the AF-dedicated imaging device as in the above embodiment, the HD signal is obtained when the camera head 10 is compatible with a high-definition (HD) television system. Since it is possible to perform AF without using a lens, it is advantageous for miniaturization and power saving.

FIG. 1 is a block diagram showing the overall configuration of an imaging system to which an autofocus system according to the present invention is applied. FIG. 2 is a diagram showing an AF frame. FIG. 3 is a flowchart showing the procedure of the tracking process in the tracking device.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Camera head, 12 ... Lens apparatus, 14 ... Frame operation part, 16 ... Tracking apparatus, 18 ... Viewfinder, 20 ... Lens CPU, 22 ... AF process part, 24 ... Imaging circuit for AF, 26 ... Image process part, 28: Image input unit, 30: Serial communication interface

Claims (3)

In the autofocus system that controls the focus of the photographic lens with the subject in the AF area of the shooting range of the camera as the target of autofocus, and automatically focuses on the subject,
Imaging means for sequentially acquiring images of subjects within the imaging range;
Target subject designating means for designating a target subject to be focused by the autofocus;
Storage means for acquiring and storing the target subject image designated by the target subject designation means as a reference image from the imaging means;
Target subject detection means for sequentially detecting the image of the target subject from images sequentially acquired by the imaging means by detecting an image that matches the reference image from images acquired by the imaging means;
AF area changing means for changing the position of the AF area so as to follow the target subject based on the position of the image of the target subject detected by the target subject detecting means within the shooting range;
Updating means for updating the reference image stored in the storage means with the image of the target subject detected by the target subject detection means;
An autofocus system characterized by comprising   The target subject detection means limits the range of the image acquired by the imaging means to a part of the range including the image range of the target subject detected last time, and selects the reference from the images in the range. 2. The autofocus system according to claim 1, wherein an image that matches the image is detected.   The target subject detection unit detects an image of the target subject by obtaining a difference between two images sequentially acquired by the imaging unit when an image matching the reference image is not detected. The autofocus system according to claim 1.

Images