JP2016058764A - Imaging device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To clearly display a distinction between subjects even if a subject out of the field depth of a captured image is made unclear by blurring, by performing consecutive imaging while changing a focal position under an imaging condition of shallow field depth.SOLUTION: An imaging apparatus includes: image capture means for capturing a plurality of images with different focal positions with respect to the same subject; distance information acquisition means for acquiring focal distance information for each divided region of the plurality of images; distance information image generation means for generating a distance information image corresponding to the distance information of the divided region; display means for displaying the captured image; and coordinate designation means for designating coordinates corresponding to the divided region in the display image. The display means displays at least a part of the distance information image for selecting an image focused on the coordinates designated by the coordinate designation means together with the captured image.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an apparatus for performing continuous shooting while changing a focus position and selecting a desired image from a plurality of acquired images.

  A camera having an AF function is known which has a function of continuously shooting a plurality of images while changing a focus position for one release operation, such as focus bracket shooting. There has also been proposed a digital camera having a configuration for obtaining an image focused on a desired subject from images captured by changing a focus position and taking a predetermined number of images (Patent Document 1). In the same configuration, information on areas obtained by dividing the screen and AF evaluation values calculated for each area are recorded as a pair together with a plurality of images taken continuously while changing the focus position. Then, the user selects one of the divided areas from the display images displayed on the monitor, and acquires an image focused on the subject in the selected area based on the AF evaluation value for the same area of each image.

JP 2009-089348 A

  However, if continuous shooting such as a focus bracket is performed under shooting conditions where the depth of field is shallow, when the acquired image is displayed, subjects outside the depth of field become blurred due to blurring and other subjects It becomes difficult to distinguish between Therefore, it is difficult to specify an area (subject) with reference to such an image.

  The present invention has been made in view of the above problems, and performs continuous shooting while changing the focal position under shooting conditions with a shallow depth of field, and subjects outside the depth of field of the acquired image are blurred. Even if the image becomes unclear, it is an object to clearly display the distinction between subjects.

  An imaging apparatus of the present invention includes an image acquisition unit that acquires a plurality of images with different focal positions for the same subject, a distance information acquisition unit that acquires focusing distance information for each divided region of the plurality of images, A distance information image generating means for generating a distance information image according to the distance information; a display means for displaying the acquired image; and a coordinate specifying means for specifying coordinates corresponding to the divided areas in the display image. Further, at least a part of the distance information image for selecting an image focused at the coordinates designated by the coordinate designating means is displayed together with the acquired image.

  It is preferable that the distance information image is displayed without being updated, and the acquired image is updated and displayed according to the selection of the coordinate designating means. Before displaying a new acquired image whose designated coordinates are updated by the coordinate designating means, at least one image created by α blending the acquired images corresponding to the preceding and following coordinate areas is displayed.

For example, the distance information image is transmitted and displayed overlaid on the entire acquired image, or the distance information image is displayed superimposed on at least a part of the acquired image. Further, the distance information image is superimposed only on the area based on the distance information indicated by the area selected by the area selection unit, for example. Further, the distance information image and the acquired image may be displayed side by side. The coordinate instruction means is, for example, a touch panel. The in-focus position may be changed automatically, for example, or manually.

  A reproducing apparatus according to the present invention includes the above-described imaging apparatus.

  According to the computer program of the present invention, a procedure for acquiring a plurality of images with different focal positions for the same subject, a procedure for acquiring in-focus distance information for each divided region of the plurality of images, and the distance information of the divided regions The procedure for generating the distance information image, the procedure for displaying the acquired image, the procedure for designating the coordinates corresponding to the divided areas in the display image, and the display means are focused at the coordinates designated by the coordinate designating means. And causing a computer to execute a procedure for displaying at least a part of a distance information image for selecting a selected image together with an acquired image.

  According to the present invention, continuous shooting is performed while changing the focal position under shooting conditions with a shallow depth of field, and even if a subject outside the depth of field of the acquired image becomes blurred due to blur, the subject The distinction between them can be clearly displayed.

1 is a block diagram illustrating a schematic configuration of a digital camera according to an embodiment of the present invention. It is a flowchart of the continuous shooting which changes the focus position including the subject selection processing of the present embodiment. It is the first half of the flowchart of a distance map creation process. It is the second half part of the flowchart of a distance map creation process. It is a figure which illustrates typically the content until contrast map preparation. It is a figure which shows the method of scanning a contrast value in the contrast map with respect to several through images. It is a graph which shows an example of the relationship between the contrast map number attached | subjected to several through images, and contrast value. It is a figure which shows an example of a distance map. It is a graph which shows a correspondence with contrast map number (horizontal axis) and each pixel value (vertical axis) of RGB. It is a figure which shows an example of a distance information image. It is a figure which illustrates typically the flow of a process from coordinate designation to image display. It is a figure which shows a state when the designated coordinate is changed. It is a figure which shows the behavior of a display image when the designated coordinate is changed. It is a figure which performs the coordinate designation | designated by swipe, and description of the stability of a display image. It is a figure which illustrates the aspect which displays both a display image and a distance information image.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing a schematic configuration of a digital camera according to an embodiment of the present invention. In this specification, a digital camera includes a camera unit mounted on a small-sized electronic device such as a single-lens reflex camera or a mobile phone, but in this embodiment, a compact camera will be described as an example.

  In the digital camera 10, the subject image is picked up by the image pickup device 12 through the photographing lens 11. The driving of the image sensor 12 is controlled by the image sensor driver 13, and the image sensor driver 13 is controlled based on a command from the control unit 14. The image acquired by the image sensor 12 is sent to the control unit 14 and subjected to various digital image processing, and is temporarily stored in, for example, the image memory 15 and stored in, for example, the recording medium 16 according to the user's decision. Is done. The image acquired by the image sensor 12 or the image stored in the image memory 15 and the recording medium 16 can be displayed on a monitor 17 such as an LCD.

  The photographing lens 11 is driven by a lens driving unit 18, and the diaphragm 19 is driven by a diaphragm driving unit 20. A contrast method is employed for the AF processing, and the control unit 14 controls the lens driving unit 18 with reference to the contrast of the image captured by the image sensor 12 and controls the aperture driving unit 20 based on the photometric value.

  The digital camera 10 also includes an image stabilization unit (camera shake correction mechanism) 22. In this embodiment, the image sensor shift method is used as the camera shake correction mechanism, but other methods such as a lens shift method may be used. The image stabilization unit 22 includes a sensor for detecting the amount of camera shake as is well known in the art, and moves the image sensor 12 so as to cancel the camera shake to reduce the occurrence of image blur (hereinafter referred to as image blur) on the imaging surface. To prevent. The drive of the image stabilization unit 22 is controlled by the control unit 14, and the drive amount in the image stabilization unit 22, for example, the amount of camera shake and the shift amount of the image sensor 12, is an image stabilization drive result for evaluation of pseudo contour component generation to be described later. It is sent to the control unit 14. Note that camera shake correction is controlled independently of the driving of the image sensor, and the driving is sufficiently faster than the frame rate.

  The digital camera 10 includes a switch group 23 and a touch panel 21 that are connected to the control unit 14 and include various switches such as a release switch, a photometric switch, a dial switch for selecting / setting various functions, and a four-way switch. And are provided.

  Next, continuous shooting for changing the focus position including subject selection processing according to the present embodiment will be described with reference to the flowcharts of FIGS. 1 and 2. The flowchart of FIG. 2 is a process executed by the control unit 14.

  In step S100, for example, it is determined whether or not a release button (not shown) is fully pressed and a release switch in the switch group 23 is turned on. When the release switch is not turned on, that is, when the release button is not fully pressed, continuous shooting is not started and the same determination is repeated. If it is determined in step S100 that the release switch has been turned on, driving of the image stabilizing unit is started in step S101.

  In a state where the image stabilization unit 22 is performing the image stabilization operation, the photographing lens 11 is moved by the lens driving unit 18 in step S102, and a plurality of positions between a short distance and a long distance (for example, infinity) are focused. In accordance with (Np locations) (for example, the lens is moved in a predetermined step), one image is taken at each position, and Np consecutive acquired images are stored in the image memory 15 as a result. Note that the process of step S102 is performed at high speed, and Np images are acquired during this time.

  Note that the shift generated between Np images due to the image stabilization drive is reduced, and the positional accuracy between the images is increased. As a result, the block formation (area division) described later can be made finer, and the accuracy of coordinate selection can be improved. It is also possible to adjust the block size (area) according to the amount of camera shake. At this time, the movement of the photographic lens 11 may be manually operated regardless of the lens driving unit 18.

  After completing the image acquisition in step S102, in step S104, a distance map creation process using the through image stored in the image memory 15 in step S102 is executed. The distance map is map data in which distance information to the subject in each region (block) of the main image (or through image) is recorded. In this embodiment, the distance information includes a contrast map number (to be described later) corresponding to the lens position. Used. The details of the distance map creation process will be described later.

  In step S105, one of the images stocked in step S102 is displayed on the single monitor 17. For example, an image focused on the subject determined to be closest is displayed. At this time, a distance information image obtained by visualizing a distance map described later is displayed on the monitor 17. In step S106, the user selects a block (area) desired to be focused, that is, a subject from the image displayed on the monitor in step S105, and obtains coordinates corresponding to the block.

  In step S107, the distance map is referred to, and the frame image having the contrast map number assigned to the coordinates specified in step S106 is called from the memory. In step S108, the image of the same frame is displayed on the monitor together with the above-described distance information image. .

  In step S109, it is determined whether or not there has been an image storage instruction by a predetermined operation of the touch panel 21 or the switch group 23. If there is an image storage instruction, the image of the frame on which the monitor display is performed in step S108 is performed in step S110. The data is stored in the recording medium 16 and a series of continuous shooting processing is completed. On the other hand, when the image saving instruction is not detected, the process returns to step S106, and the processes after step S106 are repeated. In the same iterative process, the distance information image is continuously displayed, and only the display frame image is updated in step S108, so that the change of the selected frame image, that is, the change of the subject of interest can be recognized. Then, it is possible to easily select a desired subject with reference to the distance information image again.

  In the continuous shooting, the depth of field may be evaluated from the aperture and lens information, and when the depth of field is shallow, the predetermined step of step S102 may be made finer and the number of acquired frames may be increased. .

  3 and 4 are flowcharts of the distance map creation process executed in step S104 of FIG. 2, FIG. 3 is the first half of the process, and FIG. 4 is the second half of the process. 5 to 8 are diagrams for explaining the contents of the processing performed in FIGS. 3 and 4. The distance map creation process will be described below with reference to FIGS.

  In the distance map creating process, first, in step S200, each through image stored in the image memory 15 in step S102 of FIG. 2 is subjected to a contour extracting process such as a high-pass filter or a differential filter to extract a contour component. Here, FIG. 5A is an example of a through image before contour extraction, and FIG. 5B is an example of an image obtained by performing contour extraction processing on the image of FIG. In step S202, as illustrated in FIG. 5C, the image (FIG. 5B) from which the contour component is extracted is divided into M × N blocks (regions). N is the number of blocks in the vertical direction, for example, and is a value equal to or smaller than the number of pixels in the vertical direction of the through image. M is, for example, the number of blocks in the horizontal direction, the number of blocks in the horizontal direction of the through image, and a value equal to or less than the number of pixels in the vertical direction of the through image. In addition, the block in this embodiment includes what consists of only one pixel.

  In step S204, the sum total of contour components is calculated for each block. That is, in the image from which the contour component is extracted (FIG. 5B), the sum of the pixel values in each block is calculated as the contrast value of the block (value for evaluating the contrast level). In step S206, the contrast value calculated for each block is stored in a buffer memory (not shown) of M × N size as a contrast map (contrast high / low evaluation map) for each through image. FIG. 5D schematically shows an example of a contrast map corresponding to the block shown in FIG. 5C. The brighter the block, the higher the contrast value of the block.

  Next, in step S208, it is determined whether or not the above processing has been performed on all the Np through images stocked (stored) in the image memory 15. If the above processing has not been completed for all the through images, the processing returns to step S200 and the above processing (steps S200 to S206) is performed on the unprocessed images among the stocked through images.

  On the other hand, if it is determined in step S208 that all Np through images stocked in the image memory 15 have been processed in steps S200 to S206, the process proceeds to step S210. As shown in e), Np contrast maps are retained.

  As schematically shown in FIG. 6, the distance map is created by referring to the Np contrast maps created in steps S200 to S208 for each block. First, in step S210, an array of M × N blocks for storing the distance map is secured in the memory. In step S212, an initial position of a block (target block) to be processed among the M × N blocks is set.

  For example, when M × N blocks in each of Np contrast maps are referred to a block of i rows and j columns with (i, j) as the base point in the upper left of FIG. 6, the process is i = 1, j = 1 starts. In the following processing, the block of interest is first moved from left to right in the same row, then moved to the next row, and the same movement is sequentially repeated.

  In step S214, the contrast value of the target block (i, j) is scanned in the Np contrast maps, and in step S216, the number of the contrast map that maximizes the contrast value is calculated. For example, when there are 23 contrast maps (Np = 23), when the contrast value of the target block (i, j) changes as shown in FIG. 7 with respect to the contrast map number, the contrast map number that maximizes the contrast value 20 is detected.

  In step S218, the contrast map number detected in step S216 is recorded in the corresponding block array of the distance map secured in step S210. In step S220, it is determined whether or not the above processing has been completed for all M × N blocks, that is, whether or not contrast map numbers have been detected and recorded for all M × N blocks of the distance map. If not completed, the process moves to the next block of interest in step S222, and the processes in and after step S214 are repeated.

  On the other hand, if it is determined in step S220 that the contrast map number has been detected and recorded for all of the M × N blocks, the creation of the distance map ends, and the process moves to step S224. At this time, the number of the contrast map having the highest contrast value in each block is recorded in the array corresponding to each block of the distance map, as illustrated in FIG. That is, it corresponds to the lens position considered to be the most focused in the block, and this corresponds to the distance to the subject in the block. In the example of FIG. 8, the greater the contrast map number, the greater the distance. In correspondence with FIG. 5A, the block with the contrast map number 8 corresponds to the person who is the main subject, and the block with the contrast map number 20 A background that is a distant view, a block with a contrast map number of 13 corresponds to a subject in the middle, for example, a tree immediately behind the main subject.

  In step S224, a value indicating the anti-vibration driving performance is acquired from the anti-vibration unit 22, and it is determined whether this value is below a predetermined threshold. That is, when it is considered that the amount of image blur is small and the occurrence of a pseudo contour component described later is small, a median filter having a size of 3 × 3 blocks is set in step S226. On the other hand, if it is considered that the image blur is large and the pseudo contour component generated thereby is large in step S224, a median filter having a size of 5 × 5 blocks is set in step S228. That is, when the image blur is evaluated to be large (the pseudo contour component is large), the median filter region is enlarged.

  In step S230, the median filter having a size of 3 × 3 blocks or 5 × 5 blocks selected in either step S226 or step S228 is applied to the distance map data created in steps S214 to S222. That is, in step S230, the pseudo contour component (noise) mixed in the distance map is removed from the distance map based on camera shake, and the distance map creating process (step S114) of the present embodiment is thereby completed.

  Next, with reference to FIGS. 9 and 10, an example of creating a distance information image created in order to display distance map data as a visual image in this embodiment will be described. In the present embodiment, the distance information image displays the contrast map number (distance information) assigned to each block (area) in a thermal image style by color coding. FIG. 9 is a graph showing the correspondence between contrast map numbers (horizontal axis) and RGB pixel values (vertical axis). That is, for each block of the distance information image, the RGB pixel values in FIG. 9 corresponding to the contrast number associated with the block are displayed as an image.

  In the example of FIG. 9, the R pixel value is high on the close side, the G pixel value is high in the intermediate range, and the B pixel value is high on the infinity side. Therefore, as shown in FIG. 10, the distance information image is displayed in red as the block corresponding to the closest side, blue as the block corresponding to the infinity side, and the block corresponding to the middle is displayed in green.

  Next, the flow of processing from the coordinate designation (step S106 in FIG. 2) to the image display (step S108 in FIG. 2) in this embodiment will be described with reference to FIG.

  FIG. 11A shows a desired image in the image displayed on the monitor 17 at step S106 by using the touch panel 21 provided on the monitor 17 to arbitrarily select (for example, the closest image) in step S105. A finger is touched on the subject and the coordinates in the screen (image) are designated (input). FIG. 11B schematically shows a state of referring to the distance map corresponding to the coordinates input in FIG. 11A. The position of the user's finger in the distance information image of the thermal image wind and the same coordinates are shown. The correspondence of RGB contrast is shown. In FIG. 11, a subject indicated by a solid line represents a focused subject, and a subject indicated by a broken line represents a subject that is not in focus.

  If the coordinates are designated, the corresponding block is specified, and the distance map value (contrast map value) assigned to the block is known (step S107 in FIG. 2). Thus, as shown in FIG. 11C, an image to be selected from the images held in the buffer memory is specified from the specified distance map value (contrast map value) (step S107). As shown in FIG. 11D, an image focused on the distance corresponding to the contrast map value of the designated coordinates is displayed on the monitor 17 (step S108 in FIG. 2). In this process, the distance information image is also displayed on the monitor 17 at the same time, but the state is not drawn in the monitor display of FIG. The display method of the distance information image will be described later with reference to FIG.

  Next, referring to FIG. 12 and FIG. 13, in the repetitive processing of steps S106 to S109 in FIG. 2, the display image is switched when the target object to be selected is changed and the designation (input) of coordinates is changed. It is a figure which shows a mode.

  As shown in FIG. 12, when there are three subjects (indicated by rectangular areas) S1, S2, S3 (for example, closer to the left side) in order from the left on the screen, the user first selects the subject S1. Then, the case where the selection is switched to the subject S3 will be described as an example. FIG. 13 illustrates three modes of display image switching.

  FIG. 13A shows an example in which an image focused on the subject S1 is immediately switched to an image focused on the subject S3. FIG. 13B illustrates an example in which an image focused on the subject S2 at the intermediate position is displayed, and then an image focused on the designated subject S3 is displayed. FIG. 13C shows an example in which an image obtained by alpha blending an image focused on the subject S3 with an image focused on the subject S1 is displayed once, and then an image focused on the subject S3 is displayed. Show.

  In FIG. 13 (a), the emphasis is on switching between long and short distances or when the distance difference is small. In the examples of FIGS. 13 (b) and 13 (c), the designated coordinates are changed from subject S1 to subject S3. When the screen is slid (swipe), the display image changes continuously, improving the usability. The camera adopts all these display methods, and may switch according to the user's selection, or any one display method. Further, the display switching method is not limited to this embodiment.

  With reference to FIG. 14, a behavior when the user selects a subject by a swipe operation on the touch panel 21 in the repeated processing of steps S <b> 106 to S <b> 109 in FIG. 2 will be described. FIG. 14A schematically shows how the user swipes around the subject on the touch panel 21 of the monitor 17. If a swipe is performed over a wide area in a state where the image stabilization unit 22 is not driven, the selected subject can change continuously over time, so the display image can change continuously accordingly. (FIG. 14B). When the image stabilization unit 22 is driven, there is almost no displacement of the subject between images, so that the subject shakes even when the display image is switched following the swipe as shown in FIG. There is no problem because there is no. On the other hand, when the image stabilization unit 22 is not used, the subject image is misaligned between the plurality of images taken in step S102 due to camera shake, so that the display image is switched in accordance with the swipe. The phenomenon that the displayed subject shakes occurs. Accordingly, in the present embodiment, when performing shooting in the continuous shooting mode of FIG. 2, for example, the image stabilization unit 22 is always driven.

  A method for displaying the distance information image on the monitor 17 in steps S106 to S109 will be described with reference to FIGS. 15 (a) to 15 (d). 15A to 15D exemplify different display methods, and a subject indicated by a solid line represents a focused subject and a subject indicated by a broken line represents a subject that is not in focus.

  FIG. 15A is an example in which the distance information image is superimposed at the same position and the same size on the display image selected in step S108. FIG. 15B is an example in which the reduced distance information image is displayed in the picture-in-picture format in the selected display image. FIG. 15C shows an example in which only the region corresponding to the designated subject is superimposed. FIG. 15D shows an example in which a reduced distance image is displayed in a region where there is no display image in the selected display image. It should be noted that a plurality of these display methods can be adopted and selected by the user, or only one can be adopted. Further, the display methods are not limited to these, and for example, distance information The image and the acquired image may be displayed side by side.

  As described above, according to the present embodiment, continuous shooting is performed while changing the focal position under shooting conditions with a shallow depth of field, and a subject outside the depth of field of the acquired image is blurred due to blurring. Even so, the distance information image makes the distinction between the subjects clear and easy to select.

  Further, according to the present embodiment, even if the designated coordinates are continuously updated and the display is repeated, the display image does not fluctuate, and the image switching is smooth. Furthermore, since the user can specify the coordinates (subject designation) while recognizing the distance of the subject, it is possible to provide the user with the same function as refocusing.

  In the present embodiment, the same control can be performed as long as it is not only an electronic device having a camera function, but also an apparatus or a system that can receive and calculate captured image information. For example, it is also possible to transfer data to a playback device equipped with a personal computer or a control unit using a recorded SD card or a communication function.

DESCRIPTION OF SYMBOLS 10 Digital camera 11 Shooting lens 12 Image pick-up element 13 Image pick-up element driver 14 Control part 15 Image memory 16 Recording medium 17 Monitor 18 Lens drive part 19 Aperture 20 Aperture drive part 21 Touch panel 22 Anti-vibration unit 23 Switch group

Claims (12)

Image acquisition means for acquiring a plurality of images having different focal positions for the same subject;
Distance information acquisition means for acquiring in-focus distance information for each of the divided areas of the plurality of images;
A distance information image generating means for generating a distance information image according to the distance information of the divided area;
Display means for displaying the acquired image;
A display image, comprising: coordinate designating means for designating coordinates corresponding to the divided areas;
The image pickup apparatus, wherein the display means displays at least a part of a distance information image for selecting an image focused at the coordinates designated by the coordinate designation means together with an acquired image.   The imaging apparatus according to claim 1, wherein the distance information image is displayed without being updated, and the acquired image is updated and displayed according to the selection of the coordinate designating unit.   Before displaying a new acquired image whose designated coordinates are updated by the coordinate designating means, at least one image created by alpha blending the acquired images corresponding to the coordinate areas before and after the update is displayed. The imaging apparatus according to claim 1.   The imaging apparatus according to claim 1, wherein a distance information image is transmitted and superimposed on the acquired image.   The imaging apparatus according to any one of claims 1 to 3, wherein a distance information image is displayed so as to be superimposed on at least a part of the acquired image.   The imaging apparatus according to claim 1, wherein the distance information image is superimposed only on a region based on the distance information indicated by the region selected by the region selection unit.   The imaging apparatus according to any one of claims 1 to 3, wherein the distance information image and the acquired image are displayed side by side.   The imaging apparatus according to claim 1, wherein the coordinate instruction unit is a touch panel.   The imaging apparatus according to claim 1, wherein the focal point position is automatically changed.   The imaging apparatus according to claim 1, wherein the focus position is changed manually.   A playback device comprising the imaging device according to claim 1. A procedure for acquiring a plurality of images with different focal positions for the same subject;
A procedure for acquiring in-focus distance information for each divided region of the plurality of images;
A procedure for generating a distance information image according to the distance information of the divided region;
Displaying the acquired image;
In the display image, a procedure for designating coordinates corresponding to the divided areas;
The display means causes a computer to execute a procedure of displaying at least a part of a distance information image for selecting an image focused at the coordinates designated by the coordinate designation means together with an acquired image. Computer program.