JP5659856B2 - Imaging apparatus, imaging method, and program - Google Patents

Description

  The present invention relates to an imaging apparatus, an imaging method, and a program.

  Recently, there is an increasing demand for a digital camera capable of shooting a stereoscopic image (3-Dimensional Picture, hereinafter also referred to as “3D image”). As a general 3D digital camera capable of capturing a 3D image, a 3D digital camera including two imaging units including an image sensor and a photographing lens is known (for example, see Patent Document 1).

  There is also known a 3D digital camera that generates 3D image data by connecting the data of a plurality of images shot in a panoramic manner around a predetermined point by calculating the parallax of the left and right eyes ( For example, refer nonpatent literature 1).

JP 2001-66547 A

Sony Marketing Co., Ltd., “Q & A Support / Inquiry” “What is the 3D swing panorama function?” <Online>, [Search on February 9, 2011], Internet <URL: http: // www. faq. sonydrive. jp / faq / 1040 / app / servlet / qadoc? 035265>

  However, in the techniques described in Patent Document 1 and Non-Patent Document 1, for the user, if the user does not actually visually recognize the 3D image actually obtained after the shooting / recording operation such as pressing the shutter button fully, There has been a problem that the stereoscopic effect of the 3D image cannot be grasped.

In other words, the user cannot confirm the stereoscopic effect before the shooting / recording operation even though the user wants to immediately confirm the stereoscopic effect of the 3D image before the shooting / recording operation. For this reason, in order to acquire a 3D image having a stereoscopic effect (a 3D image desired by the user), the user needs to perform a shooting / recording operation assuming a composition having a stereoscopic effect.
However, the stereoscopic effect of the 3D image changes according to the composition at that time (the situation of the subject included in the angle of view), and how much stereoscopic effect the 3D image that will be obtained after the shooting and recording operation has. It has been very difficult for the user to properly assume whether or not he / she has it. For this reason, many 3D images obtained by a single shooting and recording operation often do not have the stereoscopic effect desired by the user. Therefore, in order to finally obtain a 3D image with a desired three-dimensional effect, the user has to perform a wasteful and complicated operation such as changing the composition and repeating the photographing and recording operation many times.

  The present invention has been made in view of such circumstances, and predicts the degree of stereoscopic effect of a 3D image that will be obtained after the shooting and recording operation and presents it to the user before the shooting and recording operation. The purpose is to make this easy to implement.

In order to achieve the above object, an imaging device of one embodiment of the present invention includes:
Imaging means for capturing a plurality of subjects and obtaining images;
Feature region extraction means for extracting a plurality of feature regions, each indicating an image of the plurality of subjects included in an image picked up by the image pickup means;
A focal length detection means for detecting a focal length for the plurality of feature areas extracted by the feature area extraction means;
Of the focal lengths detected by the focal length detection means, a focal length comparison means for comparing a focal length for a main feature region and a focal length for other feature regions;
Stereoscopic effect calculating means for calculating the degree of stereoscopic effect of the stereoscopic display when it is assumed that stereoscopic display is performed using the image captured by the imaging means based on the comparison result of the focal length comparing means; ,
Presenting means for presenting the stereoscopic effect calculated by the stereoscopic effect calculating means;
It is characterized by providing.

  According to the present invention, it is possible to predict the degree of stereoscopic effect of a 3D image that will be obtained after a shooting / recording operation and present it to the user before the shooting / recording operation.

It is a block diagram which shows the structure of the hardware of the imaging device which concerns on one Embodiment of this invention. It is a functional block diagram which shows the functional structure for performing 3D effect determination processing among the functional structures of the imaging device of FIG. 3 is a flowchart illustrating a flow of mode setting processing executed by the imaging apparatus of FIG. 1 having the functional configuration of FIG. 2. 3 is a flowchart for explaining a flow of imaging processing executed by the imaging apparatus of FIG. 1 having the functional configuration of FIG. 2. 5 is a flowchart for explaining the flow of 3D effect determination processing in the imaging processing of FIG. 4 executed by the imaging apparatus of FIG. 1 having the functional configuration of FIG. 2. 6 shows a display example of a display of the output unit during the 3D effect determination process of FIG. It is a figure which shows each focal distance about each feature area shown in the example of a display of FIG. It is a figure which shows an example of the screen as which the processing result of the 3D effect determination process of FIG. 5, ie, the symbol image which shows a three-dimensional effect, was superimposed on the through image. FIG. 6 shows an example of the result of the 3D effect determination process in FIG. 5 when all of the subject distances of the subjects corresponding to the feature areas included in the through image are short. FIG. 6 shows an example of the result of the 3D effect determination process in FIG. 5 when the subject distances of the subjects corresponding to the feature regions included in the through image are separated (split). FIG. 6 shows an example of the result of the 3D effect determination process in FIG. 5 when all of the subject distances of the subjects corresponding to the feature regions included in the through image are long.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

FIG. 1 is a block diagram showing a hardware configuration of an imaging apparatus according to an embodiment of the present invention.
The imaging device 1 is configured as a digital camera, for example.

  The imaging apparatus 1 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a bus 14, an input / output interface 15, an imaging unit 16, and an input unit 17. An output unit 18, a storage unit 19, a communication unit 20, and a drive 21.

  The CPU 11 executes various processes according to a program recorded in the ROM 12 or a program loaded from the storage unit 19 to the RAM 13.

  The RAM 13 appropriately stores data necessary for the CPU 11 to execute various processes.

  The CPU 11, ROM 12, and RAM 13 are connected to each other via a bus 14. An input / output interface 15 is also connected to the bus 14. An imaging unit 16, an input unit 17, an output unit 18, a storage unit 19, a communication unit 20, and a drive 21 are connected to the input / output interface 15.

  The imaging unit 16 includes an optical lens unit 31 including a focus lens 31F and a zoom lens 31Z, and an image sensor 32, as will be described with reference to FIGS. 7 and 9 to 11 described later.

The optical lens unit 31 includes various lenses such as a focus lens 31F and a zoom lens 31Z in order to photograph a subject.
The focus lens 31F is a lens that forms an image of a subject on the light receiving surface of the image sensor 32. The zoom lens 31Z is a lens that freely changes the focal length within a certain range.
The optical lens unit 31 is also provided with a peripheral circuit for adjusting setting parameters such as focus, exposure, and white balance as necessary.

The image sensor 32 includes a photoelectric conversion element, an AFE (Analog Front End), and the like.
The photoelectric conversion element is composed of, for example, a CMOS (Complementary Metal Oxide Semiconductor) type photoelectric conversion element or the like. A subject image is incident on the photoelectric conversion element from the optical lens unit. Therefore, the photoelectric conversion element photoelectrically converts (captures) the subject image, accumulates the image signal for a predetermined time, and sequentially supplies the accumulated image signal as an analog signal to the AFE.
The AFE performs various signal processing such as A / D (Analog / Digital) conversion processing on the analog image signal. Through various signal processing, a digital signal is generated and output as an output signal of the imaging unit 16.
Hereinafter, the output signal of the imaging unit 16 is referred to as “captured image data”. The captured image data is appropriately supplied to the CPU 11.

  The operation of the imaging unit 16 is controlled by the CPU 11. For example, the CPU 11 sets various imaging conditions of the imaging unit 16. For example, the CPU 11 sets the zoom magnification as one of the imaging conditions, and controls the lens driving unit (not shown) to drive the zoom lens 31Z so that the set zoom magnification is obtained. Further, for example, the CPU 11 sets a diaphragm for adjusting the amount of light incident on the imaging unit 16 as another imaging condition.

The input unit 17 includes various buttons such as a power button, a shooting mode setting button, a stereoscopic effect setting button, a shutter button, and the like, and inputs various information according to a user's instruction operation.
The output unit 18 includes a display, a speaker, and the like, and outputs images and sounds.
The storage unit 19 is composed of a hard disk, a DRAM (Dynamic Random Access Memory), or the like, and stores various image data.
The communication unit 20 controls communication performed with other devices (not shown) via a network including the Internet.

  A removable medium 22 made of a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is appropriately attached to the drive 21. The program read from the removable medium 22 by the drive 21 is installed in the storage unit 19 as necessary. The removable medium 22 can also store various data such as image data stored in the storage unit 19 in the same manner as the storage unit 19.

FIG. 2 is a functional block diagram illustrating a functional configuration for executing the 3D effect determination process among the functional configurations of the imaging apparatus 1 as described above.
In the 3D effect determination process, each feature region is extracted from the through image, and a difference in focal length between each of the main feature region (hereinafter referred to as “important feature region”) and other feature regions is obtained. Based on this, it means a series of processes from predicting and calculating a stereoscopic effect when a 3D image is generated based on a through image and displaying the prediction calculation result superimposed on the through image. The through image and the feature area will be described later.

In the CPU 11, when the execution of the 3D effect determination process is controlled, the image acquisition unit 41, the feature region extraction unit 42, the focal length detection unit 43, the focal length comparison unit 44, the stereoscopic effect calculation unit 45, The display control unit 46 functions.
In the CPU 11, a mode setting unit 51 and an imaging condition setting unit 52 function to control the imaging unit 16.

In the imaging apparatus 1 having such a functional configuration, as a mode for automatically setting the setting conditions at the time of imaging of the imaging unit 16, a plurality of modes that can be selected by the user for each main subject, specifically, this embodiment In the form, there are “person mode”, “proximity mode”, “landscape mode”, “night scene mode”, “sport mode”, “child / pet mode”, and the like.
The mode setting unit 51 automatically sets a setting condition corresponding to the mode selected from the plurality of modes based on selection by the user operation of the shooting mode setting button of the input unit 17.

In addition, a mode “stereoscopic display mode” for executing the 3D effect determination process and displaying the prediction calculation result of the stereoscopic effect is also provided as an operation mode of the imaging apparatus 1. The user can instruct to set or cancel the stereoscopic display mode by pressing down a stereoscopic effect setting button (not shown) of the input unit 17.
That is, the mode setting unit 51 sets or cancels the stereoscopic effect display mode based on the pressing operation of the stereoscopic effect setting button of the input unit 17 by the user. If the stereoscopic effect display mode is set, the 3D effect determination process in FIG. 5 described later is executed to display the prediction result of the stereoscopic effect. If the stereoscopic effect display mode is canceled, the 3D effect determination process is performed. Execution of the process is prohibited, and the display of the prediction result of stereoscopic effect is deleted. The mode setting unit 51 provides information on the shooting mode set in this way to the imaging unit 16 and the feature region extraction unit 42.

The imaging condition setting unit 52 can individually set various setting conditions at the time of imaging of the imaging unit 16 according to each operation of the input unit 17 by the user.
That is, in the mode in which the above-described setting conditions are automatically set, various setting conditions associated with the selected mode in advance are forcibly set. In such a case, the user can give an instruction to set or update an arbitrary number of arbitrary types of conditions among various setting conditions by performing a predetermined operation on the input unit 17.
For example, when the zoom switch of the input unit 17 is pressed by the user, the imaging condition setting unit 52 sets the zoom magnification (for example, 1 to 12 times) corresponding to the pressing operation as one of the setting conditions. Set as.
Further, for example, when the user presses the aperture switch of the input unit 17, the imaging condition setting unit 52 sets the aperture corresponding to the pressing operation as one of the predetermined conditions.

The image acquisition unit 41 acquires captured image data output from the imaging unit 16 as through image data.
That is, the CPU 11 or the like displays the through image on the output unit 18 (display of the imaging apparatus 1) by executing the through imaging process and the through display process before or after the execution of the 3D effect determination process.
Here, at the time of shooting, the user presses the shutter switch of the input unit 17 to the lower limit while the imaging device 1 is fixed. The operation of pressing the shutter switch to the lower limit in this way is hereinafter referred to as “full press operation” or simply “full press”. On the other hand, the user presses the shutter switch of the input unit 17 halfway (a predetermined position that does not reach the lower limit) in order to cause the imaging apparatus 1 to execute AF (Auto Focus) processing or the like before performing the full pressing operation. Perform the operation. The operation of pressing down to the middle of the shutter switch (a predetermined position that does not reach the lower limit) is hereinafter referred to as “half-press operation” or simply “half-press”.
Specifically, for example, when the shutter switch of the input unit 17 is half-pressed, the CPU 11 or the like continues the imaging operation by the imaging unit 16. Then, the CPU 11 or the like temporarily stores the captured image data sequentially output from the imaging unit 16 in the memory (the storage unit 19 in the present embodiment) while the imaging operation by the imaging unit 16 is continued. . Such a series of control processes is the “through imaging process” referred to herein.
Further, the CPU 11 and the like sequentially read out the data of each captured image temporarily recorded in the memory (the storage unit 19 in the present embodiment) during the through imaging process, and sequentially display the captured image on the output unit 18. Let Such a series of control processes is the “through display process” here, and the captured image displayed on the output unit 18 by the through display process is the “through image” here.
That is, the image acquisition unit 41 executes up to the process of acquiring the through image data temporarily stored in the memory in the through display process, and the display control unit 46 described later outputs the through image to the output unit 18. Display.

The feature region extraction unit 42 extracts a plurality of feature regions from the through image data acquired by the image acquisition unit 41.
“Characteristic region” refers to a region indicating a characteristic object among objects (subject, background, etc.) included in the through image.
Specifically, the feature region extraction unit 42 calculates contrast change using color information or the like in the data of the through image, and based on the calculation result, an object such as a person or a plant included in the through image. Is extracted as a feature region.
Note that the feature region extraction method is not particularly limited to this, and other methods such as detecting a contour line using the pixel value of the edge of the background image and extracting the region within the contour line as the feature region are also available. It may be adopted.

Then, the feature region extraction unit 42 classifies each of the extracted feature regions into an important feature region or other feature regions.
The “important feature area” is an area including an object that the user particularly wants to confirm a stereoscopic effect. The important feature region is determined according to the imaging mode set by the mode setting unit 51 or is determined by the user directly operating the input unit 17.
“Stereoscopic” refers to the three-dimensional depth, thickness, etc. that the user can perceive when viewing the 3D display by paying attention to the important feature area, assuming that the 3D display based on the through image has been performed. This refers to the degree of index.
This stereoscopic effect is the actual position between the subject (main subject) corresponding to the priority feature region and the subject or background corresponding to the other feature region in the real world within the range of the angle of view of the imaging apparatus 1. It is obtained in consideration of the relationship (hereinafter referred to as “the positional relationship between subjects in the real world”). In other words. The method for obtaining the three-dimensional effect is not particularly limited as long as it is a method for obtaining the positional relationship between subjects in the real world. Therefore, in the present embodiment, the positional relationship between subjects in the real world is simulated by comparing the focal length for the priority feature region included in the through image with the focal length for the other feature region. The three-dimensional effect is calculated based on the calculation result.

Thus, the focal length detection unit 43 detects the focal length for each feature region including the priority feature region.
The feature region extraction unit 42 extracts a plurality of feature regions based on the imaging mode set by the mode setting unit 51. That is, the feature area extraction unit 42 extracts a feature area and an important feature area centered on a person from among objects included in a through image when “person mode” is set as a mode at the time of shooting. . Similarly, when “proximity mode” is set as the mode at the time of shooting, the feature region extraction unit 42 centers on the object on the near side (proximity side) among the objects included in the through image. A feature region and an important feature region are extracted. Similarly, when the “landscape mode” is set as the shooting mode, the feature region extraction unit 42 sets the feature region and the priority feature centered on the object in the distant view among the objects included in the through image. Extract regions.
In the present embodiment, the value indicating the three-dimensional effect increases as the focal length for the important feature region and the focal length for other feature regions are separated.
Specifically, the value indicating the stereoscopic effect is defined so as to increase steplessly within the range from “Min” to “Max”, and the higher the stereoscopic effect value, the more important features in the through image. The user can easily feel the depth and thickness of the object in the area.
Therefore, the feature region extraction unit 42 supplies the focal length detection unit 43 with information on the important feature region extracted from the through image and other feature regions together with the data of the through image.

The focal length detection unit 43 detects the focal length of the feature region to be processed using each feature region (including the priority feature region) extracted by the feature region extraction unit 42 as a processing target.
Specifically, the focal length detection unit 43 obtains position information in the optical axis direction of the focus lens 31F when the focus lens 31F moved so as to be focused in the feature region to be processed is captured by the imaging unit 16. And the focal length of the feature region to be processed is detected based on the position information. The focal length detection unit 43 supplies the focal length comparison unit 44 with the focal length information about each feature region including the priority feature region and the data of the through image.

The focal length comparison unit 44 compares the focal length for the important feature region detected by the focal length detection unit 43 with the focal length for each of the other feature regions.
Specifically, the focal length comparison unit 44 determines the important feature region and each of the other features based on the difference (distance difference) between the focal length of the important feature region and the focal length of each of the other feature regions. The relative relationship between the focal lengths for the region is obtained.
Here, the focal distance of the feature region to be processed and the actual distance from the imaging apparatus 1 to the subject corresponding to the feature region in the real world have a certain correspondence. For this reason, the relative relationship between the focal lengths of the important feature region and the other feature regions (comparison result of focal lengths) represents the positional relationship between subjects in the real world in a pseudo manner. In other words, the focal length comparison unit 44 pseudo-determines the positional relationship between subjects in the real world. The focal length comparison unit 44 supplies the comparison result of the focal length (position relation between subjects in the real world) and the data of the through image to the stereoscopic effect calculation unit 45.

The stereoscopic effect calculation unit 45 calculates the stereoscopic effect for the image captured by the imaging unit 16 based on the comparison result of the focal lengths of the focal length comparison unit 44 (the positional relationship between subjects in the real world).
In the present embodiment, the stereoscopic effect calculation unit 45 calculates the absolute value of the difference between the focal length for the priority feature region and the focal length for other feature regions as a value indicating the stereoscopic effect.

If there are a plurality of feature regions other than the important feature region, any one of the plurality of feature regions is selected, and the focal length of the selected feature region is used to generate a stereoscopic effect. Is calculated.
Specifically, for example, the display control unit 46 may display a plurality of feature regions (such as a frame indicating the feature region) extracted by the feature region extraction unit 42 on the display of the output unit 18 by superimposing them on the through image. . In this case, the user can select any one of the plurality of feature areas displayed on the display by operating the input unit 17. The stereoscopic effect calculation unit 45 may calculate the stereoscopic effect by using the focal length for the feature region selected by the user in this way.
Further, for example, the stereoscopic effect calculation unit 45 selects a feature region having the largest or smallest absolute value of the focal length difference from the priority feature region, and calculates the stereoscopic effect using the focal length of the feature region. May be.

  Further, the stereoscopic effect calculation unit 45 obtains each difference between the focal length for the priority feature region and each focal length for a plurality of other feature regions, and comprehensively considers the plurality of differences. Thus, the stereoscopic effect may be calculated.

Furthermore, the parameters necessary for calculating the stereoscopic effect are particularly limited to the difference between the focal length of the priority feature region and the focal length of the other feature region (the positional relationship between subjects in the real world). Instead, any number of any type of parameters can be employed in conjunction with or instead of this.
For example, the stereoscopic effect calculation unit 45 may calculate the stereoscopic effect by adding the imaging mode set by the mode setting unit 51 or the imaging condition set by the imaging condition setting unit 52 to one of the parameters. That is, the stereoscopic effect calculation unit 45 may calculate the stereoscopic effect by comprehensively setting the zoom magnification and the aperture setting value as well as the positional relationship between the subjects in the real world.

The display control unit 46 superimposes the data of the symbol image indicating the stereoscopic effect calculated by the stereoscopic effect calculation unit 45 on the data of the through image supplied from the imaging unit 16, and is an image expressed by the resulting data (Hereinafter, referred to as “stereoscopic superimposed image”) is displayed on the display of the output unit 18.
Note that the stereoscopic effect superimposed image data can also be output to an external device (not shown) by the communication unit 20. As a result, a stereoscopic effect superimposed image can be displayed even by an external device such as a television receiver, a personal computer, or a projector.

When image recording is instructed, the image processing unit 61 acquires data of a captured image displayed as a through image at the time of the instruction from the stereoscopic effect calculation unit 45. Then, the image processing unit 61 generates data for each image for left eye and right eye for 3D display from the data of the captured image (through image), and stores the data in the storage unit 19.
Here, when the stereoscopic effect value calculated by the stereoscopic effect calculation unit 45 is equal to or less than a predetermined threshold value, sufficient stereoscopic effect can be obtained even if 3D display is performed using each image for the left eye and the right eye. Will not be obtained. Therefore, when the stereoscopic effect value calculated by the stereoscopic effect calculation unit 45 is equal to or less than a predetermined threshold, the image processing unit 61 performs image processing (hereinafter, “ The data for each image for the left eye and the right eye is generated.
Specifically, the image processing unit 61 executes stereoscopic effect enhanced image processing so as to enhance the parallax of the enhanced feature region in each of the left-eye and right-eye images based on the comparison result of the focal length comparison unit 44. .
Here, the image processing unit 61 can perform parallax enhancement based on the stereoscopic effect calculated by the stereoscopic effect calculating unit 45. Specifically, the image processing unit 61 can perform parallax enhancement according to the difference between the above-described important feature region and other feature regions. That is, when the difference is small, the image processing unit 61 can enhance the stereoscopic effect by increasing the parallax between the right-eye image and the left-eye image according to the difference.
Then, the image processing unit 61 generates left-eye and right-eye image data from the captured image data based on the execution result of the stereoscopic image processing, and records the generated data in the storage unit 19.

  Next, a mode setting process executed by the imaging apparatus 1 having the functional configuration shown in FIG. 2 will be described with reference to FIG. FIG. 3 is a flowchart for explaining the flow of mode setting processing executed by the imaging apparatus 1 of FIG. 1 having the functional configuration of FIG.

  The mode setting process is a process that is executed independently of the shooting process described with reference to FIG. 4 (which may be executed in parallel), and the user presses the power button of the input unit 17. This is started at the timing, and then executed every predetermined unit time. That is, the following processing is repeatedly executed every predetermined unit time.

  In step S11, the mode setting unit 51 determines whether or not the shooting mode setting button has been pressed.

When the shooting mode setting button is pressed, it is determined as YES in Step S11, and the process proceeds to Step S12.
In step S12, the mode setting unit 51 sets the shooting mode. Thereby, the imaging unit 16 captures an image of the subject under various shooting conditions specified by the set shooting mode.

  As described above, when the shooting mode setting button is pressed, it is determined as YES in Step S11, and then the process of Step S12 is executed, and the process proceeds to Step S13. On the other hand, if the shooting mode setting button is not pressed, it is determined as NO in step S11, and the process proceeds to step S13 without executing the process in step S12.

  In step S13, the mode setting unit 51 determines whether or not the stereoscopic effect setting button has been pressed.

If the stereoscopic effect setting button has not been pressed, it is determined NO in step S13, and the mode setting process ends.
On the other hand, when the stereoscopic effect setting button is pressed, it is determined as YES in Step S13, and the process proceeds to Step S14.

In step S14, the mode setting unit 51 sets or cancels the stereoscopic display mode.
Specifically, when the 3D effect setting button is pressed with the 3D effect determination flag for determining whether or not to perform the 3D effect determination process of FIG. Sets the stereoscopic display mode by turning on the 3D effect determination flag. When the 3D effect determination flag is turned on in this way, it is determined as YES in step S37 of FIG. 4 described later, and the 3D effect determination process is executed.
On the other hand, when the stereoscopic effect setting button is pressed while the 3D effect determination flag is on, the mode setting unit 51 cancels the stereoscopic effect display mode by turning off the 3D effect determination flag. . When the 3D effect determination flag is turned off in this way, it is determined NO in step S37 of FIG. 4 described later, and the 3D effect determination process is not executed.
When the process in step S14 is completed, the mode setting process ends.

Next, with reference to FIG. 4, the imaging process executed by the imaging apparatus 1 having the functional configuration of FIG. 2 described above will be described.
FIG. 4 is a flowchart for explaining the flow of imaging processing executed by the imaging apparatus 1 of FIG. 1 having the functional configuration of FIG.

  The photographing process is a process executed independently of the mode setting process described with reference to FIG. 3 (may be executed in parallel), and a predetermined operation is performed on the input unit 17 by the user. This is started at the timing, and then executed every predetermined unit time. That is, the following processing is repeatedly executed every predetermined unit time.

  In step S31, the display control unit 48 starts through image display. As a result, the through image is displayed on the display of the output unit 18. In the present embodiment, it is assumed that the through image is continuously displayed on the display of the output unit 18 until the photographing process is completed.

In step S32, the imaging condition setting unit 52 determines whether or not the shutter switch is half-pressed.
If the shutter switch is not half-pressed, it is determined as NO in Step S32, and the process returns to Step S32. That is, in this embodiment, until the shutter switch is half-pressed, the determination process in step S32 is repeatedly executed, and the photographing process is in a standby state. If the shutter switch is pressed halfway during this standby state, it is determined as YES in Step S32, and the process proceeds to Step S33.

In step S33, the imaging condition setting unit 52 sets the zoom magnification.
In step S34, the imaging condition setting unit 52 performs zooming based on the zoom magnification set in the process of step S33. Specifically, the imaging condition setting unit 52 drives the zoom lens 31Z so as to achieve the set zoom magnification.

In step S35, the imaging condition setting unit 52 sets an aperture.
In step S36, the imaging condition setting unit 52 adjusts the amount of light incident on the imaging unit 16 based on the diaphragm set in the process of step S35.

In step S37, the mode setting unit 51 determines whether or not the stereoscopic effect display mode is set. Specifically, the mode setting unit 51 determines whether or not the 3D effect determination flag set in step S14 of FIG. 3 is on.
If it is not the stereoscopic effect display mode, that is, if the 3D effect determination flag is OFF, it is determined NO in step S37, and the 3D effect determination process in step S37 is not executed, and the process proceeds to step S39. Proceed to
On the other hand, if it is the stereoscopic display mode, that is, if the 3D effect determination flag is on, it is determined as YES in Step S37, and the process proceeds to Step S38. In step S38, the image acquisition unit 41 through the display control unit 46 execute a 3D effect determination process described with reference to FIG. Thereby, a process progresses to step S39.

In step S39, the imaging condition setting unit 52 determines whether the shutter switch is fully pressed. When the shutter switch is not fully pressed, it is determined as NO in Step S39, the process returns to Step S33, and the subsequent processes are repeated. That is, in this embodiment, as long as the shutter switch is half-pressed, the loop process of step S32: YES and step S39: NO is repeatedly executed until the shutter switch is fully pressed, and the imaging process is in a standby state. It becomes.
If the shutter switch is fully pressed during such a standby state, it is determined as YES in Step S39, and the process proceeds to Step S40.

  In step S40, the imaging unit 16 performs an imaging process. Specifically, the image acquisition unit 41 controls the imaging unit 16 to acquire captured image data displayed as a through image at that time. The acquired captured image data is temporarily stored in a memory such as the RAM 13 or the storage unit 19.

  In step S41, the image processing unit 61 determines whether the stereoscopic effect calculation result calculated by the stereoscopic effect calculation unit 45 is equal to or less than a predetermined threshold value.

The stereoscopic effect is calculated by the 3D effect determination process in step S38 (specifically, the process in step S65 in FIG. 5 described later).
Therefore, when the 3D effect determination process of step S38 is not executed in the first place, or when the 3D effect determination process is executed but the calculation result of the stereoscopic effect is larger than the threshold value (obtained by the process of step S41). If the original captured image remains as it is and the stereoscopic effect is sufficiently obtained), the stereoscopic effect enhanced image process in step S42 is not executed, and the process proceeds to step S43.
In step S43, the image processing unit 61 uses the data of the original captured image obtained by the imaging process of step S40, and data for each image for the left eye and for the right eye (an image on which stereoscopic effect enhancement image processing has not been performed). Is generated and stored in the storage unit 19.
When this process ends, the photographing process ends.

On the other hand, when the calculation result of the stereoscopic effect by the 3D effect determination process in step S38 is equal to or less than the threshold value, that is, when the stereoscopic effect is not sufficiently obtained with the captured image acquired in the process of step S41, the process Advances to step S42.
In step S42, the image processing unit 61 performs stereoscopic effect enhanced image processing on the captured image data acquired in step S41.
In step S43, the image processing unit 61 generates data for each image for the left eye and the right eye from the captured image data that has been subjected to the stereoscopic effect enhanced image processing in step S42, and stores the data in the storage unit 19.
When this process ends, the photographing process ends.

Next, with reference to FIG. 5, the 3D effect determination process in step S <b> 38 among such shooting processes will be described.
FIG. 5 is a flowchart for explaining the flow of the 3D effect determination process in step S38 in the imaging process of FIG. 4 executed by the imaging apparatus 1 of FIG. 1 having the functional configuration of FIG.

  In the present embodiment, the 3D effect determination process is performed in step S14 of the mode setting process in FIG. 3 when the user presses the stereoscopic effect setting button of the input unit 10 to turn on the 3D effect determination flag. The stereoscopic effect display mode is set, and as a result, the process is started when it is determined that the stereoscopic effect display mode is set in step S37 of the photographing process of FIG. That is, the following processing is executed.

  In step S <b> 61, the image acquisition unit 41 acquires through image data from the imaging unit 16.

  In step S62, the feature region extraction unit 42 extracts a plurality of feature regions from the through image acquired as data in the process of step S61.

In step S63, the feature region extraction unit 42 specifies the priority feature region from the plurality of feature regions extracted in the process of step S61.
Note that the processing in steps S61 and 62, that is, the extraction of a plurality of feature regions and the specification (selection) of the important feature regions among them may be executed by autonomous determination of the feature region extraction unit 42, It may be executed based on an instruction by a user operation. That is, when the user directly operates the input unit 17, any one or more regions can be selected as a feature region from the through image, and an important feature region can be selected from the selected region. .

  In step S64, the focal length detection unit 43 detects the focal length for each feature region extracted in the process of step S61.

  In step S65, the focal length comparison unit 44 compares the focal length for the important feature region detected in the process of step S63 with the focal length for the other feature regions.

In step S66, the stereoscopic effect calculation unit 45 calculates the stereoscopic effect based on the comparison result of step S65.
In this case, the stereoscopic effect calculation unit 45 comprehensively includes not only the comparison result in step S64 but also the shooting mode set in the process in step S12 in FIG. 3, the imaging conditions set in the processes in step S33 and step S35 in FIG. Therefore, the stereoscopic effect may be calculated in consideration of the above.

  In step S67, the display control unit 46 outputs the calculation result of the stereoscopic effect calculated in step S65 to the output unit 18. When this process ends, the 3D effect determination process ends.

  Further, the 3D effect determination process will be specifically described below with reference to FIGS. 6 to 11.

  FIG. 6 shows a display example of the display of the output unit 18 during the 3D effect determination process.

FIG. 6A illustrates an example of the display screen of the output unit 18 at the start of the 3D effect determination process.
The user can instruct setting of the stereoscopic display mode by pressing the stereoscopic setting button of the input unit 17 (FIG. 1). The mode setting unit 51 sets the stereoscopic effect display mode based on such an instruction. If the user presses the shutter button of the input unit 17 halfway with this setting maintained, it is determined as YES in the process of step S37 (FIG. 4), and the 3D effect determination process of step S38 starts.
Then, a screen as shown in FIG. 6A is displayed on the display of the output unit 18. In this case, a through image is displayed on the entire screen, and a mode start display 101 including a message “3D level is determined” is displayed on the upper left of the screen.
By visually recognizing the mode start display 101, the user can easily start the 3D effect determination process, that is, the confirmation display of the stereoscopic effect before recording for recording for 3D display. I can grasp it.
Note that the mode start display 101 only needs to be displayed after the start of the 3D effect determination process, and the erasure timing is not particularly limited. The mode start display 101 may be erased after a predetermined time has elapsed after the start of display, or a stereoscopic effect calculation result described later. May be deleted when is displayed.
However, it is preferable to display the mode start display 101 at a position that does not overlap the feature area so that the user can visually recognize the feature area at the time of and after extraction of the feature area.

FIG. 6B shows an example of the display screen of the output unit 18 in the state after the processing of steps S62 and S63 of the 3D effect determination processing of FIG.
In the process of step S62, a plurality of feature regions are extracted from the through image. In the example of FIG. 6B, the feature regions A1, A2, A3, and A4 are extracted from the through image, and are highlighted and surrounded by a dotted frame. Each of the feature areas A1, A2, A3, and A4 indicates the respective objects of “mountain”, “forest”, “stone”, and “person” included in the through image.
Here, it is assumed that the feature area A4 indicating the “person” object is identified as the important feature area in the processing of the next step S63. In this case, as shown in FIG. 6B, the emphasis feature region A4 is emphasized by being surrounded by a thick frame or a different color frame so that the user can easily distinguish it from the other feature regions A1, A2, A3. It is displayed.

Note that the extraction of the feature areas A1, A2, A3, and A4 and the process of specifying the feature area A4 as the priority feature area is executed based on what the user selects by operating the input unit 17. Alternatively, it may be executed by autonomous determination by the feature region extraction unit 42.
A method for the feature region extraction unit 42 to identify the important feature region by autonomous determination is not particularly limited. For example, a method for specifying a person or a face region as an important feature region using an arbitrary person or face detection method. Alternatively, it is possible to adopt a method of specifying the largest feature region among the feature regions included in the through image as the important feature region.

FIG. 7 shows the processing result of step S64 in FIG. 5 executed for each of the feature areas A1, A2, A3, and A4 of FIG. 6, that is, the focal lengths for the feature areas A1, A2, A3, and A4. FIG.
In the process of step S64 of the 3D effect determination process of FIG. 5, each of the focal lengths da1, da2, da3, da4 for each of the feature areas A1, A2, A3, A4 of FIG. 6 extracted by the process of step S62. Is detected.
Here, the focal lengths da1, da2, da3, da4 and the distance to each of the subjects a1, a2, a3, a4 with respect to each of the characteristic regions A1, A2, A3, A4 (hereinafter referred to as “subject distance”). There is a certain correspondence between.
In other words, the focal length of the predetermined subject corresponds to the position (lens address) of the focus lens 31F when the predetermined subject is in focus, and the subject distance increases as the lens address approaches the image sensor. There is a relationship of becoming.
In the example of FIG. 7, for each of the focal lengths (lens addresses) da1, da2, da3, da4, subject distances to the focused subjects a1, a2, a3, a4, that is, “∞”, “15 m ”,“ 5 m ”, and“ 1 m ”correspond to each other.

In this case, the relative relationship between the focal length da4 of the important feature region A4 and each of the focal lengths da1, da2, da3 of the other feature regions A1, A2, A3 is that the subject distance is “1 m”. The relative relationship that the subject distance is “5 m”, “15 m”, and “∞”, that is, the positional relationship between subjects in the real world, is represented in a pseudo manner.
That is, in the processing of the next step S65 (FIG. 5), the focal length da4 of the important feature region A4 is compared with each of the focal lengths da1, da2, da3 of the other feature regions A1, A2, A3. However, this comparison is equivalent to a pseudo determination of the positional relationship between subjects in the real world.
In step S66, a stereoscopic effect is obtained based on such a comparison result, that is, the positional relationship between subjects in the real world. In step S67, a symbol image indicating the stereoscopic effect is superimposed on the through image. Displayed.

FIG. 8 is a diagram illustrating an example of the display screen of the output unit 18 in the state after the process of step S67 of the 3D effect determination process of FIG. 5, that is, a screen in which a symbol image indicating a stereoscopic effect is superimposed on the through image. is there.
As shown in FIG. 8, a through image is displayed on the entire display screen of the output unit 18, and on the right side of the through image, a long gauge 102 arranged in the vertical direction is superimposed and displayed. ing.
A character string “Max” indicating that the stereoscopic effect is the highest value is displayed at the upper end of the gauge 102, and a character string “Min” indicating that the stereoscopic effect is the lowest value is displayed at the lower end of the gauge 102. Is displayed.
Then, a solid for displaying the position indicating the stereoscopic effect obtained in the process of step S66 of the long gauge 102 between the character string “Max” and the character string “Min” of the gauge 102 with an arrow. A feeling presentation unit 103 is displayed as one of the symbol images showing a stereoscopic effect.
Accordingly, the user visually recognizes the degree of stereoscopic effect presented by the stereoscopic effect presenting unit 103, so that the object indicated by the priority feature area included in the through image is displayed before shooting for recording for 3D display. It is possible to easily confirm whether or not photographing that feels the depth and thickness can be performed.

Next, a difference in the result of the 3D effect determination process in FIG. 5, that is, a difference in how to obtain a three-dimensional effect due to a difference in the positional relationship between subjects in the real world will be described with reference to FIGS.
FIG. 9 shows an example of the result of the 3D effect determination process of FIG. 5 when all the subject distances of the subjects corresponding to the feature areas included in the through image are close.

FIG. 9A shows an example of the display screen of the output unit 18 in a state where each feature region and important feature region are specified from the through image.
In the example of FIG. 9A, a feature region B1 indicating a “desk” object, a feature region B2 indicating a “dish” object, and a feature region B3 indicating a “fish” object are extracted from the through image. It is highlighted so that it is surrounded by a frame. The feature region B3 is specified as the important feature region, and is highlighted so as to be surrounded by a thick frame or a different color frame so that it can be clearly distinguished from the other feature regions B1 and B2.

FIG. 9B shows the processing result of step S64 in FIG. 5 executed for each feature region B1, B2, B3 of FIG. 9A, ie, each feature region B1, B2, B3. It is a figure which shows a focal distance.
In the process of step S64 of the 3D effect determination process of FIG. 5, the focal lengths db1, db2, and db3 are detected for each of the feature regions B1, B2, and B3 of FIG. 9B.
In the example of FIG. 9B, for each of the focal lengths (lens addresses) db1, db2, and db3, subject distances to the subjects b1, b2, and b3 corresponding to the feature regions B1, B2, and B3, respectively. That is, “0.2 m”, “0.15 m”, and “0.1 m” correspond to each other.
In this case, the difference in focal length can be used as the relative relationship between the focal length db3 of the important feature region B3 and the focal lengths db1 and db2 of the other feature regions B1 and B2. In this case, the difference in focal length is equivalent to indicating the difference in subject distance in the real world. That is, the difference in subject distance is adopted as the positional relationship between subjects in the real world, and the specific calculation is approximately performed by the difference in focal length.
In this example, the difference between the focal length db3 of the priority feature region B3 and the focal length db1 of the feature region B1 corresponds to the subject distance “0.1 m” of the subject b3 corresponding to the priority feature region B3 and the feature region B1. This corresponds to the difference of the subject distance “0.2 m” of the subject b1 to be performed, that is, “0.1 m”. Further, the difference between the focal length db3 of the priority feature region B3 and the focal length db2 of the feature region B2 is the subject distance “0.1 m” of the subject b3 corresponding to the priority feature region B3 and the subject corresponding to the feature region B2. This corresponds to the difference of the subject distance “0.15 m” of b2, that is, “0.05 m”.
Thus, from the difference between the focal length db3 of the priority feature region B3 and the focal lengths db1 and db2 of the other feature regions B1 and B2, the subject b3 corresponding to the priority feature region B3 and the feature region B1 It can be easily seen that the actual distance between the subject b1 corresponding to 被 写 体 and the subject b2 corresponding to the feature region B2 is a short distance such as “0.1 m” or “0.05 m”.
In this case, the focal length difference between the subject b3 corresponding to the priority feature region B3 included in the through image and the subjects b1 and b2 corresponding to the other feature regions B1 and B2, and the feature regions B1, B2, The stereoscopic effect is calculated by comprehensively considering the weighting of the focal length of B3.
Therefore, in the process of step S66, it is determined that all the subject distances of the subjects corresponding to the feature regions included in the through image are close, and a low value is obtained as the stereoscopic effect.

FIG. 9C shows the display screen of the output unit 18 in the state after the process of step S67 of the 3D effect determination process of FIG. 5 when the positional relationship between the objects in the real world is the relationship of FIG. 9B. That is, it is a figure which shows an example of the screen on which the symbol image which shows a three-dimensional effect was superimposed on the through image.
As shown in FIG. 9C, the user indicates the position indicated by the stereoscopic effect presentation unit 103 in the long gauge 102 between the character string “Max” and the character string “Min” of the gauge 102. It is easy to see that the three-dimensional effect is low. That is, before the user actually performs 3D display using the through image of FIG. 9C, the “fish” corresponding to the priority feature region B3 corresponds to each of the other feature regions B2 and B1. When it becomes a 3D display that does not appear so much as “dish” and “desk” (the depth is not felt so much), it can be easily estimated.
More generally speaking, the 3D display obtained when the user visually recognizes the stereoscopic effect presentation unit 103 in FIG. 9C and all the objects (feature regions) included in the through image are close to each other. It can be easily estimated that the feeling becomes small.

  FIG. 10 shows an example of the result of the 3D effect determination process of FIG. 5 when the subject distances of the subjects corresponding to the feature regions included in the through image are separated (split).

FIG. 10A shows an example of the display screen of the output unit 18 in a state where each feature region and important feature region are specified from the through image.
In the example of FIG. 10A, a feature region C1 indicating a “mountain” object, a feature region C2 indicating an object “person 1”, and a feature region C3 indicating an object “person 2” are extracted from the through image. And highlighted so as to be surrounded by a frame. The feature region C3 is specified as the important feature region, and is highlighted so as to be surrounded by a thick frame or a different color frame so that it can be clearly distinguished from the other feature regions C1 and C2.

FIG. 10B shows the processing result of step S64 in FIG. 5 executed for each feature area C1, C2, C3 of FIG. 10A, that is, each feature area C1, C2, C3. It is a figure which shows a focal distance.
In the process of step S64 of the 3D effect determination process in FIG. 5, the focal lengths dc1, dc2, and dc3 are detected for each of the feature regions C1, C2, and C3 in FIG.
In the example of FIG. 10B, the subject distance to each of the subjects c1, c2, and c3 corresponding to each of the characteristic regions C1, C2, and C3 with respect to each of the focal lengths (lens addresses) dc1, dc2, and dc3. That is, “∞ (infinity) m”, “5 m”, and “1 m” correspond to each other.
In this case, a difference in focal length can be used as a relative relationship between the focal length dc3 of the important feature region C3 and each of the focal lengths dc1 and dc2 of the other feature regions C1 and C2. In this case, the difference in focal length is equivalent to indicating the difference in subject distance in the real world. That is, the difference in subject distance is adopted as the positional relationship between subjects in the real world, and the specific calculation is approximately performed by the difference in focal length.
In this example, the difference between the focal length dc3 of the important feature region C3 and the focal length dc1 of the feature region C1 is the subject distance “1m” of the subject c3 corresponding to the important feature region C3 and the subject corresponding to the feature region C1. This corresponds to the difference of the subject distance “∞ (infinity) m” of c1, that is, “(∞−1) m”. Further, the difference between the focal length dc3 of the important feature region C3 and the focal length dc2 of the feature region C2 is that the subject distance “1m” of the subject c3 corresponding to the important feature region C3 and the subject c2 corresponding to the feature region C2 are different. This corresponds to the difference of the subject distance “5 m”, that is, “4 m”.
In this way, from the difference between the focal length dc3 of the important feature region C3 and each of the focal lengths dc1 and dc2 of the other feature regions C1 and C2, the subject c3 corresponding to the important feature region C3 and the feature region C1. It can be easily seen that the actual distance between the subject c1 corresponding to and the subject c2 corresponding to the feature region C2 is a long distance such as “(∞−1) m” or “4 m”.
In this case, the focal length difference between the subject c3 corresponding to the priority feature region C3 included in the through image and the subjects c1 and c2 corresponding to the other feature regions C1 and C2, and the feature regions C1, C2, The stereoscopic effect is calculated by comprehensively considering the weighting of the focal length of C3.
Therefore, in the process of step S66, the subject distances of the subjects corresponding to the feature regions included in the through image are separated, that is, the subject distances are recognized as being far, and a high value is obtained as a stereoscopic effect. It is done.

FIG. 10C shows the display screen of the output unit 18 in the state after the process of step S67 of the 3D effect determination process of FIG. 5 when the positional relationship between the objects in the real world is the relationship of FIG. That is, it is a figure which shows an example of the screen on which the symbol image which shows a three-dimensional effect was superimposed on the through image.
As shown in FIG. 10C, the user indicates the position indicated by the stereoscopic effect presentation unit 103 in the long gauge 102 between the character string “Max” and the character string “Min” of the gauge 102. It is easy to see that the three-dimensional effect is high. That is, before the user actually performs the 3D display using the through image of FIG. 10C, the “person 2” corresponding to the important feature region C3 corresponds to each of the other feature regions C2 and C1. It becomes possible to estimate easily when it becomes a 3D display that seems to jump out greatly with respect to “person 1” and “mountain” (depth is felt).
More generally speaking, when the user visually recognizes the stereoscopic effect presentation unit 103 in FIG. 10C, all objects (feature regions) included in the through image are separated (split). It can be easily estimated that the 3D display obtained in the above will have a large stereoscopic effect.

  FIG. 11 shows an example of the result of the 3D effect determination process of FIG. 5 when all of the subject distances of the subjects corresponding to the feature regions included in the through image are long.

FIG. 11A shows an example of the display screen of the output unit 18 in a state where each feature region and important feature region are specified from the through image.
In the example of FIG. 11A, a feature region D1 indicating an object of “mountain 1”, a feature region D2 indicating an object of “mountain 2”, and a feature region D3 indicating an object of “people” are extracted from the through image. And highlighted so as to be surrounded by a frame. The feature region D3 is specified as the important feature region, and is highlighted so as to be surrounded by a thick frame or a different color frame so that it can be clearly distinguished from the other feature regions D1, D2.

FIG. 11B shows the processing result of step S64 in FIG. 5 executed for each feature area D1, D2, D3 of FIG. 11A, that is, each feature area D1, D2, D3. It is a figure which shows a focal distance.
In the process of step S64 of the 3D effect determination process of FIG. 5, the focal lengths dd1, dd2, and dd3 are detected for each of the feature regions D1, D2, and D3 of FIG.
In the example of FIG. 11B, for each of the focal lengths (lens addresses) dd1, dd2, and dd3, subject distances to the subjects d1, d2, and d3 corresponding to the feature regions D1, D2, and D3, respectively. That is, “50 m”, “40 m”, and “30 m” correspond to each other.
In this case, the difference in focal length can be used as the relative relationship between the focal length dd3 of the important feature region D3 and each of the focal lengths dd1 and dd2 of the other feature regions D1 and D2. In this case, the difference in focal length is equivalent to indicating the difference in subject distance in the real world. That is, the difference in subject distance is adopted as the positional relationship between subjects in the real world, and the specific calculation is approximately performed by the difference in focal length.
In this example, the difference between the focal length dd3 of the important feature region D3 and the focal length dd1 of the feature region D1 is the subject distance “30 m” of the subject d3 corresponding to the important feature region D3 and the subject corresponding to the feature region D1. This corresponds to the difference of the subject distance “50 m” of d1, ie, “20 m”. Further, the difference between the focal length dd3 of the important feature region D3 and the focal length dd2 of the feature region D2 is that the subject distance “30 m” of the subject d3 corresponding to the important feature region D3 and the subject d2 corresponding to the feature region D2 are different. This corresponds to the difference of the subject distance “40 m”, that is, “10 m”.
Thus, from the difference between the focal length dd3 of the important feature region D3 and the focal lengths dd1 and dd2 of the other feature regions D1 and D2, the subject d3 corresponding to the important feature region D3 and the feature region D1. It can be easily seen that the actual distance between the subject d1 corresponding to and the subject d2 corresponding to the feature region D2 is a short distance such as “20 m” or “10 m”.
In this case, the difference in focal length between the subject d3 corresponding to the priority feature region D3 included in the through image and the subjects d1 and d2 corresponding to the other feature regions D1 and D2, and the feature regions D1, D2, and so on. The stereoscopic effect is calculated by comprehensively considering the weighting of the focal length of D3.
Therefore, in this case, in the process of step S66, it is determined that all of the subject distances of the subjects corresponding to the feature regions included in the through image are close, and a low value is obtained as the stereoscopic effect.

FIG. 11C shows the display screen of the output unit 18 in the state after step S67 of the 3D effect determination process of FIG. 5 when the positional relationship between the objects in the real world is the relationship of FIG. That is, it is a figure which shows an example of the screen on which the symbol image which shows a three-dimensional effect was superimposed on the through image.
As shown in FIG. 11C, the user indicates the position indicated by the stereoscopic effect presentation unit 103 in the long gauge 102 between the character string “Max” and the character string “Min” of the gauge 102. It is easy to see that the three-dimensional effect is low. That is, before the user actually performs the 3D display using the through image of FIG. 11C, the “person” corresponding to the important feature region D3 corresponds to each of the other feature regions D2 and D1. It becomes possible to estimate easily when it becomes a 3D display in which “mountain 2” and “mountain 1” jump out so much that they cannot be seen (the depth is not felt so much).
More generally speaking, when the user visually recognizes the stereoscopic effect presentation unit 103 in FIG. 11C, the 3D display obtained when all the objects (feature regions) included in the through image are far away is It can be easily estimated that the feeling becomes small.

As described above, the imaging apparatus 1 according to the present embodiment includes the imaging unit 16, the feature region extraction unit 42, the focal length detection unit 43, the focal length comparison unit 44, and the stereoscopic effect calculation unit 45. .
The imaging unit 16 captures a plurality of subjects and obtains images.
The feature region extraction unit 42 extracts a plurality of feature regions indicating each of a plurality of subject images included in the image captured by the imaging unit 16.
The focal length detection unit 43 detects focal lengths for a plurality of feature regions extracted by the feature region extraction unit 42.
The focal length comparison unit 44 compares the focal length of the main feature region (emphasis feature region) with the focal length of other feature regions out of the focal lengths detected by the focal length detection unit 43. .
Based on the comparison result of the focal length comparison unit 44, the stereoscopic effect calculation unit 45 calculates the degree of stereoscopic effect of the stereoscopic display when it is assumed that the stereoscopic display is performed using the image captured by the imaging unit 16. To do.
In this case, there is a certain correspondence between the focal distance of the feature region and the actual distance from the imaging device 1 to the subject corresponding to the feature region in the real world. For this reason, by comparing the relative relationship between the focal lengths of the important feature region and the other feature regions, the positional relationship between subjects in the real world can be estimated in a pseudo manner.
Therefore, by comparing the focal length of the important feature region included in the image captured by the imaging unit 16 with the focal length of the other feature region, it is possible to perform the post-shooting and recording operation before the shooting and recording operation. The degree of stereoscopic effect of the 3D image that will be obtained can be predicted and the result can be provided to the user.
In addition, since the stereoscopic effect is calculated based on the comparison result based on the comparison of the focal lengths, the stereoscopic effect can be calculated using the function originally provided in the imaging device. An increase in manufacturing cost can be suppressed.

Furthermore, the imaging apparatus 1 of the present embodiment further includes an imaging condition setting unit 52 that sets a zoom magnification for imaging by the imaging unit 16, and the stereoscopic effect calculation unit 45 adds to the comparison result of the focal length comparison unit 44. Thus, the degree of stereoscopic effect can be calculated based on the zoom magnification set by the imaging condition setting unit 52.
In this case, the stereoscopic effect obtained from the 3D image becomes a different stereoscopic effect depending on the zoom magnification. Therefore, by correcting the degree of stereoscopic effect calculated based on this zoom magnification, the calculated degree of stereoscopic effect can be changed. It can be closer to the real thing. Therefore, by calculating the degree of stereoscopic effect based on the set zoom magnification, the stereoscopic effect of the image captured by the imaging unit 16 can be estimated more specifically, and the result can be provided to the user.

Furthermore, the imaging apparatus 1 of the present embodiment further includes an imaging condition setting unit 52 that adjusts the amount of light incident on the imaging unit 16, and the stereoscopic effect calculation unit 45 is added to the comparison result of the focal length comparison unit 44. The degree of stereoscopic effect can be calculated based on the amount of light adjusted by the imaging condition setting unit 52, that is, the aperture amount.
In this case, since the stereoscopic effect obtained from the 3D image has a different stereoscopic effect depending on the diaphragm, the degree of the calculated stereoscopic effect can be more realistic by correcting the degree of the stereoscopic effect calculated based on the diaphragm. You can get closer to things. Therefore, by calculating the degree of stereoscopic effect based on the set aperture, it is possible to more specifically estimate the stereoscopic effect of the image captured by the imaging unit 16 and provide the result to the user.

Furthermore, the imaging apparatus 1 of the present embodiment further includes a mode setting unit 51 that sets an imaging mode to be imaged by the imaging unit 16, and the feature region extraction unit 42 includes, in addition to the comparison result of the focal length comparison unit 44, A plurality of feature regions are extracted based on the imaging mode set by the mode setting unit 51.
In this case, an object to be compared can be set according to each imaging mode by extracting the feature region according to the imaging mode. That is, in the case of an imaging mode that is a person mode, a person is determined as an important feature area, and in the case of an imaging mode that is a landscape mode, an object desired by a user is determined by determining a landscape as an important feature area. It is possible to calculate the degree of three-dimensional feeling with respect to. Accordingly, it is possible to calculate stereoscopic effect information based on the set imaging mode, more specifically estimate the stereoscopic effect of the image captured by the imaging unit 16, and provide the result to the user. .

Furthermore, the imaging apparatus 1 according to the present embodiment includes an input unit 17 that receives an instruction operation for selecting a predetermined region as a feature region from an image captured by the imaging unit 16.
Then, the feature region extraction unit 42 specifies the feature region instructed by the input unit 17 as the important feature region.
In this case, by specifying the important feature region to be extracted based on the instruction operation to the input unit 17 by the user, the object to be compared can be specified as desired by the user. That is, the user can reliably calculate the degree of stereoscopic effect with respect to an object desired by the user by performing an instruction operation on the object to be confirmed. Therefore, the degree of stereoscopic effect can be calculated based on the priority feature region of the object that is instructed and operated via the input unit 17, and the stereoscopic effect of the image captured by the imaging unit 16 is estimated more specifically. The result can be provided to the user.

Furthermore, the imaging apparatus 1 according to the present embodiment performs display control for executing control to superimpose the stereoscopic effect presenting unit 103 indicating the stereoscopic effect calculated by the stereoscopic effect calculating unit 45 on the image captured by the imaging unit 16. A part 46 is further provided.
In this case, whether or not the user can take a photograph so that the object indicated by the priority feature region feels the depth and thickness by visually recognizing the degree of the stereoscopic effect presented by the stereoscopic effect presentation unit 103 superimposed on the image. Can be easily confirmed.
Therefore, by displaying the stereoscopic effect presentation unit 103 in a superimposed manner, the degree of stereoscopic effect of the 3D image that will be obtained after the shooting and recording operation can be presented to the user at the stage before the shooting and recording operation of the 3D image.

  Furthermore, the imaging apparatus 1 of the present embodiment performs image processing that enhances the stereoscopic effect of the image captured by the imaging unit 16 when the stereoscopic effect value calculated by the stereoscopic effect calculation unit 45 is equal to or less than a predetermined threshold. A unit 61 is further provided. Therefore, when it can be estimated that the degree of stereoscopic effect of the obtained image will be small, it is possible to obtain a stereoscopic image by enhancing the parallax between the left-eye image and the right-eye image. .

  In addition, this invention is not limited to the above-mentioned embodiment, The deformation | transformation in the range which can achieve the objective of this invention, improvement, etc. are included in this invention.

  In the above-described embodiment, a feature region is extracted, the focal length with respect to the main feature region is compared with the focal length with respect to a region other than the main feature region, and information on stereoscopic effect is calculated based on the comparison result. However, it is not limited to this. For example, the distance to each feature region may be measured by measuring an actual value for each feature region using infrared rays or ultrasonic waves, and the actual value for each feature region may be compared, and information on stereoscopic effect may be calculated based on the comparison result.

  Further, the image captured by the imaging unit 16 is not limited to a still image, and sequentially acquires frames in a moving image, detects a feature region for an object existing in the acquired frame, and detects each detected You may compare the focal distance with respect to a characteristic area.

In addition, the stereoscopic effect calculation unit 45 calculates the degree of stereoscopic effect based on the comparison result between the focal lengths of the important feature regions compared by the focal length comparison unit 4 and the focal lengths of other feature regions. However, it is not limited to this. In other words, the estimated distances for a plurality of feature regions extracted by the feature region extraction unit 42 are detected using ultrasonic waves, infrared rays, or the like. Then, among the estimated distances estimated, the estimated distance for the main feature region (important feature region) is compared with the estimated distances for other feature regions.
Then, based on the comparison result of the estimated distance, the degree of stereoscopic effect of the stereoscopic display may be calculated when it is assumed that the stereoscopic display is performed using the image captured by the imaging unit 16.

In the above-described embodiment, the imaging apparatus 1 to which the present invention is applied has been described using a digital camera as an example, but is not particularly limited thereto.
For example, the present invention can be applied to general electronic devices having an imaging function. Specifically, for example, the present invention can be applied to a notebook personal computer, a printer, a television receiver, a video camera, a portable navigation device, a mobile phone, a portable game machine, and the like.

The series of processes described above can be executed by hardware or can be executed by software.
In other words, the functional configuration of FIG. 2 is merely an example and is not particularly limited. That is, it is sufficient that the imaging apparatus 1 has a function capable of executing the above-described series of processing as a whole, and what functional blocks are used to realize this function is not particularly limited to the example of FIG.
In addition, one functional block may be constituted by hardware alone, software alone, or a combination thereof.

When a series of processing is executed by software, a program constituting the software is installed on a computer or the like from a network or a recording medium.
The computer may be a computer incorporated in dedicated hardware. The computer may be a computer capable of executing various functions by installing various programs, for example, a general-purpose personal computer.

  The recording medium including such a program is not only constituted by the removable medium 22 shown in FIG. 1 distributed separately from the apparatus main body in order to provide the program to the user, but is also stored in the apparatus main body in advance. It is comprised with the recording medium etc. which are provided in. The removable medium 22 is configured by, for example, a magnetic disk (including a floppy disk), an optical disk, a magneto-optical disk, or the like. The optical disk is composed of, for example, a CD-ROM (Compact Disk-Read Only Memory), a DVD (Digital Versatile Disk), or the like. The magneto-optical disk is configured by an MD (Mini-Disk) or the like. In addition, the recording medium provided to the user in a state of being preliminarily incorporated in the apparatus main body includes, for example, the ROM 12 in FIG. 1 in which a program is recorded, the hard disk included in the storage unit 19 in FIG.

In the present specification, the step of describing the program recorded on the recording medium is not limited to the processing performed in time series along the order, but is not necessarily performed in time series, either in parallel or individually. The process to be executed is also included.
Further, in the present specification, the term “system” means an overall apparatus configured by a plurality of devices, a plurality of means, and the like.

  As mentioned above, although several embodiment of this invention was described, these embodiment is only an illustration and does not limit the technical scope of this invention. The present invention can take other various embodiments, and various modifications such as omission and replacement can be made without departing from the gist of the present invention. These embodiments and modifications thereof are included in the scope and gist of the invention described in this specification and the like, and are included in the invention described in the claims and the equivalent scope thereof.

The invention described in the scope of claims at the beginning of the filing of the present application will be appended.
[Appendix 1]
Imaging means for capturing a plurality of subjects and obtaining images;
Feature region extraction means for extracting a plurality of feature regions, each indicating an image of the plurality of subjects included in an image picked up by the image pickup means;
A focal length detection means for detecting a focal length for the plurality of feature areas extracted by the feature area extraction means;
Of the focal lengths detected by the focal length detection means, a focal length comparison means for comparing a focal length for a main feature region and a focal length for other feature regions;
Stereoscopic effect calculating means for calculating the degree of stereoscopic effect of the stereoscopic display when it is assumed that stereoscopic display is performed using the image captured by the imaging means based on the comparison result of the focal length comparing means; ,
An imaging apparatus comprising:
[Appendix 2]
A zoom magnification setting means for setting a zoom magnification for imaging by the imaging means;
The stereoscopic effect calculating means calculates the degree of stereoscopic effect based on the zoom magnification set by the zoom magnification setting means in addition to the comparison result of the focal length comparing means.
The imaging apparatus according to Supplementary Note 1, wherein
[Appendix 3]
Further comprising aperture setting means for adjusting the amount of light incident on the imaging means;
The stereoscopic effect calculation means calculates the degree of stereoscopic effect based on the amount of light adjusted by the aperture setting means in addition to the comparison result of the focal length comparison means.
The imaging apparatus according to appendix 1 or 2, characterized in that:
[Appendix 4]
Further comprising mode setting means for setting an imaging mode to be imaged by the imaging means;
The feature region extraction unit extracts the plurality of feature regions based on the imaging mode set by the mode setting unit in addition to the comparison result of the focal length comparison unit.
The imaging apparatus according to any one of appendices 1 to 3, wherein the imaging apparatus is characterized in that
[Appendix 5]
An input unit that receives an instruction operation for selecting a predetermined region as a feature region from the image captured by the imaging unit;
The feature region extraction means specifies the feature region instructed by the input means as the main feature region;
The imaging apparatus according to any one of appendices 1 to 4, wherein the imaging apparatus is characterized in that
[Appendix 6]
Display control means for executing control to superimpose and display a symbol image indicating the stereoscopic effect calculated by the stereoscopic effect calculating means on the image captured by the imaging means;
The imaging apparatus according to any one of appendices 1 to 5, further comprising:
[Appendix 7]
An image processing unit that executes image processing for enhancing the stereoscopic effect of the image captured by the imaging unit when the stereoscopic effect value calculated by the stereoscopic effect calculating unit is a predetermined threshold value or less;
The imaging apparatus according to any one of appendices 1 to 6, further comprising:
[Appendix 8]
In an imaging method executed by an imaging device that captures an image of a subject and obtains an image,
An imaging step of imaging a plurality of subjects to obtain images;
A feature region extraction step for extracting a plurality of feature regions, each indicating an image of the plurality of subjects included in the image captured by the imaging step;
A focal length detection step for detecting a focal length for the plurality of feature regions extracted by the feature region extraction step;
Of the focal lengths detected by the focal length detection step, a focal length comparison step for comparing a focal length for a main feature region with a focal length for other feature regions;
A stereoscopic effect calculating step for calculating the degree of stereoscopic effect of the stereoscopic display when it is assumed that stereoscopic display is performed using the image captured in the imaging step based on the comparison result of the focal distance comparing step; ,
An imaging method comprising:
[Appendix 9]
A computer that controls an imaging device that captures an image of a subject and obtains the image;
An imaging means for capturing a plurality of subjects to obtain images;
Feature region extraction means for extracting a plurality of feature regions, each indicating an image of the plurality of subjects included in an image picked up by the image pickup means;
A focal length detection means for detecting a focal length for the plurality of feature areas extracted by the feature area extraction means;
Of the focal lengths detected by the focal length detection means, a focal length comparison means for comparing a focal length for a main feature region with a focal length for other feature regions;
3D effect calculation means for calculating the degree of 3D effect of the 3D display when it is assumed that 3D display is performed using the image captured by the imaging means based on the comparison result of the focal length comparison means;
A program characterized by functioning as

  DESCRIPTION OF SYMBOLS 1 ... Imaging device, 11 ... CPU, 12 ... ROM, 13 ... RAM, 14 ... Bus, 15 ... Input-output interface, 16 ... Imaging part, 17 ... Input unit 18 ... Output unit 19 ... Storage unit 20 ... Communication unit 21 ... Drive 22 ... Removable media 41 ... Image acquisition unit 42 ... Features Area extraction unit 43... Focal length detection unit 44... Focal length comparison unit 45. Three-dimensional effect calculation unit 46... Display control unit 51.・ Imaging condition setting unit, 61... Image processing unit

Claims (9)

Imaging means for capturing a plurality of subjects and obtaining images;
Feature region extraction means for extracting a plurality of feature regions, each indicating an image of the plurality of subjects included in an image picked up by the image pickup means;
A focal length detection means for detecting a focal length for the plurality of feature areas extracted by the feature area extraction means;
Of the focal lengths detected by the focal length detection means, a focal length comparison means for comparing a focal length for a main feature region and a focal length for other feature regions;
Stereoscopic effect calculating means for calculating the degree of stereoscopic effect of the stereoscopic display when it is assumed that stereoscopic display is performed using the image captured by the imaging means based on the comparison result of the focal length comparing means; ,
Presenting means for presenting the stereoscopic effect calculated by the stereoscopic effect calculating means;
An imaging apparatus comprising: A zoom magnification setting means for setting a zoom magnification for imaging by the imaging means;
The stereoscopic effect calculating means calculates the degree of stereoscopic effect based on the zoom magnification set by the zoom magnification setting means in addition to the comparison result of the focal length comparing means.
The imaging apparatus according to claim 1. Further comprising aperture setting means for adjusting the amount of light incident on the imaging means;
The stereoscopic effect calculation means calculates the degree of stereoscopic effect based on the amount of light adjusted by the aperture setting means in addition to the comparison result of the focal length comparison means.
The imaging apparatus according to claim 1 or 2, wherein Further comprising mode setting means for setting an imaging mode to be imaged by the imaging means;
The feature region extraction unit extracts the plurality of feature regions based on the imaging mode set by the mode setting unit in addition to the comparison result of the focal length comparison unit.
The image pickup apparatus according to claim 1, wherein the image pickup apparatus is an image pickup apparatus. An input unit that receives an instruction operation for selecting a predetermined region as a feature region from the image captured by the imaging unit;
The feature region extraction means specifies the feature region instructed by the input means as the main feature region;
The image pickup apparatus according to claim 1, wherein the image pickup apparatus is an image pickup apparatus. It said presenting means, the imaging apparatus according to any one of claims 1 to 5, wherein the superimposing display to Turkey the symbol image that indicates the three-dimensional effect that is calculated by the three-dimensional effect calculation means. An image processing unit that executes image processing for enhancing the stereoscopic effect of the image captured by the imaging unit when the stereoscopic effect value calculated by the stereoscopic effect calculating unit is a predetermined threshold value or less;
The imaging apparatus according to claim 1, further comprising: In an imaging method executed by an imaging device that captures an image of a subject and obtains an image,
An imaging step of imaging a plurality of subjects to obtain images;
A feature region extraction step for extracting a plurality of feature regions, each indicating an image of the plurality of subjects included in the image captured by the imaging step;
A focal length detection step for detecting a focal length for the plurality of feature regions extracted by the feature region extraction step;
Of the focal lengths detected by the focal length detection step, a focal length comparison step for comparing a focal length for a main feature region with a focal length for other feature regions;
A stereoscopic effect calculating step for calculating the degree of stereoscopic effect of the stereoscopic display when it is assumed that stereoscopic display is performed using the image captured in the imaging step based on the comparison result of the focal distance comparing step; ,
A presentation step for presenting the stereoscopic effect calculated by the stereoscopic effect calculating step;
An imaging method comprising: A computer that controls an imaging device that captures an image of a subject and obtains the image;
An imaging means for capturing a plurality of subjects to obtain images;
Feature region extraction means for extracting a plurality of feature regions, each indicating an image of the plurality of subjects included in an image picked up by the image pickup means;
A focal length detection means for detecting a focal length for the plurality of feature areas extracted by the feature area extraction means;
Of the focal lengths detected by the focal length detection means, a focal length comparison means for comparing a focal length for a main feature region with a focal length for other feature regions;
3D effect calculation means for calculating the degree of 3D effect of the 3D display when it is assumed that 3D display is performed using the image captured by the imaging means based on the comparison result of the focal length comparison means;
Presenting means for presenting the stereoscopic effect calculated by the stereoscopic effect calculating means;
A program characterized by functioning as

Images