JP2013070154A - Compound-eye imaging device and operation control method therefor - Google Patents

Abstract

[Purpose] To prevent the face from overlapping the background image.
[Structure] The subject is imaged from different viewpoints to obtain a plurality of subject images. A face image is detected from the subject image obtained by imaging (step 42), and a blind spot area is detected (step 43). If the blind spot area overlaps the background object (YES in step 45), a face movement instruction is given (step 47). If it is not superimposed on the blind spot area or the background object (NO in step 45), image data representing the subject image obtained by imaging is recorded (step 50).
[Selection] Figure 2

Description

  The present invention relates to a compound eye imaging apparatus and an operation control method thereof.

  A liquid crystal display device is provided on the back of the digital camera. The captured subject image is displayed on the display screen of the liquid crystal display device. Since the display screen is formed on the back of the digital camera, when the photographer takes an image of himself / herself, the image cannot be taken while looking at the display screen. Because of this, important backgrounds may be hidden behind you. For this reason, there is a technique in which the standing position of the person to be photographed with respect to the background is set in a desired relationship in the self-portrait with the actual building or landscape as the background (Patent Document 1). In addition, when a plurality of face frames are displayed together with a plurality of faces, the stereoscopic effect of a stereoscopic observer is not disturbed (Patent Document 2), and the object can be perceived at an actual size (Patent Document 2) Document 3), and others that perform image alignment with a small amount of calculation (Patent Document 4).

JP 2011-19051 A JP 2011-77900 A JP 2010-147940 A

  In the device described in Patent Document 1, a person index must be set in advance for a background image in which no person is shown. Also, if the blind spot area resulting from photographing yourself on the background overlaps, the blind spot area cannot be seen stereoscopically.

  An object of the present invention is to prevent an important background from being concealed by itself when a photographer takes an image of himself / herself even if a preset such as a person index is unnecessary. Another object of the present invention is to prevent a background object from overlapping with a blind spot area generated by photographing itself.

  The present invention detects a face region from at least one of a plurality of subject images obtained by imaging a subject with the plurality of imaging devices in a compound eye imaging device including a plurality of imaging devices that repeat imaging from different viewpoints. Whether or not there is a blind spot area that is included in one of the plurality of subject images but not included in the other subject image adjacent to the face area detected by the face detection means. A blind spot area judging means for determining whether or not the blind spot area overlaps a background object area adjacent to the face area in the background of the face area in response to the blind spot area judging means determining that the blind spot area exists. Superimposition determination means for determining whether the blind spot area overlaps the area of the background object by the superimposition determination means. Characterized in that it comprises a first informing means for informing a movement instruction of the device itself.

  The present invention also provides an operation control method suitable for the compound eye imaging apparatus. That is, this method is a method for controlling the operation of a compound eye imaging device including a plurality of imaging devices that repeats imaging from different viewpoints. In this method, the face detection means obtains a plurality of images obtained by imaging a subject with the plurality of imaging devices. A face area is detected from at least one of the subject images, and the blind spot area determining means includes a blind spot area that is included in one of the subject images but not included in the other subject image. Is determined to be adjacent to the detected face area, and the blind spot area is detected when the blind spot area is determined to be present by the blind spot area determining means. Is determined whether or not it overlaps with the background object area adjacent to the face area, and the notification means causes the superimposition determination means to overlap the blind spot area with the background object area. It is intended to notify the movement instruction of the face or the compound-eye imaging apparatus itself in response to that is constant.

  According to the present invention, a face area is detected from a plurality of subject images captured from different viewpoints obtained by capturing a subject with a plurality of imaging devices. It is determined whether there is a blind spot area adjacent to the detected face area. If it is determined that the blind spot area exists, it is determined whether or not the blind spot area overlaps the area of the background object adjacent to the face area in the background of the face area. If it is determined that the blind spot area overlaps with such a background object area, an instruction to move the face or the compound-eye imaging device itself is notified.

  If there are a plurality of subject images taken from different viewpoints, a stereoscopic image can be obtained. However, if there is a blind spot area, the blind spot area cannot be seen three-dimensionally. When the blind spot area overlaps with the background object area, the user is instructed to move the face or the compound-eye imaging device itself, so that the photographer knows that the face overlaps the background object. By moving the face and picking up images, both the face and the background object can be picked up. In particular, since the blind spot area does not overlap the background object, it is possible to prevent the background object from being viewed stereoscopically by the blind spot area.

  There may be further provided an angle of view determination means for determining whether or not the face area detected by the face detection means falls within a predetermined angle of view which is an angle of view narrower than a shootable range. In this case, for example, the first notification means notifies the movement instruction of the face or the compound-eye imaging device itself when the face angle determination means determines that the face is within a predetermined angle of view. Is.

  The superimposition determining unit may further include a second notification unit that notifies the photographing permission when it is determined that the blind spot region does not overlap with the background object region.

  It is preferable to further include third informing means for informing the photographing permission in response to the fact that the face is determined to be within a predetermined angle of view by the angle of view determining means.

  First recording that records image data representing subject images captured by the plurality of imaging devices on a recording medium in response to determining that the blind spot area does not overlap with the image of the background object by the superimposition determination unit. Control means may be further provided.

  Second recording that records image data representing subject images captured by the plurality of imaging devices on a recording medium in response to determining that the face is within a predetermined angle of view by the angle of view determination means. Control means may be further provided.

  The image processing apparatus may further include first parallax amount adjusting means for adjusting parallax amounts of the plurality of subject images so that the parallax between the face represented by the face image and the object between the background object becomes zero. Good.

  A distance calculation unit that calculates a distance between the face represented by the face image and the background object, and a parallax amount of the image portion of the face according to whether the distance calculated by the distance calculation unit is equal to or greater than a predetermined distance You may further provide the 2nd parallax amount adjustment means which adjusts the parallax amount of said several to-be-photographed image so that may become below a predetermined parallax amount.

  The background object parallax amount calculating means for calculating the parallax amount of the background object, and the 2D image data is recorded on the recording medium when the parallax amount calculated by the background object parallax amount calculating means is a predetermined amount or more. Third recording control means may be further provided.

  A first area calculating means for calculating a total area of the background object; a second area calculating for calculating an area of a remaining portion excluding a portion where the background object is hidden by a face detected by the face detecting means; And a fourth notification means for reporting an area ratio between the area calculated by the first area calculation means and the area calculated by the second area calculation means.

  The background object is an object of the same color, for example.

It is a block diagram which shows the electric constitution of a digital camera. It is a flowchart which shows the process sequence of a digital camera. (A) shows an example of a left viewpoint image, and (B) shows an example of a right viewpoint image. (A) shows an example of a left viewpoint image, and (B) shows an example of a right viewpoint image. (A) shows an example of a left viewpoint image, and (B) shows an example of a right viewpoint image. (A) to (C) are examples of captured images. It is a flowchart which shows a superimposition determination processing procedure. It is a flowchart which shows the left viewpoint image superimposition determination processing procedure. (A) shows an example of the left viewpoint color division image, and (B) shows an example of the right viewpoint color division image. It is a flowchart which shows the parallax amount adjustment process procedure. The relationship between the subject distance range and the amount of parallax is shown. (A) shows a part of the subject image, and (B) shows the position of the subject. (A) shows a part of the subject image, and (B) shows the position of the subject. (A) shows a part of the subject image, and (B) shows the position of the subject. (A) shows a part of the subject image, and (B) shows the position of the subject. The relationship between the subject distance range and the amount of parallax is shown. It is a flowchart which shows an imaging control processing procedure. It is a flowchart which shows the process sequence of a digital camera. It is an example of a to-be-photographed image.

  FIG. 1 shows an embodiment of the present invention and is a block diagram showing the electrical configuration of a so-called compound-eye digital camera.

  The entire operation of the digital camera 1 is controlled by the CPU 10. The digital still camera 1 includes a stereoscopic imaging mode for generating a stereoscopic image, an imaging mode for performing normal two-dimensional imaging, a two-dimensional reproduction mode for performing two-dimensional reproduction, a stereoscopic reproduction mode for displaying a stereoscopic image, a setting mode, and the like. There is provided an operating device (not shown) including various buttons such as a mode setting button for setting the mode and a two-stroke type shutter / release button. An operation signal output from the operation device is input to the CPU 10.

  The digital still camera includes a left viewpoint imaging device 1 that captures a left viewpoint image viewed by a viewer of a stereoscopic image with the left eye and a right viewpoint imaging device 2 that captures a right viewpoint image viewed by the viewer of the stereoscopic image with a right eye. include.

  The left viewpoint imaging device 1 includes a CCD 4 that images a subject and outputs a left viewpoint image signal representing a left viewpoint image. In front of the CCD 4, there are provided a zoom lens 2 and a focus lens 3 whose zoom amounts and focus amounts are controlled by motor drivers 5 and 6, respectively.

  When the stereoscopic imaging mode is set, pre-imaging (through image imaging) is performed before the shutter release button is pressed, and the left viewpoint image signal is output from the CCD 4 at a constant cycle. The left viewpoint image signal output from the CCD 4 is converted into left viewpoint image data in the analog / digital conversion circuit 7. The left viewpoint image data is input to the digital signal processing device 21 by the image input controller 8.

  The right viewpoint imaging device 11 includes a CCD 14 that images a subject and outputs a right viewpoint image signal representing a right viewpoint image. In front of the CCD 14, there are provided a zoom lens 12 and a focus lens 13 whose zoom amounts and focus amounts are controlled by motor drivers 15 and 16, respectively.

  In the stereoscopic imaging mode, the right viewpoint image signal is also output from the CCD 14 during pre-imaging. The right viewpoint image signal output from the CCD 14 is converted into right viewpoint image data in the analog / digital conversion circuit 17. The right viewpoint image data is input to the digital signal processing device 21 by the image input controller 18.

  The digital signal processing device 21 performs predetermined digital signal processing on the left viewpoint image data and the right viewpoint image data. The left viewpoint image data and the right viewpoint image data output from the digital signal processing device 21 are given via a display control device 26 to a first display device 27 provided with a display screen on the back of the digital camera. An image obtained by imaging is displayed as a three-dimensional moving image on the display screen of the first display device 27 (through image display). Both the left viewpoint image data and the right viewpoint image data are given to the first display device 27, and either the left viewpoint image data or the right viewpoint image data is displayed without displaying the subject image stereoscopically. Data may be given to display the subject image two-dimensionally.

  The digital camera also includes a second display device 28 having a display screen formed in front of it.

  The left viewpoint image data and the right viewpoint image data output from the digital signal processing device 21 are also provided to the parallax amount calculation device 29 and the face detection device 30. The parallax amount calculation device 29 calculates the parallax amount between the left viewpoint image and the right viewpoint image. Further, the face detection device 30 detects the face image portion from the captured image.

  The subject is imaged by half-pressing the shutter release button, and at least one of the left viewpoint image data and the right viewpoint image data obtained by the imaging is input to the integrating device 23. In the integrating device 23, high frequency components and luminance data are integrated. The focus amounts of the focus lenses 3 and 13 are determined based on the integrated value of the high frequency components. Based on the integrated value of the luminance data, the shutter speed of the so-called electronic shutter is determined. When the shutter release button is fully pressed, the subject is imaged, and the left viewpoint image data obtained by imaging by the left viewpoint imaging device 1 and the right viewpoint image data obtained by imaging by the right viewpoint imaging device 11 are controlled by the memory. It is given to the main memory 20 via the device 19 and temporarily stored in the main memory 20. The left viewpoint image data and the right viewpoint image data are read from the main memory 20 and input to the compression / decompression processor 22. The left viewpoint image data and the right viewpoint image data are compressed by the compression / decompression processor 22, and the compressed left viewpoint image data and right viewpoint image data are recorded on the memory card 25 by the memory controller 24.

  When the stereoscopic playback mode is set, the left viewpoint image data and the right viewpoint image data recorded in the memory card 25 are read. The read left viewpoint image data and right viewpoint image data are decompressed by the compression / decompression processor 22. The expanded left viewpoint image data and right viewpoint image data are given to the display device 26, whereby a stereoscopic image is displayed.

  FIG. 2 is a flowchart showing the processing procedure of the compound-eye digital camera.

  In a compound-eye digital camera, a subject image obtained by imaging is displayed on a display screen (first display device 27) formed on the back surface. Since the display screen is formed on the rear surface of the compound-eye digital camera, when the user takes an image of himself / herself with the compound-eye digital camera, the display screen of the first display device 27 cannot be seen. The camera angle cannot be determined while viewing the subject image displayed on the display screen of the first display device 27. For this reason, an object (background object) considered to be important in the background may be hidden behind the user. In this embodiment, a compound eye digital camera is used to prevent the background object from being hidden behind the user when the user images itself.

  The user holds a compound-eye digital camera and faces it. The user who is the subject himself / herself is imaged by the left viewpoint imaging device and the right viewpoint imaging device (step 41).

  3A shows an example of a left viewpoint image 60L taken by the left viewpoint imaging device, and FIG. 3B shows an example of a right viewpoint image 60R taken by the right viewpoint imaging device.

  Since the left viewpoint image 60L and the right viewpoint image 60R are captured from different viewpoints, parallax occurs. Since parallax has occurred, a stereoscopic image can be generated from the left viewpoint image 60L and the right viewpoint image 60R. The left viewpoint image 60L includes an image 62L of the user who is the photographer and an image 61L of the car behind the user. Similarly, the right viewpoint image 60R includes a user image 62R and a car image 61R corresponding to the user image 62L and the car image 61L included in the left viewpoint image 60R.

  When the left viewpoint image 60L and the right viewpoint image 60R are obtained by imaging, face detection processing is performed on each of the obtained left viewpoint image 60L and right viewpoint image 60R (step 42 in FIG. 3). For face detection processing, template matching or the like can be used.

  4A shows an example of the left viewpoint image 60L, and FIG. 4B shows an example of the right viewpoint image 60R.

  The face image 63L and the face image 63R are detected by performing face detection processing on each of the left viewpoint image 60L and the right viewpoint image 60R.

  In FIGS. 4A and 4B, face presence defining regions 64L and 64R defined by a predetermined angle of view are also defined. In this embodiment, in order to prevent the background object from being hidden by the user's own face, the user is notified to move the face when the background object is hidden by the user's own face. However, if the face of the user is moved when the user's face is at the end of the imaging range, the user's face will not be captured. For this reason, if the user's face is within the imageable range even if the face is moved, the user is notified to move the face. When the face is in the face presence defining areas 64L and 64R, the user is notified to move the face.

  When a face is detected, a blind spot area is detected from each of the left viewpoint image 64L and the right viewpoint image 64R (step 43 in FIG. 2).

  FIGS. 5A and 5B show a left viewpoint image 60L and a right viewpoint image 60R. In these drawings, the same components as those shown in FIGS. 3A and 3B are denoted by the same reference numerals, and description thereof is omitted.

  As described above, since the left viewpoint image 60L and the right viewpoint image 60R are captured from different viewpoints, the left viewpoint image 60L is included in the left viewpoint image 60L but is included in the right viewpoint image 60R. A blind spot area 63 (right blind spot area) that is not included is included. The right blind spot area 63 is on the left side of the face image 62L and overlaps the automobile image 61L. Similarly, the right viewpoint image 60R includes a blind spot area (left blind spot area) 64 that is included in the right viewpoint image 60R but not included in the left viewpoint image 60L. The left blind spot area 64 is on the right side of the face image 62R. Correspondence of each pixel between the left viewpoint image 60L and the right viewpoint image 60R is detected, and a blind spot area is detected when there is no corresponding pixel.

  The blind spot areas 63 and 64 are included in one of the left viewpoint image 60L and the right viewpoint image 60R, but are not included in the other image. Cannot be seen three-dimensionally. In this embodiment, a blind spot area 63 adjacent to the face image 62L as shown in FIG. 5A or a blind spot area 64 adjacent to the face image 62R as shown in FIG. 5B overlaps the background object. If so, it is considered that a part of the background object is hidden by the face. For this purpose, a process for determining whether or not the blind spot area 63 or 64 overlaps the background object is performed (step 44 in FIG. 2). Details of this superimposition determination process will be described later.

  If the right blind spot area 63 or the left blind spot area 64 is not superimposed on the background object (NO in step 45), it is considered that the background object is not hidden by the user's own face. Therefore, the recording permission is notified to the user (step 48). For example, since the user takes an image of himself / herself with a compound-eye digital camera, the display screen of the first display device 27 cannot be seen. For this reason, even if the recording permission is displayed on the display screen of the first display device 27, the user does not know. Therefore, the display screen (LED may be used) of the second display device 28 provided in front of the compound-eye digital camera. ) Will be notified to the user. Voice notification may be used. When the shutter release button is pressed by the user (YES in step 49), the user who is the subject is imaged, and image data representing the left viewpoint image and the right viewpoint image obtained by the imaging are stored in the memory card 25, respectively. Recorded (step 50). When the blind spot area is not superimposed on the background object (NO in step 45), the user who is the subject is imaged, and the image data representing the left viewpoint image and the right viewpoint image obtained by the imaging are respectively stored in the memory card 25. It is preferable that the total area S of the background object 61 is calculated and data representing the total area S is also recorded on the memory card 25. As will be described later, when the background object 61 is hidden by the face 62, the user can be informed of the ratio of the hidden object.

  If the right blind spot area 63 or the left blind spot area 64 is superimposed on the background object (YES in step 45), it is considered that the background object is hidden by the user's own face. For this purpose, the user is notified to move the face (step 47) (the user may be notified to move the compound-eye digital camera itself).

  6A to 6C are examples of a three-dimensional image (not necessarily a three-dimensional image) displayed on the display screen of the first display device 27. FIG.

  Referring to FIG. 6A, when the background object 61 is hidden by the face image 62 included in the captured image 60A, the user is notified to move the face as described above. In response to the notification, the user moves his / her face. Then, as shown in FIG. 6B, since the face image 62 included in the captured image 60B moves in the horizontal direction, the ratio of the background object 61 visible increases. If the background object 61 is still hidden by the face image 62, the user is notified to move the face again. In response to the notification, the user moves the face again. As shown in FIG. 6C, the face image 62 moves in the horizontal direction and the background object 61 becomes visible.

  As described above, when the user moves the face in accordance with the face movement instruction, the overlap between the face and the background object is reduced.

  However, as described above, when the face is within the predetermined specified areas 64L and 64R (within the predetermined angle of view) so that the face after movement is within the imaging range ( If YES in step 46), the user is notified to move the face (step 47). For example, the fact is displayed on the display screen of the second display device 28 provided in front of the compound-eye digital camera. Audio notification may be used. If the face is not within the predetermined angle of view (NO in step 46), the recording permission is notified to the user (step 48).

  In the above-described embodiment, the image data obtained by imaging the subject in response to the shutter release being turned on by the user after notifying the user of the recording permission is recorded on the memory card. Automatic imaging may be performed without notifying the user of the recording permission, and the obtained image data may be recorded on the memory card. The user can concentrate on the setting of the angle of view and save the trouble of shooting.

  FIG. 7 is a flowchart showing the superimposition determination processing procedure (the processing procedure of step 44 in FIG. 2).

  The superimposition determination process is performed for each of the left viewpoint image 60L and the right viewpoint image 60R. First, the superimposition determination is performed on the left viewpoint image 60L (step 71), and then the superimposition determination is performed on the right viewpoint image 60R (step 72).

  The obtained determination results of the left viewpoint image superimposition determination process and the right viewpoint image superimposition determination process are integrated (step 73). For example, processing is performed so that a background object close to a compound-eye digital camera is not a shadow of the face.

  FIG. 8 is a flowchart showing the left viewpoint image superimposition determination processing procedure (the processing procedure of step 71 in FIG. 7). The same processing is performed for the image describing the superimposition determination processing for the left viewpoint image 60L and the right viewpoint image 60R.

  First, the color area dividing process of the left viewpoint image 60L is performed (step 81). The color area dividing process is a process of dividing an object included in an image into areas for each color. Color region division processing is also performed on the right viewpoint image 60R. For example, the k-means method can be used for the color region division processing.

  FIGS. 9A and 9B are examples of a right viewpoint image (right viewpoint color division image) 90L and a left viewpoint image (left viewpoint color division image) 90R that have been subjected to color division processing.

  Referring to FIG. 9A, by performing color division processing on the left viewpoint image 60L, a red region 91L that is the color of the car, a black region 92L that is the color of the user's hair, The color is divided into a light brown area 93L and a blue area 94L which is the background sky color. The right blind spot area 63 is also shown in the right viewpoint color division image 90L.

  Similarly, referring to FIG. 9B, by performing color division processing on the right viewpoint image 60R, a red region 91R that is the color of the automobile, a black region 92R that is the color of the user's hair, the user Are divided into areas of a light brown area 93R which is the face color and a blue area 94R which is the sky color of the background. The left blind spot area 64 is also shown in the left viewpoint color division image 90R.

  If the color area of the right viewpoint blind spot area 63 is the same color area as the adjacent color area, the right viewpoint blind spot area 63 is considered to be a part of the area adjacent to the right viewpoint blind spot area 63. For this purpose, it is determined whether the color area of the right-viewpoint blind spot area 63 is the same color area as the adjacent color area (step 82). However, if the size of the color area is equal to or larger than a predetermined size, it is considered that the background object may be hidden by the user such as the sky. For this purpose, it is also confirmed whether the size of the color area is equal to or smaller than a predetermined size (step 83).

  If the color area of the right-viewpoint blind spot area 63 is the same color area as the adjacent color area (YES in step 82), it is confirmed whether the size of the color area is equal to or smaller than a predetermined size as described above. (Step 83). If it is less than or equal to the predetermined size (YES in step 83), the right viewpoint blind spot area 63 is determined to be a part of the background object specified by the color area (step 84). Thus, it is determined that the right blind spot area 63 is superimposed on the background object specified by the color area 91L. As shown in FIG. 8A, when the color area of the right blind spot area 63 is determined to be the same color area as the adjacent color area 91L and the size of the color area 91L is equal to or smaller than a predetermined size, The right blind spot area 63 is determined to overlap the automobile image 61L specified by the color area 91L.

  If the color area of the right viewpoint blind spot area 63 is not the same color area as the adjacent color area (NO in step 82), it is determined that the right viewpoint blind spot area 63 is not part of the background object (step 85). ). For example, if the color of the right viewpoint blind spot area 63 is different from the color of the color area 93L on the right side of the right viewpoint blind spot area 63, the right viewpoint blind spot area 63 is not part of the background object specified by the color area 93. It is determined. If the size of the color area adjacent to the right viewpoint blind spot area 63 is larger than a predetermined size, the right viewpoint blind spot area 63 is not determined to be a part of the color area adjacent to the right viewpoint blind spot area 63 (step 85).

  A process similar to the process described above is also performed for the right-viewpoint color-divided image shown in FIG.

  In the above-described embodiment, in the left viewpoint image 60L or the right viewpoint image 60R, when there is a blind spot area superimposed on the background object on both sides of the face image, the background object and the face image on the left side of the face image are displayed. Since it is not known if the background object on the right side is not superimposed on the face image, the imaging permission can be notified without notifying the face movement instruction. Of course, among the background objects on both sides of the face image, a face movement instruction may be given so that the background object closer to the compound-eye digital camera is not superimposed on the face image. The moving direction of the face can be determined based on the positional relationship between the position of the face image and the blind spot area. When the blind spot area is on the left side of the face image, the subject image is notified to move to the right side, and when the blind spot area is on the right side of the face image, the face image is moved to the left side of the subject image. You may make it alert | report so that. Further, the background object is considered to be an object of the same color, for example.

  FIGS. 10 to 15A and 15B show modifications.

  FIG. 10 is a flowchart showing a parallax adjustment processing procedure. This parallax adjustment is performed, for example, in response to the shutter release button being turned on (YES in step 49 in FIG. 2). FIG. 11 shows the relationship between the subject distance range and the amount of facial parallax. FIGS. 12A, 13A, 14A, and 15A show examples of subject images (left viewpoint image or right viewpoint image) 111, 121, 131, and 141. FIG. (B), FIG. 13 (B), FIG. 14 (B), and FIG. 15 (B) show the position of the subject.

  In this modification, the amount of parallax is adjusted based on the distance between the user and the background object.

  First, the distance to the face and the distance to the background object are calculated (step 101). These distances can be calculated based on the amount of movement of the focus lens. Subsequently, the distance between the face and the background object (subject distance range) is calculated (step 102). The amount of parallax is calculated based on the distance between the face and the background object (step 103). The amount of parallax between the left viewpoint image 60L and the right viewpoint image 60R is adjusted so as to be the calculated amount of parallax (step 104). However, the parallax amount adjustment processing may be performed so that the parallax amount of the object between the face and the background object becomes zero.

  Referring to FIG. 11, when the subject distance range is zero, the parallax amount is also zero. The face position becomes the position of the cross point. Until the subject distance range reaches a predetermined value R1, the parallax amount of the face gradually increases as the subject distance range increases. When the subject distance range exceeds the predetermined value R1, a predetermined allowable parallax amount is obtained.

  If the amount of parallax of the face becomes large, the amount of parallax of the background becomes larger than that, which makes it difficult for stereoscopic viewing. Since the parallax amount of the face is suppressed to the allowable parallax amount, it is possible to prevent the background parallax amount from increasing. Not only the face but also the background subject can be viewed stereoscopically.

  FIG. 12A is an example of the subject image 111 only for the user except when the subject is empty. In this way, when the subject is only the user other than the sky, the distance between the user and the background object is 0, and the user image (face image) 112 is the position of the cross point as shown in FIG. The parallax amount is adjusted so that

  In FIG. 13A, the subject image 121 includes a user image 122 and a motorcycle image 123. As shown in FIG. 13B, when the distance between the user and the motorcycle is small, the amount of parallax is adjusted in accordance with the distance difference.

  In FIG. 14A, a subject image 131 includes a user image 132 and a building image 132. As shown in FIG. 14B, even when the distance between the user and the building is relatively close, the amount of parallax is adjusted according to the distance difference.

  In FIG. 15A, the subject image 141 includes a user image 142 and a mountain image 142. As shown in FIG. 15B, if the distance between the user and the mountain is long, stereoscopic viewing may be difficult if the amount of parallax is adjusted according to the distance difference. For this reason, as described above, the parallax amount of the user's face is suppressed to the allowable parallax amount.

  FIG. 16 shows the relationship between the subject distance range and the amount of face parallax, and corresponds to FIG.

  In the relationship shown in FIG. 11, when the subject distance range is 0, the face parallax amount is 0. However, in the relationship shown in FIG. 16, when the subject distance range is 0, the face parallax amount is 0. Is the optimum amount of parallax. The parallax amount of the face is increased as the distance range increases until the subject distance range reaches a predetermined distance range R1. As described above, the face parallax amount is suppressed to the allowable parallax amount when the distance is equal to or greater than the predetermined distance range R1.

  It is necessary to change the allowable parallax amount and the optimal parallax amount of the face parallax amount according to the size and resolution of the display screen. For example, when a stereoscopic image is displayed on a 3.5-inch display screen having 800 pixels in the horizontal direction and 640 pixels in the vertical direction, for example, the allowable parallax amount is 56 pixels and the optimal parallax amount is 24 pixels.

  FIG. 17 shows another modified example and is a flowchart showing an imaging control processing procedure. When the shutter release button is pressed by the user, the processing procedure shown in FIG. 16 is performed.

  As described above, the distance to the face and the distance to the background object are calculated (step 101), and the distance (distance range) between the distance to the face and the distance to the background object is calculated (step 102). . A parallax amount is calculated based on the calculated distance range (step 103).

  If there is a background object in which the amount of parallax becomes too large (YES in step 151), stereoscopic vision is broken, and 2D imaging is performed (step 153). If there is no background object whose parallax amount is too large (NO in step 151), 3D imaging is performed (step 152). 2D imaging may be performed with one of the left viewpoint imaging device and the right viewpoint imaging device, and 3D imaging may be performed with both the left viewpoint imaging device and the right viewpoint imaging device. .

  FIG. 18 shows another embodiment, and is a flowchart showing the processing procedure of the compound-eye digital camera. This process is performed following the process shown in FIG. FIG. 19 is an example of a captured image 60D displayed on the display screen of the first display device 27. This corresponds to the captured images 60A to 60C shown in FIGS. 6 (A) to 6 (C).

  In this embodiment, the user is notified of the degree of overlap between the background object and the face.

  As described above, the total area S of the background object is recorded, and the total area S is used. If the angle of view changes, the background object included in the imaging range may change. If the angle of view changes (YES in step 162), other processing is performed. .

  If the angle of view has not changed (NO in step 162), the subject is imaged as shown in FIG. 19, and the face image 62 is detected from the obtained captured image 60D (step 163).

  As described above, the background object 61 is found from the captured image 60D, and among the found background objects 61, the non-display area and the blind spot area that are not displayed by the face 62 are removed. Area A is calculated (step 164). The calculated area A represents an area that can be seen as a stereoscopic image.

  The display ratio R = A / S of the background object is calculated (step 165), and the calculated display ratio is notified to the user (step 166). A display screen will be formed in front of the compound-eye digital camera, and the display ratio will be displayed on the display screen, or the display ratio will be output as audio. For example, in the case shown in FIG. 19, it is notified that the ratio of the background object 61 is 80%. By knowing the ratio, the user knows how much of the background object 61 can be viewed stereoscopically. If the ratio is acceptable, the shutter release button is turned on (YES in step 167), and image data (left viewpoint image data and right viewpoint image data) obtained by imaging is recorded in the memory card 25 (step 168). ). Since the user knows the degree of overlap between the face and the background object, the user can take an image when the degree of overlap is desired.

  If the ratio of the background object 61 to be imaged is not the desired ratio (NO in step 167), the user moves the face, and the ratio of the stereoscopic view of the entire background object 61 is calculated and displayed again (step). 163-166). Even in this case, the user may be notified that the face is moved in order to change the proportion of the background object 61 that can be stereoscopically viewed.

1, 2 Imaging device
10 CPU

Claims (12)

In a compound eye imaging device having multiple imaging devices that repeat imaging from different viewpoints,
Face detection means for detecting a face region from at least one of a plurality of subject images obtained by imaging a subject with the plurality of imaging devices;
A blind spot that determines whether a blind spot area that is included in one of the subject images but not included in the other subject image exists adjacent to the face area detected by the face detection means. Area determination means,
A superimposition judging means for judging whether or not the blind spot area overlaps a background object area adjacent to the face area in the background of the face area in response to the blind spot area judging means that the blind spot area exists, First notifying means for notifying the movement instruction of the face or the compound-eye imaging device itself when it is determined by the superimposition determining means that the blind spot area overlaps the area of the background object;
A compound eye imaging device. An angle-of-view determination means for determining whether the face area detected by the face detection means is within a predetermined angle of view that is narrower than an imageable range;
The first notification means includes
In response to a determination that the face is within a predetermined angle of view by the angle of view determination means, a movement instruction for the face or the compound-eye imaging device itself is notified.
The compound eye imaging device according to claim 1. Second notifying means for notifying the photographing permission when it is determined by the superimposition determining means that the blind spot area does not overlap with the background object area;
The compound eye imaging device according to claim 1, further comprising: Third notification means for notifying permission of photographing in response to the fact that the face is determined to be within a predetermined angle of view by the angle of view determination means;
The compound eye imaging device according to claim 2, further comprising: First recording that records image data representing subject images captured by the plurality of imaging devices on a recording medium in response to determining that the blind spot area does not overlap with the image of the background object by the superimposition determination unit. Control means,
The compound eye imaging device according to any one of claims 1 to 4, further comprising: Second recording that records image data representing subject images captured by the plurality of imaging devices on a recording medium in response to determining that the face is within a predetermined angle of view by the angle of view determination means. Control means,
The compound eye imaging device according to any one of claims 2 to 4, further comprising: First parallax amount adjusting means for adjusting parallax amounts of the plurality of subject images so that parallax between the face represented by the face image and an object between the background object is zero;
The compound eye imaging device according to any one of claims 1 to 6, further comprising: A distance calculating means for calculating a distance between the face represented by the face image and the background object; and a parallax amount of the image portion of the face in accordance with the distance calculated by the distance calculating means being equal to or greater than a predetermined distance Second parallax amount adjusting means for adjusting the parallax amounts of the plurality of subject images so that is equal to or less than a predetermined parallax amount;
The compound eye imaging device according to any one of claims 1 to 7, further comprising: The background object parallax amount calculating means for calculating the parallax amount of the background object, and the 2D image data is recorded on the recording medium when the parallax amount calculated by the background object parallax amount calculating means is equal to or larger than a predetermined amount. Third recording control means,
The compound eye imaging apparatus according to claim 8, further comprising: First area calculating means for calculating a total area of the background object;
A second area calculating means for calculating an area of a remaining portion excluding a portion where the background object is hidden by the face detected by the face detecting means; and an area calculated by the first area calculating means; Fourth informing means for informing an area ratio with the area calculated by the second area calculating means;
The compound eye imaging apparatus according to any one of claims 1 to 9, further comprising:   The compound eye imaging apparatus according to claim 1, wherein the background object is an object of the same color. In a method of controlling the operation of a compound-eye imaging device having a plurality of imaging devices that repeat imaging from different viewpoints,
A face detection unit that detects a face region from at least one of a plurality of subject images obtained by imaging the subject by the plurality of imaging devices;
The blind spot area determining means includes a blind spot area that is included in one of the subject images but not included in the other subject image adjacent to the face area detected by the face detecting means. Whether or not
In response to determining that the blind spot area exists by the blind spot area determining means, the superimposition determination means determines whether the blind spot area overlaps with the background object area adjacent to the face area in the background of the face area. ,
The notification means notifies the movement instruction of the face or the compound-eye imaging device itself when it is determined by the superimposition determination means that the blind spot area overlaps the area of the background object.
Operation control method of compound eye imaging apparatus.

Images