JP2007318307A - Image processing method, photographing apparatus, and image processing apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing method capable of acquiring photographed image data and displaying a three-dimensional image, a photographing apparatus capable of obtaining image data for displaying a three-dimensional image, and an image processing apparatus.
An imaging optical system 101 generates captured images (2D images) of a focusing position (front) of a first focus lens 101c and a focusing position (rear) of a second focus lens 101i. The brightness ratio processing unit 130 generates two brightness adjustment image data representing two brightness adjustment images adjusted to a higher brightness side as the degree of focusing of each pixel in the image is higher. Two compressed image data representing two compressed images compressed at a higher compression rate in a region where the degree of focusing of the image is lower are generated by the compression rate processing unit 131 and the compression processing circuit 123, and the image display unit 13 is overlapped. Are displayed on two light-transmitting display screens.
[Selection] Figure 2

Description

  The present invention relates to an image processing method for acquiring and displaying captured image data, an imaging apparatus for generating captured image data, and an image processing apparatus for acquiring and processing captured image data.

  2. Description of the Related Art Conventionally, in a photographing apparatus, various techniques relating to a photographing optical system that forms an image of a subject and an image obtained via the photographing optical system have been proposed. For example, in an electro-development type camera that electronically and directly records and develops a subject image that has entered via a photographic optical system as a visible image instantaneously, a plurality of electro-development type recording media are different at the time of shooting. A technique has been proposed that includes a holder assembly that positions an image surface and fixes a plurality of electro-developing recording media in a single plane after completion of photographing (see Patent Document 1). According to this technique, an optical path length correcting prism is not required, and a recorded image can be easily read from a plurality of electro-developing recording media.

  Further, in measuring the position of the inspection point on the electronic component by triangulation, a technique has been proposed in which a beam splitter is provided to divide the light beam from the electronic component into two, and each camera is placed in each light beam. (See Patent Document 2). According to this technology, the distance between the optical axes of each camera can be as close as several millimeters, so even three-dimensional unevenness can be accurately measured even at inspection points on small electronic components such as semiconductor elements. can do.

  Furthermore, the stereo microscope variable tilt angle binocular tube for the stereoscopic microscope in which the tilt angle of the eyepiece tube portion is variable is provided with a fixed lens barrel portion and a movable lens barrel portion, and the fixed lens barrel portion is formed in a U shape by the movable lens barrel portion. A technique has been proposed in which an image rotation correction optical system is disposed in the enclosure and the movable lens barrel (see Patent Document 3). According to this technique, it is possible to reduce the size of the lens barrel, and it is possible to improve workability such as surgery in medicine.

  In addition, two-dimensional images generated by projecting from the viewing direction of the observer are respectively displayed on a plurality of display surfaces arranged at different depth positions as viewed from the observer, and the two-dimensional images displayed are displayed. In a three-dimensional image display method for displaying a three-dimensional image (stereoscopic image) by independently changing the luminance of each display surface, the transparency of the two-dimensional image displayed on each display surface is changed for each display surface. A technique has been proposed in which the brightness of a two-dimensional image displayed on each display surface is independently changed by changing each independently (see Patent Document 4). According to this technology, video display is possible without using glasses, and contradiction between physiological factors of stereoscopic vision (for example, contradiction between binocular parallax and convergence and focus adjustment) is suppressed. In addition, the display can be electrically rewritten.

  Furthermore, a two-dimensional image generated by projecting the display object from the viewing direction of the observer is displayed on each of a plurality of display surfaces arranged at different depth positions as viewed from the observer, and the two images displayed are displayed. An optical member having a plurality of regions in which the light deflection directions are different from each other provided on the display surface arranged at the closest position of the observer while changing the brightness of the dimensional image independently for each display surface, A technique for emitting incident light incident on the display surface in different directions has been proposed (see Patent Document 5). According to this technique, a three-dimensional image can be simultaneously displayed to a plurality of observers with a simple configuration.

  The first and second transparent self-luminous display devices, the reflection mirror, the wavelength converting means, and the filter film are provided, and the light emitted from the first and second transparent self-luminous display devices is allowed to pass therethrough. The light of a predetermined wavelength which is blocked by the filter film and converted by the wavelength conversion means is allowed to pass, and the luminance of the two-dimensional image displayed on the first and second transparent self-luminous display devices is changed to the first and second. A technique has been proposed in which each pixel group is changed independently (see Patent Document 6). According to this technology, it is possible to suppress a contradiction between physiological factors of stereoscopic vision, and it is possible to display a three-dimensional image of a color image without using spectacles, and the configuration is simple and compact. An original image display device can be provided.

  Further, in the electronic endoscope apparatus, the image light emitted from the objective lens and the optical low-pass filter is separated by the cemented prism, and the separated light is sent to the far-point CCD and the near-point CCD set to different optical path lengths. A technique for making it incident has been proposed (see Patent Document 7). According to this technique, a plurality of images in focus at different distances can be simultaneously captured and displayed, and information useful for diagnosis and treatment can be obtained.

Further, in an electronic endoscope apparatus, image light emitted from an objective lens and an optical low-pass filter is separated into three optical paths by a cross prism, and far optical paths set to different optical path lengths so that these separated lights enter. A technique has been proposed in which a point CCD and a near-point CCD are arranged, and another far-point CCD is provided at a position shifted by a half pixel with respect to the far-point CCD (see Patent Document 8). According to this technique, it is possible to simultaneously display a plurality of images in focus at different distances, and to increase the resolution, thereby obtaining information useful for diagnosis, treatment, and the like.
Japanese Patent Laid-Open No. 9-96865 JP-A-9-42946 JP-A-11-231226 JP 2001-54144 A JP 2003-66371 A JP 2004-157296 A JP-A-11-197097 Japanese Patent Application Laid-Open No. 11-197098

  The techniques proposed in Patent Documents 1, 2, 3, and 7 described above are techniques that devise a photographing optical system, and the techniques proposed in Patent Documents 4, 5, 6, and 8 display a three-dimensional image. It is a technique to do. As described above, Patent Documents 1 to 8 described above propose a technique for three-dimensionally displaying an image obtained via a device for a photographing optical system or a photographing optical system. However, these Patent Documents 1 to 8 do not describe a technique for performing image processing by generating or acquiring captured image data necessary for displaying a three-dimensional image.

  In view of the above circumstances, the present invention provides an image processing method capable of acquiring photographed image data and displaying a three-dimensional image, a photographing apparatus capable of obtaining image data for displaying a three-dimensional image, and an image. An object is to provide a processing apparatus.

An image processing method of the present invention that achieves the above object is as follows.
Obtaining a plurality of photographed image data representing a plurality of photographed images having different in-focus distances for the same subject;
Generating a plurality of brightness adjustment image data representing a plurality of brightness adjustment images in which the brightness of each pixel in each captured image is adjusted to a higher brightness side as the degree of focusing of each pixel is higher;
The plurality of brightness adjustment images are displayed one by one on a plurality of light transmissive display screens arranged in an overlapping manner.

  In the image processing method of the present invention, first, a plurality of photographed image data representing a plurality of photographed images with different distances in focus with respect to the same subject are acquired. In this way, captured image data for displaying a three-dimensional image is acquired. Next, a plurality of brightness adjustment image data representing a plurality of brightness adjustment images in which the brightness of each pixel in each captured image is adjusted to a higher brightness side as the degree of focusing of each pixel is higher is generated. As a result, when viewed from the observer, for example, when a close captured image (front image) is in focus, the brightness of each pixel of the front image is adjusted to the high luminance side and a distant captured image (rear side) The two brightness adjustment image data representing the two brightness adjustment images in which the brightness of each pixel of the image is adjusted to the low brightness side is generated. Further, in the case of this example, the two brightness adjustment images are displayed one by one on the two light transmissive display screens arranged in an overlapping manner. In this way, the captured image data is acquired to generate a plurality of brightness adjustment image data, and the plurality of brightness adjustment images are displayed one by one on the plurality of light transmissive display screens arranged in an overlapping manner. Thus, a three-dimensional image can be displayed.

  Here, in the image processing method of the present invention, a plurality of compressed image data representing a plurality of compressed images obtained by compressing each captured image at a higher compression rate in a region where the degree of focusing of each captured image is lower, It is preferable to store the plurality of compressed image data.

  The area where the degree of focus of the captured image is low is not focused, so the degree of blur is large. Therefore, the area is compressed at a high compression ratio, and the area where the degree of focus of the captured image is high is out of focus. The degree of compression is small, so compression is performed at a low compression rate. By doing in this way, the image data for displaying a three-dimensional image can be compressed efficiently.

Moreover, the 1st imaging device of the imaging devices of this invention which achieve the said objective is
A photographing unit that generates a plurality of photographed image data representing a plurality of photographed images with different distances in focus with respect to the same subject by photographing;
An image processing unit for generating a plurality of brightness adjustment image data representing a plurality of brightness adjustment images adjusted to a higher brightness side as the degree of focusing of each pixel in each captured image is higher It is characterized by that.

  Here, the first photographing apparatus of the present invention has a plurality of light transmissive display screens arranged in an overlapping manner, and displays the plurality of brightness adjustment images one by one on the plurality of display screens. An image display unit is preferably provided.

  The first photographing apparatus of the present invention generates photographed image data for displaying a three-dimensional image by a photographing unit, and determines the luminance of each pixel in each photographed image represented by the generated photographed image data. The image processing unit generates a plurality of brightness adjustment image data representing a plurality of brightness adjustment images adjusted to the higher brightness side as the degree of focusing of the pixels is higher. For this reason, when viewed from the observer, for example, when the front image is in focus, the luminance of each pixel of the front image is adjusted to the high luminance side and the luminance of each pixel of the rear image is adjusted. Two brightness adjustment image data representing the two brightness adjustment images adjusted to the low brightness side are generated. Here, when the first image capturing apparatus of the present invention is provided with the image display unit, the two brightness adjustment images are displayed on the two light-transmitting display screens arranged on the image display surface. Can be displayed one by one. By doing so, it is possible to acquire captured image data and display a three-dimensional image based on the acquired captured image data.

Furthermore, the 2nd imaging device of the imaging devices of this invention which achieve the said objective is
A shooting unit that generates a plurality of shot image data representing a plurality of shot images with different distances in focus with respect to the same subject by shooting;
An image compression unit that generates a plurality of compressed image data representing a plurality of compressed images in which each captured image is compressed at a higher compression rate in a region where the degree of focusing of each captured image is lower .

  The second image capturing apparatus of the present invention acquires captured image data for displaying a three-dimensional image by the image capturing unit, and converts each captured image represented by the acquired captured image data to the focus of each captured image. The image compression unit generates a plurality of compressed image data representing a plurality of compressed images compressed at a higher compression rate in a region having a lower degree. For this reason, the region where the degree of focusing of the captured image is low is compressed at a high compression rate, and the region where the degree of focusing of the captured image is high is compressed at a low compression rate. Therefore, it is possible to efficiently compress image data for displaying a three-dimensional image.

The image processing apparatus of the present invention that achieves the above object is
An image acquisition unit for acquiring a plurality of photographed image data representing a plurality of photographed images having different in-focus distances for the same subject;
An image processing unit for generating a plurality of brightness adjustment image data representing a plurality of brightness adjustment images adjusted to a higher brightness side as the degree of focusing of each pixel in each captured image is higher It is characterized by that.

  Here, in the image processing apparatus of the present invention, an image display that has a plurality of light transmissive display screens arranged in an overlapping manner and displays the plurality of brightness adjustment images one by one on the plurality of display screens. It is preferable to have a part.

  The image processing apparatus of the present invention acquires captured image data for displaying a three-dimensional image by an image acquisition unit, and determines the luminance of each pixel in each captured image represented by the acquired captured image data. A plurality of brightness adjustment image data representing a plurality of brightness adjustment images adjusted to the higher brightness side as the degree of focusing is higher is generated by the image processing unit. For this reason, when viewed from the observer, for example, when the front image is in focus, the luminance of each pixel of the front image is adjusted to the high luminance side and the luminance of each pixel of the rear image is adjusted. Two brightness adjustment image data representing the two brightness adjustment images adjusted to the low brightness side are generated. Here, when the image processing apparatus according to the present invention includes the image display unit, the two brightness adjustment images are displayed one by one on the two light-transmitting display screens arranged on the image display surface. Can be displayed. By doing so, it is possible to acquire captured image data and display a three-dimensional image based on the acquired captured image data.

  In the image processing apparatus of the present invention, it is also possible to generate a plurality of compressed image data representing a plurality of compressed images in which each captured image is compressed at a higher compression rate in a region where the degree of focusing of each captured image is lower. This is a preferred embodiment.

  In this way, a region where the degree of focusing of the photographed image is low is compressed at a high compression rate, and a region where the degree of focusing of the photographed image is high is compressed at a low compression rate to display a three-dimensional image Can be efficiently compressed.

  According to the present invention, there are provided an image processing method capable of acquiring photographed image data and displaying a three-dimensional image, a photographing apparatus and an image processing apparatus capable of obtaining image data for displaying a three-dimensional image. can do.

  Embodiments of the present invention will be described below.

  FIG. 1 is a top view of a digital camera as an embodiment of both the first photographing apparatus of the present invention and the second photographing apparatus of the present invention (FIG. 1 (a)), viewed from the front. It is the front view (FIG.1 (b)) and the rear view (FIG.1 (c)) seen from the back surface.

  This digital camera 1 is a digital camera that can generate two photographed image data representing two photographed images with different distances in focus with respect to the same subject and display a three-dimensional image. In addition, an embodiment of the image processing method of the present invention is applied to the digital camera 1.

  First, an outline of the digital camera 1 will be described.

  As shown in FIGS. 1A and 1B, the digital camera 1 is provided with a lens barrel 10 in the center of a camera body 1a. A power button 11 is provided on the upper surface of the camera body 1a. When the power button 11 is operated, the lens barrel 10 is extended as shown in FIGS. 1 (a) and 1 (b). Preparations for shooting are ready.

  Further, on the upper surface of the camera body 1a, in addition to the power button 11 for turning on the power, a release button 12 and a shooting mode dial 12_1 are provided around the release button 12, and the shooting mode dial 12_1 is switched to AUTO. When the release button 12 is pressed while shooting is performed, shooting conditions are automatically set in the digital camera 1 and shooting is performed. When the shooting mode dial 12_1 is switched to the moving image shooting mode, moving image shooting is performed. When the shooting mode dial 12_1 is switched to the scene position (symbol SP) side, shooting conditions corresponding to the shooting scene are set in the digital camera 1. It is automatically set and shooting is performed.

  Further, as shown in FIG. 1C, an image display unit (liquid crystal monitor) 13 to be described in detail later is provided on the back side of the camera body 1a, and on the image display unit 13 in the shooting mode. The subject is displayed or the menu is displayed. When the playback button 14 on the side of the image display unit 13 is pressed once in this shooting mode, the mode is switched to the playback mode and the already shot image is displayed on the image display unit 13, and when the playback button 14 is pressed again. Then, the shooting mode is switched and a through image is displayed on the image display unit 13. In addition, an F (photo mode) button 15 is provided next to the playback mode button 14, and a frequently used mode such as a pixel setting mode or a sensitivity setting mode can be easily switched by operating the F button 15. You can also do it.

  A cross key 16 and an OK / menu button 17 are provided below the F button 15, and a zoom switch 18 is provided above the F button 15. By operating the cross key 16 or the OK / menu button 17, the setup menu is switched to set the date and time, whether or not to display images, and the shooting menu is switched to select continuous shooting, self-timer, etc. You can do it.

  Further, below the cross key 16, a DSP / BACK button 19 used for switching the display of the image display unit 13 or stopping the operation is provided.

  Further, a photometric sensor 20 and a flash light emission window 21 are provided in front of the camera body 1a shown in FIG. 1B. When flash light emission is required, the flash light is directed toward the subject from the flash light emission window 21. Light is emitted.

  FIG. 2 is a diagram showing an internal configuration of the digital camera shown in FIG.

  The digital camera 1 includes an iris 101a, a zoom lens 101b, a first focus lens 101c, a half mirror 101d, and a first reflecting mirror that constitute a photographing optical system 101 (corresponding to an example of an imaging unit according to the present invention). 101e, a second reflecting mirror 101f, a first CCD 101g, a third reflecting mirror 101h, a second focus lens 101i, and a second CCD 101j are provided.

  The digital camera 1 also includes motor drivers 111, 112, and 113 that drive motors for the iris 101a, the zoom lens 101b, and the first focus lens 101c.

  Further, the digital camera 1 is provided with a timing generator 114 and a CPU 115. The timing generator 114 drives the first CCD 101g and the second CCD 101j at a predetermined timing according to an instruction from the CPU 115. As a result, the subject light incident on the first CCD 101g and the second CCD 101j is photoelectrically converted at a predetermined frame rate, and is output from the first CCD 101g and the second CCD 101j as an analog image signal. The detailed operation of the photographing optical system 101 including the first CCD 101g and the second CCD 101j will be described later, but the photographing optical system 101 has different distances in focus with respect to the same subject by photographing. Two photographed image data representing two photographed images are generated.

  The CPU 115 controls the entire digital camera 1. Specifically, the CPU 115 has a built-in ROM, and the operation of the entire digital camera 1 is controlled in accordance with the program procedure of the built-in ROM.

  Further, the digital camera 1 is provided with a first CDSAMP 116 and a second CDSAMP 117 for performing a process for reducing noise of analog image signals output from the first CCD 101g and the second CCD 101j, and the processes. The first A / D conversion circuit 118, the second A / D conversion circuit 119, and the first A / D conversion circuit 118, the second A / D conversion circuit 119 for analog / digital conversion of the analog image signal thus converted into a digital image signal. An image input controller 121 is provided for transferring image data composed of RGB converted into a digital image signal by the A / D conversion circuit 119 to the SDRAM 120 as a memory via a data bus.

  In addition, the digital camera 1 reads out image data from the SDRAM 120 and executes an image processing process 122 for executing an image processing process for conversion into a YC signal, and executes a JPEG process when the image processing process is completed. A compression processing circuit 123 for compressing the image data, and a Video / LCD encoder 124 for converting the image data into a video signal and guiding it to the image display unit 13. The compression processing circuit 123 compresses image data with reference to information from a compression rate processing unit 131 described later.

  Further, in the digital camera 1, the AF detection circuit 125 that detects the focus information of the image, the AE & AWB detection unit 126 that detects the luminance information and the white balance information of the image, and the memory used for the work area described above. An SDRAM 120, a VRAM 127 as a display buffer having an A area and a B area, a media controller 128 that performs control for storing image data in the recording medium 110, and a photometric sensor 20 are provided.

  Further, the digital camera 1 includes a flash light emitting unit 129, the power button 11, the release button 12, the shooting mode dial 12_1, the playback button 14, the F button 15, the cross key 16, the OK / menu button 17, and the zoom switch 18 described above. , DSP / BACK buttons 19 are provided with an operation switch group 100.

  Furthermore, in the digital camera 1, the brightness of each pixel in each photographed image of two photographed images represented by the two photographed image data generated by the photographing optical system 101 is higher as the degree of focusing of each pixel is higher. A luminance ratio processing unit 130 (corresponding to an example of the image processing unit in the present invention) that generates two luminance adjustment image data representing two luminance adjustment images adjusted to the high luminance side is provided.

  Here, the image display unit 13 has two light-transmitting display screens arranged in an overlapping manner, and displays two brightness adjustment images one by one on these two display screens. This is a three-dimensional display device capable of displaying a moving image without using the.

  The digital camera 1 also includes information for generating two pieces of compressed image data representing two compressed images obtained by compressing each photographed image at a higher compression ratio in a region where the degree of focusing of each photographed image is lower. The compression ratio processing unit 131 provided to the compression processing circuit 123 is provided. The compression processing circuit 123 refers to this information and compresses the image data. Here, the compression rate processing unit 131 and the compression processing circuit 123 correspond to an example of the image compression unit according to the present invention.

  Next, a schematic operation of the digital camera 1 will be described.

  The operation of the digital camera 1 is comprehensively controlled by the CPU 115. The CPU 115 has a built-in ROM, and a program is stored in the built-in ROM. The operation of the entire digital camera 1 is controlled by the CPU 115 in accordance with the procedure of this program. Further, the SDRAM 120 temporarily stores image data being processed when the CPU 115 controls the entire digital camera 1 according to the procedure of the program described in the ROM.

  Further, the VRAM 127 as a display buffer stores image data representing a through image in order in the A area and the B area, and the image data in these areas are alternately supplied to the Video / LCD encoder 124 based on the image data. The images are sequentially switched and displayed on the image display unit 13 as a through image. Here, the through image displayed on the image display unit 13 is a three-dimensional moving image that can be observed without using glasses, as will be described in detail later.

  Here, the flow of image data when shooting is performed with the shooting mode dial 12_1 (see FIG. 1) switched to AUTO will be described with reference to FIG.

  When the photographing mode dial 12_1 is switched to AUTO, the CPU 115 captures the subject light that has passed through the photographing optical system 101 with the first CCD 101g and the second CCD 101j and generates them with the first CCD 101g and the second CCD 101j. Image data is output to the first CDSAMP 116 and the second CDSAMP 117 at predetermined intervals, and the first A / D converter 118, the second A / D converter 119, the image input controller 121, and the luminance ratio processing The through image is displayed on the screen of the image display unit 13 after being converted into image data that can be displayed by the unit 130 and the image signal processing circuit 122. The framing is arbitrarily performed based on the through image, and the AF detection circuit 125 continuously moves the first focus lens 101c and the second focus lens 101h back and forth to perform in-focus detection as will be described later. The existing image is formed on the first CCD 101g and the second CCD 101j. At this time, the field brightness is detected by the AE & AWB detection unit 126, and the gains of the R, G, and B color signals are adjusted for white balance adjustment based on the detection result. In this way, image data representing a focused image whose exposure is adjusted according to the field luminance is sent to the first CDSAMP 116 and the second CDSAMP 117 at predetermined intervals according to the timing signal from the timing generator 114. A through image is obtained at a later stage. The photographer performs framing while viewing the through image, and performs a release operation at a photo opportunity.

  When the release operation is performed, the CPU 115 supplies a timing signal from the timing generator 114 in order to form an image at the time of the release operation on the first CCD 101g and the second CCD 101j. This timing signal informs the first CCD 101g and the second CCD 101j of the start and end of exposure and corresponds to a so-called shutter speed. At the end of the exposure, the CPU 115 outputs image data (image data composed of the three primary colors R, G, and B of RGB light) from the first CCD 101g and the second CCD 101j, and the first CDSAMP 116 and the second CDSAMP 117 in the subsequent stage. To supply. Further, the first CDSAMP 116 and the second CDSAMP 117 reduce the noise of the image data output from the first CCD 101g and the second CCD 101j, and the image data with the reduced noise is subjected to the first A / D conversion. Unit 118 and second A / D converter 119 convert the digital signal. The RGB image data converted into digital signals by the first A / D conversion unit 118 and the second A / D conversion unit 119 is supplied to the SDRAM 120 via the data bus by the image input controller 121 in the subsequent stage. The SDRAM 120 stores the image data of all the pixels of the first CCD 101g and the second CCD 101j. When the image data corresponding to all the pixel data is stored in the SDRAM 120, the CPU 115 activates the image processing process of the luminance ratio processing unit 130 and the image signal processing circuit 122 described later, and the image data of the SDRAM 120 is converted by the image processing process. It is read out and converted into a YC signal. Further, when the CPU 115 detects that the image processing process of the image signal processing unit 122 is completed, the CPU 115 activates a JPEG process of the compression rate processing unit 131 and the compression processing circuit 123 described later, and compresses the image data by the JPEG process. To do. When the CPU 115 detects that the compression by the compression rate processing unit 131 and the compression processing circuit 123 is completed, the CPU 115 starts a recording processing process and records JPEG-compressed image data via the media controller 128. It is recorded on the medium 110. In this way, the image data is processed.

  FIG. 3 is a view for explaining a state in which the front and rear two-dimensional images are simultaneously generated by the photographing optical system shown in FIG.

  In order to supply image data to the image display unit 13 which is a three-dimensional display device capable of displaying a moving image without using glasses, two display surfaces, a display surface close to a distant display surface when a three-dimensional object is viewed from an observer, are displayed. It is necessary to generate a two-dimensional image (2D image) that can be projected on a surface. The digital camera 1 of the present embodiment is a photographing device that simultaneously captures an image on a display surface (rear) that is far from the observer and an image on a display surface (front) that is closer to the observer.

  One of the features of the present embodiment is, as shown in FIG. 3, two optical systems in the first CCD 101g and the second CCD 101j, which have different focal lengths and the same optical axis and optical path length. It is in the optical system 101. In addition, one of the effects of the present embodiment is that the optical axis and the optical path length are the same, and a digital camera having the first CCD 101g and the second CCD 101j that are two imaging elements is provided with a focusing lens (second adjustment lens). 2D images necessary for three-dimensional image display (3D display) can be generated simply by inserting the focus lens 101i).

  Here, the subject A and the subject B are separated by a distance L, the subject A is located behind the subject B, and the subject A and the subject B exist so as not to overlap each other in the direction of the digital camera 1. Will be described as an example.

  The subject B is positioned in front of the subject A, and when the subject B is focused with the focus adjustment lens (first focus lens 101c), an image at the focus position of the first focus lens 101c is obtained. About half of the light is transmitted through the half mirror 101d after passing through the aperture for adjusting the amount of light (iris 101a), the lens for adjusting the magnification (zoom lens 101b), and the first focus lens 101c, and two reflecting mirrors (first The first reflecting mirror 101e and the second reflecting mirror 101f) are incident on the first CCD 101g.

  The subject A is located behind the subject B, and when the second focus lens 101i focuses on the subject A, the image at the in-focus position of the second focus lens 101i is an iris 101a and a zoom lens 101b. , Approximately half of the light is reflected by the half mirror 101d after passing through the first focus lens 101c, passes through the second focus lens 101i via the third reflecting mirror 101h, and passes through the second CCD 101j. Is incident on.

  The captured image of the first CCD 101g is focused on the subject B at the front position and is not focused on the subject A at the rear position. The captured image of the second CCD 101j is focused on the subject A at the rear position and is not focused on the subject B at the front position.

  As described above, when the photographing optical system 101 of the digital camera 1 is used, a 2D image of the in-focus position (front) of the first focus lens 101c and the in-focus position (rear) of the second focus lens 101i is obtained. The optical axis and the optical path length are the same, and only the focusing adjustment lens (second focus lens 101i) is inserted into a digital camera having two image sensors (first CCD 101g and second CCD 101j). A 2D image necessary for 3D display can be generated.

  FIG. 4 is a diagram for explaining a state in which two-dimensional images of the front surface and the rear surface are simultaneously generated by a photographing optical system different from the photographing optical systems shown in FIGS.

  4 shows a photographic optical system 201 different from the photographic optical system 101 shown in FIGS. The photographing optical system 201 is different from the photographing optical system 101 shown in FIGS. 2 and 3 in that the first reflecting mirror 101e and the second reflecting mirror 101f are deleted.

  The feature here is the two optical systems in the first CCD 101g and the second CCD 101j, which are in the photographing optical system 201 having different focal lengths, the same optical axis, and different optical path lengths. In addition, the effect here is that two reflecting mirrors (the first reflecting mirror 101e and the second reflecting mirror 101f) are unnecessary as compared with the digital camera 1 described above, and therefore a reduction in the amount of light can be prevented. At the same time, it is possible to reduce the cost by reducing the number of parts and adjustment man-hours. Further, similarly to the digital camera 1 described above, a 2D image necessary for 3D display can be generated.

  Here, subject A and subject B are separated by a distance L, subject A is located behind subject B, and subject A and subject B exist so as not to overlap each other in the direction of the digital camera. An example will be described.

  The subject B is positioned in front of the subject A, and when the subject B is focused with the focus adjustment lens (first focus lens 101c), an image at the focus position of the first focus lens 101c is obtained. About half of the light is transmitted through the half mirror 101d through the light amount adjusting diaphragm (iris 101a), the magnification adjusting lens (zoom lens 101b), and the first focus lens 101c, and is incident on the first CCD 101g. .

  The subject A is located behind the subject B, and when the second focus lens 101i focuses on the subject A, the image at the in-focus position of the second focus lens 101i is an iris 101a and a zoom lens 101b. , Approximately half of the light is reflected by the half mirror 101d after passing through the first focus lens 101c, passes through the second focus lens 101i via the third reflecting mirror 101h, and passes through the second focus lens 101i. Incident on the CCD 101j.

  The captured image of the first CCD 101g is focused on the subject B at the front position and is not focused on the subject A at the rear position. The captured image of the second CCD 101j is focused on the subject A at the rear position and is not focused on the subject B at the front position.

  As described above, when the photographing optical system 201 is used, a 2D image of the in-focus position (front) of the first focus lens 101c and the in-focus position (rear) of the second focus lens 101i is displayed on the optical axis. Necessary for 3D display by inserting a focusing lens (second focus lens 101i) into a digital camera having the same optical path length but different two image sensors (first CCD 101g, second CCD 101j) A 2D image can be generated. A three-dimensional image may be displayed using such a photographing optical system 201.

  5 is a diagram showing a signal processing flow in the luminance ratio processing unit shown in FIG. 2, and FIG. 6 is a diagram showing signal waveforms in the luminance ratio processing unit shown in FIG.

  As described above, the digital camera 1 including the photographic optical system 101 (or the digital camera including the photographic optical system 201) is the two optical systems of the first CCD 101g and the second CCD 101j, and has a focal length. Differently, the imaging optical systems 101 and 201 having the same optical axis are characterized. In addition, a 2D image necessary for 3D display is provided as a 2D image of the front surface and the rear surface captured on a single optical axis.

  Further, in the luminance ratio processing unit 130, the 2D images of the front surface and the rear surface captured on the single optical axis obtained by the photographing optical system 101 (or the photographing optical system 201) are used in the depth direction viewed from the observer. A signal processing method is provided that enables sequential information to be sequentially calculated.

  Here, when there is a subject near the front, the focus ratio at the front is high and the focus ratio at the rear is low, so the brightness signal at the front is multiplied by the focus ratio at the front to obtain the brightness signal at the rear. Is multiplied by the rear focus ratio to provide a luminance signal for 3D display. This effect is that information in the depth direction is sequentially calculated using the focus ratio, and a luminance signal for 3D display in which the depth appears to change continuously is obtained.

  Next, with reference to FIGS. 5 and 6, the flow of signal processing and the waveform of each part in the luminance ratio processing unit 130 will be described. The image luminance signal Ya captured at the in-focus position on the front surface is input to an HPF (High Pass Filter) provided in the luminance ratio processing unit 130, and the high frequency component a of the spatial frequency is extracted by the HPF, and further An absolute value A of the high frequency component a is obtained. In addition, the image luminance signal Yb captured at the in-focus position on the rear surface is input to an HPF different from the HPF provided in the luminance ratio processing unit 130, and the high frequency component b of the spatial frequency is extracted by the HPF. The absolute value B of the high frequency component b is obtained.

  In each part waveform shown in FIG. 6, a focused part is indicated by a thick line. The front image is in focus (A> B) with the front subject, and the rear subject is out of focus (A <B). Further, the rear surface image is focused on the rear surface subject (A <B), and the front surface subject is out of focus (A> B). The front-side focusing ratio A / (A + B) is multiplied by the image luminance signal Ya imaged at the in-focus position on the front surface, and the front LCD screen (first light transmittance of the image display unit 13). Image luminance signal Ya ′ (Ya ′ = Ya × A / (A + B)) (first luminance adjustment image data) to be displayed on the display screen). This image luminance signal Ya 'is a signal indicating that the in-focus portion near the front surface increases and the out-of-focus portion near the rear surface decreases. Further, the rear-side LCD screen (second light of the image display unit 13) is obtained by multiplying the focusing ratio B / <A + B) on the rear surface by the image luminance signal Yb captured at the focusing position on the rear surface. An image luminance signal Yb ′ (Yb ′ = Yb × B / (A + B)) (second luminance adjustment image data) to be displayed on the transmissive display screen). The image luminance signal Yb 'is a signal indicating that the in-focus portion near the rear surface increases and the non-focus portion near the front surface decreases. Note that the image luminance signal Ya ′ shown in FIG. 6 is generated by performing a process in which portions where the absolute value A of the high-frequency component a is large (an edge in the image and another edge) are connected. Yes.

  As described above, the luminance signal of the image projected to the front side and the image projected to the rear side can be varied according to the depth position of the subject. When these images are supplied to the image display unit 13, they are displayed as a stereoscopic subject.

  FIG. 7 is a diagram showing a flow of signal processing in the compression rate processing unit shown in FIG.

  The compression rate processing unit 131 performs processing for optimizing the compression rate when generating image data for recording from the 2D images of the front and rear surfaces imaged on a single optical axis. That is, the compression rate processing unit 131 performs image data compression rate according to the in-focus ratio (A / (A + B), B / <A + B) between the front surface and the rear surface calculated by the luminance ratio processing unit 130 described above. Adjust. Specifically, the reference compression rate Z is multiplied by the reciprocal of the focus ratio A / (A + B) on the front surface, and the front LCD screen (first light transmissive display screen of the image display unit 13) is displayed. It is assumed that the compression rate Za ′ (Za ′ = Z × (A + B) / A) for storing image data to be displayed. Further, the reference compression rate Z is multiplied by the reciprocal of the focusing ratio B / <A + B) on the rear surface, and displayed on the rear LCD screen (second light-transmitting display screen of the image display unit 13). The compression rate Zb ′ for storing image data (Zb ′ = Z × (A + B) / B) is used. That is, for each pixel (block), when the front focus ratio is higher than the rear surface, the compression ratio of the front image data is decreased and the compression ratio of the rear image data is increased. As described above, since the compression rate of each pixel (block) on the front surface and the rear surface is adjusted according to the focus ratio, the image data to be displayed in 3D can be efficiently compressed. In other words, the compression rate of image data projected onto a display surface with a high focus ratio is reduced, so sharper recording can be performed, and the compression rate of image data projected onto a display surface with a low focus ratio is increased, so the amount of data to be saved Can be further reduced. As a more desirable mode, it is preferable to adjust the image data compression rate so that the sum of the amount of stored data on the front side and the amount of stored data on the rear side becomes substantially equal. This means that the quality of the image data on the front surface and that on the rear surface are almost equal, and the quality in the depth direction of the 3D image projected on the image display unit 13 is almost equal, so that it can be displayed as a natural image.

  In the above-described embodiment, the digital camera is described as an example of the photographing apparatus of the present invention. However, the present invention is not limited to this. It may be a video camera or the like for shooting.

  Next, an embodiment of the image processing apparatus of the present invention will be described.

  FIG. 8 is a schematic diagram showing an information processing apparatus in which an embodiment of the image processing apparatus of the present invention is realized.

  In FIG. 8, a computer 200 generally called a workstation or a personal computer is shown as the information processing apparatus. The computer 200 implements an embodiment of the image processing apparatus of the present invention. Note that the image processing apparatus according to an embodiment is implemented by loading an image processing program stored in a CD-ROM 300, which will be described later, into the computer 200.

  First, the hardware configuration of the computer 200 will be described.

  The computer 200 includes a main body 201 in which a CPU (Central Processing Unit), a RAM (Random Access Memory), a hard disk, a communication board, and the like are built, and an image or character string on the display screen 202a according to an instruction from the main body 201 By designating an arbitrary position on the display screen 202a, an image display unit (liquid crystal monitor) 202 for displaying the image, a keyboard 203 for inputting a user instruction to the computer 200, and displayed at that position at the time of the designation A mouse 204 is provided for inputting an instruction corresponding to the icon or the like. Here, the image display unit 202 has two light transmissive display screens arranged in an overlapping manner, and an image display for displaying two luminance adjustment images described later on the two display screens one by one. Part.

  The main body 201 further has a flexible disk (not shown), a flexible disk loading port 201a into which the CD-ROM 200 is loaded, and a CD-ROM loading port 201b. A flexible disk and a CD-ROM drive that are accessed by driving a flexible disk loaded from the loading ports 201a and 201b and the CD-ROM 300 are also incorporated.

  FIG. 9 is a hardware configuration diagram of a computer having the appearance shown in FIG.

  The hardware configuration diagram of FIG. 9 shows a CPU 211, RAM 212, hard disk controller 213, flexible disk drive 214, CD-ROM drive 215, mouse controller 216, keyboard controller 217, display controller 218, and communication board 219. They are connected to each other by a bus 210.

  As described with reference to FIG. 8, the flexible disk drive 214 and the CD-ROM drive 215 access the flexible disk 400 and the CD-ROM 300 loaded from the flexible disk loading slot 201a and the CD-ROM loading slot 201b, respectively. Is. The communication board 219 is connected to the communication line 500.

  9 also includes a hard disk 220 accessed by the hard disk controller 213, a mouse 204 controlled by the mouse controller 216, a keyboard 203 controlled by the keyboard controller 217, and an image display unit 202 controlled by the display controller 218. It is shown.

  FIG. 10 is a conceptual diagram showing a CD-ROM storing an image processing program that causes the computer, which is the information processing apparatus shown in FIG. 8, to operate as an image processing apparatus.

  The CD-ROM 300 shown in FIG. 10 stores an image processing program 310, and the image processing program 310 includes an image acquisition routine unit 311 and an image processing routine unit 312. Details of each part of the image processing program 310 will be described together with the operation of each part of the embodiment of the image processing apparatus of the present invention.

  FIG. 11 is a diagram showing a configuration of an embodiment realized by an image processing program stored in a CD-ROM of the image processing apparatus of the present invention.

  An image processing apparatus 200_1 illustrated in FIG. 11 includes an image acquisition unit 200_11 and an image processing unit 200_12 that are realized by loading an image processing program stored in the CD-ROM 300 into the computer 200, and the image display unit 202 described above. Consists of.

  The image acquisition unit 200_11 operates under the action of the program of the image acquisition routine unit 311 shown in FIG. 10, and the same subject from the digital camera or the like via the communication line 500 and the communication board 219 shown in FIG. A plurality of photographed image data representing a plurality of photographed images having different in-focus distances are acquired.

  The image processing unit 200_12 operates in response to the program of the image processing routine unit 312 shown in FIG. 10, and the brightness of each pixel in each captured image is increased as the degree of focusing of each pixel increases. Two brightness adjustment image data representing the two adjusted brightness adjustment images are generated. Also, two compressed image data representing two compressed images obtained by compressing each photographed image at a higher compression ratio in a region where the degree of focusing of each photographed image is lower is generated, and the hard disk is transmitted via the hard disk controller 213. Save to 220.

  As described above, the image display unit 202 has two light-transmitting display screens arranged in an overlapping manner, and displays the two brightness adjustment images one by one on the two display screens.

  The image processing apparatus 200_1 of the present embodiment acquires captured image data for displaying a three-dimensional image by the image acquisition unit 200_11, and determines the luminance of each pixel in each captured image represented by the acquired captured image data. The image processing unit 200_12 generates two brightness adjustment image data representing two brightness adjustment images adjusted to the higher brightness side as the degree of focusing of each pixel is higher. For this reason, when the front image is in focus when viewed from the observer, the luminance of each pixel of the front image is adjusted to the high luminance side and the luminance of each pixel of the rear image is low. Two brightness adjustment image data representing the two brightness adjustment images adjusted to the side are generated. Further, the two brightness adjustment images are displayed one by one on the two light transmissive display screens of the image display unit 202 provided in the image processing apparatus 200_1. In this way, a three-dimensional image is displayed on the image display unit 202.

  Further, in the image processing apparatus 200_1 of the present embodiment, a region where the degree of focusing of the captured image is low is compressed at a high compression rate, and a region where the degree of focusing of the captured image is high is compressed at a low compression rate. Therefore, image data for displaying a three-dimensional image can be efficiently compressed.

1 is a top view of a digital camera that is an embodiment of both the first photographing apparatus of the present invention and the second photographing apparatus of the present invention when viewed from above, a front view when viewed from the front, and a rear view when viewed from the back. . It is a figure which shows the internal structure of the digital camera shown in FIG. It is a figure for demonstrating a mode that the two-dimensional image of a front surface and a rear surface is produced | generated simultaneously with the imaging | photography optical system shown in FIG. It is a figure for demonstrating a mode that the two-dimensional image of a front surface and a back surface is produced | generated simultaneously by the imaging optical system different from the imaging optical system shown in FIG. It is a figure which shows the flow of the signal processing in the luminance ratio process part shown in FIG. It is a figure which shows the signal waveform in the luminance ratio process part shown in FIG. It is a figure which shows the flow of the signal processing in the compression rate process part shown in FIG. 1 is a schematic diagram illustrating an information processing apparatus in which an embodiment of an image processing apparatus of the present invention is realized. It is a hardware block diagram of the computer which has the external appearance shown in FIG. FIG. 9 is a conceptual diagram illustrating a CD-ROM storing an image processing program that causes the computer, which is the information processing apparatus illustrated in FIG. 8, to operate as an image processing apparatus. It is a figure which shows the structure of one Embodiment implement | achieved by the image processing program memorize | stored in CD-ROM of the image processing apparatus of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Digital camera 1a Camera body 10 Lens barrel 11 Power button 12 Release button 12_1 Shooting mode dial 13 Image display part 14 Playback button 15 F button 16 Four-way controller 17 OK / menu button 18 Zoom switch 19 DSP / BACK button 20 Photometric sensor 21 Flash emission window 100 Operation unit 101, 201 Imaging optical system 101a Iris 101b Zoom lens 101c, 101i Focus lens 101d Half mirror 101e, 101f, 101h Reflective mirror 101g, 101j CCD
110 Recording media 111, 112, 113 Motor driver 114 Timing generator 115, 211 CPU
116,117 CDSAMP
118,119 A / D conversion circuit 120 memory (SDRAM)
121 Image Input Controller 122 Image Signal Processing Circuit 123 Compression Processing Circuit 124 Video Encoder 125 AF Detection Circuit 126 AE & AWB Detection Circuit 127 VRAM
128 Media Controller 129 Flash Light Emitting Unit 130 Brightness Ratio Processing Unit 131 Compression Ratio Processing Unit 200 Computer 200_1 Image Processing Device 200_11 Image Acquisition Unit 200_12 Image Processing Unit 201 Main Body Unit 201a Flexible Disk Loading Port 201b CD-ROM Loading Port 202 Image Display Unit 202a Display screen 203 Keyboard 204 Mouse 210 Bus 211 CPU
212 RAM
213 Hard disk controller 214 Flexible disk drive 215 CD-ROM drive 216 Mouse controller 217 Keyboard controller 218 Display controller 219 Communication board 220 Hard disk 300 CD-ROM
310 Image processing program 311 Image acquisition routine unit 312 Image processing routine unit 400 Flexible disk 500 Communication line

Claims (8)

Obtaining a plurality of photographed image data representing a plurality of photographed images having different in-focus distances for the same subject;
Generating a plurality of brightness adjustment image data representing a plurality of brightness adjustment images in which the brightness of each pixel in each captured image is adjusted to the higher brightness side as the degree of focusing of each pixel is higher;
An image processing method, wherein the plurality of brightness adjustment images are displayed one by one on a plurality of light transmissive display screens arranged in an overlapping manner.   Generating a plurality of compressed image data representing a plurality of compressed images obtained by compressing each captured image at a higher compression ratio in a region where the degree of focusing of each captured image is lower, and storing the plurality of compressed image data. The image processing method according to claim 1, wherein: A photographing unit that generates a plurality of photographed image data representing a plurality of photographed images with different distances in focus with respect to the same subject by photographing;
An image processing unit that generates a plurality of brightness adjustment image data representing a plurality of brightness adjustment images adjusted to a higher brightness side as the degree of focusing of each pixel in the captured image is higher. An imaging apparatus characterized by that.   An image display unit having a plurality of light transmissive display screens arranged in an overlapping manner and displaying the plurality of brightness adjustment images one by one on the plurality of display screens. 3. The photographing apparatus according to 3. A photographing unit that generates a plurality of photographed image data representing a plurality of photographed images with different distances in focus with respect to the same subject by photographing;
An image compression unit that generates a plurality of compressed image data representing a plurality of compressed images in which each captured image is compressed at a higher compression ratio in a region where the degree of focusing of each captured image is lower. Shooting device. An image acquisition unit for acquiring a plurality of photographed image data representing a plurality of photographed images having different in-focus distances for the same subject;
An image processing unit that generates a plurality of brightness adjustment image data representing a plurality of brightness adjustment images adjusted to a higher brightness side as the degree of focusing of each pixel in the captured image is higher. An image processing apparatus.   An image display unit having a plurality of light transmissive display screens arranged in an overlapping manner and displaying the plurality of brightness adjustment images one by one on the plurality of display screens. 6. The image processing apparatus according to 6.   7. The image processing according to claim 6, wherein a plurality of compressed image data representing a plurality of compressed images obtained by compressing each captured image with a higher compression ratio in a region where the degree of focusing of each captured image is lower is generated. apparatus.

Images