WO2018124054A1 - Imaging device and method for controlling same - Google Patents

Abstract

An imaging device (100) is provided with an imaging element (101) capable of nondestructive readout and a control unit (102) for controlling the imaging of a subject frame using an auxiliary image obtained by nondestructive readout in the subject frame. The control unit (102) may, for example, control the exposure time of the subject frame using the auxiliary image. The control unit (102) may, for example, successively perform nondestructive readout in the subject frame multiple times and stop exposure for the subject frame when the luminance of the auxiliary image exceeds a predetermined threshold.

Description

Imaging apparatus and control method thereof

The present disclosure relates to an imaging apparatus and a control method thereof.

A technique described in Patent Document 1 is known as an image sensor using an organic photoelectric conversion element.

JP 2008-042180 A

Such an image pickup apparatus is desired to be further improved.

Therefore, an object of the present disclosure is to provide an imaging apparatus or a control method thereof that can realize further improvement.

An imaging apparatus according to an aspect of the present disclosure includes an imaging element capable of nondestructive readout, and a control unit that controls imaging of the target frame using an auxiliary image obtained by nondestructive readout in the target frame. .

The present disclosure can provide an imaging apparatus or a control method thereof that can realize further improvement.

FIG. 1 is a block diagram of the imaging apparatus according to the first embodiment. FIG. 2A is a diagram illustrating an appearance example of the imaging apparatus according to Embodiment 1. FIG. 2B is a diagram illustrating an appearance example of the imaging apparatus according to Embodiment 1. FIG. 3 is a diagram illustrating a configuration of the image sensor according to the first embodiment. FIG. 4 is a circuit diagram illustrating a configuration of a pixel according to Embodiment 1. FIG. 5 is a flowchart illustrating the operation of the imaging apparatus according to the first embodiment. FIG. 6 is a diagram illustrating the operation of the imaging apparatus according to the first embodiment. FIG. 7 is a flowchart illustrating the operation of the imaging apparatus according to the second embodiment. FIG. 8 is a diagram illustrating the operation of the imaging apparatus according to the second embodiment. FIG. 9 is a diagram illustrating an example of an auxiliary image according to the second embodiment. FIG. 10 is a diagram illustrating an example of a threshold setting screen according to the second embodiment. FIG. 11 is a flowchart illustrating the operation of the imaging apparatus according to the third embodiment. FIG. 12 is a diagram for explaining subject change determination processing according to the third embodiment. FIG. 13 is a flowchart illustrating a modified example of the operation of the imaging apparatus according to the third embodiment. FIG. 14 is a flowchart illustrating a modified example of the operation of the imaging apparatus according to the third embodiment. FIG. 15 is a flowchart illustrating a modified example of the operation of the imaging apparatus according to the third embodiment. FIG. 16 is a flowchart illustrating the operation of the imaging apparatus according to the fourth embodiment. FIG. 17 is a diagram illustrating the operation of the imaging apparatus according to the fourth embodiment. FIG. 18 is a diagram illustrating an example of an image selection screen according to the fourth embodiment. FIG. 19 is a diagram illustrating the operation of the imaging apparatus according to the modification of the fourth embodiment.

Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. Each of the embodiments described below shows a preferred specific example of the present disclosure. Therefore, the numerical values, shapes, materials, components, component arrangement positions, connection forms, and the like shown in the following embodiments are merely examples, and are not intended to limit the present disclosure. Therefore, among the constituent elements in the following embodiments, constituent elements that are not described in the independent claims indicating the highest concept of the present disclosure are described as arbitrary constituent elements.

Each figure is a schematic diagram and is not necessarily shown strictly. Moreover, in each figure, the same code | symbol is attached | subjected to the substantially same structure, The overlapping description is abbreviate | omitted or simplified.

(Embodiment 1)
[Configuration of imaging device]
First, the configuration of the imaging device according to the embodiment of the present disclosure will be described. FIG. 1 is a block diagram illustrating a configuration of an imaging apparatus 100 according to the present embodiment. 2A and 2B are diagrams illustrating an example of the appearance of the imaging apparatus 100. FIG. For example, as illustrated in FIGS. 2A and 2B, the imaging apparatus 100 is a camera such as a digital still camera or a digital video camera.

1 includes an imaging element 101, a control unit 102, a display unit 103, a storage unit 104, an imaging system 106, and a light shielding unit 107. The imaging device 101 is a solid-state imaging device (solid-state imaging device) that converts incident light into an electrical signal (image) and outputs the obtained electrical signal. For example, the imaging device 101 is an organic sensor using an organic photoelectric conversion device.

The control unit 102 controls the image sensor 101. In addition, the control unit 102 performs various signal processing on the image obtained by the imaging element 101, and displays the obtained image on the display unit 103 or stores it in the storage unit 104. Note that the image output from the control unit 102 may be output to the outside of the imaging apparatus 100 via an input / output interface (not shown).

The control unit 102 is a circuit that performs information processing and is a circuit that can access the storage unit 104. For example, the control unit 102 is realized by a processor such as a DSP (Digital Signal Processor) or a GPU (Graphics Processing Unit). Note that the control unit 102 may be a dedicated or general-purpose electronic circuit. The control unit 102 may be an aggregate of a plurality of electronic circuits.

The display unit 103 displays an image obtained by the image sensor 101, a user interface, and the like. For example, the display unit 103 is a display panel provided on the back of the camera, and is a liquid crystal panel, an organic EL (electroluminescence) panel, or the like. The display unit 103 may be an electronic viewfinder (EVF) provided on the upper part of the camera. In this case, the image of the liquid crystal panel or the organic EL panel is projected through the lens. Further, the display unit 103 may be an external monitor connected to the imaging apparatus 100 via an interface such as HDMI (registered trademark), USB (registered trademark), or Wi-Fi (registered trademark).

The storage unit 104 is a general purpose or dedicated memory in which information is stored. For example, the storage unit 104 may be a magnetic disk or an optical disk, or may be expressed as a storage or a recording medium. The storage unit 104 may be a non-volatile memory or a volatile memory. In addition, the storage unit 104 is not limited to a memory built in the imaging apparatus 100, and may be a memory attached to the imaging apparatus 100. For example, the storage unit 104 may be an SD card or the like. The storage unit 104 may be a combination of these plural types of memories.

The imaging system 106 includes, for example, one or a plurality of lenses, and condenses light from the outside of the imaging apparatus 100 on the imaging element 101.

The light shielding unit 107 is, for example, a mechanical shutter, and shields light from the imaging system 106.

[Configuration of image sensor]
Next, the configuration of the image sensor 101 will be described. FIG. 3 is a block diagram illustrating a configuration of the image sensor 101.

3 includes a plurality of pixels (unit pixel cells) 201 arranged in a matrix, a vertical scanning unit 202, a column signal processing unit 203, a horizontal readout unit 204, and a row. A plurality of reset control lines 205, a plurality of address control lines 206 provided for each row, a plurality of vertical signal lines 207 provided for each column, and a horizontal output terminal 208.

Each of the plurality of pixels 201 outputs a signal corresponding to the incident light to the vertical signal line 207 provided in the corresponding column.

The vertical scanning unit 202 resets the plurality of pixels 201 via the plurality of reset control lines 205. The vertical scanning unit 202 sequentially selects the plurality of pixels 201 in units of rows via the plurality of address control lines 206.

The column signal processing unit 203 performs signal processing on the signals output to the plurality of vertical signal lines 207, and outputs the plurality of signals obtained by the signal processing to the horizontal reading unit 204. For example, the column signal processing unit 203 performs noise suppression signal processing represented by correlated double sampling, analog / digital conversion processing, and the like.

The horizontal readout unit 204 sequentially outputs a plurality of signals after the signal processing by the plurality of column signal processing units 203 to the horizontal output terminal 208.

Hereinafter, the configuration of the pixel 201 will be described. FIG. 4 is a circuit diagram illustrating a configuration of the pixel 201.

As shown in FIG. 4, the pixel 201 includes a photoelectric conversion unit 211, a charge storage unit 212, a reset transistor 213, an amplification transistor 214 (source follower transistor), and a selection transistor 215.

The photoelectric conversion unit 211 generates signal charges by photoelectrically converting incident light. A voltage Voe is applied to one end of the photoelectric conversion unit 211. Specifically, the photoelectric conversion unit 211 includes a photoelectric conversion layer made of an organic material. The photoelectric conversion layer may include a layer made of an organic material and a layer made of an inorganic material.

The charge storage unit 212 is connected to the photoelectric conversion unit 211 and stores the signal charge generated by the photoelectric conversion unit 211. Note that the charge storage unit 212 may be configured with a parasitic capacitance such as a wiring capacitance instead of a dedicated capacitance element.

The reset transistor 213 is used to reset the potential of the signal charge. The gate of the reset transistor 213 is connected to the reset control line 205, the source is connected to the charge storage unit 212, and the reset voltage Vreset is applied to the drain.

Note that the definitions of drain and source generally depend on circuit operation, and are often not specified from the element structure. In this embodiment, for convenience, one of the source and the drain is referred to as a source and the other of the source and the drain is referred to as a drain. However, in this embodiment, the drain may be replaced with the source and the source may be replaced with the drain.

The amplification transistor 214 amplifies the voltage of the charge storage unit 212 and outputs a signal corresponding to the voltage to the vertical signal line 207. The gate of the amplification transistor 214 is connected to the charge storage unit 212, and the power supply voltage Vdd or the ground voltage Vss is applied to the drain.

The selection transistor 215 is connected in series with the amplification transistor 214, and switches whether to output the signal amplified by the amplification transistor 214 to the vertical signal line 207. The selection transistor 215 has a gate connected to the address control line 206, a drain connected to the source of the amplification transistor 214, and a source connected to the vertical signal line 207.

Further, for example, the voltage Voe, the reset voltage Vreset, and the power supply voltage Vdd are voltages commonly used in all the pixels 201.

The characteristics of the organic sensor include non-destructive readout and electronic ND (Neutral Density) control. Non-destructive reading is a process of reading image data during an exposure period and continuing exposure. In conventional readout (hereinafter referred to as destructive readout), it is necessary to end exposure when performing readout. That is, only one image could be obtained by one exposure. On the other hand, by using nondestructive reading, it is possible to read the image data exposed up to that time during the exposure period and continue the exposure. Thereby, a plurality of images having different exposure times can be obtained by one exposure.

The electronic ND control is a process for electrically controlling the transmittance of the image sensor. Here, the transmittance means the proportion of light that is converted into an electrical signal in the incident light. That is, by setting the transmittance to 0%, it is possible to electrically shield the light. Specifically, the transmittance is controlled by controlling the voltage Voe shown in FIG. Thereby, exposure can be electrically terminated without using a mechanical shutter.

Note that the image sensor 101 includes a mechanical shutter (light shielding unit 107), and electronic ND control and light shielding by the mechanical shutter may be used together, or light shielding by the mechanical shutter may be used without using the electronic ND control.

In addition, here, an example in which the image sensor 101 is an organic sensor will be described. However, the image sensor 101 only needs to realize nondestructive reading or electronic ND control, and may be other than an organic sensor. That is, the photoelectric conversion layer included in the photoelectric conversion unit 211 may be made of an inorganic material. For example, the photoelectric conversion layer may be made of amorphous silicon or chalcopyrite semiconductor.

[Operation of imaging device]
Next, the operation of the imaging apparatus 100 according to the present embodiment will be described. The imaging apparatus 100 according to the present embodiment controls photographing of the target frame using an auxiliary image obtained by nondestructive readout in the target frame. Specifically, the imaging apparatus 100 performs automatic exposure (AE: Auto Exposure) of the target frame using the auxiliary image.

FIG. 5 is a flowchart showing an operation flow of the imaging apparatus 100. FIG. 6 is a diagram for explaining the operation of the imaging apparatus 100.

As illustrated in FIG. 6, the imaging device 100 captures a live image that is a moving image of a real-time subject and displays the live image on the display unit 103 in a period before time t <b> 1 at which still image capturing starts. is doing.

At time t1, for example, based on a user operation, the imaging apparatus 100 starts exposure for capturing a still image (S101). Next, the imaging apparatus 100 acquires one or a plurality of auxiliary images by performing non-destructive readout a predetermined number of times during the exposure period (S102). In FIG. 6, two nondestructive readings are performed, but the number of nondestructive readings may be arbitrary. Next, the imaging apparatus 100 performs AE control using the obtained auxiliary image (S103).

Here, conventionally, AE control has been performed using live images. Therefore, the imaging apparatus 100 can perform the AE control using the auxiliary image by the same method as the AE control using the live image. Specifically, the imaging apparatus 100 controls the exposure time T1 of the target frame using the auxiliary image. For example, the imaging apparatus 100 detects the movement of the subject based on the difference between the plurality of auxiliary images, and controls the exposure time T1 based on the obtained movement. For example, the imaging apparatus 100 sets the exposure time T1 to be shorter as the movement is larger. Alternatively, the imaging apparatus 100 controls the exposure time T1 based on the luminance level of one or more auxiliary images. For example, the imaging apparatus 100 sets the exposure time T1 longer as the luminance level is lower.

Next, at time t4 when the exposure time T1 set in step S103 has elapsed, the imaging apparatus 100 ends the exposure (S104). For example, the imaging apparatus 100 ends the exposure when the above-described electronic ND control or the mechanical shutter performs light shielding.

Next, the imaging apparatus 100 acquires an image by performing destructive reading (S105). In FIG. 6, nondestructive readout is performed immediately after the end of exposure, but it may not be immediately after.

As described above, the imaging apparatus 100 according to the present embodiment controls shooting of the target frame using the auxiliary image obtained by nondestructive reading in the target frame. Specifically, the imaging apparatus 100 performs automatic exposure of the target frame using the auxiliary image. Here, when performing automatic exposure using a live image, there is a time difference from when automatic exposure is performed to when a still image is shot. This may reduce the accuracy of automatic exposure. On the other hand, since the time difference can be reduced in the method of the present embodiment, the accuracy of automatic exposure can be improved.

Also, the method of the present embodiment is different from the conventional method in that the image used for AE control is changed from a live image to an auxiliary image obtained by nondestructive reading. Therefore, more accurate automatic exposure can be realized while diverting part of the conventional automatic exposure algorithm.

In addition, here, the case where a still image is shot has been described as an example, but a similar method may be used when shooting a moving image.

(Embodiment 2)
In the present embodiment, another method for controlling the exposure time of a target frame using an auxiliary image will be described.

FIG. 7 is a flowchart showing an operation flow of the imaging apparatus 100 according to the present embodiment. FIG. 8 is a diagram for explaining the operation of the imaging apparatus 100.

First, the imaging apparatus 100 determines a threshold value TH used for determination processing described later (S111). Details of this process will be described later.

Next, the imaging apparatus 100 starts exposure at time t1 (S112). At time t2 after the predetermined time has elapsed, the imaging device 100 acquires an auxiliary image by nondestructive reading (S113). Then, the imaging apparatus 100 determines whether the luminance of the obtained auxiliary image exceeds the threshold value TH set in step S111 (S114).

When the brightness of the auxiliary image does not exceed the threshold value TH (No in S114), the imaging apparatus 100 performs nondestructive readout again at time t3 after the time T2 has elapsed (S113), and the obtained auxiliary image It is determined whether the luminance exceeds a threshold value TH (S114). Then, the imaging apparatus 100 repeats these processes until the luminance of the auxiliary image exceeds the threshold value TH.

When the luminance of the auxiliary image exceeds the threshold value TH (Yes in S114: time t5), the imaging apparatus 100 ends the exposure by electronic ND control or mechanical shutter (S115), and acquires an image by destructive reading (S116: time). t6). Note that non-destructive reading may be performed instead of destructive reading.

FIG. 9 is a diagram illustrating an example of an auxiliary image and a luminance histogram obtained at each time. The horizontal axis of the luminance histogram shown in FIG. 9 indicates luminance, and the vertical axis indicates the frequency of each luminance. As shown in FIG. 9, the auxiliary images obtained at each time have different brightness levels because of different exposure times. In other words, the brightness level increases as the obtained time is later.

In the determination in step S114, the imaging apparatus 100 determines, for example, whether the maximum luminance value of the auxiliary image exceeds the threshold value TH. Note that the imaging apparatus 100 may compare the average value or median value of the luminance with the threshold value TH without being limited to the maximum luminance value.

In FIG. 8, the imaging apparatus 100 ends the exposure immediately after it is determined that the luminance of the auxiliary image exceeds the threshold value TH. However, the exposure may end after a predetermined time. In this case, the threshold value TH is set lower by a difference corresponding to the time.

In FIG. 8, the interval T2 at which nondestructive reading is performed is constant, but it may not be constant. For example, the imaging apparatus 100 may set an exposure time in advance by AE control or the like, and set a plurality of nondestructive readout timings so that the nondestructive readout interval at a time approaching the exposure time is shortened. Good. Alternatively, the imaging apparatus 100 may shorten the nondestructive reading interval as the luminance of the obtained auxiliary image approaches the threshold value TH. Further, the time from the end of exposure to the time when destructive readout is performed may be arbitrary.

In addition, here, the case where a still image is shot has been described as an example, but a similar method may be used when shooting a moving image.

As described above, the imaging apparatus 100 sequentially performs non-destructive reading a plurality of times in the target frame, and stops exposure of the target frame when the luminance of the auxiliary image exceeds a predetermined threshold value TH. Here, in the conventional general method, the exposure period is determined before the image is captured, and an image having a desired luminance level is not always obtained. On the other hand, by using the method of the present embodiment, it is possible to dynamically determine whether the brightness level of the image has reached a desired value during the exposure period, so that an image with the desired brightness level can be obtained with high accuracy. Can do.

Further, as a method for determining the threshold value TH in step S111, for example, the imaging apparatus 100 determines the threshold value TH according to the currently set shooting mode. Alternatively, the imaging apparatus 100 may display a user interface for setting the threshold value TH on the display unit 103 and determine the threshold value TH based on a user instruction via the user interface user. FIG. 10 is a diagram showing an example of this user interface.

For example, as shown in FIG. 10, a plurality of sample images having different luminance levels are displayed, and a threshold TH corresponding to the sample image is set by the user selecting one of the sample images. The sample image may be an image stored in advance or an auxiliary image obtained by nondestructive reading at the time of past shooting.

FIG. 10 shows an example in which the user selects the sample image, but the brightness level may be selected using an operation bar or the like for changing the brightness level. Alternatively, an operation of increasing or decreasing the luminance level with respect to the current luminance level may be used.

By providing such a user interface, the user can intuitively and easily perform a brightness level selection operation.

Note that the threshold value TH need not be variable, and a fixed value may be used as the threshold value TH.

(Embodiment 3)
In the present embodiment, another example of processing for controlling photographing of a target frame using an auxiliary image obtained by nondestructive reading in the target frame will be described. Specifically, the imaging apparatus 100 according to the present embodiment sequentially performs a plurality of nondestructive readings in a target frame, and uses a plurality of auxiliary images obtained by the plurality of nondestructive readings in the target frame. Changes in the subject are detected. For example, the imaging apparatus 100 detects a change in an undesired subject that occurs due to the occurrence of a disturbance to the camera or a tripod falling when a long night is exposed, such as a starlit night.

FIG. 11 is a flowchart showing an operation flow of the imaging apparatus 100 according to the present embodiment. FIG. 12 is a diagram illustrating an example of a change in luminance of the auxiliary image when a change in the subject occurs.

First, the imaging apparatus 100 starts exposure (S121). At time t1 after the predetermined time T3 has elapsed, the imaging device 100 acquires an auxiliary image by nondestructive reading (S122). Then, the imaging apparatus 100 determines whether the subject has changed using a plurality of auxiliary images obtained after the start of exposure (S123). If it is determined that there is no change in the subject (No in S123) and the exposure period has not ended (No in S125), the imaging apparatus 100 again performs nondestructive reading at time t2 after the time T3 has elapsed. (S122), it is determined again whether the subject has changed using the plurality of auxiliary images obtained so far (S123).

On the other hand, when it is determined that the subject has changed (Yes in S123), the imaging apparatus 100 issues a warning (S124). For example, the imaging apparatus 100 displays a message indicating that the subject has changed on the display unit 103. Note that the imaging apparatus 100 may notify the user that the subject has changed by a warning sound or a voice message.

Then, the imaging apparatus 100 repeats these series of processes until the exposure period ends (S125).

When the exposure period ends (Yes in S125), the imaging apparatus 100 ends the exposure and acquires an image by destructive reading (S126).

Hereinafter, a method for determining whether a subject has changed using a plurality of auxiliary images will be described. As shown in FIG. 12, the luminance of the auxiliary image increases linearly as the exposure time increases. That is, the luminance increases with a constant slope. On the other hand, as shown at time t0, when a change in the subject occurs, the luminance gradient changes. FIG. 12 shows a case where an instantaneous disturbance has occurred. It should be noted that the inclination of the luminance changes (increases or decreases) when the shooting scene changes due to the tripod falling or when an undesired subject enters the screen.

Specifically, for example, the imaging apparatus 100 detects a gradient of luminance using a plurality of auxiliary images obtained at a plurality of times. Then, the imaging apparatus 100 calculates the inclination based on the newly obtained luminance of the auxiliary image and the luminance of the auxiliary image obtained immediately before, and the difference between the obtained inclination and the past inclination is determined in advance. It is determined whether the threshold value is exceeded. The imaging apparatus 100 determines that the subject has changed when the difference exceeds the threshold value, and determines that the subject has not changed when the difference does not exceed the threshold value.

For example, in the example shown in FIG. 12, at the time before time t5, for example, the past inclination is calculated based on the luminance at time t3 and time t4. The difference between the slope calculated based on the luminance at time t4 and time t5 and the past slope exceeds the threshold. As a result, a warning is issued.

The imaging apparatus 100 performs the above determination using, for example, the average luminance of the auxiliary image. Note that the imaging apparatus 100 may use the maximum luminance, or may use the average luminance or the maximum luminance of a specific area of the auxiliary image. The imaging apparatus 100 may perform the above determination for each region including one or more pixels, and may determine that the subject has changed when it is determined that any region has a change.

As described above, the imaging apparatus 100 according to the present embodiment sequentially performs a plurality of nondestructive readings in a target frame, and uses a plurality of auxiliary images obtained by the plurality of nondestructive readings in the target frame. Detect changes in the subject. Thereby, for example, since a change in the subject during the exposure of the long exposure can be detected, it is possible to notify the user of the shooting failure during the long exposure at an early stage. Therefore, it is possible to prevent the exposure from being performed to the end even though the shooting has failed.

As shown in FIG. 13, when it is determined that there is a change in the subject (Yes in S123), the imaging apparatus 100 may stop shooting (S124A). As a result, the imaging apparatus 100 can automatically stop shooting, for example, when an abnormality occurs during long exposure, so that exposure is performed to the end even though shooting has failed. Can be prevented.

Alternatively, as shown in FIG. 14, when it is determined that there is a change in the subject (Yes in S123), the imaging apparatus 100 may perform re-imaging (S124B). Thereby, the imaging apparatus 100 can automatically perform re-imaging when an abnormality occurs during, for example, long exposure.

Alternatively, as illustrated in FIG. 15, when it is determined that there is a change in the subject, the imaging apparatus 100 performs correction for reducing the influence of the change in the subject on the image of the target frame obtained by shooting. May be. Specifically, when it is determined that there is a change in the subject (Yes in S123), the imaging apparatus 100 stores change information regarding the change in the subject (S124C). Then, after acquiring the image of the target frame by destructive reading (S126), the imaging apparatus 100 corrects the image using the change information (S127).

For example, the imaging apparatus 100 determines a change for each area including one or a plurality of pixels, and stores information indicating the changed area and the amount of change as change information. Then, the imaging apparatus 100 performs correction by subtracting or adding a luminance value corresponding to the amount of change with respect to the changed area.

In the process illustrated in FIG. 15, the imaging apparatus 100 determines the change and stores the change information for each non-destructive reading. However, after all the non-destructive reading is performed or the destructive reading is performed. After that, determination and change information may be stored together.

As described above, the imaging apparatus 100 can generate an image with reduced influence even when a change occurs in the subject.

Further, although a plurality of operation examples when a change in the subject is detected has been described with reference to FIGS. 11 to 15, a plurality of these may be combined.

(Embodiment 4)
In the present embodiment, another method for controlling the exposure time of a target frame using an auxiliary image will be described. In the first and second embodiments, the method for automatically controlling the exposure time is described. However, in this embodiment, the user provides information for determining the exposure time, and the exposure time is determined based on the user's operation. An example of control will be described. For example, the method of the present embodiment can be applied during long-second exposure of a still image.

FIG. 16 is a flowchart showing an operation flow of the imaging apparatus 100 according to the present embodiment. FIG. 17 is a diagram for explaining the operation of the imaging apparatus 100.

First, the imaging apparatus 100 starts exposure (S131). Next, the imaging apparatus 100 acquires an auxiliary image by nondestructive reading (S132). Next, the imaging apparatus 100 displays the obtained auxiliary image on the display unit 103 (S133).

If there is no instruction to end exposure from the user (No in S134) and the predetermined exposure period has not ended (No in S136), the imaging apparatus 100 performs nondestructive readout again after a predetermined time has elapsed. Perform (S132), and display the newly obtained auxiliary image (S133).

For example, as shown in FIG. 17, nondestructive reading is performed at a predetermined cycle T4, and the read auxiliary images are sequentially displayed. Note that the interval at which nondestructive reading is performed may not be constant.

On the other hand, when there is an instruction to end the exposure from the user (Yes in S134), the imaging apparatus 100 stops the exposure (S135) and acquires an image by destructive reading (S137). When the predetermined exposure period ends (Yes in S136), the imaging apparatus 100 acquires an image by destructive reading (S137).

As described above, the display unit 103 displays the auxiliary image during the exposure period of the target frame. Further, the control unit 102 receives an instruction to end the exposure of the target frame during the exposure period of the target frame. As a result, the user can check the luminance level at any time with reference to the auxiliary image displayed during the exposure period. For this reason, the user can stop the exposure at a desired luminance level. Therefore, the user can easily take an image with a desired luminance level.

Note that the method for instructing the end of exposure by the user is not particularly limited. For example, an instruction to end the exposure is accepted when the user presses the shutter button or operates the touch panel during the exposure period.

In addition, although an example in which the user controls the exposure time T1 during the exposure period is described here, the user selects an arbitrary image from a plurality of auxiliary images already obtained during or after the exposure period. Also good. In this case, for example, in step S133, the imaging apparatus 100 displays an auxiliary image and stores the auxiliary image.

Further, for example, in accordance with a user operation, the imaging apparatus 100 displays a user interface for selecting one of a plurality of auxiliary images on the display unit 103 as illustrated in FIG. Specifically, auxiliary images obtained during exposure are sequentially stored in the storage unit 104. If exposure is in progress, a list of a plurality of auxiliary images obtained up to that timing is displayed. If it is after the exposure, a list of all auxiliary images obtained by the exposure is displayed. The user can select an image having an arbitrary luminance level by selecting one auxiliary image via the interface. When selection is performed after exposure, the displayed image may include an image obtained by destructive readout.

Note that the number of images to be selected is not limited to one, and a plurality of images may be selected. The imaging apparatus 100 may delete auxiliary image data that has not been selected after the above selection. Further, the imaging apparatus 100 may end the exposure when a selection is made during the exposure.

In addition, as illustrated in FIG. 19, the imaging apparatus 100 sequentially performs a plurality of nondestructive readings in the target frame, and stores a plurality of images obtained by the plurality of nondestructive readings in the storage unit 104 as moving images. Also good. Further, the imaging apparatus 100 may reproduce this moving image and display the reproduced moving image on the display unit 103. That is, the imaging apparatus 100 stores a plurality of images obtained by a plurality of nondestructive readings in a moving image file format. For example, a publicly known format including a moving image encoding method using inter-screen prediction or the like is used.

In addition, the stored moving image is played back automatically after the exposure is completed or based on a user operation.

Storing a plurality of images obtained by non-destructive readout in the moving image format in this way allows the shooting process itself during the exposure period to be stored as a moving image work. In addition, when using a plurality of images obtained by destructive readout without using an image obtained by nondestructive readout, it is necessary to generate a moving image by synthesizing still images over time. By using the method, it is possible to generate an operation without performing still image composition processing. Also, by storing a plurality of images in a moving image format, the amount of data can be reduced as compared to storing a plurality of still images.

The imaging apparatus 100 according to one aspect of the present disclosure uses an imaging element 101 capable of nondestructive readout and an auxiliary image obtained by nondestructive readout in the target frame, and a control unit 102 that controls shooting of the target frame. With.

According to this, since the imaging apparatus 100 can control photographing of the frame using the auxiliary image obtained by nondestructive readout, appropriate control can be performed according to the situation during the exposure period. As described above, the imaging apparatus 100 can realize further improvement.

For example, the control unit 102 may control the exposure time of the target frame using the auxiliary image.

According to this, since the imaging apparatus 100 can control the exposure time based on the information of the image obtained during the exposure period, it is more appropriate than when the exposure time is controlled based on the information obtained before the start of exposure. The exposure time can be controlled.

For example, the control unit 102 may sequentially perform non-destructive reading a plurality of times in the target frame, and stop exposure of the target frame when the luminance of the auxiliary image exceeds a predetermined threshold.

According to this, the imaging apparatus 100 can appropriately control the exposure time based on the image information obtained during the exposure period.

For example, the imaging apparatus 100 may further include a display unit 103, and the control unit 102 may further display a user interface for setting a threshold on the display unit 103.

According to this, it is possible to easily set the exposure according to the user's preference.

For example, the control unit 102 may perform automatic exposure of the target frame using the auxiliary image.

According to this, since the imaging apparatus 100 can perform automatic exposure based on the image information obtained during the exposure period, the image capturing apparatus 100 can perform automatic exposure more appropriately than when performing automatic exposure based on the information obtained before the start of exposure. Can be exposed.

For example, the control unit 102 sequentially performs a plurality of nondestructive readings in the target frame, and detects a change in the subject in the target frame using a plurality of auxiliary images obtained by the plurality of nondestructive readings. Good.

According to this, the imaging apparatus 100 can detect, for example, a change in the subject during the long exposure using the auxiliary image obtained by nondestructive reading.

For example, the control unit 102 may issue a warning when a change in the subject is detected.

According to this, the imaging apparatus 100 can notify the user of the occurrence of an abnormality during long exposure, for example.

For example, the control unit 102 may stop shooting when a change in the subject is detected.

According to this, the imaging apparatus 100 can automatically stop photographing when, for example, an abnormality occurs during long exposure.

For example, the control unit 102 may perform re-photographing when a change in the subject is detected.

According to this, the imaging apparatus 100 can automatically perform re-imaging when, for example, an abnormality occurs during long exposure.

For example, when a change in the subject is detected, the control unit 102 may perform correction for reducing the influence of the change in the subject on the image of the target frame obtained by shooting.

According to this, even when a change occurs in the subject, the imaging apparatus 100 can generate an image with reduced influence.

For example, the imaging apparatus 100 further includes a display unit 103 that displays an auxiliary image during the exposure period of the target frame, and the control unit 102 further instructs to end the exposure of the target frame during the exposure period of the target frame. May be accepted.

According to this, the user can take an image of a desired luminance level with reference to the auxiliary image displayed during the exposure period.

For example, the control unit 102 may sequentially perform a plurality of nondestructive readings in the target frame and store a plurality of images obtained by the nondestructive readings as a moving image.

For example, the image sensor 101 may be an organic sensor.

The control method according to an aspect of the present disclosure is a control method of the imaging apparatus 100 including the imaging element 101 capable of nondestructive readout, and uses the auxiliary image obtained by nondestructive readout in the target frame, A control step for controlling frame shooting;

According to this, since the control method can control photographing of the frame by using the auxiliary image obtained by nondestructive reading, appropriate control can be performed according to the situation during the exposure period. Thus, the control method can realize further improvement.

Note that these comprehensive or specific modes may be realized by a system, a method, an integrated circuit, a computer program, or a recording medium such as a computer-readable CD-ROM, and the system, method, integrated circuit, and computer program. Also, any combination of recording media may be realized.

The imaging device according to the embodiment of the present disclosure has been described above, but the present disclosure is not limited to this embodiment.

For example, each processing unit included in the imaging apparatus according to the above embodiment is typically realized as an LSI that is an integrated circuit. These may be individually made into one chip, or may be made into one chip so as to include a part or all of them.

Further, the integration of circuits is not limited to LSI, and may be realized by a dedicated circuit or a general-purpose processor. An FPGA (Field Programmable Gate Array) that can be programmed after manufacturing the LSI or a reconfigurable processor that can reconfigure the connection and setting of circuit cells inside the LSI may be used.

Further, in each of the above embodiments, each component may be configured by dedicated hardware or may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or a processor reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory.

Also, the present disclosure may be realized as a control method executed by the imaging apparatus.

The circuit configuration shown in the circuit diagram is an example, and the present disclosure is not limited to the circuit configuration. That is, similar to the circuit configuration described above, a circuit that can realize the characteristic function of the present disclosure is also included in the present disclosure. Moreover, all the numbers used above are illustrated for specifically explaining the present disclosure, and the present disclosure is not limited to the illustrated numbers.

In addition, division of functional blocks in the block diagram is an example, and a plurality of functional blocks can be realized as one functional block, a single functional block can be divided into a plurality of functions, or some functions can be transferred to other functional blocks. May be. In addition, functions of a plurality of functional blocks having similar functions may be processed in parallel or time-division by a single hardware or software.

In addition, the order in which the steps in the flowchart are executed is for illustration in order to specifically describe the present disclosure, and may be in an order other than the above. Also, some of the above steps may be executed simultaneously (in parallel) with other steps.

As described above, the imaging device according to one or more aspects has been described based on the embodiment, but the present disclosure is not limited to this embodiment. Unless it deviates from the gist of the present disclosure, various modifications conceived by those skilled in the art have been made in this embodiment, and forms constructed by combining components in different embodiments are also within the scope of one or more aspects. May be included.

The present disclosure can be applied to an imaging apparatus such as a digital still camera or a digital video camera.

DESCRIPTION OF SYMBOLS 100 Image pick-up device 101 Image pick-up element 102 Control part 103 Display part 104 Storage part 106 Imaging system 107 Light-shielding part 201 Pixel 202 Vertical scanning part 203 Column signal processing part 204 Horizontal read-out part 205 Reset control line 206 Address control line 207 Vertical signal line 208 Horizontal Output terminal 211 Photoelectric conversion unit 212 Charge storage unit 213 Reset transistor 214 Amplification transistor 215 Selection transistor

Claims (14)

An image sensor capable of non-destructive readout;
An imaging apparatus comprising: a control unit that controls photographing of the target frame using an auxiliary image obtained by nondestructive reading in the target frame. The imaging apparatus according to claim 1, wherein the control unit controls an exposure time of the target frame using the auxiliary image. The controller is
Sequentially performing a plurality of nondestructive readings in the target frame,
The imaging apparatus according to claim 2, wherein exposure of the target frame is stopped when the luminance of the auxiliary image exceeds a predetermined threshold value. The imaging apparatus further includes a display unit,
The imaging device according to claim 3, wherein the control unit further displays a user interface for setting the threshold on the display unit. The imaging apparatus according to claim 1, wherein the control unit performs automatic exposure of the target frame using the auxiliary image. The controller is
Sequentially performing a plurality of nondestructive readings in the target frame,
The imaging apparatus according to claim 1, wherein a change in a subject in the target frame is detected using a plurality of auxiliary images obtained by the plurality of nondestructive readings. The imaging device according to claim 6, wherein the control unit issues a warning when a change in the subject is detected. The imaging apparatus according to claim 6, wherein the control unit stops shooting when a change in the subject is detected. The imaging apparatus according to claim 6, wherein the control unit performs re-imaging when a change in the subject is detected. The imaging apparatus according to claim 6, wherein, when a change in the subject is detected, the control unit performs correction for reducing an influence of the change in the subject on an image of the target frame obtained by photographing. The imaging device further includes:
A display unit for displaying the auxiliary image during an exposure period of the target frame;
The imaging apparatus according to claim 1, wherein the control unit further receives an instruction to end exposure of the target frame during an exposure period of the target frame. The controller is
Sequentially performing a plurality of nondestructive readings in the target frame,
The imaging device according to claim 1, wherein a plurality of images obtained by the plurality of nondestructive readings are stored as moving images. The imaging apparatus according to any one of claims 1 to 12, wherein the imaging element is an organic sensor. A method for controlling an imaging apparatus including an imaging element capable of nondestructive readout,
A control method including a control step of controlling photographing of a target frame using an auxiliary image obtained by nondestructive reading in the target frame.

Images