JP2016054479A - Imaging device, control method and program thereof, and imaging element - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging device capable of reducing control information to be transmitted from a control part to a signal processing circuit.SOLUTION: An imaging device includes a pixel part having a first pixel region and a second pixel region, output means for outputting image data obtained from the pixel part with added region information indicating the first pixel region and the second pixel region, and signal processing means for correcting pixel data in the first pixel region using pixel data in the second pixel region for image data read from the pixel part. The signal processing means extracts pixel data in the first pixel region and pixel data in the second pixel region in the image data using region information added to the image data received from the output means.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an imaging apparatus, a control method thereof, a program, and an imaging element.

  In recent years, with the increase in the number of pixels in an image sensor and the increase in the reading speed from the image sensor, the transmission capacity required between the image sensor and a signal processing circuit (DSP, Digital Signal Processor) connected thereto is increased. It is increasing. In response to such a request, Patent Document 1 discloses a technique for transmitting packets divided into a plurality of transmission paths in data transmission / reception between the image sensor and the DSP and improving the efficiency of data transmission / reception between circuits. Yes.

JP 2012-120158 A

  By the way, in an imaging system having an imaging device, a signal processing circuit, and a control unit (for example, CPU), it is necessary to transmit control information (for example, coordinate information) from the control unit to each circuit. In particular, since the signal processing circuit processes the image data in accordance with the size of the image output from the image sensor, the signal processing circuit is controlled when changing the image size of the image data output from the image sensor. It is necessary to receive control information such as coordinate information from the unit. Further, for example, when the imaging element has a light-shielded optical black region pixel, and the signal processing circuit corrects the pixel data of the effective pixel region using the pixel data of the region, the circuit is It is necessary to receive coordinate information for the two areas described above. At this time, since the control unit transmits the same control information to the image sensor and the signal processing circuit, the control information transmitted from the control unit to each increases as the number of processes performed in the signal processing circuit increases. Therefore, it is assumed that the system becomes complicated.

  The present invention has been made in view of the above-described problems of the related art, and an imaging apparatus capable of reducing control information transmitted from a control unit to a signal processing circuit, a control method thereof, a program, and an imaging An object is to provide an element.

  In order to solve this problem, for example, an imaging apparatus of the present invention has the following configuration. That is, a pixel portion having a first pixel region and a second pixel region, and region information indicating the first pixel region and the second pixel region are added to the image data obtained from the pixel portion and output. Signal processing means for correcting the pixel data of the first pixel region using the pixel data of the second pixel region with respect to the image data read from the pixel unit; Is characterized by extracting the pixel data of the first pixel region and the pixel data of the second pixel region in the image data using the region information added to the image data received from the output means.

  According to the present invention, it is possible to reduce the control information transmitted from the control unit to the signal processing circuit.

1 is a block diagram illustrating an example of a functional configuration of a digital camera as an example of an imaging apparatus according to an embodiment of the present invention. 1 is a block diagram showing a functional configuration example (a) and an image output unit (b) of an image sensor according to the present embodiment. Block diagram showing a functional configuration example of the DFE according to the present embodiment The flowchart which shows a series of operation | movement of the process of the imaging system which concerns on this embodiment. 7 is a flowchart showing a series of operations of packet generation processing according to the first embodiment. A flowchart for describing image processing according to the first embodiment. The figure which shows an example of the format of the packet which concerns on Embodiment 1. FIG. 3 is a diagram illustrating a configuration example of a pixel region in a pixel unit according to the first embodiment. 7 is a flowchart showing a series of image processing operations according to the second embodiment. FIG. 10 is a diagram illustrating a configuration example of a pixel region in a pixel unit according to the second embodiment. FIG. 6 is a block diagram illustrating a functional configuration example (a) and an image output unit (b) of an image sensor according to a third embodiment. 9 is a flowchart showing a series of image processing operations according to the third embodiment. The figure which shows an example of the format of the packet which concerns on Embodiment 3. The figure which shows an example of the laminated type image sensor which concerns on other embodiment. The block diagram which shows the function structural example of the mobile telephone as an example of the imaging device which concerns on other embodiment.

(Embodiment 1)
Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the drawings. In the following, an example in which the present invention is applied to a digital camera including an image sensor, a signal processing circuit, and a control unit that are connected to each other will be described as an example of an imaging apparatus. However, the imaging apparatus referred to in the present invention is not limited to a digital camera, and can be applied to any electronic device having such a configuration of an imaging element or the like. These electronic devices may include, for example, a mobile phone, a game machine, a tablet terminal, a personal computer, a clock-type or glasses-type information terminal.

(Configuration of digital camera 100)
FIG. 1 is a block diagram illustrating a functional configuration example of a digital camera 100 as an example of an imaging apparatus according to the present embodiment. One or more of the functional blocks shown in FIG. 1 may be realized by hardware such as an ASIC or a programmable logic array (PLA), or may be realized by a programmable processor such as a CPU or MPU executing software. May be. Further, it may be realized by a combination of software and hardware. Therefore, in the following description, even when different functional blocks are described as the operation subject, the same hardware can be realized as the subject.

  A signal output from each pixel of the image sensor generally includes a dark current generated due to the influence of heat or the like even when light is shielded. For this reason, the image sensor includes an optical black pixel region (OB pixel region) that is shielded so as not to react with light in order to obtain a signal (black reference signal) that is a signal level reference signal, and a signal of an effective pixel. Is processed based on the black reference signal obtained from the pixels in the OB pixel area. For example, by providing an OB pixel area above the effective pixel area on the image sensor and calculating a correction value from the output of the OB pixel, the influence of dark current on the signal output from the pixel in the effective pixel area The shading in the horizontal direction due to can be corrected. In the present embodiment, an example of performing correction processing related to dark current will be described as an example of image processing performed by the DSP. That is, an example will be described in which the image sensor outputs signals of both the effective pixel region and the OB pixel region, and the correction process related to the dark current is performed on the signal input by the DSP.

  The imaging unit 120 includes an imaging element 121 and a DFE (Digital Front End) 122. As will be described later with reference to FIG. 2, the imaging element 121 includes a pixel unit 200, an A / D conversion unit 201, an image output unit 202, and a timing generator (TG) 203. The image sensor 121 generates image data from the subject optical image formed by the optical system 102 and outputs the image data to the DFE 122.

  The DFE 122 is a signal processing processor (DSP), performs image processing on the image data output from the image sensor 121, and outputs the image data subjected to the image processing to the control unit 101.

  The control unit 101 includes, for example, a CPU or MPU, and controls the entire digital camera 100 by developing and executing a program stored in the nonvolatile memory 105 in the work area of the volatile memory 103. The control unit 101 controls each unit such as the image sensor 121, the optical system 102, and the DFE 122 in order to perform a shooting operation according to the shooting conditions set by the user. As will be described later, the control unit 101 instructs the image sensor 121 to specify a pixel region from which a signal is read from the pixel unit 200.

  The optical system 102 is electrically connected to the digital camera 100, and includes a focus lens, a zoom lens, a diaphragm, a shutter, and the like that capture light from the outside and form an image on the imaging surface of the pixel unit 200.

  The volatile memory 103 is a main storage device such as a RAM that stores temporary data that may be erased after the digital camera 100 is turned off. The volatile memory 103 is connected to the control unit 101 and holds data provided from the control unit 101 in accordance with an instruction from the control unit 101.

  The power supply unit 104 supplies power to the digital camera 100. The power supply destination is not limited to the control unit 101 illustrated in FIG. 1, and power is supplied to each unit included in the digital camera 100.

  The non-volatile memory 105 is an auxiliary storage device composed of a semiconductor memory, a magnetic disk, or the like. The non-volatile memory 105 stores captured images and videos, and turns off the power such as Tv value and Av value set in the digital camera 100. It stores information that needs to be retained. The non-volatile memory 105 is connected to the control unit 101 and stores data provided from the control unit 101 in accordance with an instruction from the control unit 101.

  The accessory shoe 106 is an assembly of metal contacts that are disposed on the top of the digital camera 100 and can be connected to a clip-on type strobe.

  The first shutter switch 107 is turned on when a shutter button provided in the digital camera 100 is being operated, so-called half-press (shooting preparation instruction), and generates a first shutter switch signal SW1. The control unit 101 starts operations such as AF (autofocus) processing, AE (automatic exposure) processing, and AWB (auto white balance) processing in response to the first shutter switch signal SW1.

  The second shutter switch 108 is turned on when the operation of the shutter button is completed, that is, when it is fully pressed (shooting instruction), and generates a second shutter switch signal SW2. In response to the second shutter switch signal SW2, the control unit 101 starts a series of imaging processing operations from reading a signal from the imaging unit 120 to writing image data in the nonvolatile memory 105.

  The operation unit 109 is a button, a dial, a touch panel, or the like for setting a setting value such as an Av value or a Tv value set in the digital camera 100. When the operation unit 109 detects a user operation, the operation unit 109 notifies the control unit 101 of the operation content or the set value. Various setting values set by user operations are stored in the nonvolatile memory 105 under the instruction of the control unit 101 as described above.

  The power switch 110 is a mechanical switch that switches the power state of the digital camera 100 on or off. When the on / off switching operation by the user is detected, power supply from the power supply unit 104 to each unit of the digital camera 100 is started or stopped.

(Configuration of the image sensor 121)
Next, the configuration of the image sensor 121 shown in FIG. 1 will be described with reference to FIG.

  The image sensor 121 receives a subject optical image formed by the optical system 102 by a plurality of pixels arranged two-dimensionally, photoelectrically converts each pixel, and outputs an analog signal. The image sensor 121 may be an image sensor such as a CCD (Charge-Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor. The pixel unit 200 of the image sensor 121 outputs an analog signal in accordance with the timing signal supplied from the timing generator 203.

  The A / D conversion unit 201 converts the analog signal output from the pixel unit 200 into a digital signal according to the operation timing generated by the timing generator 203.

  The image output unit 202 adds header information based on control information acquired from the control unit 101 to the digital signal input from the A / D conversion unit 201, and outputs image data including the header information. More specifically, the image output unit 202 acquires control information from the control unit 101, that is, first and second vertical coordinate ranges to be described later, and generates header information based on the vertical coordinate range. When outputting the image data, the image output unit 202 divides the digitized pixel data into predetermined data lengths, adds the generated header information to these data, and performs packetization. The image output unit 202 outputs the packetized data to the DFE 122 as image data.

  The timing generator 203 supplies operation timing to the pixel unit 200 and the A / D conversion unit according to the operation timing information input from the control unit 101.

(Configuration of image output unit 202)
Further, the configuration of the image output unit 202 shown in FIG. 2A will be described in more detail with reference to FIG.

  The pixel data generation unit 250 receives the digital signal output from the A / D conversion unit 201 and converts it into pixel data. The pixel data generation unit 250 forms pixel data so as to form data of each pixel in the RGB format in which the input digital signal is separated into, for example, R (red), G (green), and B (blue) color components. Is generated.

  The row counter unit 251 receives the pixel data output from the pixel data generation unit 250, counts the number of rows (number of pixel lines) by counting up the number of rows for each predetermined number of pixels, The counted row value is output to the comparison unit 252.

  The comparison unit 252 inputs the row value from the row counter unit 251, and inputs the row value from the control unit 101, the vertical coordinate range of the first pixel region (for example, the shielded OB pixel region) and the second To the vertical coordinate range of the pixel region (for example, effective pixel region). Based on the comparison, the comparison unit 252 determines whether the pixel data is included in the first pixel region and the second pixel region, and the pixel data is included in the first pixel region and the second pixel in the image illustrated in FIG. Data indicating whether it is included in the pixel area is generated as header information. In the present embodiment, data indicating whether or not included in the first pixel region and the second pixel region is referred to as first region information and second region information, respectively.

  The packet generation unit 253 generates a packet based on the pixel data output from the pixel data generation unit 250 and the header information output from the comparison unit 252. The packet generation unit 253 stores the pixel data output from the pixel data generation unit 250 in the payload of the packet, adds the header information output from the comparison unit 252 to the payload, and generates a packet.

  The packet output unit 254 outputs the packet generated by the packet generation unit 253 according to a transmission method according to a transmission path (not shown) that connects the image sensor 121 and the DFE 122.

(Configuration of DFE122)
FIG. 3 shows a more detailed functional configuration example of the DFE 122.

  The packet separation unit 300 separates header information and payload data included in the input packet.

  The pixel data extraction unit 301 extracts pixel data from the payload data separated from the packet by the packet separation unit 300.

  The first region information extraction unit 302 extracts the first region information based on the header information separated from the packet by the packet separation unit 300 and outputs the first region information to the correction value calculation unit 303.

  The correction value calculation unit 303 determines whether the pixel data input from the pixel data extraction unit 301 is included in the first pixel region based on the first region information extracted by the first region information extraction unit 302. judge. Then, a correction value for correcting the influence of the dark current is calculated based on the average value of the pixel data determined to be included in the first pixel region. The calculated correction value is output to the pixel data processing unit 305. In the present embodiment, an example in which the correction value for correcting the influence of the dark current is calculated using the average value of the pixel data is described. However, the correction value is not limited to the average value, and various values may be used depending on the purpose. Can be modified and changed.

  The second region information extraction unit 304 extracts the second region information from the header information separated from the packet by the packet separation unit 300 and outputs the second region information to the pixel data processing unit 305.

  The pixel data processing unit 305 determines whether the pixel data input from the pixel data extraction unit 301 is included in the second pixel region based on the second region information extracted by the second region information extraction unit 304. . Then, the correction value calculated by the correction value calculation unit 303 is subtracted from the pixel data determined to be included in the second pixel region, and the influence of the dark current included in the pixel data is corrected. When completing the correction process, the pixel data processing unit 305 outputs the corrected image data to the control unit 101.

(A series of operations related to imaging processing)
Next, with reference to FIG. 4, a series of operations related to imaging processing by the imaging unit 120 will be described. This process is started when, for example, a shooting instruction from the user to the second shutter switch 108 of the digital camera 100 is detected. Note that each step related to this processing is executed in response to an instruction from the control unit 101, and the control unit 101 develops and executes the program stored in the nonvolatile memory 105 in the work area of the volatile memory 103. It is realized by.

  In S1, the imaging unit 120 performs imaging. The control unit 101 controls the aperture of the optical system 102 and the exposure time, for example, to expose the pixel unit 200 with an appropriate exposure amount. As described above, the analog signal read from the pixel unit 200 is converted into a digital signal to generate pixel data.

  In S2, the image output unit 202 generates a packet obtained by dividing the image data. FIG. 7 shows a format of a packet generated by the image output unit 202. A packet consists of a header and a payload. The pixel data for one row stored in the payload stores pixel data output from the pixels in the OB pixel region or pixel data output from the pixels in the effective pixel region. Information indicating whether the pixel data stored in the payload is the pixel data output from the pixels in the first or second pixel region is added to the header information. The image output unit 202 divides the image data into pixel data for one row, stores the pixel data for one row in the payload, adds header information, and generates a packet. The packet generation process will be described later with reference to the flowchart shown in FIG. The image output unit 202 outputs the generated packet to the DFE 122 via a transmission path (not shown).

  In S <b> 3, the DFE 122 performs image processing on the image data input from the image sensor 121. The DFE 122 separates the input packet data into a payload storing pixel data and a header storing first area information and second area information, and performs image processing on the pixel data, that is, influence of dark current. Perform correction to remove. Details of the image processing will be described later with reference to a flowchart shown in FIG. When the DFE 122 outputs the corrected image data to the control unit 101, the control unit 101 ends a series of operations related to this processing.

(A series of operations related to packet generation processing)
With reference to the flowchart shown in FIG. 5, a series of operations related to the packet generation process shown as S2 in FIG. 4 will be described.

  In S <b> 21, the pixel data generation unit 250 receives the digital signal output from the A / D conversion unit, generates, for example, the above-described RGB format pixel data, and sequentially outputs the data for each pixel. The generated pixel data is counted by setting the number of pixels determined in advance by the row counter unit 251 as one row.

  In S <b> 22, the comparison unit 252 compares the number of rows generated by the row counter unit 251 (that is, the vertical coordinate of the pixel data) with the vertical coordinate range of the first pixel region input from the control unit 101. It is determined whether the vertical coordinate of the data is included in the first pixel area. When the comparison unit 252 determines that the pixel data to be processed (that is, the number of input rows) is included in the vertical coordinate range of the first pixel region, the comparison unit 252 updates the first region information. The first area information is, for example, a binary identifier whose initial value is set to 0, and is 1 when included in the vertical coordinate range of the first pixel area, and 0 when not included in the range. Is set.

  In S23, the comparison unit 252 compares the number of rows generated by the row counter unit 251 with the vertical coordinate range of the second pixel region input from the control unit 101, and the vertical coordinate of the pixel data is the second pixel. It is determined whether it is included in the area. When the comparison unit 252 determines that the pixel data to be processed is included in the vertical coordinate range of the second pixel region, the comparison unit 252 updates the second region information. The second area information is set in the same manner as the first area information described above.

  In S24, the comparison unit 252 generates header information including the first area information and the second area information.

  In S <b> 25, the packet generation unit 253 generates a packet based on the payload data including the pixel data and the header information generated by the comparison unit 252.

  In S <b> 26, the packet output unit 254 outputs the packet generated in S <b> 25 to the DFE 122. The packet output unit 254 repeats the packet transmission for each row, and when the packet transmission is completed for the entire image data, the series of operations related to this processing is completed.

(A series of operations related to image processing)
Next, the image processing shown as S3 in FIG. 4 will be described with reference to the flowchart shown in FIG.

  In S300, when the DFE 122 receives a packet transmitted from the image sensor 121, in S301, the packet separation unit 300 of the DFE 122 separates the header information and the payload data.

  In S302, the pixel data extraction unit 301 extracts pixel data included in the payload.

  In step S <b> 303, the first region information extraction unit 302 extracts the first region information included in the header information separated in step S <b> 301 and outputs the first region information to the correction value calculation unit 303.

  In S304, the correction value calculation unit 303 determines whether the vertical coordinates of the pixel data extracted from the payload in S302 are included in the first vertical area based on the extracted first area information. When the first area information indicates 1, the correction value calculation unit 303 determines that the pixel data is included in the first vertical area, advances the processing to S305, and sets the first area information to 0. When it is shown, it is determined that the pixel data is not included in the first vertical area, and the process proceeds to S307.

  In step S305, the correction value calculation unit 303 compares the horizontal coordinate of the pixel data obtained by counting the pixel data of the payload with the first horizontal coordinate range input from the control unit 101, and the horizontal coordinate of the pixel data is It is determined whether it is included in the first horizontal coordinate range. If it is determined that the horizontal coordinate of the pixel data is included in the first horizontal coordinate range, the process proceeds to S306. If it is determined that the pixel data does not include the horizontal coordinate, the process proceeds to S307.

  In S306, the correction value calculation unit 303 calculates an average value of all the pixel data in the first pixel area included in the first horizontal coordinate range, and generates a correction value for each image.

  In S307, the second region information extraction unit 304 extracts second region information included in the header information.

  In S308, the pixel data processing unit 305 determines whether the vertical coordinates of the pixel data are included in the second vertical area from the extracted second area information in the same manner as in S304, and is included in the second vertical area. If it is determined that it is, the process proceeds to S309.

  In step S <b> 309, the pixel data processing unit 305 compares the horizontal coordinate information indicated by the correction value calculation unit 303 with the second horizontal coordinate range input from the control unit 101. If it is determined that the horizontal coordinate of the pixel data is included in the input horizontal coordinate range, the process proceeds to S310.

  In step S310, the pixel data processing unit 305 performs a subtraction process on the correction value calculated in step S306 on the pixel specified by the second area information. When the pixel data processing unit 305 finishes the correction process on the pixel specified by the second region information, the pixel data processing unit 305 ends the series of operations related to the image processing.

  In this embodiment, pixel data is stored in a packet for each row, and the control unit 101 transmits the vertical coordinate range to the image output unit 202. However, pixel data is stored in a packet for each column. The horizontal coordinate range may be transmitted from the control unit 101. In this case, the row counter unit 251 counts pixel data for each column, and the comparison unit 252 may determine whether the counter value is included in the horizontal coordinate range.

  Moreover, although the example which subtracts a correction value from the pixel data contained in a 2nd pixel area was shown as a correction process in this embodiment, the said correction process is not restricted to a subtraction process, Various deformation | transformation is carried out according to the objective. And changes are possible.

  As described above, in this embodiment, the vertical coordinate information of the pixel data is input to the image sensor 121, and the image sensor 121 supplies the vertical coordinate information of the pixel data together with the pixel data. In this way, the DFE 122, which is a signal processing circuit, can perform image processing without receiving information related to vertical coordinates from the control unit 101. Therefore, control information transmitted from the control unit to the signal processing circuit can be reduced.

(Embodiment 2)
Next, Embodiment 2 will be described. In this embodiment, as an example of image processing performed by the DFE 122, processing for correcting shading in the vertical direction due to the influence of dark current will be described. The digital camera 100 of the present embodiment is different from the first embodiment in the configuration of the pixels of the image sensor and the applied image processing, but the other configurations are the same as those in the first embodiment. For this reason, the same reference numerals are assigned to the same components, and redundant descriptions are omitted, and differences will be mainly described.

  As shown in FIG. 10, the pixel unit 200 according to the present embodiment has an OB pixel region adjacent to the effective pixel region in the horizontal direction. The pixels in the OB pixel region arranged in the horizontal direction with the effective pixel region are input to the DFE 122, and the DFE 122 calculates a correction value using the output of the pixel in the OB pixel region and performs a correction process on the effective pixel region. By doing so, the shading in the vertical direction due to the influence of dark current is corrected.

  The OB pixel region (first pixel region) and the effective pixel region (second pixel region) are arranged adjacent to each other in the horizontal direction. The pixels included in the first pixel region are specified by the vertical coordinate range of the first pixel region and the horizontal coordinate range of the first pixel region, which indicate the vertical coordinate range and the horizontal coordinate range. Similarly, the pixels included in the second pixel region are specified by the vertical coordinate range of the second pixel region and the horizontal coordinate range of the second pixel region.

  A series of image processing operations according to the present embodiment will be described with reference to a flowchart shown in FIG. In FIG. 9, the same steps as those in FIG. 6 are given the same step numbers.

  The DFE 122 performs each process of S300 to S305 described in FIG.

  Next, in S901, when the correction value calculation unit 303 determines that the horizontal coordinate of the pixel data is included in the first horizontal coordinate range, the correction value calculation unit 303 calculates the average value of the pixel data of each row. The correction value calculation unit 303 outputs the calculated average value for each row to the pixel data processing unit 305 as a correction value for correcting vertical shading due to the influence of dark current.

  Then, each process of S307-S309 demonstrated in FIG. 6 is performed.

  Further, in S902, the pixel data processing unit 305 uses the correction value for each row calculated in S901 to correct the pixel data in the row to be processed among the pixel data in the second pixel region. I do. The pixel data processing unit 305 corrects the shading in the vertical direction due to the influence of the dark current of the image data, for example, by subtracting the correction value from the pixel data of the second pixel region.

  As described above, in this embodiment, the DFE 122 calculates a correction value using pixel specified from the same row for the pixel data for each row determined based on the header information, and performs the correction process. I did it. By doing in this way, even when the imaging device has the OB pixel region so as to be adjacent to the effective pixel region in the horizontal direction, the DFE 122 that is a signal processing circuit is connected to the control unit 101. Image processing can be performed without receiving information about vertical coordinates. Therefore, control information transmitted from the control unit to the signal processing circuit can be reduced.

(Embodiment 3)
Next, Embodiment 3 will be described. The third embodiment is different from the above-described embodiment in that the horizontal coordinate range for the first and second pixel regions is further input from the control unit 101 to the image sensor 121. For this reason, the structure which produces | generates the header information of a packet and the structure which receives a packet and calculates a correction value from header information differ. Since other configurations are the same as those of the above-described embodiment, the same configurations are denoted by the same reference numerals, and redundant description is omitted, and differences are mainly described. Note that the imaging device in the present embodiment has the first and second pixel regions shown in FIG. 10 in the second embodiment, and the DFE 122 performs vertical shading due to the influence of dark current as in the second embodiment. The process which corrects is performed.

  FIG. 11A shows a functional configuration example of the image sensor 121 in the present embodiment. In the embodiment described above, the image output unit 202 illustrated in FIG. 3 generates the first region information from the first vertical coordinate range and the number of rows. In contrast, the image output unit 1101 according to the present embodiment inputs the first and second horizontal coordinate ranges from the control unit 101 in addition to the first and second vertical coordinate ranges. FIG. 11B shows a functional configuration example of the image output unit 1101 according to the present embodiment. The image output unit 1101 generates first area information from the first vertical coordinate range and the first horizontal coordinate range, and the number of rows and columns of pixel data to be processed. In the present embodiment, an example in which pixel data for one pixel is stored as a payload as shown in FIG. 13 will be described. However, similarly to the above-described embodiment, pixel data for one row may be stored in the payload, and a header indicating each region information for each pixel stored in the payload may be added.

  The counter unit 1102 of the image output unit 1101 counts a row and a column every time one pixel is input with respect to the pixel data input from the pixel data generation unit 250, and compares the row and the column of the processing target pixel. 1103. The comparison unit 1103 compares the row and column counter values input from the counter unit 1102 with the coordinate ranges of the first and second pixel areas input from the control unit 101, and generates header information. The comparison unit 1103 determines whether the input row counter value is included in the first vertical coordinate range, and further determines whether the column counter value is included in the first horizontal coordinate range. When the column counter value is included in the first vertical coordinate range, the comparison unit 1103 updates the corresponding value of the first area information. When the row counter value is included in the first horizontal coordinate range, the first area information is similarly updated. The first area information is, for example, a 2-digit binary identifier whose initial value is set to 00, and is set to 1 when each of the coordinate ranges is included in the order of vertical coordinates and horizontal coordinates. Accordingly, when the row counter value and the column counter value are included in the first pixel area, the first area information is 11. The comparison unit 1103 further determines whether the pixel data is included in the second pixel area, and generates second area information. At this time, the comparison unit 1103 generates the second area information in the same manner as the generation of the first area information.

  Next, a series of operations related to image processing according to the embodiment will be described with reference to FIG. In FIG. 12, the same step numbers are assigned to the processes similar to those in FIGS.

  The DFE 122 performs each process of S300 to S304 described in FIG.

  Next, in S1201, the correction value calculation unit 303 of the DFE 122 refers to the first area information to determine whether or not the horizontal coordinate of the pixel data is included in the first horizontal area, and the first If it is determined that the image is included in the horizontal area, the process proceeds to S901. More specifically, the correction value calculation unit 303 is included in the first horizontal region when the value of the digit corresponding to the horizontal coordinate among the two-digit identifier is 1 in the first region information described above. It is determined that

  Then, each process of S901 demonstrated in FIG. 9, and S307-S308 demonstrated in FIG. 6 is performed.

  Next, in S1203, the correction value calculation unit 303 refers to the second area information, determines whether the horizontal coordinate of the pixel data is included in the second horizontal area, and includes it in the second horizontal area. If it is determined that the process has been performed, the process of S902 described with reference to FIG. 9 is performed.

  As described above, in this embodiment, the vertical coordinate information and horizontal coordinate information of pixel data are input to the image sensor 121, and the image sensor 121 supplies the vertical coordinate information and horizontal coordinate information of the pixel data together with the pixel data. I tried to do it. In this way, the DFE 122, which is a signal processing circuit, can perform image processing without receiving information on the vertical coordinate and the horizontal coordinate from the control unit 101. Control information transmitted to the circuit can be reduced.

(Other embodiments)
The present invention can also be applied to other embodiments described below. For example, the present invention can be applied to a multilayer imaging device 1400 shown in FIG. As shown in FIG. 14, in the imaging device 1400 of the present embodiment, a first semiconductor chip 1401 and a second semiconductor chip 1402 are stacked at a chip level. FIG. 14A shows an oblique projection, and FIG. 14B shows a top view of each chip. The first semiconductor chip 1401 includes a region including a pixel portion 1405, and the second semiconductor chip 1402 for a high-speed logic process can perform high-speed processing including digital data such as a column AD conversion circuit and a horizontal scanning circuit. 1403 is included. In the configuration of FIG. 1 described above, for example, the pixel portion 200 included in the imaging element 121 corresponds to the pixel portion 1405 of the first semiconductor chip 1401. Further, a circuit other than the A / D conversion unit 201, the image output unit 202, and the pixel unit 200 included in the imaging element 121 may be arranged on the second semiconductor chip 1402.

  Furthermore, the present invention can also be applied to a mobile phone 1500 shown in FIG. 15 as an example of an imaging apparatus. FIG. 15 is a block diagram showing a functional configuration of mobile phone 1500. The mobile phone 1500 has an electronic mail function, an Internet connection function, an image shooting / playback function, and the like in addition to a voice call function. The communication unit 1010 communicates audio data and image data with other telephones by a communication method according to a communication carrier contracted by the user. The voice processing unit 1020 converts voice data from the microphone 1030 into a format suitable for outgoing call and sends it to the communication unit 1010 during a voice call. Also, the voice processing unit 1020 decodes the voice data from the communication partner sent from the communication unit 1010 and sends it to the speaker 1040. The imaging unit 1050 captures an image of a subject and outputs pixel data. The imaging unit 1050 includes the imaging element 121 described above in each embodiment. Here, the imaging element 121 may be the above-described stacked imaging element. Further, the above-described packet is transmitted between the imaging unit 1050 and the image processing unit 1060. The image processing unit 1060 includes the DFE 122 described above in each embodiment, and processes the pixel data photographed by the imaging unit 1050 at the time of photographing an image, converts it into a format suitable for recording, and outputs it. In addition, when the recorded image is reproduced, the image processing unit 1060 processes the reproduced image and sends the processed image to the display unit 1070. The display unit 1070 includes a liquid crystal display panel of about several inches, and displays various screens according to instructions from the control unit 1090. The nonvolatile memory 1080 stores data such as address book information, e-mail data, and image data captured by the imaging unit 1050.

  The control unit 1090 includes a CPU, a memory, and the like, and controls each unit of the mobile phone 1500 according to a control program stored in a memory (not shown). The operation unit 1100 includes a power button, number keys, and other various operation keys for a user to input data. The memory IF 1110 records and reproduces various data on a recording medium 1120 such as a memory card. The external IF 1130 transmits data stored in the nonvolatile memory 1080 or the recording medium 1120 to the external device, and receives data transmitted from the external device. The external IF 1130 performs communication by a known communication method such as a wired communication method such as USB or wireless communication.

  Next, a voice call function in the mobile phone 1500 will be described. When making a call to the other party, the user inputs the number of the other party by operating the number key of the operation unit 1100 or displays the address book stored in the nonvolatile memory 1080 on the display unit 1070 and Select the other party and instruct outgoing. When transmission is instructed, control unit 1090 transmits to communication party 1010 to communication unit 1010. When an incoming call is received, the communication unit 1010 outputs the other party's voice data to the voice processing unit 1020 and transmits the user's voice data to the other party.

  When transmitting an e-mail, the user uses the operation unit 1100 to instruct mail creation. When mail creation is instructed, control unit 1090 displays a screen for creating mail on display unit 1070. The user inputs a transmission destination address and text using the operation unit 1100 and instructs transmission. When the mail transmission is instructed, control unit 1090 sends address information and mail body data to communication unit 1010. The communication unit 1010 converts mail data into a format suitable for communication and sends it to a transmission destination. In addition, when receiving the electronic mail, the communication unit 1010 converts the received mail data into a format suitable for display and displays the data on the display unit 1070.

  Next, a photographing function in the mobile phone 1500 will be described. When the user operates the operation unit 1100 to set the shooting mode and then instructs to shoot a still image or a moving image, the imaging unit 1050 shoots the still image data or the moving image data and sends it to the image processing unit 1060. The image processing unit 1060 processes captured still image data and moving image data, and stores them in the nonvolatile memory 1080. Further, the image processing unit 1060 sends the captured still image data and moving image data to the memory IF 1110. The memory IF 1110 stores still images and moving image data in the recording medium 1120.

  In addition, the mobile phone 1500 can transmit a file including still images and moving image data shot in this way as an attached file of an e-mail. Specifically, when an e-mail is transmitted, an image file stored in the nonvolatile memory 1080 or the recording medium 1120 is selected, and transmission is instructed as an attached file.

  In addition, the mobile phone 1500 can transmit a file including a captured still image or moving image data to an external device such as a PC or another telephone using the external IF 1130. The user operates the operation unit 1100 to select an image file stored in the nonvolatile memory 1080 or the recording medium 1120 and instruct transmission. The control unit 1090 controls the external IF 1130 so that the selected image file is read from the nonvolatile memory 1080 or the recording medium 1120 and transmitted to the external device.

  The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

DESCRIPTION OF SYMBOLS 101 ... Control part, 200 ... Pixel part, 121 ... Image sensor, 202 ... Image output part, 122 ... DFE

Claims (21)

A pixel portion having a first pixel region and a second pixel region;
Output means for adding and outputting region information indicating the first pixel region and the second pixel region to the image data obtained from the pixel unit;
Signal processing means for correcting the pixel data of the first pixel region using the pixel data of the second pixel region with respect to the image data read from the pixel unit;
The signal processing means extracts pixel data of the first pixel area and pixel data of the second pixel area in the image data using the area information added to the image data received from the output means. To
An imaging apparatus characterized by that. The output means includes
Packet generating means for generating a packet by adding the region information to the image data obtained from the pixel unit;
Packet output means for outputting the packet to the signal processing means,
The imaging apparatus according to claim 1. The output means includes
Comparing means for comparing the region in the image obtained from the region information with the region of the image data to generate information indicating the first pixel region and the second pixel region;
The packet generation means generates a packet by adding information indicating the first pixel region and the second pixel region to the image data as the region information.
The imaging apparatus according to claim 2. The signal processing means includes
Information extracting means for extracting information indicating the first pixel area and the second pixel area from the packet output from the output means;
Based on information indicating the first pixel region and the second pixel region, pixel data included in the first pixel region and the second pixel region are extracted from pixel data obtained from the packet, respectively. Extraction means to
Calculation means for calculating a correction value for correcting the pixel data included in the second pixel region using the pixel data included in the first pixel region;
The imaging apparatus according to claim 3. The area information includes at least one of a vertical coordinate range and a horizontal coordinate range of each area as information for specifying the pixels of the first pixel area and the pixels of the second pixel area. ,
The imaging apparatus according to any one of claims 1 to 4, wherein the imaging apparatus is characterized in that The packet generation unit divides the image data for each predetermined pixel data, and adds information indicating the first pixel region and the second pixel region to the divided image data as header information. ,
The imaging apparatus according to claim 3, wherein: The calculation means calculates the correction value based on an average value of pixel data of pixels specified by information indicating the second pixel region among the pixels of the second pixel region.
The imaging apparatus according to claim 4. The signal processing means subtracts the correction value from pixel data of a pixel specified by information indicating the first pixel area among the pixels of the first pixel area, thereby obtaining the second pixel area. Having correction means for correcting the output of the pixel;
The imaging apparatus according to claim 4. The pixels in the first pixel region are pixels in the effective pixel region, and the pixels in the second pixel region are pixels for correcting the output of the pixels in the effective pixel region.
The image pickup apparatus according to claim 1, wherein the image pickup apparatus is an image pickup apparatus. The pixels in the second pixel region are pixels in a light-shielded region.
The imaging apparatus according to claim 9. The pixel portion is included in a stacked imaging device in which a plurality of semiconductor chips are stacked on each other.
The image pickup apparatus according to claim 1, wherein the image pickup apparatus is an image pickup apparatus. A pixel portion having a first pixel region and a second pixel region;
Output means for adding image information obtained from the pixel unit to the image data obtained from the first pixel region and the second pixel region, and outputting the image data.
An image sensor characterized by the above. The output means includes packet generation means for generating a packet by adding the region information to the image data obtained from the pixel unit, and packet output means for outputting the packet.
The imaging device according to claim 12. The output means further comprises comparison means for comparing the area in the image obtained from the area information with the area of the image data to generate information indicating the first pixel area and the second pixel area. Have
The packet generation means generates a packet by adding information indicating the first pixel region and the second pixel region to the image data as the region information.
The imaging device according to claim 13. The area information includes at least one of a vertical coordinate range and a horizontal coordinate range of each area as information for specifying the pixels of the first pixel area and the pixels of the second pixel area. ,
The imaging device according to any one of claims 12 to 14, wherein The packet generation unit divides the image data for each predetermined pixel data, and adds information indicating the first pixel region and the second pixel region to the divided image data as header information. The image pickup device according to claim 13 or 14, The pixels in the first pixel region are pixels in the effective pixel region, and the pixels in the second pixel region are pixels for correcting the output of the pixels in the effective pixel region.
The image pickup device according to any one of claims 12 to 16, wherein the image pickup device is provided. The pixels in the second pixel region are pixels in a light-shielded region.
The imaging device according to claim 17. The image sensor is a stacked image sensor in which a plurality of semiconductor chips are stacked on each other.
The image pickup device according to any one of claims 12 to 18, wherein the image pickup device is provided. A method for controlling an imaging apparatus having a pixel portion having a first pixel region and a second pixel region,
An output step in which output means adds and outputs region information indicating the first pixel region and the second pixel region to the image data obtained from the pixel unit;
A signal processing step of correcting the pixel data of the first pixel region using the pixel data of the second pixel region with respect to the image data read from the pixel unit; ,
In the signal processing step, pixel data of the first pixel region and pixel data of the second pixel region in the image data are extracted using the region information added to the image data received from the output unit. To
And a method of controlling the imaging apparatus.   The program for functioning a computer as each means of the imaging device of any one of Claim 1 to 11.

Images