JPH08340455A - Image signal processor - Google Patents

Abstract

PURPOSE: To display a color image corresponding to the output video signal of a solid-state image pickup element on the display screen of a computer by generating respective pixel data on the basis of pixel signals from plural actual pixels in a non-grating array. CONSTITUTION: This image signal processor is equipped with a grating-array pixel data generating means which generates respective pixel data corresponding to virtual pixels on the basis of pixel signals from actual pixels Rn, Bn, and Gn of the three colors in corresponding nearly delta-shaped position relation in horizontal pixel arrays (e.g. L1 and L2, L3 and L4, and L5 and L6) that are paired adjacently in the vertical direction. In the virtual area that the actual pixels Rn, Bn, and Gn of the three colors in the nearly delta-shaped position relation on the basis of which the respective pixel data corresponding to the respective virtual pixels are calculated, this example shows that pixels in each virtual area or in mutually adjacence relation never overlap with one another. Thus, the respective pixel data are generated on the basis of the pixel signals from the actual pixels in the non-grating array positioned nearby the virtual pixels.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image signal processing apparatus for handling an image picked up by an applied image pickup device, and particularly to inputting a picked-up image to a computer (personal computer) for printing. The present invention relates to an image signal processing device having a function for

[0002]

2. Description of the Related Art A single-plate type color image pickup device to which a solid-state image pickup device such as an XY address system or a CCD is applied has been widely used. Generally, in a solid-state image pickup device applied to this type of image pickup device, pixels of each color, which is a photoelectric conversion unit area for obtaining a photoelectric conversion output corresponding to a filter color corresponding to each self, have periodicity in the horizontal direction. The horizontal pixel columns that are arranged to form a plurality of horizontal pixel columns and that are adjacent to each other in the vertical direction and that form a pair have horizontal pixel columns that correspond to the same filter color. By arranging the direction positions such that the horizontal width of one pixel is a unit pitch and is shifted by a half pitch, the pixels of three colors in the horizontal pixel row having the above-mentioned paired relationship are arranged. They are arranged so as to be close to each other and have a substantially delta-shaped positional relationship. A solid-state image sensor having such a structure is disclosed in, for example, Japanese Patent Laid-Open No. 1-274.
It is also disclosed in Japanese Patent No. 579.

Such three color filters (pixels)
The image sensor of the type in which the color filters are configured so as to form a delta positional relationship has Regarding the two-dimensional Nyquist limit, there is an advantage that it is excellent in isotropy. In addition, 3 of R, G, B
By arranging the color filters so as to have a delta-shaped positional relationship, it is possible to ensure uniformity in the number and arrangement of R, G, and B colors. Therefore, in this respect, it can be said that the solid-state image pickup device of the above method is suitable for an image input device that generally requires strict characteristics with respect to isotropicity of image reproduction and color reproducibility.

The resolution characteristics and the like of each typical color filter array system will be described in more detail below. FIG. 20 is a diagram showing a difference in Nyquist limit characteristic (limit characteristic of reproducibility) of image pickup due to a difference in color filter array method of a single-plate type color image pickup element. The figure (a) shows the vertical stripe arrangement, the figure (c) shows the mosaic arrangement, and the figure (e) shows the element applied to the invention of the delta (Δ) arrangement belonging to it. That is, in the image pickup device in the example of FIG. 20E, pixels of each color, which are photoelectric conversion unit regions, are arranged to obtain a photoelectric conversion output corresponding to the filter color corresponding to each self, and appear periodically with a horizontal direction. The horizontal positions of the pixels corresponding to the same filter color are arranged between the horizontal pixel columns which are arranged so as to form a plurality of horizontal pixel columns and which are adjacent to each other in the vertical direction and which form a pair. Half the horizontal width of one pixel is the unit pitch
By being arranged so as to have a pitch-shifted relationship,
Pixels of the three colors in the horizontal pixel row having the above-mentioned paired relationship are located close to each other to form a substantially delta-shaped positional relationship.

In the vertical stripe arrangement shown in FIG. 20A, filters of R, B, and G colors are vertically extended in a strip shape, and the R, B, and G colors are periodically arranged in the horizontal direction in this order. Therefore, the Nyquist limit characteristic (reproduction capability limit characteristic) is good for the corresponding colors of R, B, and G in the vertical direction, but only every third pixel in the horizontal direction. The Nyquist limit characteristic is inferior because the actual output of the corresponding color cannot be obtained. Such a Nyquist limit characteristic is represented by a well-known reciprocal lattice notation method (see Journal of the Television Society: Vol.46, No.5 (1992) pp.615-623).
Is shown in FIG. 20 (b). In each drawing, “d” is the distance between the centers of adjacent pixels, and
In each of FIGS. 20B, 20D, and 20F, a rectangular or polygonal closed diagram is a resolution Nyquist limit region for each corresponding color filter array system. This diagram shows the Nyquist frequencies μN and νN for the μ and ν components (vertical stripes and horizontal stripes) of the reproducible spatial frequency and the ratio (fMAX / fMIN) of the maximum and minimum components of the Nyquist frequency in all directions. is there. FIG.
As is easily understood by comparing the drawings of (b), FIG. 20 (d), and FIG. 20 (f), the delta (Δ) array exhibits the most excellent isotropic property.

On the other hand, when the image picked up by the image pickup device is used for printing, generally, primary R, B, G color signals are converted into secondary yellow, cyan, magenta, and black signals and printed. A version is created for.
Assuming that such signal conversion is performed, it is desirable that the primary R, B, and G color signals have the same S / N and data amount for each color, and the resolution is isotropic. Is desirable. Further, it goes without saying that a high image resolution is required. From the above perspective,
The vertical stripe array of FIG. 20 (a) lacks the isotropy of resolution, and the mosaic array of FIG. 20 (c) lacks each component of R and B with respect to the G component, and is still lacking in isotropic property. However, it can be said that the delta (Δ) array of FIG. 20 (e) is the method most suitable for this purpose.

[0007]

On the other hand, on the other hand,
Regarding the capture of a captured image into a computer (personal computer) and the electronic image in a digital prepress for printing plate making, it is generally required that the image data corresponds to a square pixel (pixel having a square lattice array). It costs. Therefore, it is not possible to directly apply the image pickup device of the type in which the three color pixels are arranged in a delta positional relationship as described above as an element for obtaining an electronic image to be taken into a computer. . In JP-A-1-274579, the horizontal positions of pixels corresponding to the same filter color are arranged so as to be offset by a half pitch with the horizontal width of one pixel as a unit pitch. For an image sensor in which three color filters have a delta-shaped positional relationship, the phase shift of the switching pulse for dot-sequential pixel output for each interlaced field is shifted by 180 degrees to achieve horizontal resolution. Although a technique for obtaining a high luminance signal is disclosed, there is no suggestion of a problem recognition or a means therefor for making output image data suitable for being taken into a computer.

The invention of the present application has been made in view of the above, and pixels of each color, which is a photoelectric conversion unit area for obtaining a photoelectric conversion output corresponding to a filter color corresponding to each self, are periodically cycled in the horizontal direction. Of the pixels corresponding to the same filter color are arranged between the horizontal pixel columns that are arranged to form a plurality of horizontal pixel columns that are aligned with each other and that are adjacent to each other in the vertical direction to form a pair. Pixels of three colors in the horizontal pixel row having the above-mentioned paired relationship are arranged so that the horizontal positions are offset by a half pitch with the horizontal width of one pixel as a unit pitch. It is possible to obtain image data corresponding to square pixels suitable for being taken into a computer, while applying image pickup devices configured so that they are located close to each other and have a substantially delta-shaped positional relationship. And to provide this kind of an image signal processing apparatus to so that.

[0009]

In order to achieve the above object, the image signal processing device of the present invention has one broad expression: photoelectric conversion outputs corresponding to filter colors corresponding to each self. A solid-state imaging device configured so that pixel signals from a photoelectric conversion surface in which pixels of each color, which are photoelectric conversion unit areas, are two-dimensionally arranged in a non-lattice pattern are sequentially read in a predetermined manner. And apply
An image signal processing device adapted to obtain a color image signal based on the output of the solid-state image pickup device, wherein each virtual image space of the virtual image space corresponds to one assuming that the pixel array of the solid-state image pickup device is a lattice array. Each pixel data corresponding to the pixel output of the one color at the pixel position is generated based on the pixel signals from the plurality of real pixels in the non-lattice array located in the vicinity of the virtual pixel. It is characterized by comprising a data generating means ... (1) According to one limiting expression: photoelectric conversion output adapted to obtain a photoelectric conversion output corresponding to a filter color corresponding to each self. Horizontal pixels that are arranged so as to form a plurality of horizontal pixel rows in which pixels of each color that are unit areas are arranged so as to appear with periodicity in the horizontal direction and that are adjacent to each other in the vertical direction to form a pair. The columns are arranged such that the horizontal positions of the pixels corresponding to the same filter color are offset by a half pitch with the horizontal width of one pixel as a unit pitch. Applying a solid-state image sensor configured such that pixels of three colors in a horizontal pixel row having a relationship of ??? An image signal processing device adapted to obtain a color image signal based on the one color at each virtual pixel position in a virtual image space corresponding to the one assuming that the pixel array of the solid-state image sensor is a lattice array. For generating pixel data corresponding to the pixel output of the above-mentioned pixel signals based on the pixel signals from the corresponding three-color real pixels in the above-mentioned paired horizontal pixel rows and in the above-mentioned approximate delta positional relationship. Pixel array (2) Furthermore, according to another limited expression: photoelectric conversion output corresponding to the filter color corresponding to each self is obtained. Horizontal pixel columns that are arranged so as to form a plurality of horizontal pixel columns in which pixels of each color that are photoelectric conversion unit regions are arranged so as to appear with periodicity in the horizontal direction, and are paired adjacent to each other in the vertical direction. Since the horizontal positions of the pixels corresponding to the same filter color are arranged so as to be offset from each other by a half pitch with the horizontal width of one pixel as a unit pitch, the above pairs are arranged. A solid-state image sensor configured such that pixels of three colors in a horizontal pixel row having a relationship of being formed are located close to each other to form a substantially delta-shaped positional relationship, and based on the output of the solid-state image sensor, Color image In the image signal processing device configured to obtain the solid-state image pickup device, the center position of each virtual pixel of the virtual image space corresponding to the one assuming that the pixel array of the solid-state image pickup device is a grid-like array is the solid-state image pickup device. Of these virtual pixels, assuming that they are located at the positions corresponding to the four corners of each real pixel belonging to each of the even-numbered or odd-numbered horizontal pixel columns of each horizontal pixel column of the element. Each pixel data corresponding to the output is generated by directly or indirectly depending on the pixel signal from each corresponding real pixel in the horizontal pixel row having the above-mentioned paired relationship, and thereby the total number of the real pixels is approximately It is characterized in that it is provided with a grid-like array pixel data generation means adapted to obtain data representing three times the number of virtual pixels ... (6) As another limited configuration: each To self Pixels of each color, which is a photoelectric conversion unit area adapted to obtain a photoelectric conversion output corresponding to the corresponding filter color, are arranged so as to form a plurality of horizontal pixel rows which are arranged so as to appear with periodicity in the horizontal direction, In addition, between the horizontal pixel columns that are paired vertically adjacent to each other, the horizontal position of the pixel corresponding to the same filter color is halved with the horizontal width of one pixel as a unit pitch. By arranging so as to have a pitch-shifted relationship, the pixels of the three colors in the pair of horizontal pixel rows are positioned close to each other to form a substantially delta-shaped positional relationship. Of the solid-state imaging device, and a color image signal is obtained based on the output of the solid-state imaging device, wherein the pixel array of the solid-state imaging device is assumed to be a lattice array. Each pixel data corresponding to the photoelectric conversion output of the one color at each virtual pixel position in the corresponding virtual image space is converted into three real pixels of the same color closest to each other in vertically adjacent three horizontal pixel rows. It is characterized in that it is provided with a grid-like array pixel data generating means for generating it based on the pixel signals from (18).

[0010]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be clarified by describing the embodiments of the present invention in detail with reference to the drawings. FIG.
FIG. 3 is a diagram showing a pixel array of an image sensor (solid-state image sensor) applied to an embodiment of the present invention. As shown in the figure, the array of color filters in this element is, for example, each odd row (each odd-numbered pixel column extending in the horizontal direction) 10,1.
2, ... Pixel array in the order of R → B → G is cyclically repeated, and in each even-numbered row 11, ... Therefore, the pixel arrangement in the order of R → B → G is cyclically repeated. The above configuration is realized by coating each unit area of photoelectric conversion of the image sensor with a filter of a corresponding color corresponding to the openings.
Thus, in the completed form, in the solid-state imaging device, the pixels of each color, which are photoelectric conversion unit regions, are arranged to obtain a photoelectric conversion output corresponding to the filter color corresponding to each self, and have periodicity in the horizontal direction. The horizontal positions of pixels corresponding to the same filter color are arranged between the horizontal pixel columns which are arranged so as to form a plurality of horizontal pixel columns and which are paired adjacent to each other in the vertical direction. However, with the horizontal width of one pixel as the unit pitch, it is halved.
By being arranged so as to have a pitch-shifted relationship,
Pixels of the three colors in the horizontal pixel row having the above-mentioned paired relationship are positioned close to each other to form a substantially delta-shaped positional relationship.

The solid-state image pickup device of FIG. 1 is an XY address type image pickup device, and in order to identify and read out the photoelectric conversion output from each pixel, switching is performed corresponding to each horizontal pixel column. Element groups 9a, 9b, 9c are provided, and these switching element groups 9a, 9b, 9c are selectively driven by the horizontal shift register 13 and the vertical shift register 14 to sequentially read each pixel. The output of the image sensor by this reading is output from the output lines l1 and l2 corresponding to each odd-numbered pixel column and each even-numbered pixel column.

Next, the timing of reading the signal from the image pickup device and the structure for reading the signal will be described with reference to FIGS. 2 and 3. FIG. 2A is a diagram showing a pixel array of the image pickup device and a distribution of virtual pixels which will be described in detail later. FIG. 2B shows the virtual pixels in FIG. 2A from the image pickup device. FIG. 3A is a diagram showing the timing of deriving the signal (which is output in the form of digital data) shown in FIG.
FIG. 3B is a diagram showing an example of a grid-shaped array pixel data generation unit that is a configuration for deriving pixel data at the timing of FIG. 3B, and FIG. It is a figure which shows the example implemented mainly.

In FIG. 2A, for the sake of convenience of explanation, sequential horizontal pixel columns (rows) are represented in abbreviated form as long as L1, L2, L3, L4, L5, and L6. There are many more pixel columns. As a matter of course, the number of pixels of each color in the horizontal direction is also expressed in a simplified form as long as it is shown. Figure 2
The black dots in (a) are indices indicating each virtual pixel position (center position of the virtual pixel) of the virtual image space corresponding to the one assuming that the pixel array of the solid-state image sensor is a grid array. Is. In this embodiment, the pixel data corresponding to each of these virtual pixels are paired with each other in the horizontal direction so as to form a pair of horizontal pixel columns (for example, L1 and L2;
L5 and L6) are provided with the lattice array pixel data generating means of FIG. 3 which is generated based on the pixel signals from the corresponding three-color actual pixels Rn, Bn, Gn in the above-mentioned approximate delta positional relationship. . Attention is paid to the virtual area occupied by the three-color real pixels Rn, Bn, Gn in the above-mentioned approximate delta positional relationship, which is the basis for calculating each pixel data corresponding to each virtual pixel shown in FIG. 2A. Then, in this example, the virtual areas do not overlap with each other even though they are adjacent to each other. The structure of the grid-like array pixel data generating means of FIG. 3A and the operation of reading out each pixel data corresponding to each virtual pixel indicated by a black dot in FIG. 2A at the timing of FIG. 2B. About FIG. 3 (a) and FIG.
A description will be given based on (b). The actual pixel outputs from the horizontal pixel columns (for example, L1 and L2) that form a pair of the image pickup device 110 are amplified by the preamplifiers 101a and 101b, respectively, and then the process amplifiers 102 correspondingly provided.
a and 102b are processed to form a normal image signal corresponding to each color pixel.
The outputs of 2a and 102b are A / D converters 103a and 1 respectively.
In 03b, it is converted into digital pixel data. Each A / D
The pixel data of the converters 103a and 103b is as shown in FIG.
As shown in the data derivation timings in the upper two rows, the sequential pixel data expression timings along the horizontal scanning are output in a synchronized relationship between both lines (L1, L2). Thus, the output of one A / D converter 103a is supplied to one input of the selector 105 without additional delay. The other A / D converter 1
Regarding the output of 03b, the latch pulse of the same frequency f as the frequency of each real pixel read (from the top of FIG.
The timing is shown on the row) and the delay is applied by being latched by the latch circuit 104, and the selector is formed in the form shown in the row labeled "L2 after latch" in FIG. 2B. Each is fed to the other input of 105. In the selector 105, the switching pulse of the frequency 2f (see FIG.
Based on (b), the timing of its expression is shown in the fifth row from the top), the two inputs are switched so as to be alternately output. ), The dot-sequential pixel data output is obtained (the timing of its development is shown in the bottom row of FIG. 2B). This dot-sequential pixel data is each pixel data corresponding to each virtual pixel indicated by a black dot in FIG. In the grid-shaped array pixel data generation means of FIG. 3A,
Further, for the dot-sequential pixel data as described above, the frequency 2f that arrives at each intermediate point of each pixel data output period
By latching the dot-sequential pixel data by the latch pulse of, the final dot-sequential pixel data output is obtained. This is to obtain a stable pixel data output while avoiding a period in which the level disturbance may occur due to a transient phenomenon of signal switching in the selector 105.

FIG. 3B is a diagram showing an example in which the grid array pixel data generating means having the same meaning as in FIG. 3A is embodied mainly by an analog circuit. The circuit shown in FIG. 2 is a circuit for outputting actual pixel outputs (for example, L1 and L2 that are output in synchronization) from horizontal pixel rows that form a pair with the image sensor 110.
As for L1, the delay circuit 11 is used as it is for L2.
The signal is delayed by a period corresponding to 1/2 pixel by 1 and supplied to the input of the analog switch 112 (changeover switch). In this analog switch 112, both inputs have a frequency 2f (that is, twice the actual pixel read frequency). Frequency) to selectively output. The output signal of the analog switch 112 is already a dot-sequential pixel signal having a frequency of 2f, and the dot-sequential pixel signal is further converted into a normal-level analog dot-sequential signal by the preamplifier 113 and the process amplifier 114 in the subsequent stage. After being converted into pixel signals, they are converted into digital pixel data by the A / D converter 115.

FIGS. 4, 5, 6 and 7 are views for explaining another embodiment of the present invention, and FIG. 4 uses the same notation as in FIG. 2 (a) described above. FIG. 5 and FIG. 6 are schematic views showing the positional relationship between the real pixels of a so-called Δ array and the virtual pixels of the lattice array in the image sensor applied to the invention, and FIGS. 7 is a block diagram showing the configuration of the apparatus.

As is well understood with reference to FIG. 4, in this example, for each virtual pixel in the virtual image space corresponding to the one assuming that the pixel array of the solid-state image sensor is a grid-like array, their respective center positions are , Even-numbered (L2, L4, L6) or odd-numbered (L1,
Each virtual pixel is set on the assumption that it is located at a position corresponding to the four corners of each real pixel belonging to each horizontal pixel row corresponding to one of L3 and L5). In this example, as described in detail below, each pixel data corresponding to the output of each virtual pixel (each pixel data corresponding to the output of the virtual pixel of each virtual horizontal pixel row La, Lb, Lc, Ld, ...). Is a pair of horizontal pixel columns (for example, L1 and L2; L3 and L4;
L5 and L6) are generated by directly or indirectly generating pixel signals from corresponding real pixels in L5 and L6) to obtain data representing a number of virtual pixels corresponding to the total number of real pixels. An array pixel data generation means is provided.

As shown in FIG. 4, the pixel data of the virtual horizontal pixel row La are paired horizontal pixel rows L1 and L.
2 is calculated from the corresponding real pixels belonging to 2 and, for example, the virtual pixel located on the leftmost side of the virtual horizontal pixel row La is a real pixel of three colors R1 and B having a substantially delta positional relationship.
These virtual areas R1, B1, G1 occupied by 1, G1
Of the actual virtual pixel data of the same virtual horizontal pixel row La, and the second virtual pixel located from the left side of the same virtual horizontal pixel row La is a virtual region occupied by the real pixels B1, R2, G2 of three colors in a substantially delta positional relationship. Is calculated based on the real pixel data of B1, R2, and G2, and similarly, the virtual horizontal pixel row La is calculated.
The third virtual pixel from the left side of the is calculated based on the real pixel data of these G2, R3, and B2 in the virtual area occupied by the real pixels G2, R3, and B2 of the three colors that have a substantially delta positional relationship. In the same manner, each virtual pixel data of the virtual horizontal pixel row La is calculated in the same manner. The virtual pixel located second from the left in the virtual horizontal pixel row Lb has three color real pixels G1, which are in a substantially delta positional relationship.
These virtual areas G1, Ra, B occupied by Ra, Ba
The virtual pixel which is calculated based on the actual pixel data of a and is located at the third position from the left side of the same virtual horizontal pixel row Lb is the real pixels R2, Ba, Gb of three colors which are in a substantially delta positional relationship.
Is calculated based on these real pixel data of R2, Ba, Gb in the virtual area occupied by
The fourth virtual pixel from the left side of b is calculated based on the real pixel data of these virtual pixels B2, Gb, Rc of the virtual area occupied by the real pixels B2, Gb, Rc of three colors which are in a substantially delta positional relationship. Then, each virtual pixel data of the virtual horizontal pixel row Lb is calculated in the same manner. Calculation of such virtual pixel data is performed by referring to FIG.
Each successive virtual horizontal pixel row Lc set below b,
The same applies to Ld, ... The virtual area (R1, B) occupied by the real pixels R1, B1, G1 of each three colors which is the basis of the calculation of each virtual pixel data described above with reference to FIG.
1, G1; B1, R2, G2; G2, R3, B2; ...; ...), when expressing the virtual pixel data calculating means, the horizontal pixel rows that form a pair with the solid-state image sensor ( For example, L
1 and L2; L3 and L4; L5 and L6) which are adjacent to each other and have the corresponding three color real pixels (R1, B1, G).
1; B1, R2, G2; G2, R3, B2; ...; ...), and each virtual area occupied by three real pixels (for example, R1, B1, G1) forming one virtual area. The real pixel (B1) on one side and the real pixel (B1) on the other side of the three real pixels (B1, R2, G2) forming the adjacent region of the one virtual region (B1)
) And the same pixel (B1) are assumed to be superposed on each other, the corresponding 3 pixels are regarded as corresponding to the virtual pixel located substantially at the center of each of these virtual regions.
It can be said that each of the virtual pixel data in the grid-like array is generated based on the pixel signal from the real pixel of the color.

FIG. 5 shows horizontal pixel rows L1 and L which are paired with each other for each virtual pixel data of the virtual horizontal pixel row La.
FIG. 7 is a timing diagram in the process of generating from each applicable real pixel belonging to 2, and FIG. 7 is a block diagram showing a configuration for performing this process.

As shown in FIG. 7, the actual pixel outputs of the adjacent two horizontal lines (L1, L2) derived from the solid-state image pickup device 201 are amplified by the preamplifiers 202a and 202b provided corresponding to each, and then the corresponding pixel outputs correspond to each other. Are processed by the provided process amplifiers 203a and 203b to form regular image signals corresponding to the pixels of the respective colors, and the outputs of these process amplifiers 203a and 203b are respectively converted into A / D converters 204.
a and 204b are converted into digital pixel data. Pixel data of each A / D converter 204a, 204b is shown in FIG.
As shown in the data generation timings of the upper two rows, the sequential pixel data expression timings along the horizontal scanning are output in a synchronized relationship between both lines (L1, L2). Thus, the output of one A / D converter 204a is separated into two systems, and one system is supplied to the first input of the selector 207 without additional delay and the other system. In the case of the two-stage latch circuits 205a and 206a connected in cascade, the latch pulses P1 and P having the same frequency f as the frequency of each actual pixel read
2 (as shown in the third and fifth rows from the top in FIG. 5), the latch pulses P1 and P2 are out of phase with each other by 1/2 pixel. A delay corresponding to 1/2 pixel is applied and each is supplied to the second input of the selector 207 in the form shown in the row labeled "L1 delay by double latch" in FIG. Also,
On the other hand, for the output of the A / D converter 204b, the cascaded two-stage latch circuits 205b and 206b
Each of the above-mentioned latch pulses P1 and P2 causes a delay corresponding to 1/2 pixel, and "L2 delay by twice latching" in FIG.
It is supplied to the third input of the selector 207 in the form represented by the line marked with. The selector 207 receives the above-mentioned latch pulse P2 as a switching pulse and outputs the switching pulse P2.
Based on 2, the first, second, and third inputs are sequentially switched to be output, and as a result, B pixel data is always in the B system on the output side and R pixel is always in the R system. The pixel data of G is always in the system of G, the pixel data of G is B, R,
The G lines are synchronized and output (the lower three rows in FIG. 5). The output data of the B, R, and G systems are sequentially stored in the memories 209, 210, and 211 provided correspondingly. In FIG. 7, a block 212 shown by a broken line constitutes a central functional block of the grid-shaped pixel data generating means in this embodiment. Each of the virtual pixel data of the virtual horizontal pixel row La shown in FIG. 4 is a horizontal pixel row L having a paired relationship.
The processing for generating from the corresponding real pixels belonging to 1 and L2 is performed as described above, but the respective virtual pixel data of the virtual horizontal pixel row Lb is paired with the horizontal pixel rows L2 and L3. The process of generating from each corresponding real pixel to which it belongs is generally the same, but is partially different. That is,
The corresponding real pixel data belonging to the paired horizontal pixel columns L2 and L3 are both of the lines (L
However, in the case of the pair of lines L2 and L3, different from the above description for the pair of L1 and L2, only the one line L2 is output by the 1H delay circuit 208. 1 horizontal scanning period (1
H) It is supplied to the above-mentioned function block 212 as shown in FIG. In the functional block 212, as shown in FIG. 6, pixel data (first input to the selector) obtained by further delaying the line L2 delayed by 1H by 1 pixel (1 clock), the pixel of line L3. Data (second input to selector) and pixel data of three systems (third input to selector) of pixel data obtained by further delaying this line L3 by one pixel (1 clock) are generated. The pixel data of the system is switched by the selector as described above for the pair of L1 and L2, and as a result, the B system on the output side is always B
The pixel data of R is always R pixel data in the R system,
G pixel data is always output to the G system, and B, R,
The G lines are synchronized and output (the lower three rows in FIG. 6).

FIGS. 8, 9, 10 and 11 are views for explaining still another embodiment of the present invention, and FIG. 8 uses the same notation as the above-mentioned FIG. 2 (a). A so-called Δ array of real pixels in an image sensor applied to the invention, virtual pixels of a lattice array shown by black dots, and a second virtual pixel generated based on data of each first virtual pixel shown by these black dots 8 is a schematic view showing the positional relationship of pixel positions (x) corresponding to the data of each virtual pixel of FIGS. 9 and 10, and FIG. 9 and FIG. FIG. 11 is a block diagram showing the configuration of the apparatus.

As can be understood by comparing FIG. 8 with FIG. 2A, the first virtual pixels indicated by black dots in FIG. 8 are the same as those in FIG. 2A. . Therefore, the data of each first virtual pixel shown by these black dots in FIG.
It is the dot-sequential pixel data in which the timing of its appearance is shown in the bottom row of (b). In the present embodiment in which the configuration of the device is shown in FIG. 11, based on the data of the first virtual pixels indicated by these black dots, the timing of these data is shown in FIGS. 9 and 10. By performing the image data processing, the data of the virtual pixel indicated by “x” in FIG. 8 is obtained as the data of each second virtual pixel. The image data processing will be described below. What is shown in the first row of FIG. 9 is dot-sequential pixel data whose expression timing is shown in the bottom row of FIG. 2 (b). In FIG. 8, horizontal pixel columns (rows) of actual pixels are shown. It belongs to a virtual pixel column (row) LA consisting of the first virtual pixels located between L1 and L2.
The dot-sequential pixel data (first virtual pixel data) shown in the first row is delayed by one clock (duration of each color data output each time), and the dot-sequential pixel data (shown in FIG. 9). Second row) and dot-sequential pixel data delayed by two clocks (third row in FIG. 9), respectively. Next, the point-sequential pixel data of the first row, the second row, and the third row of FIG. 9 described above are sampled by a sampling pulse having a 2-clock cycle as shown in the fourth row of FIG. Point-sequential pixel data of 2 clock cycles as shown in the 5th, 6th, and 7th rows of FIG. 9 are obtained. Further, the point-sequential pixel data of 2 clock cycles shown in the fifth row, the sixth row, and the seventh row are sequentially switched by the selector means and output, whereby the eighth row and the ninth row in FIG. As shown in the row and the 10th row, R pixel data is always in the R system, and G pixel data is always in the G system.
B pixel data is always output to the system of B, R,
G Each system is synchronized and a synchronized output of two clock cycles is obtained. The data shown in the 8th, 9th, and 10th rows of FIG. 9 is the data of each of the above-mentioned second virtual pixels. The second virtual pixel P1 is a second virtual pixel P2 by R2, G2, B1 and the second virtual pixel P is a second virtual pixel P by R3, G2, B2.
3 by R3, G2, B3 causes the second virtual pixel P4 to be R
A virtual pixel P5 is represented by 4, G4 and B3. These second virtual pixels P1 to P5 are the second virtual pixels belonging to one horizontal pixel column (row) La located between the horizontal pixel columns (rows) L1 and L2 of the actual pixels in FIG. is there.

On the other hand, in FIG. 8, horizontal pixel rows (rows) of actual pixels
Another horizontal pixel column (row) L located between L3 and L4
Image data processing for generating data of the respective second virtual pixels Pa, Pb, Pc, Pd, ... belonging to b is performed at the timing shown in FIG.

As shown in FIG. 10, the dot-sequential pixel data (first virtual pixel data) belonging to the virtual pixel column (row) LA is represented by one horizontal scanning period ( 1
H) The delayed pixel data and the dot-sequential pixel data obtained by delaying the dot-sequential pixel data shown in the first row by one clock as shown in the second row are obtained. What is shown in the third row of FIG. 10 is a virtual pixel column (row) LB located between horizontal pixel columns (rows) L3 and L4 of actual pixels in FIG.
Is the dot-sequential pixel data of each of the first virtual pixels belonging to.
The dot-sequential pixel data (FIG. 10) obtained by delaying the dot-sequential pixel data of each first virtual pixel belonging to this virtual pixel column (row) LB by one clock (duration of each color data output). (4th row) of FIG. 10 is further delayed by 1 clock, and consequently the dot-sequential pixel data of the original virtual pixel is delayed by 2 clocks to obtain dot-sequential pixel data (5th row of FIG. 10). Next, the point-sequential pixel data of the second row, the third row, and the fifth row of FIG. 10 described above are sampled by a sampling pulse of 2 clock cycles as shown in the sixth row of FIG. Of the dot-sequential pixel data of the second row, third row, and fifth row of FIG. 10, the data with black dots are extracted in time series, and the seventh row, eighth row, and Each dot-sequential pixel data of 2 clock cycles as shown in the 9th row is obtained. Further, the point-sequential pixel data of 2 clock cycles shown in the seventh row, the eighth row, and the ninth row are sequentially switched by the selector means and output, so that FIG.
As shown in the tenth row, the eleventh row, and the twelfth row of R, the R system always has R pixel data, the G system always has G pixel data, and the B system has The pixel data of B is always output, and the B, R, and G systems are synchronized to obtain a synchronized output of two clock cycles. First of FIG.
The data shown in the 0th row, the 11th row, and the 12th row are the data of the above-mentioned second virtual pixels, and, for example, the second virtual pixel is obtained by Ra, G1, Ba of this synchronized output. Pa
However, the second virtual pixel Pb becomes Rc,
The second virtual pixel Pc is changed to Rc, G3, Bc by Gb, B2.
Will cause each second virtual pixel Pd to be represented. .. belong to one horizontal pixel column (row) Lb located between the horizontal pixel columns (rows) L2 and L3 of the actual pixels in FIG. It is a pixel.

As will be understood from the above, the second horizontal pixel columns (rows) P1 to P5 which are adjacent to each other and have a positional relationship.
And the virtual pixel data of Pa to Pd are as shown in FIG. 9 or FIG.
The image data processing of FIG. 9 or FIG. 10 is performed in the same manner as the second horizontal pixel row (row) adjacent to each of the second virtual pixel data in one screen. Can be generated by

FIG. 11 has a functional section for performing the image data processing of FIGS. 9 and 10, and as a result, the virtual pixel data indicated by "x" in FIG. 8 is obtained as the data of each second virtual pixel. FIG. 3 is a configuration diagram of a system for In FIG. 11, the incident light that has passed through the lens system 301 for imaging has a diaphragm 302.
The light amount is adjusted by so that an image is formed on the photoelectric conversion surface of the solid-state image sensor 303. Solid-state image sensor 3
Numeral 03 is a photoelectric conversion unit area for obtaining the photoelectric conversion output corresponding to the filter color corresponding to each self, as described above with reference to FIG. 1, and arranged so that the pixels of each color appear with periodicity in the horizontal direction. The horizontal position of pixels corresponding to the same filter color is 1 pixel between the horizontal pixel columns which are arranged so as to form a plurality of horizontal pixel columns and which are adjacent to each other in the vertical direction to form a pair. By arranging the horizontal width as a unit pitch so as to be offset by a half pitch, the pixels of the three colors in the horizontal pixel row having the above-mentioned paired relationship are positioned close to each other. Form an approximately delta-shaped positional relationship, and the output of this image sensor is supplied to the first image data processing block 304. This first image data processing block 304 is shown in FIG.
The dot-sequential virtual pixel data as shown in the bottom row of (b) is generated in an analog manner, and each corresponding three-color real pixel in the above-described delta-shaped positional relationship of the solid-state image sensor 303. The virtual area occupied by the virtual areas corresponding to the virtual pixels located substantially at the center of each virtual area when the mutual positional relationship is assumed so that the virtual areas do not have portions overlapping with each other. Each applicable 3
A first virtual pixel data, which is each virtual pixel data of a grid-like array, is generated based on a pixel signal from a color real pixel;
It forms a grid array pixel data generation means. The output of the first image data processing block 304 is the amplification processing unit 305.
A / D conversion circuit 30 after being converted into a prescribed form through
6 is converted into dot-sequential virtual pixel data in digital form as shown in the bottom row of FIG. 2 (b), and thus in the first row of FIG. In this embodiment, the output of the A / D conversion circuit 306 is gradation-converted by a highlight balance circuit 307a, a shadow balance circuit 307b, and a look-up table (LUT) processing circuit 307c which are cascade-connected in the above-described order. It is configured to be supplied to the unit 307 and to be subjected to required gradation conversion processing. On the other hand, a focus control block for performing focusing adjustment on the above-mentioned lens system 301 and an aperture control block 309 for adjusting the aperture of the aperture 302 are provided. These focus control block, aperture control block 309, highlight balance circuit 307a of the gradation conversion unit 307, shadow balance circuit 307b, and lookup table (LUT).
The processing circuit 307c is configured to be integrally controlled by the micro computer 310. Gradation converter 3
In the highlight balance circuit 307a of 07, the R, G, and B levels of the highlight part (the brightest part) of the subject are adjusted (the R, G, and B levels for white are made uniform), and the shadow balance circuit. At 307b, the R, G, and B levels of the shadow portion (darkest portion) of the subject are adjusted. Note that the highlight balance and the shadow balance may be adjusted by adjusting which position of the density point the gradation data of the captured image is associated with. The look-up table (LUT) processing circuit 307c performs a two-dimensional density curve conversion process on the gradation data of the captured image. As this conversion, a linear or non-linear conversion is performed as required. Applied. Look-up table (L
The output data of the (UT) processing circuit 307c is supplied to the square pixel conversion circuit 311, and the input pixel data is subjected to sampling processing (pixel thinning processing). A switch means 312 is provided for selectively extracting the data processed by the square pixel conversion circuit 311, or the output data itself of the look-up table (LUT) processing circuit 307c which is not processed by the square pixel conversion circuit 311, and the switch means 312 is provided. Also performs a switching selection operation under the control of the micro computer 310. Switch means 3
A recording device 313 applying a magneto-optical recording medium or the like and a transfer signal conversion circuit 314 are connected in parallel to the output side of 12. When the switch means 312 is switched and selected to one side, the output data itself of the look-up table (LUT) processing circuit 307c is supplied to the recording device 313 and the transfer signal conversion circuit 314 and switched and selected to the other side. In the meantime, the data processed by the square pixel conversion circuit 311 is supplied. Output data from the recording device 313 and the transfer signal conversion circuit 314 is supplied to the personal computer 315.

In this embodiment, the personal computer 315 is configured to perform the image data processing shown in FIGS. 9 and 10, and therefore the personal computer 31 is used.
Reference numeral 5 designates the center position of each virtual pixel in the virtual image space corresponding to the one assuming that the pixel array of the solid-state imaging device is a grid-like array in an even-numbered or odd-numbered position of each horizontal pixel column of the solid-state imaging device. By assuming the positions corresponding to the four corners of each real pixel belonging to each one of the horizontal pixel rows corresponding to either one, it is possible to correspond to the output of these virtual pixels having a larger capacity than the first virtual pixel data. A second lattice-shaped array pixel configured to generate the second virtual pixel data, which is each pixel data, based on the first virtual pixel data generated by the first lattice-shaped pixel data generation means. It is a data generating means.

The personal computer 315 generates data of each virtual pixel (second virtual pixel data) indicated by "x" in FIG. 8 obtained by performing the image data processing of FIG. 9 and FIG. Based on the generated data, yellow, cyan, magenta, and black data are generated and supplied to the printer 316 connected to itself. The printer 316 executes the printing operation based on the supplied data. The personal computer 315 can function as a host computer or a supervisor with respect to the micro computer 310, and the personal computer 315 tells the micro computer 310 based on an operation performed by the scanner on the personal computer 315. The two are connected so that commands can be given regarding execution of shooting and signal processing. Further, in the above configuration, the square pixel conversion circuit 31 described above is used.
As shown in FIG. 12, in the sampling processing (pixel thinning processing) of the input pixel data by 1, the virtual pixels in the grid square array as shown by the black dots in FIG. Each width in the vertical direction is the unit size,
Each point separated by a distance of 6n (n is a natural number) in each of the horizontal and vertical directions is sampled (the point shown as a large black dot in the figure is a sampled point), that is, a large black dot α, This is a thinning-out process in which virtual pixels of β, γ, δ, ... Are left and other small black dot pixels are discarded. It should be noted that the sampling (thinning-out process) interval is changed to be expressed based on the horizontal and vertical widths of the actual pixel as described above, and a grid pattern as shown by black dots in FIG. Expressed with each virtual pixel position in the square array as a reference, when viewed from a virtual pixel at one sampling point (for example, a pixel having a large black dot α), every fourth pixel in the horizontal direction and the third pixel in the vertical direction. It can be said that virtual pixels are sampled for each. The output as a result of the above-described processing performed by the square pixel conversion circuit 311 becomes coarse pixel data in which the number of pixels is smaller than that which is not yet subjected to this processing. Therefore, if this coarse pixel data is derived by the switch means 312, the amount of data to be transferred to the personal computer 315 will be small, and therefore relatively coarse image data will be transferred to the personal computer 315 in an extremely short time, and a temporary image will be transmitted. It becomes possible for the personal computer 315 to display an image with a display density (image quality) suitable for confirmation. Even when the image data is stored in the recording medium of the recording device 313, the recording can be completed in a short time by performing this for the relatively rough image data. The means for obtaining image data of a degree suitable for tentatively confirming the image by the relatively coarse image data constitutes image display data generating means, and the means for confirming the image by this means serves as preview means. There is. An apparatus provided with these means represents a preview means capable of confirming the state of an image to be recorded by an image of a relatively low resolution by the image display data generating means, and the image confirmed by the preview means. The image signal processing device can be configured to include a recording unit that records the output data of the lattice-shaped array pixel data generating unit.

The system of FIG. 11 is characterized in that a relatively small number of pixel data is generated by the first image data processing block (first grid array pixel data generating means) 304. , Gradation conversion unit 307
Conversion processing by a personal computer 315
To perform image data writing processing to a recording medium applied to the recording device 313.
These processes can be performed in an extremely short time. Then, based on the virtual pixel data (first virtual pixel data) that has been subjected to these processes, virtual pixel data having a larger capacity than this (second virtual pixel data: indicated by "x" in FIG. 8). Since it is configured to generate the data of each virtual pixel), it is possible to efficiently perform the above-mentioned various processes for relatively small capacity data, and then the information by this process is included. Large-capacity second virtual pixel data is generated based on each of the first virtual pixel data thus obtained, which is different from the above-described various processes for the large-capacity second virtual pixel data from the beginning. It is remarkably excellent in overall processing efficiency and realizes high-speed processing.

In the system shown in FIG. 11, the comparatively small number of virtual pixel data generated by the first image data processing block (first grid array pixel data generating means) 304 is used as shown in FIGS. 9 and 10. The image data processing is performed to generate the data of each virtual pixel (second virtual pixel data) indicated by “x” in FIG.
Although the personal computer 315 is applied as the grid-shaped array pixel data generating means, the hardware-based means can be applied as the second grid-shaped array pixel data generating means.

FIG. 13 is a block diagram showing an example of a signal processing circuit as the second grid-shaped array pixel data generating means. In FIG. 13, the input terminal 351 has the same structure as that shown in FIG.
The dot-sequential pixel data (first virtual pixel data) is supplied at the timing shown in the first row of FIG. The supplied data is the dot-sequential pixel data whose expression timing is shown in the bottom row of FIG. 2B, which has already been described with reference to FIG. (Row) L1
And L2, which belongs to a virtual pixel column (row) LA composed of first virtual pixels. For the dot-sequential pixel data (first virtual pixel data) belonging to LA, this is performed for one clock (duration of each color data output each time)
A delay circuit 352 for delaying (this output is shown in FIG.
Of the second row of dot-sequential pixel data) and a delay circuit 353 for delaying by 2 clocks (this output is the dot-sequential pixel data of the third row of FIG. 9). Sampling and holding for performing a sampling operation corresponding to the dot-sequential pixel data (first virtual pixel data) input to the input terminal 351, the output of the delay circuit 352, and the output of the delay circuit 353, respectively. Circuits 361, 362 and 363 are provided respectively, and these sampling and holding circuits are configured to perform sampling and holding in synchronization with the sample pulse at the timing of the fourth row in FIG. 9, and these sampling and holding are performed. As a result, the data as shown in the 5th, 6th and 7th rows of FIG. 9 are obtained. The outputs of the sampling and holding circuits 361, 362 and 363 are the first selector 3
64, where each output is switched and selected,
Synchronized virtual pixel data (8th row, 9th row in FIG. 9,
And data of the second virtual pixels shown in the 10th row, in which the second virtual pixel P1 is represented by R1, G1, B1 of this synchronized output and the second virtual pixel is represented by R2, G2, B1. Pixel P2 is the second virtual pixel P3 due to R3, G2, B2
The second virtual pixel P4 is changed to R4, G4, by R3, G2, B3.
With B3, a virtual pixel P5 is to be represented, respectively). On the other hand, the delay circuit 371 for delaying the dot-sequential pixel data (for example, the dot-sequential pixel data belonging to the above-described virtual pixel row LA) supplied to the terminal 351 by one horizontal scanning period (1H), A delay circuit 372 (this output is the dot-sequential pixel data in the second row in FIG. 9) for further delaying the output data of the delay circuit 371 by one clock is provided. The output of the delay circuit 372 and the input terminal 35 described above.
Sampling / holding circuits 381, 382, 3 for performing a sampling operation corresponding to the dot-sequential pixel data input to 1 and the output of the delay circuit 353 described above.
83 are provided respectively, and these sampling and holding circuits are configured to perform sampling and holding in synchronization with the sample pulse at the timing of the sixth row in FIG. 10. As a result of performing these sampling and holding,
The data as shown in the 7th, 8th and 9th rows of FIG. 10 are obtained. Sampling and holding circuit 381,3
Outputs 82 and 383 are supplied to the second selector 384, where each output is switched and selected to synchronize the virtual pixel data (10th row, 11th row, 12th row in FIG. 9).
Line) is obtained.

14, FIG. 15, FIG. 16, FIG. 17 and FIG.
8 is a diagram for explaining another embodiment of the invention of the present application, and FIG. 14 shows a so-called Δ in the image pickup device applied to the invention by the notation similar to that of FIGS. 2A and 4 described above. FIGS. 15, 16, and 17 are schematic diagrams showing the positional relationship between the real pixels of the array and the virtual pixels of the grid array, based on the data of the virtual pixels shown by the black dots in FIG. FIG. 18 is a diagram showing the timing of the operation in the process of obtaining the data of each virtual pixel indicated by X points, and FIG. 18 shows the configuration of the device for obtaining the data of each virtual pixel indicated by X points in FIG. It is a block diagram.

As is well understood with reference to FIG. 14, in this example, for each virtual pixel of the virtual image space corresponding to the one assuming that the pixel array of the solid-state image sensor is a grid-like array, the respective center positions of those virtual pixels are arranged. The arrangement is such that, in the horizontal direction, there are the same positions as the virtual pixels of the black dots described above with reference to FIG. 2A, and there are two points (denoted by x) at equal intervals between these black dots (•). Each horizontal column (row) is formed, while in the vertical direction, each horizontal column (row) of the above-described form is arranged in parallel at an interval of 1/2 of the vertical dimension of the actual pixel. I am doing it. In other words, the virtual pixel (•) of each black dot and the virtual pixel (×) between them have their own centers at the four corners (corners of the quadrilateral real pixel area) of each real pixel. This means that each of these virtual pixels is supposed to be located.

As described above, each virtual pixel (•) indicated by a black dot in FIG. 14 is the same as that in FIG. 2 (a). Therefore, the data of each virtual pixel indicated by these black dots in FIG. 14 is dot sequential pixel data whose expression timing is shown in the bottom row of FIG. 2 (b). Also in FIG. 14, as described above with reference to FIG. 8, virtual pixel columns (rows) formed by virtual pixels located between horizontal pixel columns (rows) of the first and second actual pixels from the top in the figure. Be LA. Also,
The virtual pixel column (row) LB is also assumed to be the same as that described above with reference to FIG. Therefore, in particular, regarding the horizontal virtual pixel column (row) in FIG. 14, the above virtual pixel column (row)
Between LA and LB, the LA and LB are included, and successive virtual pixel columns (rows) parallel to these are arranged from the top in the figure to L,
L-L, L-C, L-D, and L-E are set, and similarly for sequential virtual pixel rows (rows) below LB, L-, L-, and
Set as L, L, .... As shown in the figure, the L-I is set on LA and the L-E is set on LB.

FIG. 15 shows the timing of the generation process of the data representing the virtual pixels P1, P2, P3, P4, P5, P6 on the virtual pixel column (row) LI in FIG. .
What is shown in the first row of FIG. 15 is dot-sequential pixel data whose expression timing is shown in the bottom row of FIG. 2B, and is on the virtual pixel column (row) L in FIG. Is data representing each virtual pixel (•) indicated by a black dot. As shown in the figure, some of the virtual pixels indicated by these black dots are in the same relationship as the virtual pixels indicated by (x). As shown in the second and third rows of FIG. 15, in the present example, the dot-sequential pixel data shown in the first row is clocked by one clock (each color data output is continued every time). Time) delayed dot-sequential pixel data, and
Each dot-sequential pixel data delayed by 2 clocks is obtained.
Next, the point-sequential pixel data of the first row, the second row, and the third row of FIG. 14 described above are sequentially switched by the selector means to be output, and as shown in the lower three rows of FIG. , Data representing each virtual pixel P1, P2, P3, P4, P5, ... On the horizontal virtual pixel column (row) LI in FIG. Also for the horizontal virtual pixel column (row) L e set on the horizontal virtual pixel column (row) LB, the above-mentioned virtual pixel column (row) L
Data having a similar relation to B and data representing each virtual pixel aligned on the L-e are obtained in the same manner as above.

FIG. 16 shows each virtual pixel on the horizontal virtual pixel row (row) L C located between the virtual pixel rows (rows) LA and LB in FIG. The timing of the generation processing of data representing (×, ×, ...) Is shown.

As shown in FIG. 16, pixel data obtained by delaying the dot-sequential pixel data belonging to the virtual pixel column (row) LA by one horizontal scanning period (1H) as shown in the first row of FIG. , As shown in the second row, the dot-sequential pixel data obtained by delaying the dot-sequential pixel data shown in the first row by one clock, and the dot-sequential pixel data delayed by one clock are further added by one clock. Delayed (that is, the dot-sequential pixel data shown in the first row was delayed by 2 clocks)
Each dot-sequential pixel data is obtained (FIG. 16: first row to third row)
line). As for the dot-sequential pixel data belonging to the above virtual pixel column (row) LB, as in the case of LA, 1
The dot-sequential pixel data delayed by the clock and the dot-sequential pixel data delayed by 2 clocks are obtained respectively (FIG. 16:
Lines 4-6). Next, the second of the dot-sequential pixel data (FIG. 16: first row to sixth row) obtained as described above is used.
Figure 1 shows the dot-sequential pixel data for each row, 4th row, and 6th row.
Sampling with a sampling pulse of 2 clock cycles as shown in the 7th row of No. 6 (corresponding data string and sampling interval are shown with “○” attached for convenience of confirmation of sampling correspondence by this) 16 to obtain the dot-sequential pixel data for each of the ninth, tenth, and eleventh rows in FIG. Similarly, the first of the dot-sequential pixel data (FIG. 16: first row to sixth row) obtained as described above.
Figure 1 shows the dot-sequential pixel data for each row, the third row, and the fifth row.
Sampling pulse of a certain two clock cycles having a 1/2 cycle phase shift from the sampling pulse (○) as shown in the eighth row of No. 6 (corresponding data for convenience of confirmation of sampling correspondence by this) Columns and sampling intervals are indicated by adding "△"), and the 12th row, the 13th row, and the 14th row in FIG. 16 are sampled.
Obtain pixel data for each point in a row. Furthermore, these 9th line ~
Of the dot-sequential pixel data of 2 clock cycles shown in the 14th row, each two consecutive data strings (9th and 12th rows, that is, the first row of the ○ data row and the first row of the Δ data row). 10th and 13th rows, that is, the 2nd row of the ○ data column and the 2nd row of the Δ data column; 11th and 14th rows, that is, the 3rd row of the ○ data column and Δ
By sequentially switching and outputting (the third row of the data string) by the selector means, the 15th row, 16th row,
In addition, three systems of dot-sequential pixel data as shown in the 17th row are obtained. The dot sequential pixel data of these three systems are sequentially switched and output by the selector means in the same manner as described above with reference to FIG.
As shown in the 19th and 20th lines, R pixel data is always in the R system, G pixel data is always in the G system, and B pixel data is always in the B system. Will be output and the data representing the respective virtual pixels (×, ×, ...) On the horizontal virtual pixel column (row) L C, which has been output and has been synchronized with each system of B, R, and G, will be obtained.

Next, the horizontal virtual pixel columns (rows) L and L
The data representing each virtual pixel (×, ×, ...) On the horizontal virtual pixel column (row) L and between the two are on the horizontal virtual pixel column (row) L obtained as described above. Interpolation processing is performed based on the data representing each virtual pixel and the data representing each virtual pixel on the horizontal virtual pixel column (row) L (where L is at an intermediate position at the same distance from L and L). Therefore, the averaging process is performed as a result) to obtain data in which the B, R, and G systems are synchronized as shown in FIG. As shown in FIG. 14, the positional relationship between the horizontal virtual pixel columns (rows) L and L and the horizontal virtual pixel columns (rows) L and B between them, and the horizontal virtual pixel columns (rows) L and L The data representing each virtual pixel on the horizontal virtual pixel row (row) L is also similar to the positional relationship between the horizontal virtual pixel row (row) L and the horizontal virtual pixel row (row). This can be obtained by performing the same interpolation processing (averaging process) as described above for the horizontal virtual pixel column (row) LB based on L-C and L-E.

As described above, the virtual pixel column (row) is set to L
Parallel sequential virtual pixel row (row) L set between A and LB
Data representing each pixel of (i), (l), (l), (l), (d), and (e) can be obtained, but the sequential virtual pixel row (row) below LB
The data representing L, L, L, L, L, and so on are obtained in the same manner as above, paying attention to the similarity of the sequences with L, L, L, L, L, and E. be able to.

FIG. 18 shows the data of each virtual pixel indicated by the point x in FIG. 14 based on the data of the virtual pixel indicated by the black dots in FIG. 14 with reference to FIG. 15, FIG. 16 and FIG. It is a block diagram which shows an example of a structure of the system for performing and obtaining the process. In FIG. 18, the input terminal 40
1, the dot-sequential pixel data is supplied at the timing shown in the first row of FIG. The supplied data is the dot-sequential pixel data whose expression timing is shown in the bottom row of FIG. 2 (b), which has already been described with reference to FIG. Virtual horizontal pixel row (row) L located between horizontal pixel rows (rows) of actual pixels in the second row
It belongs to A. Here, for convenience of explanation, the virtual pixel column (row) LB is set at a time point 1H (horizontal scanning period) after the point-sequential pixel data belonging to the virtual horizontal pixel row (row) LA is supplied to the input terminal 401. It is shown that data representing each pixel of the virtual pixel rows (rows) L, L, L, L, and L is sequentially obtained at the timing when the belonging dot-sequential pixel data is supplied to the input terminal 401. With respect to the dot-sequential pixel data belonging to LB (LA), a delay circuit 402 for delaying the dot-sequential pixel data by 1 clock (duration of each color data output each time) (this output is shown in FIG.
Of the second row of dot-sequential pixel data) and a delay circuit 403 for further delaying by one clock (this output is the dot-sequential pixel data of the third row of FIG. 15). . A time-series selection operation is performed on the output of the three systems of the dot-sequential pixel data that is the input to the input terminal 401, the output of the delay circuit 402, and the output of the delay circuit 403, and on the virtual pixel column L Data representing each x point is obtained (at the timing when the dot-sequential pixel data belonging to the virtual horizontal pixel column (row) LA is supplied to the input terminal 401, as shown in the lower three rows of FIG. Each virtual pixel P1, P2, P3, P4 on the virtual horizontal pixel column (row) L i
A selector 404 for obtaining data representing P5, ... Is provided. The output of the selector 404 is the delay circuit 40.
5 and the delay circuit 406 in this order as pixel outputs belonging to the virtual pixel column (row) L during the corresponding one horizontal scanning period (in the corresponding other horizontal scanning period, the virtual pixel column (row) L). (As a pixel output belonging to B)) is output.

On the other hand, the sampling and holding for performing the sampling operation corresponding to each of the three outputs of the dot-sequential pixel data input to the input terminal 401, the output of the delay circuit 402, and the output of the delay circuit 403. Circuit 41
1, 412 and 413 are provided respectively. These sampling and holding circuits are sampling pulses marked with ○ and △ in this figure (○ and △ in FIG. 16, 7th and 8th rows).
Sampling and holding are performed in synchronism with the same sampling pulse with.

The signal supplied to the input terminal 401 (therefore, the input to the delay circuit 402) is also branched and supplied to the 1H (1 horizontal scanning period) delay circuit 421. A delay circuit 422 for delaying the dot-sequential pixel data output from the 1H delay circuit 421 by one clock (duration of each color data output each time) (this output is shown in the second row of FIG. 16). A dot-sequential pixel data) and a delay circuit 423 (this output is the dot-sequential pixel data in the third row of FIG. 16) for further delaying by one clock are provided. The dot-sequential pixel data output from the 1H delay circuit 421, the output from the delay circuit 422, and the delay circuit 423 (3)
Performing a time-series selection operation for the output of the grid,
The data representing each x point on the virtual pixel column L of 4 is obtained (at the timing when the dot-sequential pixel data belonging to the virtual horizontal pixel column (row) LA is supplied to the input terminal 401).
Selector 424 for obtaining data representing each virtual pixel P1, P2, P3, P4, P5, ... On the virtual horizontal pixel column (row) LI in FIG. 14 as shown in the bottom three rows. It is provided. The output of the selector 424 is input to the delay circuit 425 described later.

On the other hand, the dot-sequential pixel data which is the output of the 1H delay circuit 421 (that is, the input to the delay circuit 422),
Sampling and holding circuits 431, 432, 433 for performing sampling operations corresponding to the outputs of the delay circuit 422 and the output of the delay circuit 423, respectively, of the three systems.
Are provided respectively. These sampling and holding circuits perform sampling and holding in synchronism with the sampling pulses with circles and triangles in this figure (the same as the sampling pulses with circles and triangles in the seventh and eighth rows in FIG. 16). .

The sampling and holding circuit 411 described above.
And 431, a selector 441 for switching both outputs, a sampling and holding circuits 412 and 432 for switching both outputs, and a sampling and holding circuits 413 and 433 for switching both outputs.
Are provided, and these three selectors 441, 442, 4
By the switching operation at 43, the dot-sequential pixel data of three systems shown in the 15th to 17th rows of FIG. 16 is obtained from the pixel data shown in the 9th to 14th rows of FIG. A selector 450 that performs the time-series signal selection operation similar to that described above with reference to FIG. 13 for the dot-sequential pixel data of the three systems that are the outputs of the three selectors 441, 442, and 443.
16 is provided as an output of the selector 450.
18th row, 19th row, and 20th row of, the R system always has R pixel data, the G system always has G pixel data, and the B system has The pixel data of B is always output, and the B, R, and G systems are synchronized, and the virtual pixels (×, ×,
...) corresponding data will be obtained.

The outputs of the three systems (B, G, R) of the delay circuit 405 and the outputs of the three systems of the selector 450 (B, G, R) are added to perform the addition processing. Adders 461 and 46 corresponding to (B, G, R)
2, 463 are provided, and these adders 461, 46 are further provided.
A divider 464 to which the outputs of 2,463 are input is provided. The divider 464 is 1/2 of the sum of the inputs of both systems.
Division processing (averaging processing) is performed. This processing includes data corresponding to each virtual pixel on the virtual horizontal pixel column (row) L e obtained as the output of the delay circuit 405 described above and the selector described above. Horizontal virtual pixel column (row) L obtained as the output of 450
Interpolation processing based on the data corresponding to each virtual pixel on C. (In order to obtain L data at an intermediate position where the distances from L and L are equal, the averaging process results. ) Is performed, and the data of B, R, and G systems are synchronized according to the timing of data generation corresponding to each virtual pixel on a virtual horizontal pixel column (row) LB shown in FIG. Is to get.

Similarly, the outputs of the three systems (B, G, R) of the delay circuit 425 and the outputs of the three systems of the selector 450 (B, G, R) are added for the addition processing. Adders 471 and 47 corresponding to (B, G, R)
2, 473 are provided, and these adders 471, 47 are further provided.
A divider 474 to which the outputs of 2, 473 are input is provided. The divider 474 is a half of the sum of the inputs of both systems.
Division processing (averaging processing) is performed. This processing includes data corresponding to each virtual pixel on the virtual horizontal pixel column (row) L i obtained as the output of the delay circuit 425 described above and the selector described above. Horizontal virtual pixel column (row) L obtained as the output of 450
Interpolation processing based on the data corresponding to each virtual pixel on C. (In order to obtain the data of L, which is at the intermediate position where the distances from L and L are equal, the averaging process results. ) Is performed to obtain data in which the B, R, and G systems are synchronized at the timing shown in FIG.

A delay circuit 452 for giving a predetermined delay to the output of the selector 450 is connected to the output side thereof. In the above configuration, in the time section corresponding to one horizontal scanning period (the time section in which LB is input to the input terminal 401), the output of the subtractor 474 is set on each of the virtual horizontal pixel columns (rows) LB. Data corresponding to the virtual pixel is output as the output of the delay circuit 452 to the horizontal virtual pixel column (row).
The data corresponding to each virtual pixel on the L c is the subtractor 464.
Of data corresponding to each virtual pixel on the virtual horizontal pixel row (row) L d, and data corresponding to each virtual pixel on the horizontal virtual pixel row (row) L e of the output of the delay circuit 406. Will be obtained. Further, in a time section (a time section in which LC is input to the input terminal 401) corresponding to one horizontal scanning period subsequent to the time section, it corresponds to each virtual pixel on the horizontal virtual pixel column (row) Lho. Instead of the data, the data corresponding to each virtual pixel on L can be obtained. Similarly, instead of the horizontal virtual pixel column (row) L, L and L
Instead of (b), data corresponding to each virtual pixel on L is obtained.

The delay circuits 405, 406 and 42 described above are used.
Reference numerals 5 and 452 are for delaying the transmission of signals (data) by a time corresponding to the corresponding signal selection processing, addition processing, and subtraction processing, and performing necessary timing adjustment for each corresponding processing.

Next, the embodiment described with reference to FIG.
As in the above-described embodiment, the pixels of each color, which are the photoelectric conversion unit areas adapted to obtain the photoelectric conversion output corresponding to the filter color corresponding to each self, are arranged so as to appear in the horizontal direction with periodicity. The horizontal position of the pixels corresponding to the same filter color is one pixel between the horizontal pixel columns that are arranged so as to form a plurality of horizontal pixel columns that are adjacent to each other and that are paired adjacent to each other in the vertical direction. Are arranged so as to be offset by a half pitch with the width in the horizontal direction as a unit pitch, so that the pixels of three colors in the horizontal pixel row having the above-mentioned paired position are positioned close to each other. 1. An image signal processing apparatus adapted to obtain the color image signal based on the output of the solid-state image pickup device shown in FIG. 1, which is configured to form a substantially delta positional relationship. And , The pixel array corresponding to the photoelectric conversion output of the one color at each virtual pixel position of the virtual image space corresponding to the one assuming that the pixel array of the solid-state image sensor is a grid array, The present invention is characterized in that it is provided with a grid-like array pixel data generating means for generating based on pixel signals from three to four real pixels of the same color closest to each other in the horizontal pixel row.

The above-mentioned grid-like array pixel data generating means can be constituted by the corresponding functional part of the microcomputer, but as described above, the three adjacent pixels of the same color in the three vertically adjacent horizontal pixel columns are in the closest relationship. To the photoelectric conversion output of the one color at each virtual pixel position of the virtual image space corresponding to the one assuming that the pixel array of the solid-state image sensor is a grid array based on pixel signals from four real pixels The process of generating each pixel data will be described with reference to FIG.

As shown in the drawing, the horizontal pixel rows (rows) of the real pixels of the solid-state image pickup device are arranged in order from the top, as shown in FIG.
Let them be L3, L4, L5 and L6. Now here the horizontal pixel column (row) L
Focusing on 2 and L1 and L3 above and below it, the three Gs which are closest to each other in terms of the position of the actual pixel R2 are seen.
It can be seen that the pixels G1, G2 and Gb correspond to the actual pixels of the system. Paying attention to this relationship, the data corresponding to the virtual pixel G (R2) of the G system in which its own position (x) is the same as the position of the real pixel R2 is set to the degree of correlation with these real pixels G1, G2 and Gb. The following is the calculation for the calculation.

[0051]

[Equation 1]

Similarly to the above, the self position (x) is set to the real pixel B.
The data corresponding to the virtual pixel G (B2) of the G system which is the same as the position of 2 can be calculated according to the degree of correlation with the real pixels G2, G2 and Gb. Similarly, the data corresponding to the virtual pixels of each G system on the horizontal pixel column (row) L2 can be obtained. The same applies to the R-system virtual pixel and the B-system virtual pixel.

Next, paying attention to the horizontal pixel column (row) L3 and L2 and L4 above and below it, the four G systems which are closest to each other in the intermediate position (x) between the actual pixels Rc and Bc. Pixels G3, Gb, Gc and Gb correspond to the actual pixels of. Paying attention to this relationship, data corresponding to the virtual pixel G (RcBc) of the G system at the intermediate position (x) between the real pixels Rc and Bc is
The following is the calculation for calculating the correlation with the actual pixels G3, Gb, Gc and Gd.

[0054]

[Equation 2]

Further, on the horizontal pixel column (row) L3, the intermediate position (x) between the real pixels Gb and Rc is the three G system real pixels which are closest to each other as seen from here, and the pixels G3, Gb,
And Gc are applicable. Paying attention to this relationship, the actual pixel Gb
The calculation for calculating the data corresponding to the virtual pixel G (GbRc) of the G system at the intermediate position (x) between Rc and Rc according to the degree of correlation with these real pixels G3, Gb, and Gc is shown below. It is a thing.

[0056]

(Equation 3)

Further, on the horizontal pixel column (row) L3, the G system virtual pixel G (BcGd) at the intermediate position (x) between the real pixels Bc and Gd.
The data corresponding to
The same calculation is performed according to the degree of correlation with each of the G real pixels G3, Gc, and Gd.

On the horizontal pixel column (row) L3, the virtual pixels of the R system and the virtual pixels of the B system are calculated in the same manner as the calculation of the virtual pixels of each G system.
Pixel data corresponding to the virtual pixels of the system can be obtained. Horizontal pixel columns (rows) L2 and L3
Each successive horizontal pixel column (row) Ln is calculated by the same calculation as the above calculation for obtaining the data corresponding to the above virtual pixel.
The data corresponding to the upper virtual pixel can be determined.
As described above, the data corresponding to each virtual pixel of the G system, B system and R system can be obtained, and the virtual pixels of the square lattice array are represented by these data.

According to the embodiment described above with reference to FIG.
Since the degree of correlation between each virtual pixel data to be calculated by the grid-shaped array pixel data generation means is taken into account for each basic real pixel data, the accuracy of the calculated data is improved and a false signal is generated. Moire components generated in the reproduced image can be suppressed.

The embodiment described above with reference to FIG. 19 relates to the actual pixel data while the corresponding actual pixel data is being supplied to the grid array pixel data generating means which can be constituted by the corresponding functional unit of the microcomputer. Alternatively, a configuration may be employed in which means for performing processing including any of gradation conversion, data transfer to an external device, or data writing to a recording medium is interposed.

Further, each pixel data generated by the grid-shaped array pixel data generating means is viewed in both directions in the horizontal and vertical directions of the virtual image space, with the length of one side of one real pixel as a unit of interval. It may be configured to include image display data generation means for outputting data sampled at intervals of 6n (n is a natural number) as image display data. According to such a configuration, a pixel array having a relatively coarse density in a square lattice shape suitable for rapid loading into a computer, display on a display screen of a computer, or selection and confirmation of an image to be printed. The virtual pixel data corresponding to (i.e., the virtual pixel data in which the data amount representing one image is smaller than that corresponding to the high-quality image) can be obtained by a simple process.

Further, instead of generating the image display data by calculating each virtual pixel in the virtual image space and then sampling these virtual pixels, the XY address as the solid-state image pickup device is used. Type solid-state image sensor is applied, and the readout operation of this solid-state image sensor is controlled to directly extract the data of the actual pixel at the position corresponding to each virtual pixel corresponding to the image display data extracted by the above sampling. It may be configured to include a control unit (configured by a computer or the like) that performs a control operation for allocating as image display data. According to such a configuration, the XY address is compared with the above-described method in which each virtual pixel in the virtual image space is calculated and then the virtual pixels are sampled to generate image display data. By controlling the readout operation by utilizing the characteristics of the solid-state image sensor, the sampling of the pixel signal itself is performed to obtain the data of the actual pixel at the position corresponding to each virtual pixel corresponding to the image display data. By directly extracting and assigning this real pixel data as image display data, a quick display can be obtained with an extremely simple configuration. Further, each pixel data generated by the grid-shaped array pixel data generating means is sampled every four pixels by a virtual pixel in the horizontal direction of the virtual image space and every three pixels by a virtual pixel in the vertical direction. An image display data generating means for outputting the data as image display data may be provided. According to such a configuration, virtual pixel data corresponding to a square lattice pixel array suitable for being taken into a computer, displayed on a display screen of a computer, or quickly selecting and confirming an image to be printed. Can be obtained by a simple process.

Furthermore, a preview means capable of confirming the state of the image to be recorded by the image of relatively low resolution by the image display data generating means, and the grid pattern representing the image confirmed by the preview means. It may be configured to include a recording unit that records the output data of the arrayed pixel data generation unit. With such a configuration, the image to be recorded or printed can be quickly selected and confirmed by the preview unit, and the image thus selected and confirmed can be recorded by the recording unit in a high quality state. .

[0064]

According to the first aspect of the present invention, a color image corresponding to the output video signal of the solid-state image pickup device having a non-lattice-like pixel array which is not normally adapted to the display screen of the computer is displayed on the display screen of the computer. Can be projected on.

According to the invention of claim 2, a color image corresponding to the output video signal of the solid-state image pickup device of the non-lattice pixel array having the specific color filter array as defined in the above item is displayed on the computer. The image can be displayed on the display screen or on a monitor screen having a display pixel array similar to this.

According to the third aspect of the present invention, by utilizing the characteristics of the XY address type solid-state image pickup device, the output of a specific pixel smaller than the total number of pixels is read so as to be sampled when the pixel output is read and driven. By doing so, it is possible to very easily obtain the image data suitable for transmitting the simple external output image data for a short time in a short time or for simply recognizing the image in a short time. it can.

According to the invention of claim 4, the pixel output of the two horizontal actual pixel columns (rows) treated as a pair is read out to calculate the virtual pixel data, because of a special delay line or temporary storage. It is possible to adopt a configuration that does not require the storage element, and the device can be simplified.

According to the fifth aspect of the present invention, the output data of the apparatus of the present invention can be treated as it is as it is suitable for the display screen of a general computer, and the image can be easily displayed on the display screen of the computer. Can be issued.

According to the invention of claim 6, it is possible to reproduce (print or the like) an image with a quality not inferior to the image based on the actual pixel data, and obtain virtual pixel data suitable for the display screen of the computer. You can

According to the invention of claim 7, there is provided one specific configuration for obtaining virtual pixel data as defined in claim 6. According to the invention of claim 8, it is possible to obtain virtual pixel data for forming an image of higher quality than an image based on the virtual pixel data as defined in claim 6.

According to the invention of claim 9, there is provided one specific configuration for obtaining virtual pixel data as defined in claim 8. According to the invention of claim 10, it is possible to obtain relatively dense virtual pixel data for forming an image of higher quality than the image based on the virtual pixel data as defined in claim 6, and to perform gradation conversion. Alternatively, it is possible to obtain relatively coarse virtual pixel data suitable for easily and quickly performing data transfer to an external device or data writing to a recording medium.

According to the eleventh aspect of the invention, it corresponds to a pixel array of a square lattice which is suitable for loading into a computer, displaying on a display screen of a computer, or quickly selecting and confirming an image to be printed. Virtual pixel data
It can be obtained by a simple process.

According to the twelfth aspect of the invention, instead of generating the image display data by calculating each virtual pixel of the virtual image space and then sampling these virtual pixels, an XY address type is used. Utilizing the characteristics of the solid-state image sensor, the readout operation is controlled to directly extract the data of the actual pixel at the position corresponding to each virtual pixel corresponding to the image display data extracted by the sampling, and the image is displayed. It is possible to obtain a quick display by allocating it as data for use and using an extremely simple structure.

According to the thirteenth aspect of the present invention, the pixel array is suitable for a square lattice, which is suitable for loading into a computer, displaying on a display screen of a computer, or quickly selecting and confirming an image to be printed. Virtual pixel data
It can be obtained by a simple process.

According to the fourteenth aspect of the present invention, the image to be recorded or printed is selected and confirmed quickly by the preview means, and the image thus selected and confirmed is recorded by the recording means in a high quality state. can do.

According to the fifteenth aspect of the invention, it is possible to obtain extremely dense virtual pixel data for forming a high quality image suitable for printing or the like. According to the sixteenth aspect of the present invention, it is possible to obtain relatively dense virtual pixel data for forming a high-quality image suitable for printing or the like, and to perform gradation conversion or data to an external device. It is possible to obtain relatively coarse virtual pixel data suitable for simple and quick transfer or data writing to the recording medium.

According to the seventeenth aspect of the invention, it is possible to obtain extremely dense virtual pixel data for forming a high quality image suitable for printing or the like. According to the invention of claim 18, a color image corresponding to an output video signal of a solid-state image pickup device having a non-lattice pixel array having a specific color filter array as defined in the same paragraph is displayed on a display screen of a computer. Alternatively, it can be displayed on a monitor screen having a display pixel array similar to this.

According to the nineteenth aspect of the invention, instead of generating the image display data by calculating each virtual pixel in the virtual image space and then sampling these virtual pixels, an XY address type is used. Utilizing the characteristics of the solid-state image sensor, the readout operation is controlled to directly extract the data of the actual pixel at the position corresponding to each virtual pixel corresponding to the image display data extracted by the sampling, and the image is displayed. It is possible to obtain a quick display by allocating it as data for use and using an extremely simple structure.

According to the twentieth aspect of the invention, a special delay line is used for calculating the virtual pixel data by reading out the pixel outputs of three vertically adjacent horizontal real pixel columns (rows) treated as one set. It is possible to adopt a configuration that does not require a storage element for temporary storage or to simplify the device.

According to the twenty-first aspect of the present invention, the output data of the apparatus of the present invention can be treated as it is as adapted to a general computer display screen, and the image can be easily displayed on the computer display screen. Can be issued.

According to the twenty-second aspect of the invention, it is possible to obtain virtual pixel data which can reproduce an image (at the time of printing, etc.) with higher quality than that based on the actual pixel data and which is also suitable for the display screen of the computer. You can

According to the twenty-third aspect of the invention, there is provided one specific configuration for obtaining the virtual pixel data as defined in the eighteenth aspect. According to the invention of claim 24, it is possible to obtain relatively dense virtual pixel data for constructing an image of higher quality than an image based on the virtual pixel data as defined in claim 18, and to perform gradation conversion. Alternatively, it is possible to obtain relatively coarse virtual pixel data suitable for easily and quickly performing data transfer to an external device or data writing to a recording medium.

According to the twenty-fifth aspect of the present invention, it corresponds to a pixel array of a square lattice which is suitable for loading into a computer, displaying on a display screen of a computer, or quickly selecting and confirming an image to be printed. Virtual pixel data
It can be obtained by a simple process.

According to the twenty-sixth aspect of the invention, instead of generating the image display data by calculating each virtual pixel of the virtual image space and then sampling these virtual pixels, an XY address type is used. Utilizing the characteristics of the solid-state image sensor, the readout operation is controlled to directly extract the data of the actual pixel at the position corresponding to each virtual pixel corresponding to the image display data extracted by the sampling, and the image is displayed. It is possible to obtain a quick display by allocating it as data for use and using an extremely simple structure.

According to the twenty-seventh aspect of the present invention, it corresponds to a pixel array of a square lattice shape suitable for rapid loading into a computer, display on a display screen of a computer, or quick selection and confirmation of an image to be printed. Virtual pixel data
It can be obtained by a simple process.

According to the twenty-eighth aspect of the invention, the image to be recorded or printed is selected and confirmed quickly by the preview means, and the image selected and confirmed by this is recorded by the recording means in a high quality state. can do.

[Brief description of drawings]

FIG. 1 is a diagram showing a pixel array of an image sensor (solid-state image sensor) applied to an embodiment of the present invention.

FIG. 2 is a diagram for explaining a timing of reading a signal from an image sensor and a configuration for reading the signal.

FIG. 3 is a diagram for explaining a timing of reading a signal from an image sensor and a configuration for reading the signal.

FIG. 4 is a schematic diagram showing a positional relationship between so-called Δ-array real pixels and lattice-like virtual pixels in an image sensor applied to another embodiment of the present invention.

5 is a diagram showing an operation timing in a process of obtaining data of a virtual pixel indicated by a black dot in FIG.

6 is a diagram showing an operation timing in a process of obtaining data of a virtual pixel indicated by a black dot in FIG.

FIG. 7 is a block diagram showing a configuration of an apparatus of an embodiment to which the image pickup device shown in FIG. 4 is applied.

FIG. 8 is a so-called Δ array of real pixels and virtual pixels in an image sensor applied to still another embodiment of the present invention, and pixel positions corresponding to data of each virtual pixel generated based on data of each virtual pixel. It is a schematic diagram which shows the positional relationship of.

9 is a diagram showing an operation timing in the process of obtaining data of the virtual pixel in FIG.

10 is a diagram showing an operation timing in the process of obtaining data of the virtual pixel in FIG.

11 is a block diagram showing a configuration of an apparatus of an embodiment to which the image pickup device shown in FIG. 8 is applied.

FIG. 12 is a schematic diagram showing how sampling is performed on virtual pixels in the embodiment of the present invention.

FIG. 13 is a block diagram showing an example of a signal processing circuit of an apparatus according to an embodiment of the present invention.

FIG. 14 is a schematic diagram showing a positional relationship between so-called Δ array of real pixels and lattice-like array of virtual pixels in an image pickup device applied to another embodiment of the present invention.

15 is a diagram showing an operation timing in the process of obtaining data of the virtual pixel in FIG.

16 is a diagram showing operation timing in the process of obtaining data of the virtual pixel in FIG.

FIG. 17 is a diagram showing operation timing in the process of obtaining data of the virtual pixel in FIG.

FIG. 18 is a block diagram showing a configuration of an apparatus of an embodiment to which the image pickup device shown in FIG. 14 is applied.

FIG. 19 is a schematic diagram showing a positional relationship between real pixels and virtual pixels in a grid array in an image sensor applied to still another embodiment of the present invention.

FIG. 20 is a diagram showing a difference in Nyquist limit characteristic (limit characteristic of reproducibility) of image pickup due to a difference in color filter array method of a single plate type color image pickup element.

[Explanation of symbols]

 101a ……………………………… Preamplifier 101b ……………………………… Preamplifier 102a …………………………… Process amplifier 102b …………………………… Process amplifier 103a ……………………………… A / D converter 103b ……………………………… A / D converter 104 ……………………………… Latch circuit 105 …………………………………… Selector 202a ……………………………… Preamplifier 202b …………………………… Preamplifier 203a ……………… ……………… Process amplifier 203b ……………………………… Process amplifier 204a ……………………………… A / D converter 204b ……………………………… A / D converter 205a ……………………………… Latch circuit 205b ……………………………… Circuit 206a ……………………………… Latch circuit 206b …………………………… Latch circuit 207 ……………………………… Selector 212 ……………… ……………… Function block

Claims (28)

[Claims] 1. A photoelectric conversion surface in which pixels of each color, which are photoelectric conversion unit areas, are arranged to obtain a photoelectric conversion output corresponding to a filter color corresponding to each self, and which are two-dimensionally arranged in a non-lattice pattern. An image signal processing apparatus adapted to obtain a color image signal based on the output of the solid-state image sensor by applying a solid-state image sensor configured to sequentially read out the pixel signals of 1. Each pixel data corresponding to the pixel output of the one color at each virtual pixel position of the virtual image space corresponding to the one assuming that the pixel array of the solid-state image sensor is a grid array is located in the vicinity of the virtual pixel. An image signal processing apparatus, comprising: a grid-shaped array pixel data generation means for generating based on pixel signals from a plurality of real pixels in a non-grid array. 2. A plurality of horizontal pixels, each of which is a photoelectric conversion unit area adapted to obtain a photoelectric conversion output corresponding to a filter color corresponding to its own, and arranged in such a manner that pixels of respective colors appear with periodicity in the horizontal direction. The horizontal position of pixels corresponding to the same filter color is a unit of the horizontal width of one pixel between the horizontal pixel columns that are arranged to form a pixel column and are adjacent to each other in the vertical direction to form a pair. 2 as the pitch
By arranging so as to have a relationship of being shifted by one-half pitch, the pixels of three colors in the horizontal pixel rows having the above-mentioned paired relationship are positioned close to each other and form a substantially delta-shaped positional relationship. An image signal processing device that applies a solid-state image sensor configured as described above and obtains a color image signal based on the output of the solid-state image sensor, wherein the pixel array of the solid-state image sensor is a grid-like array. Pixel data corresponding to the pixel output of the one color at each virtual pixel position in the virtual image space corresponding to the hypothesized one is stored in each pair of the horizontal pixel rows in the paired relationship. An image signal processing apparatus comprising: a grid-shaped array pixel data generating unit that generates the pixel signals based on pixel signals from real pixels of three colors having a positional relationship. 3. An XY address type solid-state image pickup device is applied as the solid-state image pickup device.
Alternatively, the image signal processing device according to item 2. 4. A solid-state image pickup device, which is capable of simultaneously selecting vertically adjacent two pixels by a horizontal shift register and a vertical shift register, is applied, and a pixel signal according to this selection is made at a predetermined timing by a horizontal line selection switching means. 4. The image signal processing apparatus according to claim 1, wherein a solid-state image sensor configured to be switched alternately and led out to the outside is applied. 5. The image signal processing device according to claim 1, wherein the lattice-like arrangement of each virtual pixel in the virtual image space is a square lattice-like arrangement. 6. A plurality of horizontal pixels, each pixel being a photoelectric conversion unit area adapted to obtain a photoelectric conversion output corresponding to a filter color corresponding to its own, and arranged in such a manner that pixels of respective colors appear with periodicity in the horizontal direction. The horizontal position of pixels corresponding to the same filter color is a unit of the horizontal width of one pixel between the horizontal pixel columns that are arranged to form a pixel column and are adjacent to each other in the vertical direction to form a pair. 2 as the pitch
By arranging so as to have a relationship of being shifted by one-half pitch, the pixels of three colors in the horizontal pixel rows having the above-mentioned paired relationship are positioned close to each other and form a substantially delta-shaped positional relationship. An image signal processing device that applies a solid-state image sensor configured as described above and obtains a color image signal based on the output of the solid-state image sensor, wherein the pixel array of the solid-state image sensor is a grid-like array. For each virtual pixel in the virtual image space corresponding to the assumed one, the respective center positions thereof correspond to each horizontal pixel row corresponding to either the even-numbered or odd-numbered one of the horizontal pixel rows of the solid-state image sensor. The pixel data corresponding to the output of each of these virtual pixels, assuming that they are located at the positions corresponding to the four corners of each real pixel to which they belong, are converted into corresponding real pixels in the horizontal pixel row having the above-mentioned paired relationship. Picture from It is characterized by further comprising grid-like array pixel data generating means adapted to obtain data representing a number of virtual pixels corresponding to the total number of actual pixels by directly or indirectly generating the elementary signals. Image signal processing device. 7. The lattice-shaped array pixel data generating means is occupied by real pixels of corresponding three colors which are in a mutually adjacent positional relationship in horizontal pixel rows which form a pair of the solid-state image pickup elements. Each virtual region is represented by a real pixel on one side of the three real pixels forming the one virtual region and a real pixel on the other side of the three real pixels forming an adjacent region of the one virtual region. Pixel signals from the corresponding real pixels of each of the three colors are assumed to correspond to the virtual pixels located substantially at the centers of these virtual regions when it is assumed that the pixels overlap with each other in the same pixel relationship. 7. The image signal processing device according to claim 6, wherein the image signal processing device is configured to generate each of the virtual pixel data in a grid-like array based on the above. 8. The grid-shaped array pixel data generating means includes three corresponding color elements which are adjacent to each other in a pair of horizontal pixel rows of the solid-state image pickup device and which are in a substantially delta positional relationship. The virtual area occupied by the actual pixels is assumed to correspond to the virtual pixel located approximately at the center of each virtual area when the mutual positional relationship is assumed so that the virtual areas do not have portions that overlap each other. , Each of the above 3
A first virtual pixel data, which is each virtual pixel data of a grid-like array, is generated based on a pixel signal from a color real pixel;
And the center position of each virtual pixel in the virtual image space corresponding to the one assuming that the pixel array of the solid-state image sensor is a grid array among the horizontal pixel columns of the solid-state image sensor. By assuming the positions corresponding to the four corners of each real pixel belonging to each horizontal pixel column corresponding to either the even-numbered or odd-numbered, the data having a larger capacity than the first virtual pixel data is obtained. The second virtual pixel data, which is each pixel data corresponding to the output of the virtual pixel, is generated based on the first virtual pixel data generated by the first lattice-shaped array pixel data generating means. 7. The image signal processing device according to claim 6, further comprising a second grid-shaped array pixel data generating means. 9. The first grid array pixel data generating means dot-sequentially data obtained by sampling an output signal from each real pixel of the solid-state image pickup device at a frequency twice the output frequency of the output signal. 9. The image signal processing apparatus according to claim 8, wherein the image signal processing apparatus is configured so as to be derived as the image output of. 10. The first virtual pixel data generated by the first grid-shaped array pixel data generation means is supplied to the second grid-shaped array pixel data generation means while the first virtual pixel data is supplied to the second grid-shaped array pixel data generation means. 9. The image signal processing apparatus according to claim 8, wherein a means for performing processing including any of gradation conversion, data transfer to an external device, and data writing to a recording medium is intervened with respect to pixel data. . 11. The pixel data generated by the grid-shaped array pixel data generating means is viewed in both directions in the horizontal and vertical directions of the virtual image space when the length of one side of one real pixel is taken as a unit of interval. 3. The image signal processing apparatus according to claim 2, further comprising image display data generating means for outputting data sampled at 6n intervals (n is a natural number) as image display data. 12. An XY address type solid-state image pickup device is applied as the solid-state image pickup device, and a read operation of the solid-state image pickup device is controlled to conform to each virtual pixel corresponding to image display data extracted by the sampling. The image signal processing apparatus according to claim 11, further comprising a control unit that performs a control operation for directly extracting the data of the real pixel at the different position and assigning the data as image display data. 13. The pixel data generated by the grid-shaped array pixel data generating means is sampled every four pixels by virtual pixels in the horizontal direction of the virtual image space and every three pixels by virtual pixels in the vertical direction. 3. The image signal processing apparatus according to claim 2, further comprising image display data generating means for outputting the resulting data as image display data. 14. A preview means capable of confirming a state of an image to be recorded by an image of a relatively low resolution by the image display data generating means, and the grid-like array representing the image confirmed by the preview means. 14. The image signal processing device according to claim 11, further comprising a recording unit that records the output data of the pixel data generating unit. 15. The grid-shaped array pixel data generation means comprises:
The virtual areas occupied by the corresponding real pixels of the corresponding three colors in the mutually adjacent positional relationship in the horizontal pixel rows that form a pair of the solid-state image sensor are arranged so that the virtual areas do not overlap each other. Based on the pixel signals from these corresponding three-color real pixels, each virtual of the lattice array of normal density is assumed to correspond to the virtual pixel located substantially at the center of each virtual region when the positional relationship is assumed. A normal density grid array pixel data generating means for generating primary virtual pixel data which is pixel data, and a high density grid pixel array 12 times denser than the normal density grid array corresponding to the primary virtual pixel data. High-density grid array pixel data generation means for generating each secondary virtual pixel data representing each virtual pixel based on each primary virtual pixel data described above. The image signal processing apparatus according to claim 2 that. 16. The gradation of the primary virtual pixel data while the primary virtual pixel data generated by the normal density grid array pixel data generation means is supplied to the high density grid array pixel data generation means. 15. The image signal processing apparatus according to claim 14, further comprising means for performing processing including any of conversion, data transfer to an external device, and data writing to a recording medium. 17. The grid-shaped array pixel data generation means comprises:
The virtual areas occupied by the corresponding real pixels of the corresponding three colors in the mutually adjacent positional relationship in the horizontal pixel rows that form a pair of the solid-state image sensor are arranged so that the virtual areas do not overlap each other. Assuming that the positional relationship is assumed, the virtual regions are approximately the centers of the respective virtual regions, and the corresponding three colors corresponding to the virtual pixels whose own centers are located on the intersections of the boundaries of the real pixels of the three colors. Intersection pixel data generating means for generating intersection pixel data, which is each virtual pixel data of the grid array based on the pixel signals from the real pixels, and a line segment that vertically connects each virtual pixel represented by each intersection pixel data. Interpolation pixel data generating means for generating interpolation pixel data, which is each virtual pixel data corresponding to each virtual pixel whose center is located at the middle point, by interpolation processing based on each intersection pixel data. It is configured such that each of the intersection pixel data and the interpolation pixel data can represent each virtual pixel of the virtual image space corresponding to the one assuming that the pixel array of the solid-state image sensor is a grid array. The image signal processing device according to claim 2. 18. A plurality of horizontal pixels, each pixel of each color being a photoelectric conversion unit area adapted to obtain a photoelectric conversion output corresponding to a filter color corresponding to its own, are arranged so as to appear with periodicity in the horizontal direction. The horizontal position of pixels corresponding to the same filter color is the width of one pixel in the horizontal direction between the horizontal pixel columns that are arranged so as to form a pixel column and are adjacent to each other in the vertical direction and paired with each other. Are arranged so as to be offset by a half pitch with the unit pitch as a unit pitch, so that the pixels of the three colors in the horizontal pixel rows having the above-described paired relationship are positioned close to each other and have a substantially delta shape. Applying a solid-state image sensor configured to form the positional relationship of
An image signal processing device adapted to obtain a color image signal based on the output of the solid-state image pickup device, wherein each virtual image space of a virtual image space corresponding to the one assuming that the pixel array of the solid-state image pickup device is a grid-like array. Each pixel data corresponding to the photoelectric conversion output of the one color at the pixel position is generated based on the pixel signals from the three closest real pixels of the same color in the vertically adjacent three horizontal pixel rows. An image signal processing device comprising: 19. The image signal processing apparatus according to claim 18, wherein an XY address type solid-state image pickup device is applied as the solid-state image pickup device. 20. As the solid-state image pickup device, a device capable of simultaneously selecting vertically adjacent two pixels by a horizontal shift register and a vertical shift register is applied, and a pixel signal according to this selection is made at a predetermined timing by horizontal line selection switching means. The image signal processing apparatus according to claim 18, wherein a solid-state image sensor configured to be switched alternately and led out to the outside is applied. 21. A lattice-like arrangement of the virtual pixels in the virtual image space is a square lattice-like arrangement.
8. The image signal processing device according to item 8. 22. The grid-shaped array pixel data generating means,
Regarding a horizontal pixel column located in the middle of each set of horizontal pixel columns consisting of horizontal pixel columns of three real pixels adjacent to each other in the vertical direction of the solid-state image pickup device, the horizontal pixel columns are respectively arranged between the actual pixels of the same color belonging to the column. It is assumed that there are two virtual pixels of the same color corresponding to the positions of two adjacent real pixels of another color, and the data representing these two virtual pixels is set to the above 1
For each set of horizontal pixel columns, the calculation is performed according to the degree of correlation of each real pixel with respect to the virtual pixel based on the three real pixel data of the same color that are located closest to each of the two virtual pixels. The data representing each virtual pixel corresponding to each position to be respectively calculated and to form a grid-like array together with each virtual pixel respectively calculated is located closest to each of these virtual pixels. Data representing more virtual pixels than the total number of actual pixels is obtained by performing calculations according to the degrees of correlation of the actual pixels with respect to the virtual pixel based on the three actual pixel data of the same color. 19. The invention according to claim 18, characterized in that
The image signal processing device according to. 23. The grid-like array pixel data generation means outputs dot-sequential image data obtained by sampling an output signal from each real pixel of the solid-state image pickup device at a frequency twice the output frequency of the output signal. 19. The image signal processing apparatus according to claim 18, further comprising means for deriving as. 24. While the actual pixel data from the solid-state image pickup device is supplied to the grid array pixel data generating means,
19. The image signal according to claim 18, wherein means for performing a process including any one of gradation conversion, data transfer to an external device, and data writing to a recording medium is interposed with respect to the actual pixel data. Processing equipment. 25. The pixel data generated by the grid-shaped array pixel data generating means is viewed in both directions in the horizontal and vertical directions of the virtual image space when the length of one side of one real pixel is taken as a unit of interval. 19. The image signal processing apparatus according to claim 18, further comprising image display data generating means for outputting data sampled at 6n intervals (n is a natural number) as image display data. 26. An XY address type solid-state image pickup device is applied as the solid-state image pickup device, and a read operation of the solid-state image pickup device is controlled to conform to each virtual pixel corresponding to image display data extracted by the sampling. 19. The image signal processing apparatus according to claim 18, further comprising a control unit that performs a control operation for directly extracting the data of the real pixel at the position and assigning the data as image display data. 27. Sampling each pixel data generated by the grid-shaped array pixel data generating means in the horizontal direction of the virtual image space every 4 pixels of virtual pixels and in the vertical direction every 3 pixels of virtual pixels. 19. The image signal processing apparatus according to claim 18, further comprising image display data generating means for outputting the resulting data as image display data. 28. Preview means capable of confirming the state of an image to be recorded by the image of relatively low resolution by the image display data generating means, and the grid-like array representing the image confirmed by the preview means. The image signal processing device according to claim 25 or 27, further comprising: a recording unit that records the output data of the pixel data generating unit.