JP4499499B2 - Image processing apparatus, image processing method, and image processing program - Google Patents

Description

  The present invention relates to an image processing apparatus, an image processing method, and an image processing program for performing compression encoding of still images and moving images.

  In image compression methods that are currently mainstream, such as still image compression method JPEG and moving image compression method MPEG, compression using DCT conversion is used. This is a technique for reducing the amount of data by deleting a component having a high spatial frequency. In this method, the lower the spatial frequency in the screen, the more effectively the compression can be performed.

In addition, the data amount can be further reduced by reducing the number of pixels in advance by performing pixel thinning before performing these image compressions. For example, an example in which pixel thinning is applied before normal compression is disclosed in Japanese Patent Laid-Open No. 5-260436. Here, as shown in FIG. 25, by thinning the image from the original image in a checkered pattern, the image resolution is reduced to 1/2 with less visual resolution degradation, and the empty gap is horizontally packed, The amount of image data is further reduced by compressing the image using another general method.
JP-A-5-260436

  However, when an image with a gap formed by checkered pixel thinning is filled with a gap formed by horizontal filling, the image is distorted. For example, when an original image as shown in FIG. 20 is thinned into a checkered pattern as shown in FIG. 21 and horizontally aligned, as shown in FIG. 22, a jagged edge is formed at the edge portion in the image. When trying to compress this image by JPEG or MPEG using DCT conversion, the spatial frequency is higher than that of the original image, so the compression efficiency deteriorates. Even when vertical packing is used instead of horizontal packing, the same problem occurs as shown in FIG.

  Such a problem occurs because the pixel thinning position differs depending on the line in the checkered pixel thinning, and it is necessary to consider the pixel thinning position for each line. Nothing is described about this point of view.

  The present invention has been made to solve the above-described problems, and an image processing apparatus and an image processing capable of efficiently performing image compression even when the positions where pixels are thinned out differ from line to line. A method and an image processing program are provided.

Hereinafter, the configuration, corresponding examples, operations, and effects of the image processing apparatus, the image processing method, or the image processing program according to the invention of claims 1 to 4 will be described.

(Claim 1)
An image processing apparatus according to a first aspect of the invention includes a subframe generation unit that generates a plurality of subframes by dividing each input image frame of a moving image input from an imaging unit in units of predetermined lines, and the subframe includes: A subframe determining unit that determines whether the subframe is a first subframe or a second subframe of the plurality of subframes related to the same input image frame; the first subframe; and the first subframe, A motion vector computing unit that computes a motion vector from a correlation with an input image frame preceding the subframe, and a shift vector that outputs a difference in pixel position between the first subframe and the second subframe as a shift vector an output unit, based on the sub-frame determination section according to the determination result, when the processing target sub-frame is the first subframe the motion vector Apply the, when a second sub-frame has a moving image compression section for performing compression on the displacement vector for each said sub-frame by applying the reference picture of the first sub-frame.

(Example of corresponding invention)
An embodiment relating to the present invention corresponds to the embodiment 2. Here, the imaging unit is the imaging unit 41 of the block diagram of FIG. 14, the subframe generation unit is the subframe generation unit 43 of the block diagram of FIG. 14, and the subframe determination unit is the subframe determination unit of the block diagram of FIG. 62, the motion vector calculation unit corresponds to the motion search unit 63 in the block diagram of FIG. 16, and the shift vector output unit corresponds to the fixed motion vector calculation unit 64 in the block diagram of FIG.

(Effect)
Except for the first sub-frame, since a fixed value is used as a motion vector by the shift vector output unit, it is not necessary to perform complicated motion search processing by block matching, so a moving image divided into a plurality of sub-frames is compressed. The processing for doing so can be simplified.

(Claim 2)
The subframe generation unit according to claim 1, wherein the subframe generation unit generates a subframe related to the first subframe from only one of an odd-numbered line group and an even-numbered line group of each input image frame, and the subframes of the subframe, and wherein Rukoto forming raw from the other line group.

(Example of corresponding invention)
An embodiment relating to the present invention corresponds to the embodiment 2. A subframe related to the first subframe is generated from only one of the odd-numbered line group and the even-numbered line group of each input image frame, and the subframe related to the second subframe is generated from the other line group. Is generated from S4, S5, S6, and S7 in the flowchart of FIG. 3 by outputting the difference in pixel position between the first subframe and the second subframe as a shift vector. This corresponds to S24 in the flowchart of FIG.

(Effect)
Since the shift vector output unit uses a fixed value as the motion vector in the second subframe, it is not necessary to perform complicated motion search processing by block matching, so the moving image divided into two subframes is compressed. Can be simplified.

(Claim 3)
An image processing method according to a third aspect of the present invention generates a plurality of subframes by dividing each input image frame of a moving image input from an imaging unit in a predetermined line unit, and the subframes are divided into the same input image frame. A first subframe of the plurality of subframes or a second subframe other than the first subframe, and the first subframe and an input image frame preceding the first subframe A motion vector is calculated from the correlation. Based on the determination result, the motion vector is applied when the processing target subframe is the first subframe, and the first subframe is applied when the processing subframe is the second subframe. the displacement vector is the difference between the pixel positions of the first subframe and the second subframe by applying a reference picture child perform compression for each of the sub-frame The features.

(Example of corresponding invention)
An embodiment relating to the present invention corresponds to the embodiment 2. Correspondence with embodiment are the same as claim 1.

(Effect)
Since a fixed value is used as a motion vector except for the first subframe, it is not necessary to perform complicated motion search processing by block matching, and thus the processing for compressing a moving image divided into a plurality of subframes is simplified. Can be

(Claim 4)
According to a fourth aspect of the present invention, there is provided an image processing program that divides each input image frame of a moving image input from an image pickup unit into a predetermined line unit to generate a plurality of subframes. A process of determining whether the first subframe of the plurality of subframes related to the input image frame is a second subframe other than the first subframe, the first subframe, and the first subframe preceding A process of calculating a motion vector from a correlation with an input image frame. Based on the determination result, when the processing target subframe is the first subframe, the motion vector is applied, and when the processing target subframe is the second subframe, applying displacement vector which is the difference between the pixel position and wherein the first subframe a second subframe of the first subframe as a reference picture Characterized in that to execute the process, to perform compression for each of the sub-frame Te.

(Example of corresponding invention)
An embodiment relating to the present invention corresponds to the embodiment 2. Correspondence with embodiment are the same as claim 1.

(Effect)
Since a fixed value is used as a motion vector except for the first subframe, it is not necessary to perform complicated motion search processing by block matching, so that the processing for compressing a moving image divided into a plurality of subframes is simplified. can do.

  According to the image processing apparatus, the image processing method, and the image processing program of the present invention, even when the positions where pixels are thinned out are different for each line, such as checkered pixel thinning, image compression is performed efficiently. It becomes possible.

  Embodiments of the invention will be described with reference to the drawings.

FIG. 1 is a diagram illustrating a configuration of an image processing apparatus according to the first embodiment of the present invention. Reference numeral 1 denotes a camera device that forms an imaging compression block that can capture both still images and moving images, 2 denotes a storage unit for recording data, and 3 denotes a display device that forms an image expansion / display block.
FIG. 2 is a block diagram illustrating the configuration of the imaging compression block in the image processing apparatus according to the first embodiment of the present invention. The imaging compression block indicates a block from imaging to outputting compressed data.

  In the present embodiment, the imaging compression block includes a plurality of different thinning-out pixels constituting an imaging unit 11 having an imaging device such as a CCD and an input image frame composed of a plurality of lines input from the imaging unit 11. A pixel number reduction unit 12 to be thinned out by a pattern, a subframe generation unit 13 for generating a subframe with thinned pixels for each of the thinning patterns, and a memory 14 for storing the subframe generated by the subframe generation unit 3 And an image compression unit 15 that performs compression for each subframe, and a compressed data output unit 16 that converts the data compressed by the image compression unit 15 into a predetermined format and outputs the data.

  As the imaging unit 11, for example, a single-plate camera device that obtains a color image signal by attaching a color filter to each pixel of the photoelectric conversion element is used. As the color filter, for example, a color filter in which R (red), G (green), and B (blue) are arranged in a mosaic pattern, for example, a Bayer array is used.

FIG. 3 is a flowchart from image data input to compression in the imaging compression block.
First, an image captured by the imaging unit 11 is converted into a digital signal, and RGB Bayer format image data is output to the pixel number reduction unit 2 (step S1).

  The pixel number reduction unit 12 reduces the number of pixels by thinning out the pixels of the image data in a checkered pattern, and outputs the image data as a YUV420 color data format by predetermined image format conversion (step S2).

  In this embodiment, the number of pixels is reduced by thinning out pixels. However, another method may be adopted as long as the number of pixels can be reduced in a checkered pattern. For example, with respect to RGB Bayer data output from the imaging unit, Bayer conversion is performed using adjacent R and B data only at pixel positions where RGB G data exists, and then converted to YUV format and output. By doing so, it is also possible to output data with the number of pixels reduced in a checkered pattern. Here, Bayer conversion is, for example, when a color filter composed of R, G, and B is used in the single-plate camera device as described above, and R, G, Since only one color information of B is obtained, interpolation calculation is performed on other color information of the image signal so that R, G, and B data can be obtained for each pixel. Say the technique. For example, in Japanese Patent Laid-Open No. 10-178650, in a configuration using an R, G, B mosaic filter, an average value is calculated for each R, G, B pixel data within a 5 × 5 pixel region, A technique for obtaining an interpolation value has been disclosed.

First, the sub-frame generation unit 13 divides the checkered YUV image data input from the pixel number reduction unit 12 into units of lines (step S3).
In step S4, it is determined whether the divided lines are even lines. If the divided line is an even line (Y in step S4), it is in the first subframe area in the memory 14 (step S5), and if it is an odd line (N in step S4), the first line in the memory 14 In the second subframe region (step S6), it is sorted and stored.

  The above distribution is executed until the end of the line for one frame of image data in the end determination in step S7 (Y in step S7), whereby the first subframe and the second subframe are constructed. .

The image compression unit 15 performs image compression on the subframes input from the subframe generation unit 13 in the order of the first frame and the second frame (step S8). In the first embodiment, JPEG is assumed as the compression method, but other methods can be applied as long as the compression is based on intra-screen correlation, such as an MPEG I picture. Further, the present embodiment can be applied to both still images and moving images.
The compressed data output unit 16 converts the data compressed by the image compression unit 15 into a predetermined format and outputs it (step S9).

Next, a subframe division method for luminance signals will be described with reference to FIG.
In the following description, the lines are numbered 0, 1, 2,... Starting from the top line. For example, in FIG. 4, the uppermost line is 0th.

  First, as shown in FIG. 4A, pixels are thinned out in a checkered pattern. At this time, the positions where pixels remain in the odd-numbered lines and the even-numbered lines are different, but only the even-numbered lines are collected as shown in FIG. 4B to form the first subframe, and FIG. As shown in c), only the odd-numbered lines are collected to form the second subframe. Thereby, it is possible to generate two luminance component subframes whose pixel positions are aligned in the vertical method.

  In the present embodiment, the number of pixels is reduced on the checkerboard pattern and divided into two subframes. However, another thinning pattern is applied as a method for reducing the number of pixels, and the number of types of thinning patterns is determined. It is also possible to perform a number of subframe divisions. For example, it is possible to use a thinning pattern as shown in FIG. 5A and divide into four subframes as shown in FIGS. 5B, 5C, 5D, and 5E.

Next, a subframe division method for color difference signals will be described with reference to FIG.
FIG. 6 shows an example of a sub-frame generation method for the color difference component. In addition, although the example of U component is shown in FIG. 6, it can consider similarly about V component.

  First, the pixels are thinned out in a checkered pattern. In this embodiment, since the YUV420 color data format is used, it is sufficient that the color difference component has a data amount that is 1/4 of the luminance component.

  Therefore, as shown in FIG. 6A, the pixels at the positions (4m, 4n) and (4m + 1, 4n + 1) are left, and the other pixels are thinned out. However, m and n are integers of 0 or more, and the notation of (x, y) is the pixel position when the upper left pixel is the origin, the horizontal right direction position is x, and the vertical lower direction position is y. It is assumed that the coordinates represent.

  As shown in FIG. 6B, only the 4nth line is collected to form the first subframe, and as shown in FIG. 6C, only the 4n + 1th line is collected to form the second subframe. Thereby, it is possible to generate two color difference component subframes having the same pixel position.

  In addition to the above, the method shown in FIG. 7 may be used as the sub-frame division method for the color difference component in the YUV420 color data format. FIG. 7 shows another example of a sub-frame generation method for color difference components. In addition, although the example of U component is shown also in FIG. 7, it can consider similarly about V component. As described above, the pixels are first thinned out in a checkered pattern. However, since the YUV420 color data format is also used in this example, the color difference component only needs to have a data amount that is 1/4 of the luminance component.

That is, as shown in FIG. 7A, the pixels at the positions (4m, 4n) and (4m + 2, 4n + 2) are left, and the other pixels are thinned out.
Then, only the 4nth line is collected as shown in FIG. 7B to form the first subframe, and only the 4n + 2nd line is collected as shown in FIG. 7C to form the second subframe. Thereby, it is possible to generate two color difference component subframes having the same pixel position.

  Compared with the luminance component sub-frame generation method of FIG. 4, the chrominance component sub-frame generation method of FIG. 6 and the chrominance component sub-frame generation method of FIG. 7 will be compared.

  In the luminance component shown in FIG. 4, a first subframe and a second subframe are formed on the even-numbered line and the odd-numbered line, respectively. In the method of taking the color difference component of FIG. 6, when divided into the first and second subframes, the first subframe and the second subframe are formed by the even-numbered line and the odd-numbered line, respectively. Therefore, it corresponds to the first and second subframes of the luminance component in an array. On the other hand, in the method of taking the color difference component of FIG. 7, when divided into the first and second subframes, the first and second subframes are formed only by the even-numbered lines. Does not correspond to the first and second sub-frames of the luminance component, and therefore when the image is decompressed and restored on the display block side (during sub-frame synthesis), a positional shift occurs between the luminance component and the color difference component, resulting in color Deviation occurs. On the other hand, when the color difference component shown in FIG. 6 is adopted, there is no color misalignment between the luminance component and the color difference component when the image is decompressed and restored on the display block side (subframe synthesis).

Next, an image decompression / display block that decompresses compressed image data compressed by the imaging compression block and displays an image will be described.
FIG. 8 is a block diagram of the image decompression / display block. The image decompression / display block temporarily stores, for example, JPEG compressed image subframe data in the imaging compression block shown in FIG. 2 in the storage unit, and then stores the stored JPEG compressed data.

  The image decompression / display block includes a compressed data input unit 21 to which a plurality of JPEG-compressed subframe data is input, an image decompression unit 22 that decompresses a plurality of JPEG-compressed subframe data, and a plurality of decompressed plurality of subframe data. A subframe synthesis unit 23 for synthesizing subframe data, a memory 24 for storing the subframe synthesized by the subframe synthesis unit 23, a pixel interpolation unit 25 for interpolating a thinned-out pixel portion in the synthesized subframe data, And an image display unit 26 that converts the data interpolated by the pixel interpolation unit 25 into display data and displays the data on a display.

FIG. 9 shows a flowchart of expansion / display.
First, JPEG-compressed image subframe data is input from the compressed data input unit 21 (step S11), and is decompressed by the image decompression unit 22 into the YUV420 color data format (step S12).

  Next, the subframe synthesizing unit 23 synthesizes the two subframes to generate a synthesized frame in which pixels are missing in a checkered pattern (step S13), and the pixel interpolating unit 25 missing the pixels. The interpolated portion is interpolated (step S14), and the image display unit 26 converts the image into a predetermined format and displays the image (step S15).

Next, with reference to FIG. 10, subframe synthesis and pixel interpolation in the luminance signal in the subframe synthesis unit 23 and the pixel interpolation unit 25 will be described.
FIG. 10 shows a pixel interpolation method after the luminance component sub-frame synthesis.

  Each line of the first subframe is set as an even line of the combined frame, and each line of the second subframe is set as an odd line of the combined frame, thereby forming a combined frame.

  Then, the pixel data of (2m, 2n + 1) and (2m + 1, 2n) where the pixel data is lost due to pixel thinning is interpolated using the surrounding four pixels as shown in the following equation.

Y (2m, 2n + 1) = {Y (2m, 2n + 1) + Y (2m-1, 2n + 1) + Y (2m + 1, 2n + 1) + Y (2m, 2n + 2)} / 4
Y (2m + 1, 2n) = {Y (2m + 1, 2n-1) + Y (2m, 2n) + Y (2m + 2, 2n) + Y (2m + 1, 2n + 1)} / 4
For example, the thinned pixel Y21 is interpolated by obtaining an average value of the surrounding four pixels Y11, Y31, Y20, and Y22.

  Next, sub-frame synthesis and pixel interpolation when the color difference signal is thinned out as shown in FIG. 6 will be described with reference to FIG. FIG. 11 shows an example of a pixel interpolation method after color-difference component subframe synthesis.

  In FIG. 11, the color difference signal in the color data format of YUV420 requires pixel data at a position surrounded by a thick frame. That is, the data format of YUV420 is to take one U and one V for 4 pixels of 2 × 2. Therefore, as shown in the following equation, pixel data at the position (4m + 1, 4n + 1) is used as the pixel data at the position (4m + 2, 4n + 2).

U (4m + 2, 4n + 2) = U (4m + 1, 4n + 1)
V (4m + 2, 4n + 2) = V (4m + 1, 4n + 1)
Next, the pixel data at the positions (4m, 4n + 2) and (4m + 2, 4n) are interpolated using the surrounding four pixels.

U (4m, 4n + 2) = {U (4m, 4n) + U (4m-2,4n + 2) + U (4m + 2, 4n + 2) + U (4m, 4n + 4)}) ÷ 4
V (4m, 4n + 2) = {V (4m, 4n) + V (4m-2,4n + 2) + V (4m + 2, 4n + 2) + V (4m, 4n + 4)}) ÷ 4
U (4m + 2,4n) = {U (4m + 2,4n-2) + U (4m, 4n) + U (4m + 4,4n) + U (4m + 2,4n + 2)}) ÷ 4
V (4m + 2,4n) = {V (4m + 2,4n-2) + V (4m, 4n) + V (4m + 4,4n) + V (4m + 2,4n + 2)}) ÷ 4
Specifically, among the two pixels adjacent in the diagonal direction, the pixels U11, U15, U51, U55,... Are pixel data of the pixels U22, U26, U62, U66,. Use. In this state, the pixels U02, U20, U06, U24, U42, U60, U46, U64,. For example, the pixel U24 is interpolated using four pixels U04, U44, U22, and U26.

  Next, sub-frame synthesis and pixel interpolation when the color difference signal is thinned out as shown in FIG. 7 will be described with reference to FIG. FIG. 12 shows another example of a pixel interpolation method after sub-frame synthesis of color difference components.

  Also in the case of FIG. 12, in the color difference signal in the YUV420 color data format as in the example of FIG. 11, pixel data at a position surrounded by a thick frame is necessary. That is, as described above, the data format of YUV420 is to take one U and one V for each 2 × 2 four pixels. Therefore, the pixel data at the positions (4m, 4n + 2) and (4m + 2, 4n) are interpolated using the surrounding four pixels.

U (4m, 4n + 2) = {U (4m, 4n) + U (4m-2,4n + 2) + U (4m + 2,4n + 2) + U (4m, 4n + 4)}) ÷ 4
V (4m, 4n + 2) = {V (4m, 4n) + V (4m-2,4n + 2) + V (4m + 2,4n + 2) + V (4m, 4n + 4)}) ÷ 4
U (4m + 2,4n) = {U (4m + 2,4n-2) + U (4m, 4n) + U (4m + 4,4n) + U (4m + 2,4n + 2)}} ÷ 4
V (4m + 2,4n) = {V (4m + 2,4n-2) + V (4m, 4n) + V (4m + 4,4n) + V (4m + 2,4n + 2)}) ÷ 4
Specifically, pixels U02, U20, U06, U24, U42, U60, U46, U64,. For example, the pixel U42 is interpolated using four pixels U22, U62, U40, and U44.

  By the above method, it is possible to decompress and display an image compressed by the method of this embodiment.

  According to the first embodiment of the present invention, thinning with different thinning patterns, generating a plurality of subframes with thinned pixels for each thinning pattern, and performing compression for each subframe, Even if the positions where pixels are thinned out differ from line to line, such as pattern-like pixel thinning, image compression can be performed efficiently without increasing the spatial frequency.

Next, a second embodiment of the present invention will be described.
FIG. 13 is a schematic diagram of an image processing apparatus according to the second embodiment of the present invention. Reference numeral 31 denotes a camera device constituting an imaging compression block capable of capturing a moving image, and 32 denotes an image display display device constituting an image expansion / display block. A difference from FIG. 1 is that there is no storage unit for data recording between the camera device 31 and the display device 32 in order to capture moving image data in real time. In this embodiment, each component is connected wirelessly, but a wired connection is also possible.

  FIG. 14 is a block diagram illustrating a configuration of an imaging compression block in the image processing apparatus according to the second embodiment of the present invention. The imaging compression block indicates a block from imaging to outputting compressed data.

  In the second embodiment, the imaging compression block includes an imaging unit 41 having an imaging device such as a CCD, and pixels constituting an input image frame composed of a plurality of lines input from the imaging unit 41. A pixel number reduction unit 42 to be thinned out with a thinning pattern, a subframe generation unit 43 that generates a subframe with thinned pixels for each thinning pattern, and a memory that stores the subframe generated by the subframe generation unit 43 44, an image compression unit 45 that performs compression for each subframe, and a compressed data output unit 46 that converts the data compressed by the image compression unit 45 into a predetermined format and outputs the data.

  As the imaging unit 41, for example, a single-plate camera device that obtains a color image signal by attaching a color filter to each pixel of the photoelectric conversion element is used. As the color filter, for example, a color filter in which R (red), G (green), and B (blue) are arranged in a mosaic pattern, for example, a Bayer array is used.

  In the second embodiment, the entire configuration is the same as that of the first embodiment shown in FIG. 2, but the image to be handled is a moving image, and the compression method of the image compression unit 45 is an inter-screen such as MPEG. We shall use compression.

  FIG. 15 is a block diagram of the image decompression / display block. The image decompression / display block receives data of image subframes, for example, MPEG-compressed by the imaging compression block shown in FIG.

  The image decompression / display block includes a compressed data input unit 51 to which a plurality of MPEG-compressed subframe data is input, an image decompression unit 52 that decompresses a plurality of MPEG-compressed subframe data, and a plurality of decompressed plurality of subframe data. A subframe combining unit 53 that combines the subframe data, a memory 54 that stores the subframe combined by the subframe combining unit 53, a pixel interpolation unit 55 that interpolates a thinned-out pixel portion in the combined subframe data, An image display unit 56 that converts the data interpolated by the pixel interpolation unit 55 into display data and displays it on a display is provided.

  FIG. 16 is a block diagram of the image compression unit 45. The image compression unit 45 includes a demultiplexer (hereinafter referred to as DEMUX) 61, a subframe determination unit 62, a motion search unit 63 as a motion vector calculation unit, a fixed motion vector calculation unit 64 as a shift vector output unit, a multiplexer (Hereinafter referred to as MUX) 65, a DCT conversion unit 66, and a quantum encoding unit 67 as a moving image compression unit.

FIG. 17 is a flowchart from image data input to compression in the image compression unit 45.
First, subframe image data is input to the DEMUX 61 and the subframe determination unit 62 (step S21). The subframe determination unit 62 determines whether or not the input subframe is the first subframe, and notifies the DEMUX 61 and MUX 65 (step S22).

  Based on the notified determination result, the DEMUX 61 proceeds to the motion search unit 63 if it is the first subframe (Y in step S22), and to the fixed motion vector calculation unit 64 if it is not the first subframe (in step S22). N) Output the image data of each subframe.

  The motion search unit 63 calculates a motion vector by a well-known block matching method as a method for obtaining a motion vector using the previous first subframe as a reference image, and outputs the motion vector to the MUX 65. Also, the difference image data from the reference image obtained based on the calculated motion vector is simultaneously output to the MUX 65 (step S23).

  The fixed motion vector calculation unit 64 calculates a pixel position difference when the first subframe and the second subframe are originally one image frame, and outputs the difference to the MUX 65 as a motion vector fixed value. To do. Also, the difference image data from the first sub-frame obtained based on this motion vector fixed value is simultaneously output to the MUX 65 (step S24).

  Based on the determination result from the subframe determination unit 62, the MUX 65 alternately outputs the difference image data of the first subframe and the difference image data of the second subframe to the DCT conversion unit 66, respectively. Further, the MUX 65 alternately outputs the motion vector of the first subframe and the motion vector fixed value of the second subframe to the quantum encoding unit 67, respectively.

  The DCT transform unit 66 performs DCT transform on the input difference image data in units of subframes (step S25), and the quantum coding unit 67 performs quantization and coding (step S26). The compressed data is output (step S27).

Next, a method for calculating a motion vector will be described.
In the second embodiment, since the first subframe and the second subframe originally constituted one image frame, the images are similar.
For example, when an image frame as shown in FIG. 20 is reduced in the number of pixels in a checkered pattern as shown in FIG. 21 and divided into two first and second subframes as shown in FIG. The image of the frame and the image of the second sub-frame are similar to those moved by the difference in pixel position when forming the original image frame.

Therefore, in the second embodiment, as shown in FIG. 18, the difference between the pixel positions is used as a fixed value of the motion vector of the second subframe relative to the first subframe. The motion search by the conventional block matching is performed only when the motion vector between the first subframes is obtained.
Thereby, the complicated process of the motion search by block matching can be simplified.

In the second embodiment, the first subframe belonging to the same frame is used as the reference image of the second subframe. However, as shown in FIG. 19, the first subframe belonging to the previous frame is used. It is also possible to use two subframes as reference images. In this case, as the motion vector, the same motion vector as the motion vector of the first subframe obtained immediately before (the motion vector is fixed as a motion vector value in FIG. 19) can be used.
Or block matching between the first subframe of the previous frame and the first subframe of the current frame, and between the second subframe of the previous frame and the second subframe of the current frame, respectively. A motion vector may be obtained using motion search, an average motion vector may be obtained from these results, and used to compress the current frame.

  INDUSTRIAL APPLICABILITY The present invention can be widely used as a very effective technique when reducing the number of pixels in image data and reducing redundancy in an apparatus that processes image data such as an imaging apparatus.

1 is a diagram illustrating a configuration of an image processing apparatus according to Embodiment 1 of the present invention. 1 is a block diagram illustrating a configuration of an imaging compression block in an image processing apparatus according to Embodiment 1 of the present invention. The flowchart from the input of image data to compression in an imaging compression block. The figure explaining an example of the sub-frame production | generation method in a luminance signal. The figure explaining the other example of the sub-frame production | generation method in a luminance signal. The figure explaining an example of the sub-frame production | generation method in a color difference signal. FIG. 6 is a diagram for explaining another example of a subframe generation method for color difference signals. 1 is a block diagram illustrating a configuration of an image expansion / display block in an image processing apparatus according to a first embodiment of the present invention. 6 is a flowchart from compressed data input to image display in an image decompression / display block. The figure explaining the pixel interpolation method after the sub-frame synthesis | combination of a luminance component. The figure explaining an example of the pixel interpolation method after the sub-frame synthetic | combination of a color difference component. The figure explaining the other example of the pixel interpolation method after the sub-frame synthetic | combination of a color difference component. FIG. 6 is a schematic diagram of an image processing apparatus according to Embodiment 2 of the present invention. The block diagram which shows the structure of the imaging compression block in the image processing apparatus of Example 2 of this invention. The block diagram of an image expansion / display block. The block diagram of an image compression part. 6 is a flowchart from input of image data to compression in an image compression unit. The figure explaining an example of the simplification method of a motion estimation process. The figure explaining the other example of the simplification method of a motion estimation process. The figure which shows the original image for demonstrating a thinning image. The figure which shows a thinning image. The figure which shows the thinning image horizontally packed. The figure which shows the thinning image vertically packed. The figure which divided | segmented the thinned image of FIG. 22 into the 1st, 2nd sub-frame. The figure explaining a prior art example.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 ... Imaging part 12 ... Pixel number reduction part 13 ... Sub-frame production | generation part 14 ... Memory 15 ... Image compression part 62 ... Sub-frame determination part 63 ... Motion search part (motion vector calculating part)
64: Fixed motion vector calculation unit (deviation vector output unit)
67. Quantum encoding unit (moving image compression unit)
Agent Patent Attorney Susumu Ito

Claims (4)

A sub-frame generation unit that generates a plurality of sub-frames by dividing each input image frame of a moving image input from the imaging unit in a predetermined line unit;
A subframe determination unit that determines whether a subframe is a first subframe of the plurality of subframes related to the same input image frame or a second subframe other than the first subframe;
A motion vector computing unit that computes a motion vector from the correlation between the first subframe and the input image frame preceding the first subframe;
A shift vector output unit that outputs a difference in pixel position between the first subframe and the second subframe as a shift vector;
Based on the determination result by the subframe determination unit, the motion vector is applied when the processing target subframe is the first subframe, and the first subframe is used as a reference image when the processing target subframe is the second subframe. a moving image compression section for performing compression for each of the sub-frame by applying the displacement vector,
An image processing apparatus comprising: The sub-frame generation unit generates a sub-frame related to the first sub-frame from only one of an odd-numbered line group and an even-numbered line group of each input image frame, and sets the second sub-frame as the second sub-frame. the image processing apparatus according to sub-frame, in claim 1, characterized in Rukoto forming raw from other line groups according. Each input image frame of the moving image input from the imaging unit is divided in predetermined line units to generate a plurality of subframes, and the subframe is a first of the plurality of subframes related to the same input image frame. And a second subframe other than the first subframe, a motion vector is calculated from the correlation between the first subframe and the input image frame preceding the first subframe, and the determination result The motion vector is applied when the processing target subframe is the first subframe , and when the processing target subframe is the second subframe, the first subframe and the An image processing method, wherein compression is executed for each subframe by applying a shift vector, which is a difference in pixel position from the second subframe. A process of generating a plurality of sub-frames by dividing each input image frame of a moving image input from the imaging unit into a predetermined line unit on a computer, wherein the sub-frame is a plurality of sub-frames related to the same input image frame A process for determining whether the first subframe is within the second subframe, and calculating a motion vector from the correlation between the first subframe and the input image frame preceding the first subframe Based on the determination result, the motion vector is applied when the processing target subframe is the first subframe , and when the processing target subframe is the second subframe, the first subframe is used as the reference image . by applying the displacement vector which is the difference between pixel positions of the second subframe and the first subframe processing to perform the compression for each of the sub-frame The image processing program for causing the execution.

Images