JPH10243406A - Image coder, image coding method, image decoder, image decoding method and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To obtain a decoded image more improved in image quality. SOLUTION: A prediction tap generating circuit 32 generates prediction taps of a plurality of patterns to a noticeable picture element selected among picture elements being components of a low definition(SD) image obtained by interleaving a high definition(HD) image at an interleave circuit 31, and a classification adaptive processing circuit 33 conducts adaptive processing to obtain a predicted value of the HD image through linear combination of a predicted tap and a prescribed predicted coefficient. Then a prediction error calculation circuit 34 calculates a predicted error of a predicted value obtained from each of the predicted taps of a plurality of the patterns with respect to the HD image, and a tap pattern code addition circuit 36 adds a pattern code corresponding to the predicted tap by which a minimum predicted error is obtained among the predicted taps of a plurality of the patterns to a picture element value of the noticeable picture element.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] The present invention relates to an image encoding device and an image encoding method, an image decoding device and an image decoding method, and a recording medium. In particular, the present invention relates to an image encoding device and an image encoding method for thinning and compressing and encoding an image so as to obtain a decoded image substantially the same as an original image, an image decoding device and an image decoding method, and a recording medium.

[0002]

2. Description of the Related Art For example, a standard resolution or low resolution image (hereinafter, appropriately referred to as an SD image) is converted into a high resolution image (hereinafter, appropriately referred to as an HD image),
In the case of enlarging an image, interpolation (compensation) of pixel values of missing pixels is performed by a so-called interpolation filter or the like.

However, even if interpolation of pixels is performed by an interpolation filter, it is not possible to restore a component (high-frequency component) of an HD image that is not included in the SD image, and thus it is difficult to obtain a high-resolution image. there were.

Therefore, the present applicant has previously proposed an image conversion device (image conversion circuit) for converting an SD image into an HD image including high-frequency components not included therein.

[0005] In this image conversion apparatus, by performing an adaptive process for obtaining a predicted value of a pixel of an HD image by a linear combination of an SD image and a predetermined prediction coefficient, a high frequency component not included in the SD image is obtained. Has been made to be restored.

That is, for example, a prediction value E [y] of a pixel value y of a pixel forming an HD image (hereinafter, appropriately referred to as an HD pixel) is converted into some SD pixels (pixels forming the SD image). Pixel value (hereinafter, appropriately referred to as learning data)
x ₁ , x ₂ ,... and predetermined prediction coefficients w ₁ , w ₂ ,.
Is determined by a linear first-order combination model defined by a linear combination of In this case, the predicted value E [y] can be expressed by the following equation.

E [y] = w ₁ x ₁ + w ₂ x ₂ +... (1)

Therefore, for generalization, a matrix W composed of a set of prediction coefficients w, a matrix X composed of a set of learning data, and a matrix Y ′ composed of a set of predicted values E [y] are

(Equation 1) Defines the following observation equation.

XW = Y '(2)

[0010] Then, it is considered that a least square method is applied to this observation equation to obtain a predicted value E [y] close to the pixel value y of the HD pixel. In this case, a matrix Y consisting of a set of true pixel values y of HD pixels serving as teacher data and a matrix E consisting of a set of residuals e of predicted values E [y] for the pixel values y of HD pixels are represented by:

(Equation 2) From equation (2), the following residual equation is established.

XW = Y + E (3)

In this case, a prediction coefficient w _i for obtaining a prediction value E [y] close to the pixel value y of the HD pixel is a square error.

(Equation 3) Can be obtained by minimizing.

Therefore, when the above-mentioned squared error is differentiated by the prediction coefficient w _i , the result becomes 0, that is, the prediction coefficient w _i that satisfies the following equation is a prediction value E [y] close to the pixel value y of the HD pixel.
Is the optimum value.

[0014]

(Equation 4) ... (4)

Therefore, first, the equation (3) is calculated using the prediction coefficient w _i
By differentiating with, the following equation is established.

[0016]

(Equation 5) ... (5)

From equations (4) and (5), equation (6) is obtained.

[0018]

(Equation 6) ... (6)

Further, the learning data x, the prediction coefficient w, the teacher data y, and the residual e in the residual equation of the equation (3) are obtained.
In consideration of the relationship, the following normal equation can be obtained from Expression (6).

[0020]

(Equation 7) ... (7)

The normal equation of equation (7) can be set as many as the number of prediction coefficients w to be obtained. Therefore, solving equation (7) (however, to solve equation (7) , (7), a matrix composed of coefficients related to the prediction coefficient w needs to be regular), and an optimum prediction coefficient w can be obtained. In solving equation (7), for example, a sweeping-out method (Gauss-Jordan elimination method) or the like can be applied.

As described above, an optimal set of prediction coefficients w is obtained, and the set of prediction coefficients w
According to the equation (1), the predicted value E close to the pixel value y of the HD pixel is obtained.
The adaptive processing is to determine [y] (however, it is also assumed that a set of prediction coefficients w is determined in advance and a predicted value is determined from the set of prediction coefficients w in the adaptive processing).

Note that the adaptive processing differs from the interpolation processing in that components not included in the SD image but included in the HD image are reproduced. That is, the adaptive processing is the same as the interpolation processing using the so-called interpolation filter as far as only the equation (1) is viewed, but the prediction coefficient w corresponding to the tap coefficient of the interpolation filter is obtained by using the teacher data y. Therefore, the components included in the HD image can be reproduced because they are obtained by learning. That is, a high-resolution image can be easily obtained. From this, it can be said that the adaptive processing has a so-called image creation action.

FIG. 25 shows an example of the configuration of an image conversion apparatus that converts an SD image into an HD image by the above-described adaptive processing based on the features (classes) of the image.

The SD image is supplied to a class classification circuit 101 and a delay circuit 102. In the class classification circuit 101, the SD pixels constituting the SD image are sequentially set as a pixel of interest, and the pixel of interest is , Into a predetermined class.

That is, first, the class classification circuit 101 forms a block by collecting some SD pixels around the target pixel (hereinafter referred to as a processing block as appropriate), and forms the processing block. A value assigned in advance to the pattern of pixel values of all SD pixels is supplied to the address terminal (AD) of the coefficient ROM 104 as the class of the pixel of interest.

More specifically, as shown in FIG. 26, for example, the classifying circuit 101 is a 5 × 5 SD pixel centered on the pixel of interest (indicated by a circle in FIG. 26) as shown by a dotted rectangle. Is extracted from the SD image,
Values corresponding to the pixel value patterns of these 25 SD pixels are output as the class of the pixel of interest.

Here, to represent the pixel value of each SD pixel,
For example, when a large number of bits such as 8 bits are assigned, the number of patterns of pixel values of 25 SD pixels is
(2 ⁸ ) A huge number of ²⁵ cases, making it difficult to speed up subsequent processing.

Therefore, as a pre-process before class classification, a process block is a process for reducing the number of bits of SD pixels constituting the process block, for example, ADRC (Ad
aptiv Dynamic Range Coding) processing.

That is, in the ADRC process, first, out of the 25 SD pixels constituting the processing block, the largest one (hereinafter, appropriately referred to as a maximum pixel) and the smallest one (hereinafter, appropriately referred to as a minimum pixel) of the pixel value. ) Is detected. Then, the pixel value MAX of the maximum pixel and the pixel value MIN of the minimum pixel
DR (= MAX−MIN) is calculated, and this D
Let R be the local dynamic range of the processing block. Based on the dynamic range DR, each pixel value forming the processing block is requantized to K bits smaller than the original number of allocated bits. That is, the pixel value MIN of the minimum pixel from each pixel value forming the process block is subtracted, the subtraction value is divided by DR / 2 ^K.

As a result, each pixel value forming the processing block is represented by K bits. Therefore, for example, when K = 1, the number of patterns of the pixel values of the 25 SD pixels is (2 ¹ ) ²⁵ , which is a very small number of patterns as compared with the case where the ADRC processing is not performed. can do. The ADRC processing for setting the pixel value to K bits in this manner is hereinafter referred to as K-bit ADRC processing as appropriate.

The coefficient ROM 104 stores, for each class, a set of prediction coefficients obtained by performing learning in advance. When a class is supplied from the class classification circuit 101, the coefficient ROM 104 stores an address corresponding to the class. The stored set of prediction coefficients is read and supplied to the prediction calculation circuit 105.

On the other hand, in the delay circuit 102, the timing at which the set of prediction coefficients is supplied from the coefficient ROM 104 to the prediction operation circuit 105 coincides with the timing at which the prediction tap is supplied from the prediction tap generation circuit 103 described later. The SD image is delayed by a time necessary for the SD image to be supplied to the prediction tap generation circuit 103.

The prediction tap generation circuit 103 extracts, from the SD image supplied thereto, SD pixels used for obtaining a prediction value of a predetermined HD pixel in the prediction calculation circuit 105, and uses the extracted SD pixels as prediction taps. 105
Supplied to That is, in the prediction tap generation circuit 103,
From the SD image, for example, the same processing block as that extracted by the class classification circuit 101 is extracted, and the SD pixels constituting the processing block are supplied to the prediction operation circuit 105 as prediction taps.

In the prediction operation circuit 105, the coefficient ROM 10
Prediction coefficients w, w ₂ from 4, and ..., the prediction taps x ₁ from the prediction tap generating circuit 103, x _2, with the ..., computation shown in equation (1), i.e., adaptive By performing the processing, a predicted value E [y] of the target pixel y is obtained,
This is output as the pixel value of the HD pixel.

That is, here, for example, a 3 × 3 H centered on the pixel of interest, which is surrounded by a solid square in FIG.
The prediction value of the D pixel (indicated by a dot in the figure) is determined from one prediction tap. In this case, the prediction calculation circuit 105 calculates the equation (1) for the nine HD pixels. Is performed. Therefore, the coefficient ROM
In 104, nine sets of prediction coefficients are stored at addresses corresponding to one class.

Hereinafter, the same processing is performed using other SD pixels as target pixels, thereby converting an SD image into an HD image.

Next, FIG. 27 shows the coefficient ROM 10 of FIG.
4 illustrates a configuration example of a learning device that performs a learning process of calculating a set of prediction coefficients for each class to be stored in No. 4.

HD to be teacher data y in learning
The image is supplied to the thinning circuit 111 and the delay circuit 114.
The D image is reduced, for example, by thinning out the number of pixels, and is thereby made an SD image. This SD image is supplied to the class classification circuit 112 and the prediction tap generation circuit 113.

In the class classification circuit 112 or the prediction tap generation circuit 113, the same processing as in the case of the class classification circuit 101 or the prediction tap generation circuit 103 in FIG. 25 is performed, whereby the class of the pixel of interest or the prediction tap is output. Is done. The class output from the class classification circuit 112 is supplied to the address terminals (AD) of the prediction tap memory 115 and the teacher data memory 116, and the prediction tap output from the prediction tap generation circuit 113 is supplied to the prediction tap memory 115.

In the prediction tap memory 115, the prediction tap supplied from the prediction tap generation circuit 113 is stored at the address corresponding to the class supplied from the class classification circuit 112.

On the other hand, in the delay circuit 114, the HD image is delayed by the time when the class corresponding to the pixel of interest is supplied from the classifying circuit 112 to the teacher data memory 116. Are supplied to the teacher data memory 116 as teacher data only.

In the teacher data memory 116, teacher data supplied from the delay circuit 114 is stored at an address corresponding to the class supplied from the class classification circuit 112.

Hereinafter, the same processing is repeated until all the SD pixels constituting the SD image obtained from all the HD images prepared for learning in advance are set as the target pixels.

As described above, the prediction tap memory 11
26 or the same address in the teacher data memory 116, the SD pixel indicated by a circle in FIG.
In the above, the SD pixel or the HD pixel having the same positional relation as the HD pixel indicated by the mark is stored as the learning data x or the teacher data y.

In the prediction tap memory 115 and the teacher data memory 116, a plurality of pieces of information can be stored at the same address, whereby the same address is classified into the same class. A plurality of learning data x and teacher data y can be stored.

Thereafter, the arithmetic circuit 117 reads the prediction tap as learning data or the pixel value of the HD pixel as teacher data stored at the same address from the prediction tap memory 115 or the teacher data memory 116 and uses them. Then, a set of prediction coefficients that minimizes the error between the predicted value and the teacher data is calculated by the least square method. That is, in the arithmetic circuit 117, a normal equation shown in Expression (7) is established for each class, and by solving this, a set of prediction coefficients for each class is obtained.

As described above, the set of prediction coefficients for each class obtained by the arithmetic circuit 117 is the coefficient R shown in FIG.
It is stored in the OM 104 at the address corresponding to the class.

In the learning process described above, there may be a class in which the number of normal equations necessary for obtaining a set of prediction coefficients cannot be obtained. A set of prediction coefficients obtained by ignoring and solving a normal equation is used as a default set of prediction coefficients.

By the way, according to the image conversion apparatus shown in FIG. 25, as described above, high-frequency components not included in the SD image obtained by reducing the number of pixels of the HD image by thinning them out are also used. Although it is possible to obtain an HD image including the original HD image, there is a limit in approaching the original HD image. The reason may be that the pixel value of the pixel of the SD image (SD pixel), which is obtained by thinning out the number of pixels of the HD image, is not optimal for restoring the original HD image.

Therefore, the applicant of the present application has previously proposed compression (encoding) of an image using adaptive processing in order to obtain a decoded image having an image quality closer to the original HD image ( For example, Japanese Patent Application No. 8-206552.

That is, FIG. 28 shows that the HD image is compressed (encoded) into an optimal SD image so that a decoded image closer to the original HD image can be obtained by the adaptive processing.
1 shows an example of the configuration of an image encoding device to be used.

The HD image to be encoded is output from the thinning unit 121.
And supplied to the error calculation unit 43.

In the thinning section 121, the HD image is converted to an SD image by simply thinning it, for example, and supplied to the correcting section 41. Upon receiving the SD image from the thinning unit 121, the correction unit 41 first outputs the SD image to the local decoding unit 122 as it is. The local decoding unit 122 has, for example, the same configuration as that of the image conversion apparatus shown in FIG. 25, and performs the above-described adaptive processing using the SD image from the correction unit 41, thereby obtaining an HD image.
The prediction value of the pixel is calculated and output to the error calculation unit 43. The error calculation unit 43 receives the HD from the local decode unit 122.
A prediction error (error information) of the prediction value of the pixel with respect to the original HD pixel is calculated and output to the control unit 44. Control unit 44
Controls the correction unit 41 in accordance with the prediction error from the error calculation unit 43.

That is, the correction unit 41 corrects the pixel value of the SD image from the thinning unit 121 under the control of the control unit 44 and outputs the corrected pixel value to the local decoding unit 122. The local decoding unit 122 uses the corrected SD image supplied from the correction unit 41 to obtain the predicted value of the HD image again.

Hereinafter, the same processing is repeated until, for example, the prediction error output from the error calculating section 43 becomes equal to or less than a predetermined value.

When the prediction error output from the error calculation section 43 becomes equal to or smaller than a predetermined value, the control section 44 sets the correction section 41
, Whereby the corrected SD image when the prediction error is equal to or less than a predetermined value is output as the optimal encoding result of the HD image.

Therefore, according to the SD image after this correction,
By applying the adaptive processing thereto, it is possible to obtain an HD image whose prediction error is equal to or less than a predetermined value.

Here, as described above, the SD image output from the image encoding apparatus in FIG. 28 can be said to be the most suitable for obtaining a decoded image closer to the original HD image. Correction unit 41, local decoding unit 122, error calculation unit 43, and control unit 44 of the encoding device
Can be called optimization processing.

[0060]

By the way, in the adaptive processing, a prediction tap is constituted by SD pixels around the HD pixel, and the prediction value of the HD pixel is obtained using the prediction tap. SD used as prediction tap
Pixels were to be selected independently of the image.

That is, in the prediction tap generation circuit 103 of the image conversion apparatus of FIG. 25 and the local decoding unit 122 of FIG. 28 configured similarly to the image conversion apparatus, prediction taps of a constant pattern are always generated (formed). Was to be done.

However, the image often has locally different characteristics. Therefore, if the characteristics are different, it is better to perform the adaptive processing using the prediction tap corresponding to the characteristics.
It is considered that a decoded image closer to the image quality can be obtained.

The present invention has been made in view of such a situation, and it is an object of the present invention to obtain a decoded image with higher image quality.

[0064]

According to a first aspect of the present invention, there is provided an image encoding apparatus, wherein one of pixels constituting a compressed image signal is set as a target pixel, and a plurality of pixels are used by using pixels in the vicinity of the target pixel. A first forming unit that forms a prediction tap of a pattern, a first prediction unit that predicts an original image signal from each of the prediction taps of a plurality of patterns and a predetermined prediction coefficient, and outputs a prediction value for each of the prediction taps of the plurality of patterns. And a first calculating means for calculating a prediction error of a predicted value for each of a plurality of patterns of prediction taps with respect to an original image signal, and a prediction tap for obtaining a minimum prediction error among the plurality of patterns of prediction taps. An adding means for adding a corresponding pattern code to the pixel value of the target pixel.

According to a thirteenth aspect of the present invention, one of pixels constituting a compressed image signal is set as a target pixel, and a plurality of patterns of prediction taps are formed using pixels in the vicinity of the target pixel. A first forming step, a first prediction step of predicting an original image signal from each of the plurality of patterns of prediction taps and a predetermined prediction coefficient, and outputting a predicted value for each of the plurality of patterns of prediction taps; A first calculation step of calculating a prediction error with respect to an original image signal of a prediction value for each of the prediction taps of the pattern, and a pattern code corresponding to a prediction tap from which a minimum prediction error is obtained among prediction taps of a plurality of patterns, An adding step of adding to the pixel value of the target pixel.

According to a twenty-fifth aspect of the present invention, there is provided an image decoding apparatus, wherein one of pixels constituting a compressed image signal included in encoded data is set as a target pixel, and a pattern code added to a pixel value of the target pixel. Forming a prediction tap of a pattern corresponding to the target pixel using pixels in the vicinity of the pixel of interest, a prediction tap formed by the formation unit, and a prediction coefficient, predicting an original image signal, and calculating a prediction value of the original image signal. And a predicting means to be obtained.

The image decoding method according to claim 32, wherein one of the pixels constituting the compressed image signal included in the encoded data is set as a target pixel, and the pattern code added to the pixel value of the target pixel. Forming a prediction tap of a pattern corresponding to the target pixel using pixels in the vicinity of the pixel of interest, a prediction tap formed by the formation step, and a prediction coefficient to predict an original image signal, and predicting the prediction value And a prediction step to be determined.

In the recording medium according to the thirty-ninth aspect, the encoded data is obtained by predicting a plurality of patterns by using one of pixels constituting a compressed image signal as a pixel of interest and using a pixel near the pixel of interest. Forming a tap, predicting an original image signal from each of the plurality of prediction taps and a predetermined prediction coefficient, outputting a prediction value for each of the plurality of prediction taps, and calculating a prediction value for each of the plurality of prediction taps; , By calculating a prediction error for the original image signal, and adding a pattern code corresponding to a prediction tap for which a minimum prediction error is obtained among prediction taps of a plurality of patterns to a pixel value of a pixel of interest. There is a feature.

In the image coding apparatus according to the first aspect, the first forming means sets one of the pixels constituting the compressed image signal as a target pixel and uses a pixel near the target pixel. , Forming a plurality of patterns of prediction taps,
The prediction means of each of a plurality of prediction taps,
An original image signal is predicted from a predetermined prediction coefficient, and a predicted value for each of a plurality of prediction taps is output. The first calculating means calculates a prediction error of a predicted value for each of the plurality of patterns of prediction taps with respect to the original image signal, and the adding means calculates a prediction tap for which a minimum prediction error is obtained among the plurality of patterns of prediction taps. The corresponding pattern code is added to the pixel value of the target pixel.

In the image coding method according to the thirteenth aspect, one of the pixels constituting the compressed image signal is set as a target pixel, and prediction taps of a plurality of patterns are formed by using pixels near the target pixel. Forming an original image signal from each of the plurality of patterns of prediction taps and a predetermined prediction coefficient, outputting a prediction value for each of the plurality of patterns of prediction taps; Calculating the prediction error for the image signal,
A pattern code corresponding to a prediction tap from which a minimum prediction error is obtained among prediction taps of a plurality of patterns is added to the pixel value of the target pixel.

In the image decoding apparatus according to the twenty-fifth aspect, the forming means sets one of the pixels constituting the compressed image signal included in the encoded data as a target pixel and adds the pixel value to the pixel value of the target pixel. A prediction tap of a pattern corresponding to the pattern code being formed is formed using pixels near the pixel of interest, and the prediction unit predicts an original image signal from the prediction tap formed by the formation unit and the prediction coefficient. And
The prediction value is obtained.

In the image decoding method according to the thirty-second aspect, one of pixels constituting a compressed image signal included in encoded data is set as a target pixel, and a pattern added to the pixel value of the target pixel. A prediction tap of a pattern corresponding to a code is formed using pixels near the pixel of interest, an original image signal is predicted from the prediction tap and a prediction coefficient, and a prediction value is obtained.

In the recording medium according to the thirty-ninth aspect, one of the pixels constituting the compressed image signal is set as a pixel of interest.
A prediction tap of a plurality of patterns is formed using pixels in the vicinity of the pixel of interest, and an original image signal is predicted from each of the prediction taps of the plurality of patterns and a predetermined prediction coefficient, and prediction for each of the prediction taps of the plurality of patterns is performed. A value is output, a prediction error of the prediction value for each of the plurality of patterns of prediction taps is calculated, and a prediction error with respect to the original image signal is calculated. Coded data obtained by adding the pixel value to the pixel value of the target pixel is recorded.

[0074]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiments of the present invention will be described below, but before that, the correspondence between each means of the invention described in the claims and the following embodiments will be clarified. For this reason, the features of the present invention are described as follows by adding the corresponding embodiment (however, an example) in parentheses after each means.

That is, an image coding apparatus according to claim 1 is an image coding apparatus for coding an image signal, wherein the compression means generates a compressed image signal having a smaller number of pixels than the number of pixels of the original image signal. (For example, the thinning circuit 31 shown in FIG. 5) and one of the pixels constituting the compressed image signal is set as a target pixel, and a plurality of patterns of prediction taps are formed using pixels in the vicinity of the target pixel. An original image signal is formed from first forming means (for example, the prediction tap generation circuit 32 shown in FIG. 5 and the prediction tap generation circuit 61 shown in FIG. 22), prediction taps of a plurality of patterns, and predetermined prediction coefficients. Predicting means for predicting a plurality of patterns and outputting predicted values for each of a plurality of prediction taps (for example, a class classification adaptive processing circuit 33 shown in FIG. 5 or a class classification adaptive processing circuit shown in FIG. 22). 2) and first calculation means (for example, a prediction error calculation circuit 34 shown in FIG. 5 or a prediction error shown in FIG. 22) for calculating a prediction error of a prediction value for each of a plurality of prediction taps with respect to an original image signal. Calculation circuit 63) and a plurality of patterns of prediction taps.
An adding unit (for example, a tap pattern code adding circuit 36 shown in FIG. 5) for adding a pattern code corresponding to a prediction tap that can obtain a minimum prediction error to a pixel value of a target pixel.
And a tap pattern code change circuit 64 shown in FIG. 22).

According to a third aspect of the present invention, there is provided an image encoding apparatus comprising:
Is characterized in that it has an operation means (for example, an operation circuit 87 shown in FIG. 11) for obtaining a prediction coefficient by performing an operation based on the original image signal and the compressed image signal.

The image coding apparatus according to the fourth aspect is characterized in that the first
Predicting means classifies the target pixel into a predetermined class (for example, the classifying circuit 8 shown in FIG. 11).
2), and a prediction value is obtained from the prediction coefficient corresponding to the class of the pixel of interest and the prediction tap, and the calculating means calculates the prediction coefficient for each class based on the original image signal and the compressed image signal. It is characterized by seeking.

The image coding apparatus according to claim 7 further comprises an optimizing means (for example, an optimizing unit 23 shown in FIG. 3) for converting a compressed image signal into an optimal compressed image signal. And

In the image coding apparatus according to the present invention, the optimizing means forms the prediction tap of the pattern corresponding to the pattern code added to the pixel value of the pixel of interest (for example, FIG. The prediction tap generation circuit 42 shown in FIG.
A), a prediction tap formed by the second forming means, and a prediction coefficient, the second predicting means for predicting the original image signal and outputting the predicted value (for example, the class classification shown in FIG. 16) An adaptive processing circuit 42B), a second calculating means (for example, an error calculating circuit 43 shown in FIG. 16) for calculating a prediction error of the predicted value obtained by the second predicting means with respect to the original image signal, The image processing apparatus further includes a correction unit (for example, a correction unit 41 shown in FIG. 16) that corrects the pixel value of the target pixel in accordance with the prediction error calculated by the second calculation unit.

An image coding apparatus according to a tenth aspect of the present invention provides an image coding apparatus that calculates an original value of a prediction value obtained by a second prediction unit based on a compressed image signal obtained each time an optimization process is performed and an original image signal. A correction unit (for example, the adaptive processing unit 24 shown in FIG. 3) for correcting the prediction coefficient is further provided so as to reduce the prediction error with respect to the image signal, and the first and second prediction units are corrected by the correction unit. It is characterized in that a prediction value is obtained using a prediction coefficient.

The image coding apparatus according to the eleventh aspect provides an output unit (for example, the multiplexing unit 27 shown in FIG. 3) that outputs a compressed image signal output by the optimization unit and a prediction coefficient output by the correction unit. Etc.).

The image encoding apparatus according to the twelfth aspect provides an output means for outputting a compressed image signal to which a pattern code is added and a prediction coefficient (for example, the multiplexing unit 27 shown in FIG.
Etc.).

An image decoding apparatus according to the twenty-fifth aspect generates a compressed image signal having a smaller number of pixels than the number of pixels of the original image signal, and sets one of the pixels constituting the compressed image signal as a target pixel. A prediction tap of a plurality of patterns is formed using pixels in the vicinity of the pixel of interest, and an original image signal is predicted from each of the prediction taps of the plurality of patterns and a predetermined prediction coefficient, and a prediction value for each of the prediction taps of the plurality of patterns is formed. Is calculated, and a prediction error of a prediction value for each of the plurality of prediction taps with respect to the original image signal is calculated, and a pattern code corresponding to a prediction tap for which a minimum prediction error is obtained among the plurality of prediction taps is noted. An image decoding device for decoding encoded data including a compressed image signal obtained by adding to a pixel value of a pixel, wherein the encoded data includes A prediction tap of a pattern corresponding to a pattern code added to a pixel value of the pixel of interest is formed by using one of pixels constituting the compressed image signal of interest as a pixel of interest using pixels near the pixel of interest. A prediction means (for example, a prediction tap generating circuit 73 shown in FIG. 24), a prediction tap formed by the formation means, and a prediction coefficient for predicting an original image signal and obtaining a prediction value thereof (for example, , A classification adaptive processing circuit 74 shown in FIG. 24).

According to a twenty-eighth aspect of the present invention, the predicting means includes class classifying means for classifying a target pixel into a predetermined class (for example, a class classification circuit 91 shown in FIG. 17). Prediction coefficients corresponding to the classes of
A prediction value is obtained from the prediction tap, and the prediction coefficient is obtained for each class based on the original image signal and the compressed image signal.

According to the image decoding apparatus of the present invention, the encoded data also includes a prediction coefficient, and a separating means for separating a compressed image signal and a prediction coefficient from the encoded data (for example, as shown in FIG. 24). (Separating section 72 etc.).

Note that, of course, this description does not mean that each means is limited to those described above.

FIG. 1 shows the configuration of an embodiment of an image processing apparatus to which the present invention is applied. The transmission device 1 is supplied with image data of a digitized HD image. The transmission device 1 compresses and encodes the input image data by thinning out (decreasing the number of pixels), and converts the resulting image data of the SD image as encoded data of the HD image into, for example, an optical disc. Alternatively, the data is recorded on a recording medium 2 such as a magneto-optical disk, a magnetic tape, or the like, or transmitted, for example, via a terrestrial wave, a satellite line, a telephone line, a CATV network, or another transmission path 3.

The receiving device 4 reproduces the coded data recorded on the recording medium 2 or receives the coded data transmitted via the transmission path 3 and expands and decodes the coded data. The resulting decoded HD image is supplied to a display (not shown) for display.

The above-described image processing apparatus is, for example, an optical disk device, a magneto-optical disk device, a magnetic tape device, or another device for recording / reproducing images, or, for example, a videophone device, , A television broadcasting system, a CATV system, and other devices for transmitting images. Further, as described later, since the data amount of the encoded data output from the transmission device 1 is small, the image processing device in FIG. 1 has a low transmission rate, such as a mobile phone or other portable terminal that is convenient for movement. Is also applicable.

FIG. 2 shows an example of the configuration of the transmitting device 1.

An I / F (InterFace) 11 performs a process of receiving image data of an externally supplied HD image,
A process of transmitting encoded data to the recording device 16 is performed. ROM (Read Only Memory) 1
Reference numeral 2 stores a program for IPL (Initial Program Loading) and others. RAM (Random Access Me)
mory) 13 stores a system program (OS (Operating System)) and an application program recorded in the external storage device 15, and also stores a CPU (CeC).
The data required for the operation of the ntral processing unit 14 is stored. The CPU 14 has the ROM 1
According to the IPL program stored in the external storage device 15, the system program and the application program are loaded from the external storage device 15 into the RAM 13, and the application program is executed under the control of the system program. The encoding process described below is performed on the image data. The external storage device 15 is, for example, a magnetic disk device and stores the system programs and application programs executed by the CPU 14 as described above, and also stores data necessary for the operation of the CPU 14. The transmitter / recording device 16 includes an I / F 11
Is recorded on the recording medium 2,
Alternatively, the data is transmitted via the transmission path 3.

The I / F 11, the ROM 12, the RAM 1
3. The CPU 14 and the external storage device 15 are mutually connected via a bus. Further, in FIG. 2, the transmitting device 1 has a configuration using a CPU, but may be configured with hard logic.

In the transmission device 1 configured as described above, when the image data of the HD image is supplied to the I / F 11, the image data is supplied to the CPU 14. CP
U14 encodes the image data, and supplies an SD image as encoded data obtained as a result to the I / F 11. When receiving the encoded data, the I / F 11 supplies the encoded data to the transmitter / recording device 16. In the transmitter / recording device 16, the encoded data from the I / F 11 is recorded on the recording medium 2 or transmitted via the transmission path 3.

FIG. 3 is a functional block diagram of a portion of the transmitting device 1 of FIG. 2 except for the transmitter / recording device 16.

An HD image as image data to be encoded is supplied to a pre-processing unit 21, an optimizing unit 23, an adaptive processing unit 24, and a prediction tap pattern determining unit 26.

The pre-processing unit 21 performs pre-processing, which will be described later, on the HD image supplied thereto, for example, in units of one frame (or one field), and obtains the resulting SD image or a plurality of patterns. The set of prediction coefficients w for each class for each of the prediction taps is supplied to the terminal a of the switch 22 or 25. An SD image output from the preprocessing unit 21 or the prediction tap pattern determination unit 26 is supplied to the terminal a or b of the switch 22, respectively. In the pre-processing unit 21, the switch 22
The terminal a is selected only when the image is subjected to the preprocessing and the SD image is output, and otherwise, the terminal b is selected, and the preprocessing unit 21 or the predicted tap pattern determination unit 26 outputs Is supplied to the optimization unit 23.

The optimizing unit 23 performs the optimizing process described with reference to FIG. 28 on the SD image supplied from the switch 22, and outputs the resulting optimal SD image to the adaptive processing unit 25 The data is supplied to a pattern determination unit 26 and a multiplexing unit 27. The adaptive processing unit 24 includes the optimum SD image from the optimization unit 23 and the original HD image.
Optimum SD by performing adaptive processing using images
A set of prediction coefficients w for each class for reducing the prediction error of the prediction value of the HD image obtained by linear combination with the pixel value of the image is calculated for each of a plurality of patterns of prediction taps, and output to the terminal b of the switch 25. Has been made.

The switch 25 is operated by the pre-processing unit 21
The terminal a is selected only when a certain HD image is subjected to pre-processing, thereby outputting a set of prediction coefficients w for each class for each of a plurality of prediction taps,
In other cases, the terminal b is selected, and a set of prediction coefficients w for each class for each of the plurality of patterns of prediction taps output by the preprocessing unit 21 or the adaptive processing unit 24 is transmitted to the optimization unit 23 and the prediction taps. The data is supplied to the pattern determination unit 26 and the multiplexing unit 27.

The prediction tap pattern determination section 26 forms a plurality of patterns of prediction taps from the optimal SD image supplied from the optimization section 23, and performs an adaptive process using each of the plurality of prediction taps. The prediction values of a plurality of HD images are obtained. Further, the prediction tap pattern determination unit 26 determines, from among the plurality of patterns of prediction taps, one that minimizes the prediction error of the prediction values of the plurality of HD images, and in accordance with the determination result, A pattern code, which will be described later, is added to the pixel value of the optimal SD image, and is supplied to the terminal b of the switch 22.

In a predetermined case, the multiplexing unit 27 calculates a prediction coefficient w for each class for each of the optimum SD image supplied from the optimizing unit 23 and a plurality of patterns of prediction taps supplied via the switch 25. Multiplex with the set of
The result of the multiplexing is output to the transmitter / recorder 16 (FIG. 2) as encoded data.

Next, referring to the flowchart of FIG.
The operation will be described.

The HD image to be coded is
When supplied to the optimizing unit 23, the adaptive processing unit 24, and the predicted tap pattern determining unit 26, the preprocessing unit 21 performs preprocessing on the HD image in step S1.

That is, the pre-processing unit 21 configures an SD image by reducing the number of pixels of the HD image and compressing the SD image. To form a plurality of patterns of prediction taps. Further, the preprocessing unit 21 obtains a set of prediction coefficients w for each class by setting up and solving the normal equation shown in Expression (7) for each of the plurality of patterns of prediction taps. Then, the preprocessing unit 21 uses the prediction taps of the plurality of patterns and the set of prediction coefficients w of each class obtained for each of the classes to obtain the linear 1 By calculating the following equation, prediction values of a plurality of HD images obtained from each of the prediction taps of the plurality of patterns are obtained. further,
The preprocessing unit 21 detects a prediction tap of a plurality of patterns that minimizes a prediction error of a prediction value of a plurality of HD images, and, for example, a 2-bit prediction tap that is associated with the pattern of the prediction tap in advance. Is added to the SD pixel that is the target pixel and output.

As described above, the SD image to which the tap pattern code has been added is supplied to the terminal a of the switch 22 and the prediction tap of each of the plurality of patterns obtained by solving the normal equation. The set of prediction coefficients w is output to the terminal a of the switch 25, respectively.

As described above, the switches 22 and 25 are connected to the terminal a at the timing when the preprocessing unit 21 outputs the set of prediction coefficients w for each class for each of the SD image and the prediction taps of a plurality of patterns. Therefore, the SD image output by the pre-processing unit 21 is supplied to the optimization unit 23 via the switch 22. The set of prediction coefficients w for each class is output to the optimization unit 23 and the prediction tap pattern determination unit 26 via the switch 25.

Upon receiving the SD image and the set of prediction coefficients w for each class for each of the prediction taps of a plurality of patterns, in step S2,
The optimization processing is performed using them. That is, the optimization unit 23
Performs adaptive processing using a set of prediction coefficients w for each class for each of an SD image and a plurality of patterns of prediction taps, and generates an SD image such that a prediction error of a prediction value of an HD image obtained as a result is reduced. Is corrected. Then, the optimal SD image obtained as a result is supplied to the adaptive processing unit 24 and the prediction tap pattern determining unit 26.

Upon receiving the optimal SD image from the optimizing unit 23, the adaptive processing unit 24 performs an adaptive process in step S3 to obtain an H obtained using the optimal SD image.
A prediction coefficient w for each class for each of a plurality of patterns of prediction taps for reducing the prediction error of the prediction value of the D image
Is calculated. That is, the adaptive processing unit 24 determines the optimal S
A prediction tap is formed for each pixel of interest by sequentially setting each of the SD pixels constituting the D image as a pixel of interest. In addition,
At this time, a prediction tap having a pattern corresponding to the tap pattern code added to the target pixel is formed. Then, for each of the plurality of patterns of prediction taps, the adaptive processing unit 24 obtains a set of prediction coefficients w for each class of each of the plurality of patterns of prediction taps by forming a normal equation from the prediction taps and solving it. . The set of prediction coefficients w for each class for each of the plurality of prediction taps is supplied to the terminal b of the switch 25.

After the above processing, the process proceeds to step S4, in which both the switches 22 and 25 are switched from the terminal a to the terminal b, whereby the prediction taps of the plurality of patterns obtained by the adaptive processing unit 24 are obtained. The set of the prediction coefficient w for each class is transmitted to the optimization unit 23 and the prediction tap pattern determination unit 26 via the switch 25.
Will be supplied to

Then, the prediction tap pattern determination section 26
Receives the optimal SD image from the optimization unit 23,
When a set of prediction coefficients w for each class for each of the plurality of patterns of prediction taps is received from the adaptive processing unit 24, in step S5, each of the SD pixels constituting the optimal SD image is set as a target pixel. The optimal pattern is determined.

That is, the prediction tap pattern determination unit 26
A plurality of prediction taps are formed for each pixel of interest, with each of the SD pixels forming the optimal SD image as a pixel of interest. Further, a prediction tap pattern determination unit 2
6 is a formula (1) using a set of prediction coefficients of a predetermined class among a set of prediction coefficients w for each class corresponding to the prediction tap from the adaptive processing unit 24 for each of the plurality of patterns of prediction taps. ) Is calculated to obtain prediction values of a plurality of HD images obtained from each of a plurality of patterns of prediction taps. The prediction tap pattern determination unit 26 detects a prediction tap of a pattern corresponding to the smallest one of prediction errors of prediction values of a plurality of HD images obtained using each of the prediction taps of the plurality of patterns, and performs the prediction. In the tap pattern code corresponding to the tap, S
The tap pattern code already added to the D pixel is changed. That is, in this case, since a tap pattern code has already been added to the SD pixel,
A tap pattern code of a prediction tap that minimizes the prediction error is added.

As described above, the SD image whose tap pattern code has been changed is output to the terminal b of the switch 22.

The switch 22 is switched in step S4 to select the terminal b, so that the SD image output from the predictive tap pattern determining section 26 is supplied to the optimizing section 23 via the switch 22. In step S6, the optimizing unit 23 performs an optimizing process in the same manner as in step S2, thereby outputting an optimal SD image. In this case, in the optimization unit 23, the adaptive processing is performed as described in step S2. This adaptive processing is performed for a plurality of patterns of prediction taps supplied from the adaptive processing unit 24 via the switch 25. This is performed using a set of prediction coefficients w for each class.

The optimum SD image output from the optimizing unit 23 is output to the adaptive processing unit 24 and the prediction tap pattern determining unit 2.
6 and in the adaptive processing unit 24, in step S7, as in the case of step S3, the optimization unit 2
The adaptive processing is performed using the optimal SD image output by the third unit 3, whereby a set of prediction coefficients w for each class for each of a plurality of prediction taps is obtained. The prediction tap pattern is output to the prediction tap pattern determination unit 26.

Thereafter, the process proceeds to step S8, where
It is determined whether the processes from 5 to S8 have been performed a predetermined number of times. If it is determined in step S8 that the processes of steps S5 to S8 have not been performed a predetermined number of times, the process returns to step S5, and the above processes are repeated. Also, in step S8, step S8
When it is determined that the processes of 5 to S8 have been performed a predetermined number of times, the process proceeds to step S9, where the multiplexing unit 27 determines, in the previous process of step S6, the optimal SD image output by the optimization unit 23, The set of prediction coefficients w for each class for each of the plurality of patterns of prediction taps used at that time is multiplexed and output as encoded data, and the process ends.

The above processing is repeated, for example, in units of one frame.

In the above case, in step S8, it is determined whether or not the processes of steps S5 to S8 have been performed a predetermined specified number of times. In step S8, for example, the optimization unit It is determined whether or not the sum of the absolute values of the prediction errors of the prediction values of the HD image obtained by performing the adaptive processing using the optimal SD image output from 23 for one frame is equal to or less than a predetermined threshold value. If the value is equal to or less than the threshold value, the process proceeds to step S9. If the value is not equal to or less than the threshold value, the process returns to step S5. That is, steps S5 to S8
May be repeated until the sum of the absolute value of the prediction error of the prediction value of the HD image obtained by performing the adaptive process using the optimal SD image for one frame is equal to or less than a predetermined threshold value. It is possible.

FIG. 5 shows an example of the configuration of the pre-processing unit 21 shown in FIG.

The HD image to be encoded is output from the thinning circuit 3
1. It is supplied to a class classification adaptive processing circuit 33 and a prediction error calculation circuit 34.

The thinning circuit 31 calculates the number of pixels of the HD image by
For example, the number is reduced by thinning to form an SD image, which is supplied to the prediction tap generation circuit 32 and the tap pattern code addition circuit 36. That is,
The thinning circuit 31 converts, for example, an HD image into 3 ×
The image is divided into square blocks each including three pixels of nine pixels, and an SD image is configured by using the average value of the nine pixels in each block as the pixel value of the center pixel. Thereby, in the thinning circuit 31, for example, an HD image composed of HD pixels indicated by a mark in FIG.
An SD image composed of D pixels is configured.

Note that the thinning circuit 31 extracts only the pixel at the center of the above-mentioned block.
It is also possible to configure a D image.

The prediction tap generation circuit 32 includes a thinning circuit 3
Each of the SD pixels constituting the SD image from No. 1 (indicated by a circle in FIG. 6) is sequentially set as a target pixel, and a plurality of patterns of prediction taps are formed for each target pixel. That is, in the present embodiment, for example, as shown in FIGS. 7 to 10, four patterns of 3 × 3 pixels, 5 × 3 pixels, 3 × 5 pixels, or 7 × 5 pixels centering on the pixel of interest. Of prediction taps are formed. These four prediction taps are supplied to the classification adaptive processing circuit 33.

The class classification adaptive processing circuit 33 classifies each of the four patterns of prediction taps supplied from the prediction tap generation circuit 32, and furthermore, for each class, an HD image is used as shown in equation (7). By setting up and solving the normal equation, a set of prediction coefficients w for each class for each of the four prediction taps is obtained. In addition, the class classification adaptive processing circuit 33 includes, among the set of prediction coefficients w for each class obtained for each of the obtained four patterns of prediction taps, a predetermined class of prediction coefficients w and four patterns of prediction taps. By calculating the linear linear expression shown in Expression (1), the prediction values of a plurality of HD images obtained from each of the four prediction taps are obtained, and output to the prediction error calculation circuit 34. ing.

In the class classification adaptive processing circuit 33, a set of prediction coefficients w for each class obtained for each of the four prediction taps is supplied to the memory 35.

In the present embodiment, in the class classification adaptive processing circuit 33, for each of the four prediction taps, the normal equation is surrounded by a dotted line in FIG. 6, for example, regardless of the prediction tap pattern. As shown,
3 × 3 H centered on the SD pixel that is the pixel of interest
It is designed to obtain a predicted value of the D pixel. Therefore, in the classification adaptive processing circuit 33, 3
A set of prediction coefficients for each class is obtained for each of four patterns of prediction taps for generating a prediction value of × 3 HD pixels. This classification adaptive processing circuit 33
The detailed configuration will be described later.

The prediction error calculation circuit 34 calculates, for each pixel of interest, H obtained from each of the four prediction taps.
The prediction error of the prediction value of the D image with respect to the pixel value of the original HD image is obtained. That is, for each of the four prediction taps, for example, the sum of squares of the difference between the prediction value of the nine HD pixels and the pixel value of the nine HD pixels is calculated. Then, the prediction error calculation circuit 3
Reference numeral 4 detects a prediction tap having the smallest prediction error (sum of squares of the difference) among the four prediction taps. Further, the prediction error calculation circuit 34 stores the 2-bit tap pattern code corresponding to the pattern of the prediction tap having the smallest prediction error in the memory 35 and the tap pattern code adding circuit 3.
6 is output.

The memory 35 includes a class classification adaptive processing circuit 3
A set of prediction coefficients w for each class obtained for each of the four prediction taps supplied from 3 is temporarily stored. And the memory 35
For example, when the processing of the HD image of one frame (or one field) is completed (that is, a tap pattern code is added to all SD pixels), prediction for each class obtained for each of the four prediction taps is performed. The set of the coefficient w is read and output to the terminal a of the switch 25.

The tap pattern code adding circuit 36 applies a prediction error calculating circuit 3 to the SD image supplied thereto.
4 is added to the tap pattern code. That is, the tap pattern code addition circuit 36 generates the LSB (Least Sign) of the pixel value (for example, composed of 8 bits) of the SD pixel serving as the target pixel.
2 bits on the side of the significant bit are deleted, and a 2-bit tap pattern code is arranged there. The SD image added with the tap pattern code in the tap pattern code adding circuit 36 is
Is output to the terminal a of

Here, the configuration of the class classification adaptive processing circuit 33 will be described. The classification adaptive processing circuit 33
Classification adaptive processing circuit that performs processing on each of the four prediction taps (calculation of prediction coefficients and prediction values)
have. That is, the classification adaptive processing circuit 33
Has four independent class classification adaptive processing circuits (prediction coefficient, prediction value calculation) for each of the four prediction taps. 11 and 12 show one of them.
Classification adaptive processing circuit (calculation of prediction coefficient and prediction value)
Is shown. The four class classification adaptive processing circuits (prediction coefficient and prediction value calculation) have the same configuration except that four different prediction taps are supplied. Value calculation), and the others are omitted.

The class classification adaptive processing circuit (calculation of prediction coefficients and prediction values) shown in FIGS. 11 and 12 includes a class classification circuit 82, a delay circuit 84, a prediction tap memory 85, a teacher data memory 86, an arithmetic circuit 87, and a delay circuit. The circuit 88 (FIG. 1)
1) and a class classification circuit 91, a coefficient RAM 94, and a prediction calculation circuit 95 (FIG. 12).

The class classification circuit 82, the delay circuit 84, the prediction tap memory 85, the teacher data, except the delay circuit 88, which constitutes a part of the class classification adaptive processing circuit (prediction coefficient, prediction value calculation) shown in FIG. The memory 86 or the arithmetic circuit 87 includes a class classification circuit 112, a delay circuit 114, a prediction tap memory 115,
The configuration is the same as that of the teacher data memory 116 or the arithmetic circuit 117. However, since the prediction tap is supplied from the prediction tap generation circuit 32, the delay circuit 8 is used instead of the prediction tap generation circuit 113 shown in FIG.
8 is provided, and the prediction tap from the prediction tap generation circuit 32 is supplied to the delay circuit 88. Then, in the delay circuit 88, similarly to the delay circuit 84, the prediction tap is delayed by the time when the class for the target pixel is supplied from the class classification circuit 82 to the prediction tap memory 85, and is supplied to the prediction tap memory 85. It is made to be memorized.

A class classification circuit 91 or a prediction calculation circuit 95, which is another part of the class classification adaptive processing circuit (prediction coefficient and prediction value calculation) shown in FIG. Classification circuit 1 shown
01 or the prediction operation circuit 105 respectively. The coefficient RAM 94 corresponds to the arithmetic circuit 87 of FIG.
Is stored as a set of prediction coefficients for each class.

In the class adaptive processing circuit (predictive coefficient, predictive value calculation) configured as described above, one frame of H
Data for the D pixel is stored in the prediction tap memory 85 and the teacher data memory 86 in substantially the same manner as in FIG. 27, and a set of prediction coefficients for each class is generated. The generated set of prediction coefficients for each class is supplied to and stored in the coefficient RAM 94 in FIG. 12, and is also supplied to and stored in the memory 35 of the preprocessing unit 21 in FIG. As described above, a set of prediction coefficients for each class for each of the four prediction taps is generated by an independent circuit, so that a set of prediction coefficients for each class for each of the four prediction taps is generated. It is supplied to and stored in the coefficient RAM 94 of FIG. 12, and is also supplied to and stored in the memory 35 of the preprocessing unit 21 of FIG.

The class classification circuit 91, coefficient RAM 94, and prediction calculation circuit 95 which form part of the class classification adaptive processing circuit (calculation of prediction coefficients and prediction values) shown in FIG.
When the set of prediction coefficients for each class is stored in the coefficient RAM 94, the class classification circuit 10 of the image conversion apparatus shown in FIG.
1, a coefficient RAM 104 or a prediction operation circuit 105;
The same processing is performed for each of them, whereby the predicted value of the HD image is obtained. That is, the set of prediction coefficients for each class for each of the four prediction taps is shown in FIG.
Stored in the coefficient RAM 94 of the classifying circuit 91
, A class classification is performed, and the class information is a coefficient RA
M94. The coefficient RAM 94 outputs a set of prediction coefficients corresponding to the supplied class information and supplies it to the prediction calculation circuit 95. The prediction calculation circuit 95 obtains prediction values of a plurality of HD images by calculating the linear linear expression shown in Expression (1) from the supplied prediction tap and the set of prediction coefficients.

Since the class classification circuit 82 and the class classification circuit 91 have the same configuration, only one of them may be provided.

Next, the processing of the pre-processing unit 21 will be described with reference to the flowchart of FIG.

When an HD image to be coded is input to the preprocessing unit 21, the HD image is subjected to a thinning circuit 31, a class classification adaptive processing circuit 33, and a prediction error calculating circuit 34.
Supplied to Upon receiving the HD image, the thinning circuit 31 thins out the number of pixels in step S11, and
Construct D image.

That is, in step S11, as shown in the flowchart of FIG.
In, the HD image is divided into, for example, 3 × 3 pixel HD image blocks, and the process proceeds to step S22.

In this embodiment, the HD image is composed of, for example, a luminance signal Y and color difference signals U and V. In step S21, a luminance signal block and a color difference signal block are composed. It has been made like that.

In step S22, one of the blocks is set as a target block, and 3
The average value of the pixel values of the × 3 HD pixels is calculated. Further, in step S22, the average value is used as the pixel value of the pixel (SD pixel) at the center of the block of interest.
Proceed to 23.

In step S23, it is determined whether or not the target block is a block of a luminance signal. If it is determined in step S23 that the block of interest is a block of a luminance signal, the process proceeds to step S24, where the LSB side 2 bits of the pixel value (here, the luminance signal) of the center pixel of the block of interest as the SD pixel Is cleared to, for example, 0 to add a tap pattern code, and the process proceeds to step S25. Also, in step S23,
If it is determined that the block of interest is not a block of a luminance signal, that is, if the block of interest is a block of a chrominance signal, step S24 is skipped and step S25 is skipped.
Proceed to.

In this embodiment, a plurality of patterns of prediction taps are prepared only for the luminance signal, and a fixed pattern prediction tap is used for the color difference signal. Therefore, the tap pattern code is added only for the luminance signal.
Since the tap pattern code is not added to the color difference signal, the two bits on the LSB side are not cleared.

In step S25, it is determined whether or not all the blocks formed in step S21 have been processed as the block of interest. If it is determined that all the blocks have not been processed as the block of interest, the process proceeds to step S22. Then, the same processing is repeated with a block that has not been set as a target block as a new target block. If it is determined in step S25 that all blocks have been processed as the block of interest, that is, if an SD image has been formed, the process returns.

Returning to FIG. 13, in step S11,
The SD image constructed as described above is output to the thinning circuit 3
1 is supplied to the prediction tap generation circuit 32 and the tap pattern code addition circuit 36. Predictive tap generation circuit 3
When the SD image is received from the thinning circuit 31, in step S12, the SD pixels constituting the SD image are sequentially set as target pixels, and the target pixels are shown in FIGS.
Are formed (generated) and supplied to the classification adaptive processing circuit 33.

As described above, four prediction taps are formed only for the luminance signal, and for the color difference signal, for example, the 7 × as shown in FIG.
Only 5 pixel prediction taps are always formed.

In step S13, the class classification adaptive processing circuit 33 first obtains, for each of the four prediction taps (in the case of a luminance signal) supplied from the prediction tap generation circuit 32, as shown in FIG. 11 and FIG. Classification is performed by each of the configured classification adaptive processing circuits (calculation of prediction coefficients and prediction values).

Here, in the present embodiment, in classifying circuits 82 and 91 (FIGS. 11 and 12), 4
For each prediction tap of the pattern, for example, the following tap for class classification (hereinafter, appropriately referred to as a class tap) is configured to perform the class classification.

That is, as for the luminance signal, for any of the four prediction taps, for example, FIG.
As shown by a dotted line, a class tap is formed by five SD pixels in a diamond-shaped range centered on the target pixel. Then, a difference between the maximum value and the minimum value of the pixel values of the five pixels is defined as a dynamic range DR, and three pixels arranged vertically in the class tap are used by using the dynamic range DR (FIG. 15A). Are subjected to one-bit ADRC processing. Then, a pattern obtained by adding a tap code corresponding to the prediction tap to the pixel value pattern of the three pixels is set as the class of the target pixel. Therefore, in this case, among the class taps, a pattern of pixel values obtained by performing 1-bit ADRC processing on three pixels arranged vertically is expressed by 3 bits, and since the tap code is 2 bits, the luminance signal is , 32 (= 2 ⁵ ) classes.

On the other hand, as for the color difference signal, for example, FIG.
As shown by the dotted line in FIG. 5 (B), a class tap is constituted by nine SD pixels in a square range centered on the target pixel. Then, a difference between the maximum value and the minimum value of the pixel values of the nine pixels is defined as a dynamic range DR, and the dynamic range DR is used to define a diamond-shaped range of the class tap centered on the target pixel. 5
SD pixel (5 pixels surrounded by a solid line in FIG. 15B)
Are subjected to 1-bit ADRC processing. Then, the pattern of the pixel values of the five pixels is set as the class of the target pixel. Therefore, in this case, since a pixel value pattern obtained by performing 1-bit ADRC processing on 5 pixels centered on the target pixel among the class taps is expressed by 5 bits, the color difference signal is also similar to the luminance signal. , 32 (= 2 ⁵ ) classes.

In the class classification adaptive processing circuit 33, the class of the pixel of interest is determined as described above, whereby the address corresponding to each address of the prediction tap memory 85 or the teacher data memory 86 (FIG. 11). A prediction tap or HD pixel (teacher data) of the class is stored. Then, in the arithmetic circuit 87 (FIG. 11), for each of the four prediction taps, the prediction tap or the HD image (teacher data) stored in the prediction tap memory 85 or the teacher data memory 86 for each class.
Is used to solve the normal equation of Expression (7), and by solving the equation, a set of prediction coefficients w for each class for each of the four prediction taps is obtained. 4
A set of prediction coefficients w for each class for each of the four prediction taps obtained using each of the pattern prediction taps is supplied to the memory 35 and the coefficient RAM 94 and stored.

Thereafter, the classification adaptive processing circuit 33
In step S14, from each set of prediction coefficients w for each class for each of the four prediction taps obtained using the four prediction taps, and from each of the four prediction taps, Equation (1) is used. By calculating the calculated linear linear equation, a prediction value of the HD image obtained from each of the four prediction taps is obtained, and output to the prediction error calculation circuit.

That is, in step S14, the thinning circuit 3
With one of the SD pixels constituting the SD image output from the target pixel 1 as a target pixel, the prediction tap generated by the prediction tap generation circuit 32 is classified into a class classification circuit 91 (FIG. 12).
Is read from the coefficient RAM 94 (FIG. 12) stored at the address corresponding to the class output by. Then, in the prediction calculation circuit 95 (FIG. 12), the linear linear expression of the expression (1) is calculated by using the set of the prediction coefficient w from the coefficient RAM 94 and the prediction tap for the pixel of interest. The prediction values of the nine HD pixels around the pixel of interest described with reference to FIG. 6 are obtained and supplied to the prediction error calculation circuit 34.

In the classification adaptive processing circuit 33, a prediction value is obtained for each of the four prediction taps.

The prediction error calculation circuit 34 determines in step S15
In, the prediction error of the prediction value of the HD image for each of the four prediction taps supplied from the classification adaptive processing circuit 33 with respect to the pixel value of the original HD image is determined. That is, for example, for each of the four prediction taps, the prediction value of nine HD pixels and the original H
The sum of squares of the difference between the pixel value of the D pixel and the pixel value is obtained as a prediction error. Then, the process proceeds to step S16, and a prediction tap with a minimum prediction error is detected for the target pixel. further,
The prediction error calculation circuit 34 calculates 2
The bit tap pattern code is output to the tap pattern code adding circuit 36.

In the tap pattern code adding circuit 36,
In step S17, the LS of the pixel value of the target pixel among the SD pixels constituting the SD image from the thinning circuit 31 (however, in the present embodiment, only for the luminance signal)
The two bits on the B side are output as a tap pattern code (a 2-bit tap pattern code is added instead of the two LSB bits of the pixel value of the target pixel).

Thereafter, the flow advances to step S18 to determine whether or not the tap pattern code has been added to all the SD pixels. If it is determined that the tap pattern code has not yet been added to all the SD pixels, the process proceeds to step S14. Then, the processing of steps S14 to S18 is repeated with any one of the SD pixels to which the tap pattern code is not added as a new target pixel. On the other hand, if it is determined in step S18 that the tap pattern code has been added to all SD pixels, the memory 35 outputs a set of prediction coefficients w for each class for each of the four prediction taps in step S19. Then, the process ends.

As described above, the pre-processing unit 21 uses the average value of the 3 × 3 HD pixels as the pixel value in the SD pixels constituting the SD image output from the thinning circuit 31 as described above. ), A tap pattern code of a prediction tap that minimizes a prediction error is temporarily added.

Next, FIG. 16 shows an example of the configuration of the optimizing unit 23 shown in FIG. Note that, in the figure, the same reference numerals are given to portions configured basically in the same manner as in the image encoding device in FIG. That is, the optimization unit 23
Has no thinning circuit 121 and has a local decoding unit 12
The configuration is basically the same as that of the image encoding device in FIG. 28 except that a local decoding unit 42 is provided in place of 2.

The local decoding section 42 is composed of a prediction tap generating circuit 42A and a class classification adaptive processing circuit 42B, to which an SD image is supplied from the correcting section 41. Predictive tap generation circuit 42A
Is the L of the SD pixel of the SD image supplied from the correction unit 41.
A prediction tap is formed (generated) in accordance with the tap pattern code arranged on the SB side, and is supplied to the classification adaptive processing circuit 42B. The class classification adaptive processing circuit 42B is supplied with a set of prediction coefficients w for each class of SD pixels for class classification and four types of prediction taps in addition to the prediction taps. The class classification adaptive processing circuit 42B classifies the pixel of interest forming the prediction tap using the SD pixel for class classification as described with reference to FIG.
By calculating the linear linear expression shown in Expression (1) from the set of prediction coefficients w corresponding to the class and the prediction tap, the pixel of interest is indicated by a dotted line in FIG. A prediction value of a pixel value of a 3 × 3 HD pixel centered on the SD pixel is obtained. This prediction value is supplied to the error calculation unit 43.

FIG. 17 shows an example of the configuration of the classification adaptive processing circuit 42B of FIG. In the figure,
Parts corresponding to those in FIG. 12 are denoted by the same reference numerals. That is, the classification adaptive processing circuit 42
B has the same configuration as a part of one class classification adaptive processing circuit (prediction coefficient, prediction value calculation) constituting the class classification adaptive processing circuit 33 shown in FIG. 12, and a description thereof will be omitted.

Next, the operation will be described with reference to the flowchart of FIG.

When the optimizer 23 receives the SD image,
One of the SD pixels constituting the SD image is set as a target pixel, and in step S31, a variable △ representing a correction amount for correcting the pixel value of the target pixel is initialized to, for example, 0. In step S31, a variable S representing a change amount for changing the correction amount (hereinafter, appropriately referred to as an offset amount) is set.
For example, 4 or 1 is set as an initial value.

That is, as described above, since the two bits on the LSB side of the luminance signal are the tap pattern code and do not constitute the pixel value, the offset amount S is 4 (= 2 ² ). Is set. This is not the case for the color difference signal, and all the bits constitute a pixel value.
2 ⁰ ) is set.

In step S31, a variable i for counting the number of corrections of the target pixel is set to-
1 is set, and the process proceeds to step S32. Step S3
In step 2, the number i is incremented by one, and the process proceeds to step S33. When the adaptive processing is performed using the correction value obtained by correcting the pixel value of the target pixel by the correction amount △, the HD pixel affected by the correction is subjected to the adaptive processing. A prediction error E of the prediction value is calculated.

That is, in this case, the correction unit 41 adds, for example, a correction amount △ to the pixel value of the pixel of interest, and uses the added value as the pixel value of the pixel of interest as the local decoding unit 42.
Output to Here, when the process of step S33 is first performed on the target pixel, that is, when the number of times i = 0, the correction amount △ remains 0, which is the initial value set in step S31. The correction unit 41 outputs the pixel value of the target pixel as it is.

In the local decoding section 42, a prediction tap is formed in the prediction tap generation circuit 42A in accordance with the tap pattern code arranged in the two LSB bits of the pixel value of the target pixel, and the classification adaptive processing is performed. Output to the circuit 42B. Classification adaptive processing circuit 42B
First, the target pixel is classified into the same class as in the case of the classification adaptive processing circuit 33 shown in FIG. Further, the class classification adaptive processing circuit 42B computes the linear linear expression shown in Expression (1) from the prediction coefficient corresponding to the class and the prediction tap from the prediction tap generation circuit 42A, thereby obtaining the HD pixel. Is calculated.

That is, in the class classification adaptive processing circuit 42B, in the class classification circuit 91 (FIG. 17), the class tap described in FIG. 15 is formed from the SD pixels forming the prediction tap from the prediction tap generation circuit 42A. Then, classification is performed. The class obtained as a result of the classification by the classification circuit 91 is a coefficient RAM
94 (FIG. 17).

The coefficient RAM 94 (FIG. 17)
A set of prediction coefficients for each class for each of the four patterns of prediction taps supplied via 5 is stored as a set of prediction coefficients corresponding to the class from the class classification circuit 91 and added to the pixel of interest. The set of prediction coefficients for the prediction tap corresponding to the tap pattern code being read is read. This set of prediction coefficients is supplied to the prediction calculation circuit 95 (FIG. 17).

The prediction operation circuit 95 uses the set of prediction coefficients from the coefficient RAM 94 and the prediction tap supplied from the prediction tap generation circuit 42A to calculate the linear 1
By calculating the following equation, a predicted value of the HD pixel is obtained.

In the classification adaptive processing circuit 42B, when the pixel value of the target pixel is corrected by the correction amount △,
For HD pixels affected by the correction, a predicted value is similarly obtained.

That is, for example, it is assumed that the SD pixel A is corrected as a target pixel as shown in FIG. In the present embodiment, the range of the prediction tap is the widest in FIG.
As shown in FIG. 0, when a prediction tap is composed of 7 × 5 SD pixels, and when a prediction tap is composed of 7 × 5 SD pixels, an SD pixel A is included in the prediction tap. Is included, and the SD pixel farthest from the SD pixel A is set as the target pixel because the SD pixels B, C,
In this case, D and E are regarded as target pixels, and prediction taps of 7 × 5 pixels are formed. Then, the SD pixels B, C, D,
In the case where E is the pixel of interest and a prediction tap of 7 × 5 pixels is configured, in the present embodiment, 3 × 3 HDs in the ranges b, c, d, and e surrounded by solid lines in FIG. A predicted value of the pixel is obtained. Therefore, when the pixel value is corrected using the SD pixel A as the target pixel, the correction is affected in the worst case by the ranges b, c, d, and e.
Are the predicted values of 21 × 15 HD pixels within the range indicated by the dotted line in FIG.

Therefore, in the present embodiment, in the classification adaptive processing circuit 42B, such a 21 × 15 H
A predicted value of the D pixel is obtained.

The predicted value of the HD pixel obtained by the classification adaptive processing circuit 42B is supplied to the error calculator 43.
The error calculator 43 subtracts the true pixel value of the corresponding HD pixel from the predicted value of the HD pixel from the classification adaptive processing circuit 42B, and obtains, for example, the sum of squares of the prediction error as the subtracted value. Then, this sum of squares is supplied to the control unit 44 as error information E.

Upon receiving the error information from the error calculating unit 43, the control unit 44 determines in step S34 that the number of times i is 0.
Is determined. In step S34,
When it is determined that the number of times i is 0, that is, the control unit 44
If the received error information E is obtained without correcting the target pixel, the process proceeds to step S35, and the error information obtained without correcting the target pixel (error information at the time of uncorrection). Is set to a variable E ₀ for storing the error information, and the error information E is also set to a variable E ′ for storing the error information obtained last time. Further, step S3
In 5, the correction amount △ is incremented by the offset amount S, and the control unit 44 controls the correction unit 41 to correct the pixel value of the target pixel by the correction amount △ obtained thereby. Thereafter, the process returns to step S32, and thereafter, the same processing is repeated.

In this case, in step S32, the number i is incremented by 1 to become 1. In step S34, it is determined that the number i is not 0, and the flow advances to step S36. In step S36, it is determined whether the number i is one. In this case, since the count i is 1, the count i is determined to be 1 in step S36, the process proceeds to step S37, and the previous error information E ′ is obtained.
Is greater than or equal to the current error information E. In step S37, the previous error information E '
If it is determined that it is not more than the current error information E, ie,
By correcting the pixel value of the target pixel by the correction amount △,
If the current error information E is larger than the previous error information E ′ (here, error information without correction), the process proceeds to step S38, and the control unit 44 sets the offset amount S to − 1 is multiplied by a new offset amount S
Further, the correction amount △ is incremented by twice the offset amount S, and the process returns to step S32.

That is, by correcting the pixel value of the target pixel by the correction amount △ (in this case, △ = S), if the error increases compared to the case without correction, the sign of the offset amount S is changed. Inverted (in the present embodiment, step S
Since a positive value is set to the offset amount S at 31, the sign of the offset amount S is changed from positive to negative at step S38). Further, the correction amount あ っ た, which was S the previous time, is changed to −S.

In step S37, when it is determined that the previous error information E 'is equal to or greater than the current error information E, that is, by correcting the pixel value of the target pixel by the correction amount △, the current error information E' is corrected. If the error information E has decreased from the previous error information E '(or if it is the same as the previous error information E'), the process proceeds to step S39, and the control unit 44
Increments the correction amount だ け by the offset amount S, updates the previous error information E ′ by setting the current error information E, and returns to step S32.

In this case, in step S32, the number i is further incremented by 1 to 2, so that in step S34 or S36, it is determined that the number i is not 0 or 1, respectively.
The process proceeds from S36 to S40. In step S40, it is determined whether the number i is two. Now, since the number of times i is 2, in step S40, it is determined that the number of times i is 2, and the process proceeds to step S41, where the uncorrected error information E ₀ is equal to or less than the current error information E, and It is determined whether the offset amount S is negative.

[0178] In step S40, or less uncorrected when the error information E ₀ is the current error information E, and if the offset amount S is determined to be negative, i.e., be corrected pixel of interest + S only If the error increases by -S compared to when no correction is made, the process proceeds to step S42, where the correction amount △ is set to 0, and the process proceeds to step S47.

[0179] Further, in step S40, if the error information E ₀ at the time of non-correction is determined or not less than the current error information E, or offset amount S is not negative, the process proceeds to step S44, the previous error information E Is greater than or equal to the current error information E. Step S44
In the above, when it is determined that the previous error information E ′ is equal to or greater than the current error information E, that is, by correcting the pixel value of the target pixel by the correction amount △, the current error information E ′ is corrected.
Is smaller than the previous error information E ′, step S
The control unit 44 proceeds to 45 and sets the correction amount △ to the offset amount S.
Is incremented and the previous error information E '
Then, the current error information E is set, and the process returns to step S32.

In this case, in step S32, the number of times i is further incremented by 1 to 3, so that in the following, step S34, S36 or S40
In this case, it is determined that the number i is not 0, 1, or 2, and as a result, the process proceeds from step S40 to S44. Therefore, in step S44, the previous error information E '
Are not greater than or equal to the current error information E, until steps S32 through S34, S36, S40, S44, S
Forty-five loop processes are repeated.

In step S44, when it is determined that the previous error information E 'is not more than the current error information E, that is, by correcting the pixel value of the target pixel by the correction amount △, the current error If the information E has increased from the previous error information E ', the process proceeds to step S46, where the control unit 44 decrements the correction amount △ by the offset amount S, and proceeds to step S47. That is, in this case, the correction amount △ is a value before the error increases.

In step S47, the control section 44 controls the correction section 41 to execute step S42 or step S42.
The pixel value of the target pixel is corrected by the correction amount △ obtained in step 46, whereby the pixel value of the target pixel is optimized to minimize the prediction error in order to obtain a prediction value through adaptive processing. Will be corrected.

Then, the process proceeds to a step S48, wherein it is determined whether or not the processing has been performed with all the SD pixels as target pixels. If it is determined in step S48 that all the SD pixels have been set as the target pixel and the processing has not been performed yet, the process returns to step S31, and the same processing is performed by setting the SD pixel that has not been set as the target pixel as a new target pixel. repeat. In step S48, all S
If it is determined that the process has been performed with the D pixel as the target pixel, the process ends.

As described above, the pixel value of the SD image is
It is optimized to the optimal one for obtaining the predicted value of the HD image.

Next, FIG. 20 shows an example of the configuration of the adaptive processing section 24 of FIG.

The prediction tap generation circuit 51 includes an optimization unit 2
3 is supplied, where the tap pattern code arranged in the 2 bits on the LSB side of the pixel value is provided in the same manner as in the prediction tap generation circuit 42A of FIG. Is detected, a prediction tap is formed according to the tap pattern code, and is supplied to the classification adaptive processing circuit 52.

In addition to the prediction taps, the optimum SD image and the original HD image used for the classification are also supplied to the classification adaptive processing circuit 52. The pixel classification is performed, for example, in the same manner as described with reference to FIG. 15, and for each class obtained as a result, the normal equation shown in Expression (7) is obtained using the prediction tap and the HD image. It has been made to be. Then, the class classification adaptive processing circuit 52 obtains and outputs a set of prediction coefficients w for each of the four new prediction taps by solving a normal equation for each class.

Next, the operation will be described with reference to the flowchart of FIG. Prediction tap generation circuit 51
Receives the optimal SD image, in step S51, detects (extracts) the tap pattern code added to each SD pixel constituting the optimal SD pixel, and proceeds to step S52 to add the extracted tap pattern code to the extracted tap pattern code. Based on this, a prediction tap is formed. Then, the prediction tap generation circuit 51 outputs the formed prediction tap to the classification adaptive processing circuit 52. Classification adaptive processing circuit 5
2. In step S53, a prediction coefficient w is obtained by performing a class classification of a pixel of interest forming a prediction tap and, for each class obtained as a result, forming and solving a normal equation using the prediction tap and the HD image. And output, and the process ends.

As a result, the adaptive processing unit 24 determines the optimal S
In order to obtain the original HD image from the D image, a set of prediction coefficients w for each of the four patterns of prediction taps that minimize the prediction error is obtained. This 4
As described above, the set of prediction coefficients w for each class for each prediction tap of the pattern is determined by the optimization unit 23
Are supplied to the prediction tap pattern determination unit 26 and are used for adaptive processing (calculation of a linear linear expression shown in Expression (1)).

In the embodiment shown in FIG. 20, the prediction tap generation circuit 51 detects a tap pattern code arranged in two bits on the LSB side of the pixel value, and forms a prediction tap according to the tap pattern code. However, the prediction tap generation circuit 51 can be configured similarly to the prediction tap generation circuit 32 of the preprocessing unit 21 in FIG. That is, the prediction tap generation circuit 51 includes:
All four prediction taps can be configured and supplied to the classification adaptive processing circuit 52. In this case, the classification adaptive processing circuit 52 can be constituted by a classification adaptive processing circuit (for a luminance signal) for calculating four prediction coefficients corresponding to each of the four prediction taps. Each of the adaptive processing circuits can be configured in the same way as a part of the class classification adaptive processing circuit (prediction coefficient, prediction value calculation) shown in FIG.

In that case, prediction taps of each pattern corresponding to each HD pixel constituting the HD image are supplied to each class classification adaptive processing circuit, and the optimum SD pixels constituting the prediction tap are used. Thus, class taps are formed, and each class is classified. Further, in each class classification adaptive processing circuit, one frame of HD pixels and prediction taps for the HD pixels are stored in the teacher data memory 86 and the prediction tap memory 85 for each class. Thereafter, in each of the class classification adaptive processing circuits, a new set of prediction coefficients for each class is generated for four patterns of prediction taps in the same manner as in the learning device of FIG.

Next, FIG. 22 shows an example of the configuration of the prediction tap pattern determination section 26 of FIG.

As shown in the figure, the prediction tap pattern determination section 26 comprises a prediction tap generation circuit 61, a class classification adaptive processing circuit 62, a prediction error calculation circuit 63, and a tap pattern code change circuit 64. These prediction tap generation circuit 61 and class classification adaptive processing circuit 6
2. The prediction error calculation circuit 63 or the tap pattern code change circuit 64 is a prediction tap generation circuit 32, a classification adaptive processing circuit 33, a prediction error calculation circuit 34, or a tap pattern code addition circuit of the preprocessing unit 21 in FIG. It is basically configured in the same manner as 36.

Next, the operation will be described with reference to the flowchart of FIG.

The prediction tap pattern determination section 26 is supplied with an optimal SD image, a set of prediction coefficients for each class for each of the four prediction taps, and an HD image. The prediction tap generation circuit 61 and the tap pattern code changing circuit 64 are supplied to the prediction tap generation circuit 61 and the tap pattern code change circuit 64. The prediction coefficient set or HD image for each class for each of the four prediction taps is obtained by the classification adaptive processing circuit 62 or the prediction error calculation circuit 63, respectively.

When the prediction tap generation circuit 61 receives the optimum SD image, in step S61, as in the prediction tap generation circuit 32 of FIG. 5, one of the pixels is set as a target pixel, and the target pixel is set as shown in FIGS. The four prediction taps shown in FIG. 10 are formed. The four prediction taps are output to the classification adaptive processing circuit 62.

Upon receiving the four patterns of prediction taps formed for the pixel of interest, the class classification adaptive processing circuit 62 determines in step S62 that each of the four patterns of prediction taps has a corresponding prediction coefficient w for each class. Using each of the sets, a linear linear expression represented by Expression (1) is calculated, whereby a prediction value of 9 pixels of the HD image obtained from each of the four prediction taps is obtained, and a prediction error calculation circuit 63 is output.

In the prediction error calculation circuit 63, step S6
3 or S64, the prediction error calculation circuit 34 of FIG.
Are performed in the same manner as in step S15 or S16 in FIG. 13, whereby the tap pattern code of the four prediction taps that minimizes the prediction error is transmitted to the tap pattern code change circuit 64. Is output.

In the tap pattern code change circuit 64,
In step S65, the target pixel (S
The tap pattern code added to the 2 bits on the LSB side of (D pixel) is changed to the tap pattern code supplied from the prediction error calculation circuit 63, and the process proceeds to step S66.

In step S66, it is determined whether or not processing has been performed with all SD pixels as the target pixel. If it is determined that not all SD pixels have been set as the target pixel, the process returns to step S61. The same processing is repeated with the SD pixel that has not been set as a new target pixel. On the other hand, if it is determined in step S66 that the process has been performed with all SD pixels as the pixel of interest, the process ends.

As described above, the predictive tap pattern determining unit 26 uses the set of predictive coefficients w for each of the four predictive taps obtained by the adaptive processing unit 24 to change the tap pattern code Is changed to the one corresponding to the prediction tap in which is smaller.

Next, FIG. 24 shows a configuration example of the receiving device 4 of FIG.

[0203] In the receiver / reproducing device 71, the encoded data recorded on the recording medium 2 is reproduced, or the encoded data transmitted via the transmission path 3 is received.
It is supplied to the separation unit 72. The separation unit 72 separates the coded data into a set of prediction coefficients w for each class of the image data of the SD image and the four prediction taps, and the image data of the SD image is transmitted to the prediction tap generation circuit 73. The set of prediction coefficients w for each class for each of the four prediction taps is supplied to the classification adaptive processing circuit 74.

The prediction tap generation circuit 73 or the class classification adaptive processing circuit 74 includes the prediction tap generation circuit 4 constituting the local decoding section 42 of the optimization section 23 shown in FIG.
The configuration is the same as that of 2A or the class classification adaptive processing circuit 42B (FIG. 17). Therefore, a predicted value of the HD image is obtained in the same manner as in the case of the local decoding unit 42, and this is output as a decoded image. This decoded image is almost the same as the original image, as described above.

On the receiving side, even if the receiving apparatus is not the receiving apparatus 4 as shown in FIG. 24, the apparatus for decoding the decimated image by simple interpolation does not use prediction coefficients,
A decoded image can be obtained by performing normal interpolation.
However, the decoded image obtained in this case has image quality (resolution)
Is deteriorated.

As described above, one of the pixels constituting the SD image obtained by compressing the HD image is set as a pixel of interest, and a plurality of patterns of prediction taps are formed for the pixel of interest to perform prediction. Adaptive processing for obtaining a predicted value of an HD image is performed by a linear combination of a tap and a prediction coefficient,
A prediction error of a prediction value obtained from each of the prediction taps of the plurality of patterns is calculated, and a tap pattern code corresponding to the prediction tap of the plurality of patterns that provides the minimum prediction error is added to the pixel value of the pixel of interest. As a result, adaptive processing is performed using prediction taps corresponding to local characteristics of an image, and as a result, a decoded image with higher image quality can be obtained.

Further, since the 2-bit tap pattern code is arranged instead of the 2 bits on the LSB side of the pixel value, it is possible to prevent an increase in the data amount. Since the tap pattern code is arranged on the LSB side of the pixel value, there is not so much deterioration in image quality.

Furthermore, the optimization unit 23 optimizes the SD image by performing the adaptive processing using the prediction tap that minimizes the error, so that the original HD image can be converted to a substantially identical decoded image. It is possible to obtain.

In the adaptive processing unit 24, the optimum SD
An adaptive process is performed using the image, and a set of prediction coefficients for each class for each of a plurality of prediction taps is updated (corrected) to a more appropriate one, so to speak, and the prediction tap pattern determination unit 26 updates the set. The prediction tap is determined again using the set of prediction coefficients for each class for each of the prediction taps of the plurality of patterns, so that it is possible to obtain a decoded image with further improved image quality.

[0210] The case where the present invention is applied to the image processing apparatus for encoding / decoding HD images has been described above.
The present invention is also applicable to encoding / decoding a standard resolution image or the like such as an SD image. That is, for example, the present invention is also applicable to encoding / decoding of a television signal of a standard system such as the NTSC system. However, the present invention is particularly effective when encoding / decoding a so-called high-vision television signal or the like having a large data amount. Further, the present invention is also applicable to a case where so-called hierarchical coding is performed.

In this embodiment, prediction taps of a plurality of patterns are prepared only for luminance signals, and only prediction taps of 5 × 7 pixels are used for color difference signals. It can be processed in the same way as a signal.

Further, in the present embodiment, the tap pattern code has two bits, but the tap pattern code is not limited to two bits. However, it is desirable that the number of bits be smaller.

Further, in the present embodiment, the pixel value LS
Although the tap pattern code is arranged instead of the two bits on the B side, the tap pattern code can be recorded or transmitted separately from the pixel value.

Also, in the present embodiment, the prediction coefficient is updated using the optimal SD image pre-processed by the pre-processing unit 21 and optimized by the optimization unit 23, and the prediction coefficient is updated again using the prediction coefficient. Although the tap pattern code is determined again, the optimal SD image that has been pre-processed by the pre-processing unit 21 and optimized by the optimizing unit 23 can be directly used as encoded data. In this case, the image quality (S / N) of the decoded image is slightly degraded as compared with the case where the tap pattern code is determined again, but it is possible to speed up the processing.

Further, in the present embodiment, 3 × 3, 5 ×
Although four prediction taps of 3, 3 × 5, 7 × 5 pixels are used, other than this, for example, 1 × 5 or 5 ×
It is also possible to use a prediction tap such as one pixel. Further, the pattern of the prediction tap is not limited to four types.

Further, although not particularly mentioned in the present embodiment, after adding a tap pattern code to a pixel value, the LSB side bit to which the tap pattern code is added is set to a predetermined value. The processing may be performed as a pixel value, or the processing may be performed with the pixel value including the tap pattern code. According to an experiment conducted by the present inventor, the S / N is slightly higher when a pixel value is included including the tap pattern code than when the tap pattern code portion is set to 0 as a predetermined value. Deterioration is obtained, but the result is that the gradation is slightly improved.

In FIG. 18, the pixel value of the target pixel is corrected by 4 or 1 as the offset amount S, thereby detecting the correction amount な る at which the prediction error E becomes the minimum first. Alternatively, for example, the prediction error E may be obtained for all possible values of the pixel value of the target pixel, the minimum value thereof may be detected, and the pixel value of the target pixel may be corrected by the correction amount 場合 in that case. It is possible.
In this case, the processing takes time, but more S /
A decoded image with a high N can be obtained.

Further, when the prediction error E is obtained for all the possible values of the pixel value of the target pixel,
The initial value of the pixel value of the target pixel may be any value (however, a value within a range that the pixel value of the target pixel can take). That is, in this case, a correction value に す る that minimizes the prediction error E can be obtained regardless of the initial value.

Various modifications and application examples can be considered without departing from the gist of the present invention. Therefore, the gist of the present invention is not limited to the above embodiment.

[0220]

According to the image encoding apparatus and the image encoding method of the present invention, one of the pixels constituting the compressed image signal is set as a target pixel, and a plurality of pixels are used by using pixels in the vicinity of the target pixel. A pattern prediction tap is formed, and from each of the plurality of pattern prediction taps and a predetermined prediction coefficient,
An original image signal is predicted, and a prediction value for each of a plurality of patterns of prediction taps is output. Then, a prediction error of a prediction value for each of the plurality of patterns of prediction taps with respect to the original image signal is calculated. It is added to the pixel value. Accordingly, by forming a prediction tap in accordance with the pattern code and performing decoding, a decoded image with higher image quality can be obtained.

According to the image decoding apparatus and the image decoding method of the present invention, one of the pixels constituting the compressed image signal included in the encoded data is set as a target pixel, and added to the pixel value of the target pixel. The prediction tap of the pattern corresponding to the pattern code is formed using pixels near the pixel of interest, the original image signal is predicted from the prediction tap and the prediction coefficient, and the predicted value is obtained. Therefore,
It is possible to obtain a predicted value closer to the original image signal.

In the recording medium according to the thirty-ninth aspect, one of the pixels constituting the compressed image signal is set as a pixel of interest.
A prediction tap of a plurality of patterns is formed using pixels in the vicinity of the pixel of interest, and an original image signal is predicted from each of the prediction taps of the plurality of patterns and a predetermined prediction coefficient, and prediction for each of the prediction taps of the plurality of patterns is performed. A value is output, a prediction error of the prediction value for each of the plurality of patterns of prediction taps is calculated, and a prediction error with respect to the original image signal is calculated. Coded data obtained by adding the pixel value to the pixel value of the target pixel is recorded. Accordingly, by forming a prediction tap in accordance with the pattern code and performing decoding, a decoded image with higher image quality can be obtained.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of an embodiment of an image processing apparatus to which the present invention has been applied.

FIG. 2 is a block diagram illustrating a configuration example of a transmission device 1 of FIG.

FIG. 3 is a block diagram illustrating a functional configuration example of a transmission device 1 of FIG. 2;

FIG. 4 is a flowchart for explaining an operation of the transmission device 1 of FIG. 3;

FIG. 5 is a block diagram illustrating a configuration example of a preprocessing unit 21 in FIG. 3;

FIG. 6 is a diagram for explaining the processing of the thinning circuit 31 of FIG. 5;

FIG. 7 is a diagram illustrating a configuration example of a prediction tap.

FIG. 8 is a diagram illustrating a configuration example of a prediction tap.

FIG. 9 is a diagram illustrating a configuration example of a prediction tap.

FIG. 10 is a diagram illustrating a configuration example of a prediction tap.

11 is a block diagram illustrating an example of a partial configuration of a class classification adaptive processing circuit (calculation of prediction coefficients and predicted values) included in the class classification adaptive processing circuit 33 of FIG. 5;

12 is a block diagram showing another configuration example of another part of the class classification adaptive processing circuit (calculation of prediction coefficients and prediction values) constituting the classification adaptive processing circuit 33 of FIG. 5;

FIG. 13 is a flowchart illustrating a process of a preprocessing unit 21 in FIG. 5;

FIG. 14 is a flowchart for explaining the processing in step S11 in FIG. 13 in more detail;

FIG. 15 is a diagram illustrating a configuration example of a class tap for performing class classification.

16 is a block diagram illustrating a configuration example of an optimization unit 23 in FIG.

17 is a block diagram illustrating a configuration example of a class classification adaptive processing circuit 42B of FIG. 16 and a class classification adaptive processing circuit 74 of FIG. 24;

FIG. 18 is a flowchart illustrating a process performed by the optimizing unit 23 of FIG. 16;

FIG. 19 is a diagram for explaining the processing in step S33 in FIG. 18;

20 is a block diagram illustrating a configuration example of an adaptive processing unit 24 in FIG.

FIG. 21 is a flowchart for explaining processing of an adaptive processing unit 24 in FIG. 20;

22 is a block diagram illustrating a configuration example of a prediction tap pattern determination unit 26 in FIG.

FIG. 23 is a flowchart illustrating a process of a prediction tap pattern determination unit 26 in FIG. 22.

24 is a block diagram illustrating a configuration example of a receiving device 4 in FIG.

FIG. 25 is a block diagram showing a configuration example of an image conversion device previously proposed by the present applicant.

FIG. 26 is a diagram for explaining the processing of the class classification circuit 101 in FIG. 25;

FIG. 27 is a block diagram illustrating a configuration example of a learning device previously proposed by the present applicant.

FIG. 28 is a block diagram illustrating a configuration example of an image encoding device previously proposed by the present applicant.

[Explanation of symbols]

1 transmission device, 2 recording medium, 3 transmission line, 4
Receiver, 11 I / F, 12 ROM, 13
RAM, 14 CPU, 15 external storage device, 1
6 transmitter / recording device, 21 preprocessing unit, 22 switch, 23 optimization unit, 24 adaptive processing unit, 25
Switch, 26 predictive tap pattern determining unit, 3
1 thinning circuit, 32 prediction tap generation circuit, 33
Classification adaptive processing circuit, 34 prediction error calculation circuit, 35 memory, 36 tap pattern code addition circuit, 41 correction unit, 42 local decoding unit, 42A prediction tap generation circuit, 42B class classification adaptive processing circuit, 43 error calculation unit, 44 Control unit,
51 prediction tap generation circuit, 52 class classification adaptive processing circuit, 61 prediction tap generation circuit, 62 class classification adaptive processing circuit, 63 prediction error calculation circuit, 6
4 Tap pattern code change circuit, 71 receiver /
Playback device, 72 separation unit, 73 prediction tap generation circuit, 74 class classification adaptive processing circuit, 82 class classification circuit, 84 delay circuit, 85 prediction tap memory, 86 teacher data memory, 87 arithmetic circuit,
88 delay circuit, 91 class classification circuit, 94 coefficient RAM, 95 prediction operation circuit

Claims (40)

[Claims] 1. An image encoding apparatus for encoding an image signal, comprising: compression means for generating a compressed image signal having a smaller number of pixels than the number of pixels of an original image signal; A first forming unit that forms a plurality of patterns of prediction taps using a pixel in the vicinity of the one pixel as a target pixel, and a plurality of prediction taps of the plurality of patterns and a predetermined prediction coefficient, A first method of predicting the original image signal and outputting a predicted value for each of the plurality of patterns of prediction taps;
Prediction means, a first calculation means for calculating a prediction error of the prediction value for each of the plurality of pattern prediction taps with respect to the original image signal, and a minimum prediction error of the plurality of pattern prediction taps is An image encoding apparatus comprising: an adding unit that adds a pattern code corresponding to the obtained prediction tap to a pixel value of the target pixel. 2. The image code according to claim 1, wherein the adding unit arranges the pattern code in place of N bits on a Least Significant Bit (LSB) side of a pixel value of the target pixel. Device. 3. The first predicting unit performs an operation based on the original image signal and the compressed image signal,
2. The image coding apparatus according to claim 1, further comprising a calculation unit that calculates the prediction coefficient. 4. The method according to claim 1, wherein the first prediction unit further includes a class classification unit configured to classify the pixel of interest into a predetermined class, and a prediction coefficient corresponding to a class of the pixel of interest and the prediction tap. The image coding apparatus according to claim 3, wherein the prediction value is obtained, and the calculation unit obtains the prediction coefficient for each of the classes based on the original image signal and the compressed image signal. 5. The image coding apparatus according to claim 3, wherein said calculating means calculates said prediction coefficient for each of said plurality of patterns of prediction taps. 6. The calculation means obtains the prediction coefficient for each of the plurality of patterns of prediction taps for each of the classes, and the first prediction means calculates the prediction coefficients of the plurality of patterns and the target pixel. The image coding apparatus according to claim 4, wherein a prediction value for each of the plurality of patterns of prediction taps is obtained from prediction coefficients corresponding to the class. 7. The image encoding apparatus according to claim 1, further comprising an optimization unit configured to convert the compressed image signal into an optimal compressed image signal. 8. The optimizing unit is formed by a second forming unit that forms a prediction tap of a pattern corresponding to the pattern code added to the pixel value of the pixel of interest, and the second forming unit. A second prediction unit that predicts the original image signal from the prediction tap and the prediction coefficient, and outputs a prediction value of the original image signal; A second calculating unit that calculates a prediction error for the image signal; and a correcting unit that corrects a pixel value of the pixel of interest in accordance with the prediction error calculated by the second calculating unit. The image encoding device according to claim 7, wherein: 9. The optimization means, wherein the second formation means forms a prediction tap of a pattern corresponding to the pattern code added to the pixel value of the pixel of interest, and wherein the second prediction means , The prediction tap formed by the second forming means, and the prediction coefficient,
Calculating the predicted value; calculating, by the second calculating means, a prediction error of the predicted value calculated by the second predicting means with respect to the original image signal; 9. An optimization process for correcting a pixel value of the pixel of interest in accordance with the prediction error calculated by a means, until the compressed image signal becomes an optimal compressed image signal. An image encoding device according to claim 1. 10. A prediction error of said predicted value obtained by said second prediction means with respect to said original image signal, based on a compressed image signal obtained each time said optimization processing is performed and said original image signal. Correction means for correcting the prediction coefficient, so that the first and second prediction means obtain the prediction value by using the prediction coefficient corrected by the correction means. The image encoding device according to claim 9. 11. The image encoding apparatus according to claim 10, further comprising an output unit that outputs a compressed image signal output by said optimization unit and a prediction coefficient output by said correction unit. 12. The image encoding apparatus according to claim 1, further comprising an output unit that outputs the compressed image signal to which the pattern code is added and the prediction coefficient. apparatus. 13. An image encoding method for encoding an image signal, comprising: a compression step of generating a compressed image signal having a smaller number of pixels than the number of pixels of an original image signal; A first formation step of forming a plurality of patterns of prediction taps using pixels in the vicinity of the one pixel as a pixel of interest, and a plurality of prediction taps of the plurality of patterns and a predetermined prediction coefficient, First predicting the original image signal and outputting a predicted value for each of the plurality of patterns of prediction taps;
A first calculating step of calculating a prediction error of the prediction value for each of the plurality of patterns of prediction taps with respect to the original image signal; An adding step of adding a pattern code corresponding to the obtained prediction tap to the pixel value of the target pixel. 14. The image code according to claim 13, wherein in the adding step, the pattern code is arranged instead of N bits on the LSB (Least Significant Bit) side of the pixel value of the pixel of interest. Method. 15. The method according to claim 13, wherein the first prediction step includes a calculation step of calculating the prediction coefficient by performing a calculation based on the original image signal and the compressed image signal. Image coding method. 16. The first prediction step further includes a class classification step of classifying the pixel of interest into a predetermined class, wherein: a prediction coefficient corresponding to the class of the pixel of interest; and the prediction tap, 16. The image encoding method according to claim 15, wherein the prediction value is obtained, and in the calculating step, the prediction coefficient is obtained for each of the classes based on the original image signal and the compressed image signal. 17. The image encoding method according to claim 15, wherein, in the calculating step, the prediction coefficient is obtained for each of the plurality of prediction taps. 18. In the calculating step, the prediction coefficient is obtained for each of the classes in each of the plurality of patterns of the prediction taps. In the first prediction step, each of the plurality of patterns of the prediction taps and the target pixel 2. A prediction value for each of the plurality of patterns of prediction taps is obtained from a prediction coefficient corresponding to the class of
7. The image encoding method according to item 6. 19. The image encoding method according to claim 13, further comprising an optimization step of converting said compressed image signal into an optimal compressed image signal. 20. The optimization step is formed by a second forming step of forming a prediction tap of a pattern corresponding to the pattern code added to the pixel value of the target pixel, and the second forming step. Predicting the original image signal from the prediction tap and the prediction coefficient,
A second prediction step of outputting the predicted value, a second calculation step of calculating a prediction error of the predicted value obtained in the second prediction step with respect to the original image signal, and the second calculation 20. The image encoding method according to claim 19, further comprising: a correcting step of correcting a pixel value of the pixel of interest in accordance with the prediction error calculated in the step. 21. In the optimizing step, in the second forming step, a prediction tap of a pattern corresponding to the pattern code added to the pixel value of the pixel of interest is formed. Calculating the predicted value from the prediction tap formed by the second forming step and the prediction coefficient; and, in the second calculating step, calculating the predicted value of the predicted value obtained by the second predicting step. Calculating a prediction error with respect to the original image signal; and in the correcting step, performing an optimization process of correcting a pixel value of the pixel of interest in accordance with the prediction error calculated in the second calculating step. 21. The image encoding method according to claim 20, wherein the repetition is performed until the compressed image signal becomes an optimal compressed image signal. 22. A prediction error of the predicted value obtained in the second prediction step with respect to the original image signal, based on a compressed image signal obtained each time the optimization process is performed and the original image signal. Further comprising a correction step of correcting the prediction coefficient so as to reduce the prediction coefficient, wherein in the first and second prediction steps, the prediction value is obtained by using the prediction coefficient corrected by the correction step. 22. The image encoding method according to claim 21. 23. The image encoding method according to claim 22, further comprising an output step of outputting a compressed image signal obtained in said optimizing step and a prediction coefficient obtained in said correcting step. 24. The image encoding apparatus according to claim 13, further comprising an output step of outputting the compressed image signal to which the pattern code is added and the prediction coefficient. Method. 25. A compressed image signal having a smaller number of pixels than the number of pixels of an original image signal is generated, and one of pixels constituting the compressed image signal is set as a target pixel, and a pixel near the target pixel is used. Forming a plurality of patterns of prediction taps, predicting the original image signal from each of the plurality of patterns of prediction taps and a predetermined prediction coefficient, and outputting a prediction value for each of the plurality of patterns of prediction taps; Calculating a prediction error of the prediction value for each of the plurality of pattern prediction taps with respect to the original image signal; of the plurality of pattern prediction taps, a pattern code corresponding to a prediction tap from which the minimum prediction error is obtained, An image decoding apparatus for decoding encoded data including the compressed image signal obtained by adding to a pixel value of a target pixel, comprising: One of the pixels constituting the compressed image signal included in the encoded data is set as a target pixel, and the prediction tap of the pattern corresponding to the pattern code added to the pixel value of the target pixel is set as the target tap. Forming means for forming by using pixels in the vicinity of the pixel; predicting means for predicting the original image signal from the prediction tap formed by the forming means and the prediction coefficient, and obtaining a predicted value thereof; An image decoding device characterized by the above-mentioned. 26. An LSB (Least Sign) of a pixel value of a pixel constituting the compressed image signal included in the encoded data.
26. The image decoding apparatus according to claim 25, wherein the pattern code is arranged instead of the N bits on the significant bit) side. 27. The image decoding apparatus according to claim 25, wherein the prediction coefficient is obtained based on the original image signal and the compressed image signal. 28. The prediction means, comprising: classifying means for classifying the pixel of interest into a predetermined class; and calculating the prediction value from a prediction coefficient corresponding to the class of the pixel of interest and the prediction tap. 28. The image decoding apparatus according to claim 27, wherein the prediction coefficient is obtained for each of the classes based on the original image signal and the compressed image signal. 29. The image decoding apparatus according to claim 27, wherein the prediction coefficient is obtained for each of the plurality of patterns of prediction taps. 30. The prediction coefficient is calculated for each of the classes for each of the prediction taps of the plurality of patterns, and the prediction means calculates the prediction code for the pattern code added to the pixel value of the pixel of interest. The image decoding according to claim 28, wherein the prediction value is obtained from the prediction tap of the corresponding pattern and a prediction coefficient corresponding to the class of the pixel of interest among prediction coefficients of the prediction tap. apparatus. 31. The coded data also includes the prediction coefficient, and further comprising a separation unit for separating the compressed image signal and the prediction coefficient from the coded data. 30. The image decoding device according to any one of 30. 32. A compressed image signal having a smaller number of pixels than the number of pixels of an original image signal is generated, and one of pixels constituting the compressed image signal is set as a target pixel, and a pixel near the target pixel is used. Forming a plurality of patterns of prediction taps, predicting the original image signal from each of the plurality of patterns of prediction taps and a predetermined prediction coefficient, and outputting a prediction value for each of the plurality of patterns of prediction taps; Calculating a prediction error of the prediction value for each of the plurality of pattern prediction taps with respect to the original image signal; of the plurality of pattern prediction taps, a pattern code corresponding to a prediction tap from which the minimum prediction error is obtained, An image decoding method for decoding encoded data including the compressed image signal obtained by adding to a pixel value of a pixel of interest, comprising: One of the pixels constituting the compressed image signal included in the encoded data is set as a target pixel, and the prediction tap of the pattern corresponding to the pattern code added to the pixel value of the target pixel is set as the target tap. A forming step of forming using a pixel in the vicinity of the pixel; and the prediction tap formed by the forming step;
A prediction step of predicting the original image signal from the prediction coefficient and obtaining a prediction value thereof. 33. An LSB (Least Sign) of a pixel value of a pixel constituting the compressed image signal included in the encoded data.
33. The image decoding method according to claim 32, wherein the pattern code is arranged instead of the N bits on the significant bit) side. 34. The image decoding method according to claim 32, wherein the prediction coefficient is obtained based on the original image signal and the compressed image signal. 35. The predicting step includes a class classifying step of classifying the pixel of interest into a predetermined class. The predicting step calculates the predicted value from a prediction coefficient corresponding to the class of the pixel of interest and the prediction tap. 35. The image decoding method according to claim 34, wherein the prediction coefficient is obtained for each of the classes based on the original image signal and the compressed image signal. 36. The image decoding method according to claim 34, wherein said prediction coefficient is obtained for each of said plurality of patterns of prediction taps. 37. The prediction coefficient is obtained for each of the classes for each of the prediction taps of the plurality of patterns, and the prediction step includes the step of adding the prediction code to the pattern code added to the pixel value of the pixel of interest. 36. The image decoding according to claim 35, wherein the prediction value is obtained from the prediction tap of a corresponding pattern and a prediction coefficient corresponding to the class of the pixel of interest among prediction coefficients of the prediction tap. Method. 38. The coded data also includes the prediction coefficient, further comprising a separating step of separating the compressed image signal and the prediction coefficient from the coded data. 38. The image decoding method according to any one of 37. 39. A recording medium on which encoded data obtained by encoding an image is recorded, wherein the encoded data generates a compressed image signal having a number of pixels smaller than the number of pixels of an original image signal. One of the pixels constituting the image signal is set as a target pixel, a plurality of patterns of prediction taps are formed using pixels in the vicinity of the target pixel, and each of the plurality of patterns of prediction taps has a predetermined prediction coefficient. Predicting the original image signal, outputting a prediction value for each of the plurality of patterns of prediction taps, calculating a prediction error for the original image signal of the prediction value for each of the plurality of patterns of prediction taps, Adding, to the pixel value of the target pixel, a pattern code corresponding to a prediction tap from which the minimum prediction error is obtained among pattern prediction taps. A recording medium characterized by being obtained by: 40. The recording medium according to claim 39, wherein said prediction coefficient is also recorded.