JPH06348838A - Picture synthesizing method - Google Patents

Abstract

(57) [Abstract] [Purpose] To suppress the bulge of the extracted image embedded in the background image in the composite image, and to make the jaggedness of the boundary part inconspicuous by a simple algorithm. [Structure] When the extracted image is incorporated into a background image and synthesized, edge strength E ₁ , ... And intensity (for example, luminance) Kc ₁ for pixels G ₁ , ... Located on the outermost peripheral portion of the extracted image,
, And the intensity Kh ₁ , ... Of the pixel at the position corresponding to the pixel G ₁ , ... Of the background image, and further the ratio α ₁ (= E ₁ / E ₁ / = E ₁ / = to the maximum value Em of the edge intensities E ₁ , ...
Em), ..., K ₁ = α ₁ × Kc ₁ + (1-α ₁ ) × K
h _1, strength K ₁ determined by the ... operation, ... a, the pixel G _1,
Replace the intensity Kc ₁ , ... Of. For this reason, a sharp portion having a high edge strength in the outermost peripheral portion of the extracted image becomes sharp, and an unclear portion is blurred. Further, since the intensities of the pixels G ₁ , ... Change in consideration of the intensity of the background image, the connection with the boundary portion with the background image becomes natural.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image synthesizing method used in an image simulation system such as a landscape simulation and a design simulation, an electronic catalog exhibition system, and the like. When embedding in the corresponding area and synthesizing, processing for making the jaggedness of the contour portion of the extracted image inconspicuous (anti-aliasing (anti-al
easing)).

[0002]

2. Description of the Related Art Generally, in a composite image obtained by incorporating an extracted image cut out from an original image by image extraction into a corresponding area of a background image and combining the extracted image and the background image, a boundary portion (particularly, a shaded portion or a peripheral portion) There is a problem in that a phenomenon called aliasing such as flicker is conspicuous, which causes a sense of discomfort in a viewer.

In order to solve such a problem, anti-aliasing processing for suppressing the aliasing phenomenon is performed. Conventional anti-aliasing processing methods include
In order to blur the jagged edges to make them inconspicuous, a method of filling the jagged parts with pixels having an intermediate gradation of brightness or a method of blurring the periphery by applying a certain low-pass filter to the extracted image is performed. It was

[0004]

However, in the former anti-aliasing processing method, if this is automatically performed, as shown in FIG. 6, a combined image 34 obtained by combining the extracted image 30 and the background image 32 is obtained. In, extracted image 3
A jagged portion 36 on the side of the background image 32 at the boundary between 0 and the background image 32 is detected.

Then, the brightness of the intermediate gradation is calculated using the brightness data of the pixels around the detected notched portion 36, and the notched portion 36 is filled with the calculated brightness, so that the algorithm is complicated. In some cases, there is a problem that one treatment may not be sufficient. Further, there is a problem that the extracted image 30 cannot be prevented from expanding.

Further, in the latter anti-aliasing processing method, since the periphery of the jagged portion is blurred at the time of extraction, there is a problem that the extracted portion swells and an extra background area is cut out. Furthermore, there is a problem in that a certain low-pass filter causes excess or deficiency in the blurring method depending on the location.

The present invention has been made in view of the above-mentioned problems, and suppresses the bulge of the extraction area, does not cut out an extra background area, and surely makes the jaggedness of the boundary portion inconspicuous by a simpler algorithm. It is an object of the present invention to provide an image synthesizing method capable of performing the above.

[0008]

According to a first aspect of the present invention, in a method of incorporating an extracted image cut out from an original image by image extraction into a corresponding area of a background image and synthesizing the extracted image, the extracted image is incorporated into the background image. Pixels G ₁ , G ₂ , G ₃ , ... Which are located at the outermost periphery of the extracted image when they are combined
Edge strengths E ₁ , E ₂ , E ₃ , ... And strength Kc ₁ ,
, Kc ₂ , Kc ₃ , ... And the intensity K of the pixel at the position corresponding to the pixels G ₁ , G ₂ , G ₃ , ... Of the background image.
h ₁ , Kh ₂ , Kh ₃ , ... Are obtained, and the edge strength E ₁ ,
The ratio α ₁ (= E ₁ / E) of the maximum value Em of E ₂ , E ₃ , ...
m), α ₂ (= E ₂ / Em), α ₃ (= E ₃ / Em), ..., and K ₁ = α ₁ × Kc ₁ + (1-α ₁ ) × Kh ₁ , K ₂ = α
₂ × Kc ₂ + (1-α ₂ ) × Kh ₂ , K ₃ = α ₃ × Kc ₃ +
The intensities K ₁ , K ₂ , obtained by the calculation of (1-α ₃ ) × Kh ₃ , ...
K _3, ... in the pixel _{G 1, G 2, G 3} , ... strength Kc ₁ of
It is characterized in that Kc ₂ , Kc ₃ , ... Are replaced.

According to a second aspect of the present invention, in the first aspect of the invention, the information regarding the contour line stored in the contour line storage unit is provided with three types of information corresponding to the image extraction contour line and the outside and inside thereof. An image extraction operation for extracting the corresponding region of the original image within the image extraction contour line is performed by using the information in which the pixels are labeled differently (for example, "0", "1", and "2" are labeled). Of the pixels inside the image extraction outline in the outline storage unit, the label of the outermost peripheral pixel adjacent to the image extraction outline is different from the three types of labels (for example, a label of “3”). By replacing the pixel intensities at the time of image composition.

[0010]

In the image synthesizing method according to the first aspect of the present invention, when the extracted image is incorporated into the corresponding area of the background image to synthesize the image, first, the pixel G ₁ located at the outermost peripheral portion of the extracted image,
G _2, G _3, the edge intensity for _{... E 1, E 2, E} 3, ...
And the intensities (for example, brightness) Kc ₁ , Kc ₂ , Kc ₃ , ... And the pixels G ₁ , G ₂ among the pixels of the background image,
Intensities Kh ₁ , Kh ₂ , K of pixels at positions corresponding to G ₃ , ...
h ₃ , ...

Then, the ratio of the edge strengths E ₁ , E ₂ , E ₃ , ... To the maximum value Em is α ₁ (= E ₁ / Em), α ₂ (= E _2).
/ Em), α ₃ (= E ₃ / Em), ..., and K ₁ = α ₁ ×
Kc ₁ + (1-α ₁ ) × Kh ₁ , K ₂ = α ₂ × Kc ₂ + (1-
α ₂ ) × Kh ₂ , K ₃ = α ₃ × Kc ₃ + (1-α ₃ ) × K
The calculation of h ₃ , ... Is performed, and the intensities K ₁ , K obtained by this calculation are calculated.
₂ , K ₃ , ..., Intensities Kc ₁ , K of the pixels G ₁ , G ₂ , G ₃ ,.
Replace c ₂ , Kc ₃ , ... In other words, composite pixel G ₁ at the outermost peripheral portion of the extracted image in the image, G _2, G _3, the strength of ... is the original intensity _{Kc 1, Kc 2, Kc 3} , ... from the intensity K ₁ determined by the calculation , K ₂ , K ₃ , ...

Therefore, the intensity (eg, K ₁ ) of the pixel (eg, G ₁ ) at the outermost periphery of the extracted image is larger than that of the pixel having a large edge intensity ratio α (eg, α ₁ = E ₁ / Em). The intensity (eg K) of the corresponding pixel (eg G ₁ )
The contribution from c ₁ ) becomes large, and the contribution from the intensity (eg, Kh ₁ ) of the corresponding pixel in the background image becomes large for the pixel having a small edge intensity ratio α (eg, α ₁ ).

Therefore, in the composite image, a sharp portion having a large edge strength in the outermost peripheral portion of the extracted image is sharp, and an unclear portion is blurred. Further, since the intensity of the outermost peripheral portion of the extracted image has a contribution component from the intensity of the background image, the connection with the boundary portion with the background image becomes natural.

In the image synthesizing method according to the second aspect of the invention,
In addition to the action according to the invention of claim 1, the following action is provided. That is, first, the information about the contour line stored in the contour line storage unit is labeled differently on the three types of pixels corresponding to the image extraction contour line and its outside and inside (for example, “1”, “0”, “ The image extraction operation of extracting the corresponding region of the original image within the image extraction contour is performed by using the information labeled as “2”).

Next, among the pixels inside the contour line for image extraction in the contour storage unit, the outermost peripheral pixels adjacent to the contour line for image extraction are labeled with three types of labels (for example, "1",
By substituting a label (for example, a label of “3”) different from the “0” and “2” labels), the pixel intensities at the time of image combination are replaced.

[0016]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of an image synthesizing method according to the present invention will be described below with reference to FIGS. 2 shows an example of the configuration of an apparatus for carrying out one embodiment of the method of the present invention. In this figure, 10 is an image input section mainly composed of a television camera, an image scanner, etc., and this image input section 10 is shown in FIG. The original image 11 shown in (a) is taken in and the corresponding original image information is output.

Reference numeral 14 is a microprocessor as an example of an image processing unit, 16 is a RAM (for example, VRAM), 18 is a CRT display as an example of an image display unit, 20 is a tablet as an example of a pointing device, and the RAM 16 is An original image storage unit 16a, a contour line storage unit 16b, and a background image storage unit 16c are provided.

Reference numeral 22 denotes a ROM coupled to the microprocessor 14, and a program for causing the microprocessor 14 to perform known thinning processing, image extraction processing and image synthesis processing is stored in the ROM 22. . In the ROM 22, the microprocessor 14 further includes the following (a), (b),
A program for performing the processes shown in (c), (d), and (e) is stored.

(A) Pixel G ₁ in the outermost peripheral portion of the extracted image,
Intensity for G ₂ , G ₃ , ... (For example, brightness, and so on)
Kc ₁ , Kc ₂ , Kc ₃ , ... And the pixel G _{1 of} the background image,
Intensities Kh ₁ and K of pixels at positions corresponding to G ₂ , G ₃ , ...
A pixel intensity detection function for detecting h ₂ , Kh ₃ , ...

(B) The pixel G ₁ of the outermost peripheral portion of the extracted image,
G _2, G _3, the edge intensity for _{... E 1, E 2, E} 3, ...
Edge strength calculation function to calculate using an edge detection operator.

(C) The maximum value Em of the edge strengths E ₁ , E ₂ , E ₃ , ... Obtained in (b) above (Em is E ₁ , E ₂ ,
The ratio α ₁ (= representing the maximum of E ₃ , ...) (=
E ₁ / Em), α ₂ (= E ₂ / Em), α ₃ (= E ₃ / E)
m), ... An edge strength ratio calculation function.

(D) An arithmetic expression using the values obtained in (a), (b) and (c) above, K ₁ = α ₁ × Kc ₁ + (1-α ₁ ) ×
Kh ₁ , K ₂ = α ₂ × Kc ₂ + (1-α ₂ ) × Kh ₂ , K ₃ =
A calculation function for performing calculation of α ₃ × Kc ₃ + (1-α ₃ ) × Kh ₃ , ....

(E) When the images are combined by incorporating the extracted images, the intensities K ₁ , K ₂ , K ₃ , ... Obtained in (d) above.
Then, the intensities Kc ₁ , Kc ₂ , K of the pixels G ₁ , G ₂ , G ₃ ,.
Image intensity replacement function for replacing c ₃ , ....

Next, the operation of FIG. 2 will be described with reference to FIGS.
Will be explained together. (A) First, in the same manner as in the conventional example, the original image information about the original image 11 captured by the image input unit 10 is stored in the RAM 16.
Is written in the original image storage unit 16a in the
The T display 18 displays the original image 11 as shown in FIG.

(B) Next, based on the input from the tablet 20, the information about the rough contour line is stored in the contour line storage unit 16
In addition to being written in b, the rough contour line is superimposed and displayed in the original image 12 on the CRT display 18.

(C) Next, the microprocessor 14
3 thins the rough contour line by the thinning function based on the thinning processing program stored in the ROM 22.
The information about the image extracting contour line 24 as shown in (b) of FIG. 3 is stored in the contour line storage unit 16b and displayed on the CRT display 18, and then, as shown in (b) of FIG. The contour 24 and its outside and inside, respectively, are labeled with logical "1", "0" and "2".

(D) Next, the microprocessor 14
Is an image 12 of the area labeled with logic “2” in (c) of the original image 11 stored in the original image storage unit 16 by the pixel extraction function based on the pixel extraction program stored in the ROM 22. To extract. This extracted image 1
2 is a CRT display 18 as shown in FIG.
Is displayed.

(E) Next, the microprocessor 14
Is a pixel G of the outermost peripheral portion (that is, a portion adjacent to the image extraction contour line 24) of the extracted image 12 extracted in (d) above by the pixel intensity detection function based on the pixel intensity detection program stored in the ROM 22. Intensity of ₁ , G ₂ , G ₃ , ... (For example, brightness, the same applies hereinafter) Kc ₁ , Kc ₂ , Kc ₃ ,
, And the intensity K of the pixel at the position corresponding to the pixels G ₁ , G ₂ , G ₃ , ... Of the background image 26 as shown in FIG.
_{h 1, Kh 2, Kh 3} , to detect ... and.

(F) Next, the microprocessor 14
Is a pixel G ₁ , by an edge strength calculation function based on an edge strength calculation program stored in the ROM 22.
Edge strengths E ₁ , E ₂ , E ₃ , ... Of G ₂ , G ₃ , ... Are calculated using an edge detection operator. This edge strength calculation uses the information in the contour line storage unit 16b in the RAM 16 as shown in FIG.
From the information labeled for image extraction shown in (a) of
The information labeled for image composition shown in FIG. 4B is changed to prevent duplicate calculations.

That is, first, the pixels g ₁ and g ₂ labeled "2" in the vicinity of each pixel (pixel labeled "1") constituting the contour line 24 for image extraction of FIG. ,
g _3, attention ..., this pixel _{g 1, g 2, g 3} , pixels G ₁ in the extracted image 12 corresponding to ..., G _2, G _3, ... edge intensity calculation is performed for, finished calculation Labels “1” are sequentially assigned to the pixels g ₁ , g ₂ , g ₃ , ...
4 to (3) (labels other than “0”, “1”, and “2” already attached to (a) of FIG. 4) can be changed, and as shown in (b) of FIG. become.

In addition, the above-described calculation using the edge detection operator is to calculate the sum or difference by weighting the intensities in the neighborhood area of the pixel of interest, and to calculate the sum or difference of the 8 neighborhoods. A shape is given, and this is done by multiplying each of the 3x3 regions in the image by the matrix and adding them together.

(G) Next, the microprocessor 14
Is the maximum value Em of the edge intensities E ₁ , E ₂ , E ₃ , ... Of the pixels G ₁ , G ₂ , G ₃ , ... By the edge intensity ratio operation function based on the edge intensity ratio operation program stored in the ROM 22. (Em represents the maximum of E ₁ , E ₂ , E ₃ , ...) And the ratio α ₁ (= E ₁ / Em), α ₂ (= E ₂
/ Em), α ₃ (= E ₃ / Em), ...

(H) Next, the microprocessor 14
Is an arithmetic expression using the values obtained in the above (e), (f), and (e) by the arithmetic function based on the arithmetic program stored in the ROM 22, K ₁ = α ₁ × Kc ₁ + (1- α ₁ ) ×
Kh ₁ , K ₂ = α ₂ × Kc ₂ + (1-α ₂ ) × Kh ₂ , K ₃ =
The calculation of α ₃ × Kc ₃ + (1-α ₃ ) × Kh ₃ , ... Is performed.

(I) Next, the microprocessor 14
The image synthesizing function based on the image synthesizing program stored in the ROM 22 causes the extracted image 12a to be incorporated into the corresponding area of the background image 26 read from the background image storage unit 16c as shown in (e) of FIG. Composite image 28
Is formed and displayed on the CRT display 18.

Next, the microprocessor 14
By the image intensity replacement function based on the image intensity replacement program stored in the ROM 22, the intensity K ₁ , K ₂ , K ₃ , ... , The intensities K of the pixels G ₁ , G ₂ , G ₃ , ...
Replace c ₁ , Kc ₂ , Kc ₃ , ... Therefore, in the boundary portion 12a of the outermost peripheral pixels G ₁ , G ₂ , G ₃ , ... Adjacent to the background image 26 of the extracted image 12, the strength is changed as shown in FIG. It does not swell unlike the conventional example shown in FIG.

In the above embodiment, after the edge strength of the pixels G ₁ , G ₂ , G ₃ , ... Of the outermost peripheral portion of the extracted image has been calculated, the label of the contour line storage section is changed (“2” in FIG. 4). Is changed from “.” To “3”), but the edge strength may be calculated at the same time as changing the label.

In the above-mentioned embodiment, the image display section is formed by a CRT display, but the present invention is not limited to this, and the image display section is formed by a liquid crystal display, a plasma display, an electroluminescent display or the like. You may do it.

In the above embodiment, the ROM is used for thinning processing,
By storing programs for image extraction, image intensity detection, edge intensity calculation, edge intensity ratio calculation, calculation of predetermined calculation formulas, pixel combination and pixel intensity replacement, and executing these programs Although the corresponding function is achieved, the present invention is not limited to this, and the program is stored in a storage medium other than the ROM (for example, a hard disk, a floppy disk, etc.), and the corresponding function is executed by executing these programs. May be achieved, or high-speed processing may be performed using a dedicated processor. .

[0039]

The image synthesizing method according to the first aspect of the invention is
As described above, when the images are combined, the intensity of the pixel (eg, G ₁ ) at the outermost peripheral portion (boundary portion with the background image) of the extracted image is calculated from the original intensity (eg, Kc ₁ ) ( for example replaced with K _1), the corresponding pixel of the replacement intensity (e.g. K ₁₎ the ratio alpha (e.g. alpha ₁₎ is large is extracted image to the maximum edge intensity Em of the edge intensities E (e.g. E ₁₎ The contribution from the intensity (for example, G ₁ ) of (for example, Kc ₁ ) becomes large, and when the ratio α (for example, α ₁ ) is small, the contribution from the intensity (for example, Kh ₁ ) of the corresponding pixel of the background image becomes large. Configured to.

For this reason, in the composite image, a sharp portion having a large edge strength in the outermost peripheral portion of the extracted image becomes sharp, and an unclear portion is blurred. Also, since the strength of the outermost peripheral portion of the extracted image has a contribution from the strength of the background image, the connection with the boundary portion with the background image becomes natural. Therefore, it is possible to suppress the bulge of the extraction region, cut out an extra background region, and reliably make the jaggedness of the boundary portion inconspicuous by a simpler algorithm.

The image synthesizing method according to the invention of claim 2 has the following effect in addition to the effect according to the invention of claim 1. That is, the information about the contours stored in the contour storage unit is used as the information labeled for clearly identifying the image extraction contours during the image extraction operation, and the strength replacement is performed when the images are embedded and combined. Since the information is labeled to clearly identify the pixels to be
It is possible to easily and reliably specify the extracted image and the pixel that replaces the intensity of the outermost peripheral portion of the extracted image.

[Brief description of drawings]

FIG. 1 is a flowchart showing an embodiment of an image synthesizing method according to the present invention.

FIG. 2 is a schematic configuration diagram of an apparatus for performing the method of FIG.

3A and 3B are explanatory diagrams illustrating the operation of FIG. 2, in which FIG. 3A is an explanatory diagram of an original image stored in an original image storage unit, and FIG. 3B is an image extraction contour line stored in the contour storage unit. Explanatory drawing (labeled for image extraction), explanatory drawing of an extracted image, (d) explanatory drawing of a background image, (e) explanatory drawing of a synthetic image.

4A and 4B are explanatory views for explaining labeling of the contour line storage unit, FIG. 4A is an enlarged view of a part of FIG. 3B, and FIG. 4B is an extracted image of FIG. FIG. 5 is an explanatory diagram illustrating a state in which the label “2” corresponding to the pixel at the outermost peripheral portion of the label “2” corresponding to the pixel of is changed to “3” and labeling is performed for image extraction.

5 is an enlarged view showing a part of the combined image of FIG.

FIG. 6 is a partially enlarged view of a composite image composed by a conventional method.

[Explanation of symbols]

11 ... Original image, 12, 30 ... Extracted image, 12a ... Boundary portion consisting of pixels of outermost peripheral portion of the extracted image 12, 14 ...
Microprocessor (an example of image processing unit), 16 ... RA
M, 16a ... Original image storage unit, 16b ... Contour line storage unit,
16c ... Background image storage unit, 18 ... CRT display (an example of image display unit), 20 ... Tablet (an example of pointing device), 22 ... ROM, 24 ... Image extracting contour line, 26, 32 ... Background image, 28, 34 ... composite image, 36 ... jagged portions, the pixel of the outermost peripheral portion of the _{G 1, G 2, G 3} ... extracted image _{12, g 1, g 2,} g 3 ... G 1,
A pixel area in the contour storage unit corresponding to G ₂ and G ₃ .

Claims (2)

[Claims] 1. A method of incorporating an extracted image cut out from an original image by image extraction into a corresponding region of a background image and synthesizing the extracted image, wherein when the extracted image is incorporated into the background image and synthesized, the outermost periphery of the extracted image is included. Edge intensities E ₁ , E ₂ , for pixels G ₁ , G ₂ , G ₃ , ...
E _3, ... and strength _{Kc 1, Kc 2, Kc 3} , ... with seek and the background the image pixel _{G 1, G 2, G 3} , intensity Kh ₁ position of the pixel corresponding to ..., Kh ₂ , Kh ₃ , ... And further, ratios α ₁ (= E ₁ / Em), α ₂ (= E ₂ / E) of the edge strengths E ₁ , E ₂ , E ₃ , ... To the maximum value Em.
m), α ₃ (= E ₃ / Em), ..., and K ₁ = α ₁ × Kc
_{1 + (1-α 1)} × Kh 1, K 2 = α 2 × Kc 2 + (1-
α ₂ ) × Kh ₂ , K ₃ = α ₃ × Kc ₃ + (1-α ₃ ) × K
h _3, intensity K ₁ determined by the ... operation, K _2, K _3, ... in the pixel _{G 1, G 2, G 3} , ... strength Kc of _1, Kc _2, Kc _3, ...
An image synthesizing method characterized in that 2. Image extraction by using the information regarding the contours stored in the contour storage unit as the information regarding the contours for image extraction, and the three types of pixels corresponding to the outside and inside of the contours, respectively. An image extraction operation is performed to extract the corresponding region of the original image within the contour line for use, and among the pixels inside the contour line for image extraction in the contour storage unit, the outermost pixel adjacent to the contour line for image extraction. 2. The image synthesizing method according to claim 1, wherein the label is replaced with a label different from the three types of labels to replace the pixel intensities at the time of image synthesizing.