JP2001144968A - Multimedia information converter - Google Patents

Abstract

(57) [Summary] [Problem] To provide a conversion device for converting a video (hereinafter, a personal computer terminal video) displayed by a personal computer into a compressed video signal such as MPEG, and to convert a still image of the personal computer terminal video in the conversion device. To reduce the amount of calculation by utilizing the feature of being large, and to make a specific media area clear when generating a compressed video signal. An image data acquisition unit acquires image data of a personal computer terminal video, and an encoding image generation unit converts the acquired image data into an encoding format.
4 and an encoding unit 105 for generating a compressed video signal, thereby converting a personal computer terminal video into a compressed video signal. Further, the processing is changed for each area by providing an area classification unit that classifies the area by detecting the area of the specific medium from the image data. As a result, a personal computer terminal image can be converted into a compressed image signal in which an area of an arbitrary medium is sharpened.

Description

DETAILED DESCRIPTION OF THE INVENTION TECHNICAL FIELD OF THE INVENTION

The present invention relates to a technique for encoding an image displayed by a general personal computer (hereinafter, referred to as a personal computer terminal) into a compressed image signal.

[0002]

2. Description of the Related Art In the recent era of multimedia,
There is an increasing demand for a system for performing communication using multimedia, such as a video conference or a lecture using a PC terminal, by communicating with a remote place using a TV or a PC terminal. In addition to transmitting images and voices of humans who communicate,
There is an increasing demand for a system capable of more effective communication by transmitting video such as presentation software, spreadsheet software, and CAD displayed on a personal computer terminal. Furthermore, in order to popularize such a video conference system and a system for giving a lecture using a personal computer terminal, it is necessary to realize a system using a low-cost terminal.

[0003] To this end, a transmission device using a low-cost personal computer terminal receives images of applications such as presentation software, spreadsheet software, and CAD operating on the personal computer terminal with another low-cost receiving device. It is necessary to realize a device capable of converting the data into a format that can be reproduced and transmitting the converted data.

Conventionally, in order to transmit an image of an application displayed by a personal computer terminal to a remote place and display the image on a receiving terminal, as a first conventional method, an application of a transmitting terminal is executed on a personal computer terminal. A general method is to transmit command information for display to a receiving terminal over a network when operating, and to perform display expansion and display based on the information transmitted by the receiving terminal, a general method. There is.

[0005] As a second conventional method, a technique for sharing computer images between different models is disclosed in Japanese Patent Laid-Open No. 9-2942.
No. 55 is described. According to this method, the video of the terminal is captured as a video signal by the scan converter, converted into an NTSC signal, and input to a video conference system that operates as an input with the NTSC signal.
Computer images can be shared between different models.

[0006]

However, in the method of the conventional method 1, information necessary for creating an image (a state before the image is created) is transmitted, and the receiving side uses the information based on the information. Since it is processed so that it can be displayed,
It is necessary for the receiving side to expand the information sent from the transmitting side and necessary to create the video, and the receiving side also needs a PC terminal with the same performance as the transmitting side. There were challenges.

As a first problem, there arises a problem that a reproducing terminal on the receiving side is required to have many functions and a high-cost reproducing terminal is required.

In the conventional method 2, unlike the conventional method 1, an image is transmitted in a state where an image is formed. However, in the case of a different model, an image is acquired by a scan converter due to an image format and the like. Sometimes it is necessary to acquire an analog signal and convert it to an NTSC signal, which significantly reduces the amount of information.

As a second problem, there is a problem that image quality deteriorates because processing is performed based on image data once converted into an analog signal.

Further, when attempting to acquire image data displayed on a display by a personal computer terminal as a digital signal, a large amount of image data is acquired from a physical memory. In addition to this, a high-performance CPU is required for the operation, and a high-cost transmitting terminal is required.
Therefore, as a third problem, a problem occurs in that a large amount of computation is required to acquire an image, and a high-cost transmission terminal is required.

Further, images displayed by a personal computer terminal are artificially created images containing many high-frequency components, and are often difficult to exist in the natural world. It is not suitable for the target compression coding technology. In particular, when quantizing an image containing many high-frequency components such as characters and line drawings, encoding at a high compression rate causes a lot of mosquito noise, which causes a problem that characters are difficult to read. Therefore, as a fourth problem, a problem has arisen that it is necessary to perform encoding for clearing the image quality of an image area such as a character or a line drawing.

SUMMARY OF THE INVENTION The present invention has been made in view of the above-mentioned conventional problems. A first object of the present invention is to convert image data generated as a digital signal by a personal computer terminal to be displayed on a display device into an analog signal. Get directly before being converted,
This is to enable encoding into one compressed video signal such as MPEG.

Further, a second object of the present invention is to provide the above-described third embodiment.
In order to solve the problem of the above, image data displayed by a personal computer terminal is acquired as a digital signal at high speed, and MP
This is to enable encoding into one compressed video signal such as EG.

Further, a second object of the present invention is to solve the above-mentioned fourth problem by converting image data displayed by a personal computer terminal into a digital signal, such as MPEG.
An object of the present invention is to enable a target area to have high image quality when encoding into one compressed video signal.

[0015] The MPEG in this specification is an encoding that reduces temporal redundancy between frames by motion compensation and reduces spatial redundancy in a frame by DCT (discrete cosine transform). Refers to an encoding method adopted in an image compression standard such as MPEG1 or MPEG2.

[0016]

Therefore, in the present invention, in order to solve the above four problems, 1) acquiring image data displayed on a display by a personal computer terminal as a digital signal, and 2) acquiring the acquired digital data. Compression encoding based on signal image data. By repeating the above operation, a compressed video signal having a plurality of frames is generated, and the video displayed by the personal computer terminal is converted into one compressed video signal conforming to a format such as a moving image and MPEG. After being converted into one compressed video signal, it is transmitted to a receiving terminal by a conventional method, so that display on another terminal is realized. In addition, it is possible to share video among multiple terminals by incorporating it into a video conference system using a compressed video signal.

Further, by converting an image displayed on a display by a personal computer terminal into one compressed video signal, which is a feature of the present invention, the receiving terminal decodes the video signal or the compressed video signal, Only minimal functions for playback are required, and playback on low-cost terminals is possible.

Further, since image data of the personal computer terminal is obtained and encoded as a digital signal without using a scan converter or the like, there is no deterioration in image quality due to conversion into analog.

Further, according to the present invention, (a) the image data displayed on the display of the personal computer terminal generally has many stationary areas including presentation software and the homepage of the Internet;
(B) When image data is acquired as a digital signal, the feature is used that the data in a stationary area has no noise or the like and the same data is always acquired even if data is acquired at different times. It has the following two features.

<Characteristic 1> That is, by having a change detection unit for detecting a change in image data displayed on a display by a personal computer terminal, a changed area is detected and only image data of the changed area is obtained. In other areas, the previously acquired image data is reused and synthesized, thereby reducing the amount of data to be acquired when acquiring an image.

In particular, when a change is detected by accessing a video memory or the like in which image data to be obtained is stored, image data in a predetermined area for detecting a change in the image data to be obtained. Is acquired, and the change is detected by comparing the acquired image data with the same area of the previously acquired image data.

Thus, while minimizing the amount of image data acquired from the memory storing the image to be displayed on the display by the personal computer terminal, it is equivalent to that obtained when all the image data is acquired from the video memory or the like. Image data having the above information amount can be obtained.

According to this feature, in the case of a personal computer terminal which requires a lot of processing time when acquiring data from a video memory or the like, the amount of data acquisition from the video memory or the like is reduced. Processing time can be reduced.

<Characteristic 2> Further, using the detection result of the in-screen change detection unit, post-acquisition processing such as encoding is performed on the changed area, and the remaining area is processed in the past after acquisition. By reusing and synthesizing the processed data, the amount of calculation can be reduced by omitting the processing of the stationary part of the input image data.

The above two features of the present invention make it possible to reduce the amount of calculation of the entire conversion apparatus of the present invention, and realize a system with a lower cost transmitting terminal.

Further, according to the present invention, when encoding into a single compressed video signal such as MPEG, a means for classifying media into regions in an acquired image is provided, so that regions can be classified. By changing the pre-processing for each area and weighting the image and changing the compression ratio for each area at the time of coding, it is possible to perform coding in which the area of the selected medium is sharp. In particular, it classifies character and line drawing areas and natural picture areas that are often found in images displayed by personal computer terminals, and performs pre-processing by applying a low-pass filter to the natural picture areas for high compression. By reducing the amount of bits to be allocated for text and line processing, preprocessing is not performed, and the amount of bits for encoding is increased by the amount of bits allocated to the area of natural images. And coding in which a line drawing area is clear.

Thus, it is possible to generate a compressed video signal in which the image quality of an image area such as a character or a line drawing in an image displayed by the personal computer terminal is clear.

[0028]

Embodiments of the present invention will be described below with reference to FIGS. 1 to 35. The present invention is not limited to these embodiments at all, and can be implemented in various modes without departing from the gist thereof.

<< First Embodiment >> In the present embodiment, 1) an image displayed on a display is acquired by a personal computer terminal, and 2) the acquired image data is converted into one frame of an MPEG video signal. A device that generates a compressed video signal having a plurality of frames by repeating the above operation to convert a video displayed by a personal computer terminal into one compressed video signal conforming to the MPEG format will be described.

A personal computer terminal used in the present embodiment is a device that creates and displays image data to be displayed on a display device, and includes a storage unit that stores image data to be displayed on the display device. It is provided.

First, the overall configuration of the present apparatus will be described. FIG. 1 is a block diagram showing the configuration of the multimedia information conversion device according to the first embodiment of the present invention. FIG.
Is a functional block diagram of the present apparatus. As shown in FIG. 1, the apparatus includes a drawing information acquisition unit 100, a copied image data storage unit 101, an image data acquisition unit 102, a post-acquisition data storage unit 103, an encoding image generation unit 104, , An encoding unit 105, an output compressed video signal storage unit 106, and an overall operation control unit 107.

As shown in FIG. 1, the present apparatus operates on a personal computer terminal having a drawing information creating section 11, an image data generating section 12, an image data storing section 13, and a display section 14. (Note that the display unit 14 has a main body (11, 1
(2, 13) and the display (14) may be devices independent of the terminal, and in this embodiment, the personal computer terminal is merely described as including the display unit.) .

The drawing information creating section 11 is means for generating drawing information for generating image data for a personal computer terminal, which is image data to be displayed on a display device by a personal computer terminal. The image data generating unit 12 is a unit that generates personal computer terminal image data based on the drawing information generated by the drawing information generating unit 11, and is a device driver for generating drawing information. The image data storage unit 13 stores the personal computer terminal image data generated by the image data generation unit 12, and usually exists as a memory such as a video memory or a VRAM. The display unit 14 is a means for displaying the personal computer terminal image data stored in the image data storage unit 13 on a display device.

The drawing information obtaining unit 100 always obtains the drawing information generated by the drawing information generating unit 11 and generates the personal computer terminal image data in the same manner as the processing of generating the personal computer terminal image data by the image data generating unit 12. This is a means for sending the obtained drawing information to the image data generation unit 12. The post-copy image data storage unit 101 is a storage unit that stores the personal computer terminal image data generated by the drawing information acquisition unit 100. The image data acquisition unit 102 is a unit that acquires the personal computer terminal image data stored in the post-copy image data storage unit 101. Post-acquisition data storage unit 103
Is a storage unit for storing personal computer terminal image data acquired by the image data acquisition unit 102. The encoding image generation unit 104 converts the personal computer terminal image data stored in the acquired data storage unit 103 into YUV (Y: luminance, U: color difference, V: color difference), which is a format for encoding to MPEG.
This is means for generating image data for encoding which is a signal of 4: 2: 0 format. The encoding unit 105 is a unit that generates a compressed video signal by performing general MPEG encoding on the encoding image data generated by the encoding image generation unit 104. The output compressed video signal storage unit 106 is a storage unit that stores the compressed video signal generated by the encoding unit 105. The overall operation control unit 107 is a means for mainly determining the start and end of the conversion processing of the present apparatus.

Next, the flow of processing in which the drawing information acquisition unit 100 of the present apparatus generates personal computer terminal image data in the copied image data storage unit 101 will be described. The flow of the process of the drawing information acquisition unit 100 of the present apparatus is shown in the flowchart of FIG. As shown in FIG. 2, the drawing information acquisition unit 100 of this apparatus generates personal computer terminal image data in the copied image data storage unit 101 according to the following flow.

Step 101: First, the drawing information acquisition unit 100 checks whether or not the drawing information generated by the drawing information creation unit 11 has been transmitted. If there has been no transmission, the process returns to step 101. Step 102: If there is a transmission, the drawing information obtaining unit 100 obtains the drawing information generated by the drawing information generating unit 11 instead of the image data generating unit 12. Step 103: Next, the drawing information acquisition section 100 generates PC terminal image data in the copied image data storage section 101 by the same processing as the image data generation section 12 generates PC terminal image data. Step 104: Next, the drawing information acquisition unit 100 outputs the drawing information acquired in Step 102 to the image data generation unit 12.
Send to Step 105: Next, the drawing information acquisition unit 100 checks whether or not all conversion processing of the present apparatus has been completed,
If not completed, the process returns to step 101, and if completed, the process ends. As a result, the drawing information acquisition unit 100 of the present apparatus stores the image data storage unit 13 in the copied image data storage unit 101.
, And the same image data as the personal computer terminal image data generated by the image data generation unit 12 can be always generated.

Next, the overall processing flow of the present apparatus will be described. FIG. 3 is a flowchart showing the overall processing flow of this apparatus. As shown in FIG. 3, the present apparatus converts a video displayed by a personal computer terminal into one compressed video signal conforming to the MPEG format in the following flow.

Step 1001: First, the overall operation control unit 107 determines whether or not all the conversion processes of the present apparatus are to be terminated, and if so, terminates. Step 1002: If not, to acquire image data The unit 102 acquires the personal computer terminal image data stored in the post-copy image data storage unit 101. Step 1003: Next, the image data acquisition unit 102
The post-acquisition data storage unit 103 stores the personal computer terminal image data generated in step 1001. Step 1004: Next, the encoding image generation unit 104
Is 4: 2: YUV (Y: luminance, U: color difference, V: color difference) which is a format for encoding from the personal computer terminal image data stored in the data storage unit 103 into MPEG.
Generate encoding image data that is a 0-type signal. Step 1005: Next, the encoding unit 105 performs step 1
A compressed video signal is generated from the encoding image data generated in 003. Step 1006: Next, the encoding unit 105 performs step 1
The compressed video signal generated in 004 is stored in the output compressed video signal storage unit 106, and the process returns to step 1001.

Next, the flow of the processing of generating image data for encoding (step 1003) of the present apparatus is shown in the flowchart of FIG. As shown in FIG. 4, the encoding image generation unit 104 of the present apparatus generates encoding image data from personal computer terminal image data in the following flow. Step 1011: First, the encoding image generation unit 104
From the image format of the personal computer terminal image data stored in the acquired image data storage unit 103,
Convert to UV 4: 2: 0 format. Step 1012: Next, the encoding image generation unit 104
If the resolution of the personal computer terminal image data stored in the acquired image data storage unit 103 is the same as the compressed video signal to be output, the process ends; otherwise, Step 1013: the encoding image generation unit 104 The resolution is converted by performing enlargement and reduction processing.

In this embodiment, step 101
Step 101 for the YUV data after the processing of step 1
Although the processing of step 3 is performed, the image format may be converted in step 1011 after steps 1012 and 1013 are performed first.

As is clear from the above description, according to the apparatus of the present embodiment, it is possible to convert a video displayed by a personal computer terminal into one compressed video signal conforming to the MPEG format. This eliminates the need for complicated processing required for decoding and displaying various types of data, which was conventionally required to view presentation software and spreadsheet software images displayed on PC terminals on other terminals. It is possible to obtain a useful effect that browsing can be performed only by decoding and displaying one compressed video signal such as an MPEG decoder. In addition, since the image data of the personal computer terminal is obtained and encoded as a digital signal without using a scan converter or the like, there is no deterioration in image quality due to conversion to analog, so high-quality conversion can be performed.
The useful effect described above can be obtained.

FIG. 5 is a functional block diagram in the case where the image data acquisition unit 102 acquires the personal computer terminal image data directly from the image data storage unit 13 when acquiring the personal computer terminal image in the processing of this apparatus. Show. In the functional block diagram shown in FIG.
0 and a post-copy image data storage unit. in this case,
This can be realized by the image data obtaining unit 102 of the present apparatus directly obtaining the personal computer terminal image data stored in the image data storage unit 13. As an acquisition method for PC terminal image data, the OS running on the PC terminal
If you have a function to acquire PC terminal image data, use it. Further, if it is possible to directly access the memory of the image data storage unit 13, it is possible to obtain a simple memory copy.

The time required for the apparatus of the present embodiment to generate one frame of the compressed video signal does not always match the display time per frame calculated from the frame rate of the compressed video signal. Thus, the video displayed by the personal computer terminal can be faithfully converted along the time axis. At this time, in addition to determining the start and end of the conversion processing of the present apparatus, the overall operation control unit 107 includes a unit for acquiring the current time, and determines the time when the generation of the compressed video signal by the encoding unit 105 ends. Means for generating a compressed video signal to be output from the apparatus according to the MPEG frame rate based on the display time per frame of the compressed video signal, which is known from the frame rate of the generated compressed video signal. is there.

First, when the time for generating one frame of the compressed video signal by the present apparatus is shorter than the display time of one frame of the compressed video signal, it is matched by the following method. After the processing of generating one frame of the compressed video signal by the present apparatus is completed, the overall operation control unit 107 determines the display time per frame of the compressed video signal and the time required for the present apparatus to generate one frame of the compressed video signal. Difference time, image data acquisition unit 1
The process waits without moving to the process of step 1002 of 02. next,
If the time during which the apparatus generates one frame of the compressed video signal is longer than the display time of one frame of the compressed video signal, it is matched by the following method.

If the display time per frame of the compressed video signal has passed before the apparatus has completed the processing of generating one frame of the compressed video signal, the processing of generating one frame of the compressed video signal has ended. In the meantime, the overall operation control unit 107 instructs the encoding unit 105 to generate a frame having information indicating that the frame uses a copy of the previous frame or is the same as the previous frame. Here, the frame having the information that the frame is the same as the previous frame is referred to as a P frame or a B frame in MPEG, and in a frame generated by predictive coding, all macroblocks are the same as the previous frame. They have information. Also, a frame using a copy of the previous frame becomes a frame of the compressed video signal only by rewriting the header with respect to the data of the copy of the previous frame, and thus can be generated with a small amount of calculation.
In addition, since a frame having information that is the same as the previous frame does not require most processing required for encoding if it is prepared in advance, it requires a small amount of computation such as simple copying and rewriting of a header. Can be generated.

In this embodiment, after the image data acquisition unit 102 stores the image data in the acquired image data storage unit 103, the encoding image generation unit 104 generates the encoding image. Is a post-acquisition image data storage unit 10
By saving the image data directly into the encoding image generation unit 104 without storing the image data in
At the same time that the image data acquisition unit 102 acquires the image data, the encoding image generation unit 104 may generate the encoding image. In the present embodiment, the apparatus of the present embodiment converts a video displayed by a personal computer terminal into one compressed video signal conforming to the MPEG format. It is also possible to convert to an encoding format such as 261 or H.263 for compressing an image.

In the present embodiment, the video signal is encoded into a compressed video signal. However, the encoding unit 105 does not always need to compress the video signal, and can generate an uncompressed digital video signal. It is.

<< Second Embodiment >> In the present embodiment,
In the first embodiment, when the image data acquisition unit 102 directly acquires the personal computer terminal image data stored in the image data storage unit 13 shown in FIG. A method for reducing the amount of acquisition will be described. In particular, in the case of a personal computer terminal that requires a large amount of processing time when acquiring data from a video memory or the like, reducing the amount of data acquired from the video memory or the like leads to a reduction in the processing time. Speeding up of the processing of 102 is realized.

In the present embodiment, the image data obtaining unit 10
2 is that, because of the characteristics of the personal computer terminal image data, the image data obtained in a stationary area is exactly the same even when the data is obtained at different times, so that a change in the image data is detected and only the changed area is detected. The image data acquired beforehand is reused in the other areas, thereby realizing the acquisition of the personal computer terminal image data with a reduced amount of data acquisition.

Here, the characteristics of the personal computer terminal image data to be used will be described. For example, when it is compared with a video taken by a video camera or the like (hereinafter, referred to as a natural image) or the like, it becomes clearer particularly when attention is paid to a background video behind the shooting target. The image of the background when photographed with no change is hardly seen from human eyes, but the background image does not change at all due to the fluctuation of the air or the change of the lighting. Regardless of this, a device for generating a natural image such as a video camera does not intentionally provide a background image with the same data. A natural image is an image that has been shot and stored, and is provided in this case as a series of images with almost no change. In contrast, the PC terminal image data is provided with exactly the same data for a video that does not need to be changed and displayed.

Based on these, the characteristics of the personal computer terminal image data can be simply summarized as the following two points. 1) In general, there are many stationary areas including presentation software and Internet homepages. 2) When the personal computer terminal image data is acquired in the form of digital data, the data in the stationary area has no noise or the like, and the same data is always acquired even if the data is acquired at different times.

At this time, in the present embodiment, the image data obtaining unit 102
A part of the personal computer terminal image data stored in the image data storage unit 13 is acquired and compared with the personal computer terminal image data acquired in the past.

First, the overall configuration of the image data acquisition unit 102 in the present embodiment will be described. A portion surrounded by a dotted line in FIG. 6 is a functional block diagram of image data obtaining section 102 in the present embodiment. As shown in FIG. 6, in the present embodiment, the image data acquisition unit 102
It comprises a change detection acquisition area determination unit 111, an in-screen change detection unit 112, a partial image data acquisition unit 113, and an acquired image synthesis unit 114.

The change detection acquisition area determination unit 111 determines a change detection acquisition area, which is a target area for comparing whether or not there has been a change from the previously acquired image data, which is necessary for change detection. . In the present embodiment, as shown in FIG. 7, in a block unit of a designated resolution, the N-th row in the block is designated as a change detection acquisition area. In this embodiment, the resolution of the block is 8 pixels vertically and 8 pixels horizontally, as shown in FIG. It may change for each pixel. The Nth row here is an arbitrary integer in a range from 1 to the number of pixels of the vertical resolution of the block.

The in-screen change detection unit 112 acquires the image data of the change detection acquisition area determined by the change detection acquisition area determination unit 111 by the partial image data acquisition unit 113, and stores the acquired image data in the post-acquisition data storage unit 103. By comparing with the image data of the same area of the image data being
This is a means for detecting the presence or absence of a change. The partial image data acquisition unit 113 acquires image data of a change detection acquisition area for detecting a change by the in-screen change detection unit 112 from the image data storage unit 13, and acquires the in-screen change detection unit 1.
A unit 12 obtains image data of the block when a change is detected as a result of the change detection.
The acquired image synthesizing unit 114 includes a partial image data acquiring unit 113
Is a means for replacing the acquired image data with the image data of the same area stored in the post-acquisition data storage unit 103.

Next, the overall processing flow of the image data acquisition unit 102 in this embodiment will be described. FIG. 8 is a flowchart illustrating the overall processing flow of the image data acquisition unit 102 according to the present embodiment. As shown in FIG. 8, the image data acquisition unit 102 in the present embodiment implements acquisition of PC terminal image data with a reduced data acquisition amount in the following flow.

Step 2001: First, the change detection acquisition area determination unit 111 sets the information of the Nth row in the target block as a change detection acquisition area using a block of a predetermined resolution as one unit. . Step 2002: Next, the in-screen change detection unit 112
The image data of the change detection acquisition area is acquired from the image data storage unit 13 by operating the partial image data acquisition unit 113. Step 2003: Next, the in-screen change detection unit 112
A change is detected by comparing with the image data stored in the post-acquisition data storage unit 103 in the same area as the image data acquired in the process of step 2002. In this case, the two image data to be compared are compared on a pixel-by-pixel basis,
If any pixel has changed, the target block is determined to have changed. Step 2004: Next, if no change is detected in the process of step 2003, the process proceeds to step 2007, and if a change is detected, step 2005: the partial image data obtaining unit 113 Image data is stored in the image data storage 1
Obtain from 3. Step 2006: Next, the acquired image combining unit 114 replaces the image data of the block to be acquired by the partial image data acquiring unit 113 with the image data of the same area stored in the post-acquisition data storage unit 103. You. Step 2007: Next, the change detection acquisition area determination unit 1
11 determines whether there are any remaining blocks,
If there are remaining blocks, the process proceeds to step 2001, and if there are no remaining blocks, the process ends.

In the process of detecting a change in the screen in step 2003, the presence or absence of a change in one pixel is determined by subtracting the value representing the target pixel from the value of the target pixel in the previously acquired image data. The detected value is detected by determining that there is a change when the value is 0, and not changing otherwise. Also, a 24-bit R representing the color of one pixel by 24 bits
When there is a plurality of pieces of information representing the color of one pixel, as in the GB format, the value of each pixel is compared with the value of the target pixel of the previously acquired image data, and if any one is not the same. Makes that pixel a pixel with a change.

As is clear from the above description, the image data acquisition section 102 in the present embodiment acquires a part of the personal computer terminal image data stored in the image data storage section 13 and acquires a part of the image data in the past. Detects changes in the image data by comparing it with the image data of the personal computer terminal that has been acquired.Based on the results of the detection of the changes, only the changed area is obtained, and the other areas are By reusing, image acquisition of PC terminal image data with a reduced amount of data acquisition is realized.

In the present embodiment, the change detection acquisition area determination unit 111 changes the value of N in the process of step 2001 every time the frame to be acquired changes, so that Area determination unit 11
It becomes possible to detect a change that has occurred in an area other than the change detection acquisition area determined by 1, and it is possible to reduce the loss of change detection and improve the accuracy of change detection.

The missing of the change detection means that the in-screen change detection unit 112 uses the image data in the change detection acquisition area to detect a change in the PC terminal image data. Is a state in which a change occurring in the area cannot be detected. Also, a video of a presentation material provided by presentation software has a feature that the entire screen does not change for a certain period of time, and when there is a change, the video of the entire screen often changes.

For a video having the above characteristics, if there is at least one block determined to have changed by the change detection in the entire image data of the personal computer terminal, it is necessary to acquire the personal computer terminal. By acquiring the entire image data, it is possible to omit the calculation required for change detection and improve the accuracy of change detection.

<< Third Embodiment >> In the present embodiment,
In the processing of the image data acquisition unit 102 according to the second embodiment, the number of missing changes can be reduced without reducing the processing speed, and the amount of acquired data can be reduced without increasing the number of missing changes. A method for realizing the acquired PC terminal image data will be described.

The image data acquisition unit 1 according to the second embodiment
In the process of 02, there is a possibility that an area where the change of the image of the personal computer terminal image data is not reflected in the image data after acquisition may occur due to missing of the change detection. In addition, if the acquisition area for change detection is enlarged to reduce the loss of change detection, the amount of PC terminal image data acquired for change detection increases, so that the amount of calculation increases and the processing speed decreases.

In this embodiment, the image data acquisition unit 10
In order to solve the above problem, the operation of 2
It has two features. Among them, the first feature is that, in the processing of the image data acquisition unit 102, the pixels to be compared for change detection are predicted without increasing missing changes by predicting the latest changes from change history information. Reduce and increase processing speed. The second feature is that, in the processing of the image data acquisition unit 102, the change area information is used, and the latest change information after the change detection described in the change area information is processed, so that the change detection is performed. The purpose of the present invention is to reduce the number of missing pixels in the change detection without lowering the processing speed due to increasing the number of pixels to be compared.

Here, "processing the latest change information (after detecting the change)" means processing the change area information based on the change area information itself. For example,
For the information 11101111 (; information A),
Since it is the fourth “0” and is continuous with “1” before and after it, there is a high possibility that it has been missed (; information A
Information that can be understood / guessed by itself)
Instead of changing (processing) to “1”. The change history information is, as shown in FIG.
This is information that describes the history of past changes in personal computer terminal image data in block units. The change area information is shown in FIG.
As shown in FIG. 1, the latest change of the personal computer terminal image data is described in block units.

First, the overall configuration of the image data acquisition unit 102 in the present embodiment will be described. A portion surrounded by a dotted line in FIG. 9 is a functional block diagram of image data obtaining section 102 in the present embodiment. As shown in FIG. 9, the apparatus includes a change detection acquisition area determination unit 111,
An in-screen change detection unit 112, a change area information and change history information storage unit 301, a partial image data acquisition unit 113,
And an acquired image synthesizing unit 114.

The change detection acquisition area determination unit 111 determines a change detection acquisition area that is necessary for change detection and is a target area for comparing whether or not there has been a change from the previously acquired image data. Means. In the present embodiment, the change detection acquisition area determination unit 111 determines, for each block of the specified resolution, a block that needs change detection based on change history information, For a certain block, as shown in FIG. 5, the Nth row in the block is designated as a change detection acquisition area. The in-screen change detection unit 112 is a unit that detects whether or not there is a change in the personal computer terminal image data, generates change area information based on the presence or absence of the change, and generates change history information. is there. Further, the in-screen change detection unit 112 converts the image data of the change detection acquisition area determined by the change detection acquisition area determination unit 111 into a partial image data acquisition unit in order to detect the presence or absence of a change in the personal computer terminal image data. 113, and the acquired data storage unit 10
3 is compared with the image data of the same area of the image data stored in No. 3. Change area information and change history information storage unit 30
Reference numeral 1 denotes a storage unit that stores change area information and change history information generated by the in-screen change detection unit 112. Partial image data acquisition unit 113 and acquired image synthesis unit 114
Is the same as in the second embodiment.

Next, the overall processing flow of the image data acquisition unit 102 in this embodiment will be described. FIG. 12 is a flowchart illustrating the overall processing flow of the image data acquisition unit 102 according to the present embodiment. As shown in FIG. 12, the image data acquisition unit 102 in the present embodiment
In the following flow, it is possible to realize the acquisition of PC terminal image data with reduced data acquisition amount without increasing the loss of change detection and without increasing the data acquisition amount for change detection. It realizes the acquisition of image data of personal computer terminals with reduced detection of change detection.

Step 3001: First, the in-screen change detecting unit 112 detects the presence or absence of a change in the personal computer terminal image data in accordance with the change detecting acquisition region determined by the change detecting acquisition region determining unit 111, thereby changing the image. Generate area information and change history information. Step 3002: Next, the in-screen change detection unit 112
Process the change area information and the change history information. Step 3003: Next, the in-screen change detection unit 112
The change area information and the change history information are stored in the change area information and the change history information storage unit 301. Step 3004: Next, the partial image data acquisition unit 113
Acquires the image data of the target block from the image data storage unit 13, and the acquired image combining unit 114 stores the image data of the target block acquired by the partial image data acquisition unit 113 in the post-acquisition data storage. The image is synthesized by replacing the image data stored in the unit 103 with the image data of the same area.

Now, the detailed operation of the present apparatus will be described.
Explanation of the process of generating the change area information and the change history information (Step 3001), the process of processing the change area information and the change history information (Step 3002), and obtaining the image data of the area that has changed in the change area information And a description of the combining process (step 3004). First, the flow of the process of generating the change area information and the change history information (step 3001) will be described in detail. FIG. 13 is a flowchart illustrating a flow of a process of generating the change area information and the change history information. As shown in FIG. 13, the change detection acquisition region determination unit 111 and the in-screen change detection unit 112 generate change region information and change history information in the following flow.

Step 3101: First, the change detection acquisition area determination unit 111 refers to the change history information, and the value of the target block exceeds a predetermined threshold value and is predetermined. If the value is not a multiple of the value (eg, “4” or more and cannot be divided by “5”), it is always determined that the area has a change.
The process moves to step 3107. If the value of the target block is lower than a predetermined threshold value and is not a multiple of the predetermined value (for example, it is not divisible by "-4" or less and "5") ) It is always determined that there is no change, and the process proceeds to step 3106 as a block having no change. If the value is a multiple of a predetermined value ("5"), the block is treated as a block requiring change detection, and the process proceeds to step 3102. In other cases (Example: "-3" or more and "3" or less)
Is set as an area requiring change detection in step 310
Move to the processing of 2.

Step 3102: Next, the in-screen change detection unit 112 sets the change detection acquisition area in blocks of the designated resolution as shown in FIG.
The line is specified as the change detection acquisition area. Step 3103: Next, the in-screen change detection unit 112
The image data of the change detection acquisition area is acquired from the image data storage unit 13 by operating the partial image data acquisition unit 113. Step 3104: Next, the in-screen change detection unit 112
A change is detected by comparing with the image data stored in the post-acquisition data storage unit 103 in the same area as the image data acquired in the process of step 2002. In this case, if the difference between the two data to be compared is not zero, it is determined that there is a change. Step 3105: Next, the in-screen change detection unit 112
As a result of detecting the change, if there is no change in the target block, the process proceeds to step 3106. On the other hand, if there is a change in the target block, the process moves to step 3108. Step 3106: Next, the in-screen change detection unit 112 sets the flag indicating the target block of the change area information to 0.
To Step 3107: Next, the in-screen change detection unit 112 adds information indicating that there is no change to the value indicating the block as the target of the change history information. Step 3108: Next, the in-screen change detection unit 112 sets the flag indicating the block targeted for the change area information to 1
To Step 3109: Next, the in-screen change detection unit 112 adds information indicating that there is a change to the value indicating the block targeted for the change history information. Step 3110: Next, change detection acquisition area determination unit 1
11 determines whether there are any remaining blocks,
If there are remaining blocks, the process proceeds to step 3101. If there are no remaining blocks, the process ends.

Further, a detailed description will be given of a case where the value of the target block described in the change history information, which is set in step 3101, is not less than “4” and cannot be divided by “5”. If the value cannot be divided by “4” or more and “5”, the change detection acquisition area determination unit 111 always determines that the block has a change if the value of the target block is 4 or more, and detects the change. Is not performed, and there is a change, but this means that a change detection process is performed once every five times. That is, the change detection acquisition area determination unit 111
However, this is to prevent a block that is once determined to always have a change from being subjected to a change detection process.

Similarly, if the value is not smaller than “−4” and cannot be divided by 5, the change detection acquisition area determination unit 111 determines that the target block has a value of 4 or less. It is determined that there is no change without performing the change detection processing, but this means that the change detection processing is performed once every five times. That is, the change detection acquisition region determination unit 111
This is to prevent a change from being missed by not performing a change detection process on the block once determined to be always unchanged.

Here, regarding the detailed operation of the process (step 3001) for generating the change area information and the change history information, the in-screen change detection unit 112 determines that the value indicating the block to be the target of the change history information has no change. (Step 3107), a process in which the in-screen change detection unit 112 adds information indicating that there is a change in a value indicating a block targeted for change history information (step 31)
09) will be described.

First, a detailed description will be given of the flow of the process (step 3107) in which the in-screen change detection unit 112 adds information indicating that there is no change to the value indicating the target block of the change history information. FIG. 14 is a flowchart showing the flow of a process (step 3107) in which the in-screen change detection unit 112 adds information indicating that there is no change to a value indicating a block targeted for change history information. As shown in FIG.
The in-screen change detection unit 112 adds information indicating that there is no change to the value indicating the target block of the change history information in the following flow.

Step 3201: First, when the in-screen change detecting unit 112 indicates that the value indicating the target block of the change history information is 0 or more, the process proceeds to step 3202. If not, the process proceeds to step 3203. Step 3202: Next, the in-screen change detection unit 112 sets a value indicating a block as a target of the change history information to −1. Step 3203: Next, the in-screen change detection unit 112 subtracts 1 from the value indicating the block as the target of the change history information.

Next, the flow of the process (step 3109) in which the in-screen change detecting unit 112 adds information indicating that there is a change to the value indicating the block targeted for the change history information will be described in detail. FIG. 15 is a flowchart showing the flow of the process (step 3109) in which the in-screen change detection unit 112 adds information indicating that there is a change to the value indicating the target block of the change history information. As shown in FIG.
The in-screen change detection unit 112 adds information indicating that there is a change to the value indicating the target block of the change history information in the following flow.

Step 3301: First, if the value indicating the block as the target of the change history information by the in-screen change detection unit 112 is 0 or less, the process proceeds to the step 3302. Otherwise, the process moves to step 3303. Step 3302: Next, the in-screen change detection unit 112 sets the value indicating the block to be the target of the change history information to 1. Step 3303: Next, the in-screen change detection unit 112 adds 1 to the value indicating the block that is the target of the change history information.

Next, the flow of processing (step 3002) for processing the change area information and the change history information will be described in detail. FIG. 16 is a flowchart showing the flow of the process (step 3002) in which the in-screen change detection unit 112 processes the change area information and the change history information. As shown in FIG. 16, the in-screen change detection unit 112 processes the change area information and the change history information in the following flow.

Step 3401: In-screen change detecting section 11
2 determines whether or not the value of the flag indicating the target block of the change area information is 0. If it is 0, the process proceeds to step 3402. If it is not 0, the process moves to step 3405. Step 3402: Next, the in-screen change detection unit 112
Whether both the values of the flags indicating the blocks located above and below the block targeted for the change area information are 1,
If either the upper or lower block does not exist and the remaining one is 1, the process proceeds to step 3403. If not, the process proceeds to step 3405. Step 3403: Next, the in-screen change detection unit 112
The value of the flag indicating the target block of the change area information is set to 1. Step 3404: The in-screen change detection unit 112 adds information indicating that there is a change to the value indicating the block targeted for the change history information. (Same as the processing in step 3109) Step 3405: Next, the in-screen change detection unit 112
It is determined whether or not there are remaining blocks. If there are remaining blocks, the process proceeds to step 3101. If there are no remaining blocks, the process ends.

Next, a process of acquiring and synthesizing the image data of the area changed by the changed area information (step 3004)
Will be described in detail. FIG. 17 shows the flow of the process (step 3004) in which the partial image data obtaining unit 113 and the obtained image synthesizing unit 114 obtain, from the image data storage unit 13, the image data of the area changed by the change area information and synthesize them. Shown in the flowchart. As shown in FIG.
Partial image data acquisition unit 113 and acquired image synthesis unit 114
Obtains and combines image data of an area changed by the changed area information in the following flow.

Step 3501: First, the partial image data acquisition unit 113 determines the value of the flag indicating the block to be changed area information, and if the value is 1, the process proceeds to step 3502. Otherwise, the process proceeds to step 3504. Step 3502: Next, the partial image data acquisition unit 113
Acquires the image data of the target block from the image data storage unit 13. Step 3503: Next, the acquired image synthesizing unit 114 replaces the image data of the target block acquired by the partial image data acquiring unit 113 with the image data of the same area stored in the acquired data storage unit 103. You. Step 3504: Next, the partial image data acquisition section 11
3 judges whether there are any remaining blocks. If there are any remaining blocks, the process proceeds to step 3501, and if there are no remaining blocks, the process ends.

As is clear from the above description, according to the apparatus of the present embodiment, the change history information and the change area information are used in the processing of the image data acquisition section 102 of the second embodiment. As a result, it is possible to realize the acquisition of PC terminal image data with a reduced data acquisition amount without increasing the loss of change detection, and to miss the change detection without increasing the data acquisition amount for change detection. It is possible to realize a reduced acquisition of PC terminal image data.

Note that a predetermined number of frames (example: 1)
By acquiring all the regions once per 0 frame) without performing the change detection, it is possible to reduce missed changes. In addition, by reducing the change detection acquisition area based on the change history information, the possibility of detecting sudden changes in PC terminal image data increases compared to the state where change detection is completely omitted. Missed detection can be reduced. Therefore, in the process of step 3101, the change detection acquisition region determination unit 111 determines that the value in the change history information of the target block is lower than a predetermined threshold (eg, “−4”). "Below" Judgment that there is a high possibility that there is no change, and step 3
The acquisition area for change detection determined in 102 is reduced.

<< Fourth Embodiment >> In the present embodiment,
By using the changed area information generated in the process of the image data acquisition unit 102 according to the third embodiment, 1) Only the changed area is subjected to encoding such as resolution conversion or image format conversion. To reduce the amount of calculation in the process of generating the image for encoding. 2) Encode only the changed area and encode the previously encoded frame. A description will be given of a case where the processing of encoding a region where there is no change is omitted by combining with the subsequent data, and the amount of calculation of the process of generating a compressed video signal is reduced.

First, the basic configuration of the entire system in the present embodiment will be described. FIG. 18 is a functional block diagram of the entire system according to the present embodiment. As shown in FIG. 18, the present apparatus includes an image data storage unit 13, an image data acquisition unit 102, a post-acquisition data storage unit 103, a change area information and change history information storage unit 30.
1, a change adaptive coding image generation unit 404, a coding partial image data storage unit 405, a change adaptive coding unit 406, and an output compressed video signal storage unit 106.

The image data storage section 13, the image data acquisition section 102, the post-acquisition data storage section 103, and the change area information and change history information storage section 301 are the same as the means described in the third embodiment. is there. The output compressed video signal storage unit 106 is the same as the unit described in the first embodiment. The change adaptive coding image generation unit 404 generates N * N pixels for coding (16 * 16 in the present embodiment) based on the change region information and the change region information stored in the change history information storage unit 301. In the encoding block unit, which is a pixel unit, a changed block is determined, and encoding partial image data, which is image data in the encoding block unit converted for encoding, is generated. Means. The encoding partial image data storage unit 405 is a storage unit that stores encoding partial image data. The change adaptive encoding unit 406 and a unit that performs encoding in units of encoding blocks based on the partial image data for encoding, and combines the encoded data with the encoded data of the previously encoded frame.

Next, the overall processing flow in this embodiment will be described. The processing of the image data acquisition unit 102 is the same as in the third embodiment. FIG. 19 is a flowchart showing the flow of processing performed by the change adaptive coding image generation unit 404 and the change adaptive coding unit 406 in the present embodiment. As shown in FIG. 19, the image generation unit for change adaptive coding 404 and the change adaptive coding unit 4 according to the present embodiment.
In the flow 06, the amount of calculation of the process of generating the compressed video signal is reduced by omitting the conversion process necessary for encoding and the encoding process.

Step 4001: First, the change-adaptive coding image generation unit 404 uses the change region information and the change region information stored in the change history information storage unit 301 to determine the target in units of coding blocks. It is determined whether or not a corresponding area has changed in the post-acquisition data, which is the PC terminal image data stored in the post-acquisition data storage unit 103, for the encoding block. The image data of the corresponding area on the subsequent data is acquired and converted for encoding, thereby generating encoding partial image data. Step 4002: Next, the change adaptive coding section 406
Obtains the encoding partial image data stored in the encoding partial image data storage unit 405, encodes the acquired partial image data, and then encodes the previously encoded frame on the encoded data.
The compressed video signal is generated by replacing the encoded data at the corresponding position in units of encoding blocks.

From now on, the detailed operation of this apparatus will be described.
Processing for generating partial image data for encoding (step 40)
01) and the process of generating the compressed video signal (step 4002) will be described in detail. First, the image generation unit 404 for change adaptive coding in the present embodiment
The basic configuration will be described. An area surrounded by a dotted line in FIG. 20 is a functional block diagram of the change adaptive coding image generation unit 404 according to the present embodiment.

As shown in FIG. 20, the image generation unit 404 for change adaptive coding according to the present embodiment includes a post-acquisition image data portion acquisition unit 411, a resolution conversion unit 412, an image format conversion unit 413, Position information adding unit 414
Consists of The post-acquisition image data acquisition unit 411 obtains the post-acquisition data for the target encoding block in units of encoding blocks based on the change area information and the change area information stored in the change history information storage unit 301. Storage unit 10
3 is a means for judging whether or not there is a change in the corresponding area on the post-acquisition data stored in No. 3 and, if there is a change, acquiring image data of the corresponding area on the post-acquisition data. The resolution conversion unit 412 is a unit that performs enlargement or reduction processing on the image data acquired by the acquired image data partial acquisition unit 411 to convert the resolution. The image format conversion unit 413 converts the format of the image with respect to the image data whose resolution has been converted by the resolution conversion unit 412 (when the image data is RGB 24-bit data after acquisition and the output video signal is MPEG, it is converted to the YUV format). Means. The position information adding unit 414 adds the position information of the target coding block on the coding frame to the converted image data in units of coding blocks, thereby forming a code as shown in FIG. This is means for generating partial image data for conversion.

Next, the image generation unit 40 for change adaptive coding
The flow of the process of generating the partial image data for encoding (step 4001), which is the process of No. 4, will be described in detail. FIG. 21 is a flowchart showing the flow of processing for generating encoding partial image data. As shown in FIG. 21, the change adaptive encoding image generation unit 404 generates encoding partial image data according to the following flow.

Step 4101: First, the change-adaptive coding image generation unit 404 uses the information described in the change area information stored in the change area information and change history information storage unit 301 to set the target coding If any one of the flags indicating the block including the area on the post-acquisition image data corresponding to the area on the one frame of the image data for encoding of the block is "1", the step is performed. 41
Move to the process of 02. Otherwise, step 4106
Move on to processing. Step 4102: Next, the change adaptive coding image generation unit 404 sets the target coding block on the post-acquisition image data corresponding to the region on one frame of the coding image data of the coding block. After the image data of the area is acquired, the image data is acquired from the image data storage unit 103. Step 4103: Next, the resolution conversion section 412 converts the image data obtained in the processing of step 4102 into the resolution of the encoding block. Step 4104: Next, the image format converter 41
3 converts the format of the image data whose resolution has been converted in step 4103 into an encoding image format (YUV format when the output video signal is MPEG). Step 4105: Next, the position information adding unit 414 adds the position information of the target coding block on the coding frame to the image data whose format has been converted in step 4104, thereby performing coding. For generating partial image data for encoding and storing the partial image data for encoding 405
To be stored. Step 4106: Next, the post-acquisition image data portion acquisition unit 411 determines whether there is any remaining coding block. If there is any remaining coding block,
The process moves to step 4101 and ends if there are no remaining coding blocks.

Next, the flow of the process of generating a compressed video signal (step 4002) will be described in detail. First,
The basic configuration of change adaptive encoding section 406 according to the present embodiment will be described. An area surrounded by a dotted line in FIG. 23 is a functional block diagram of the change adaptive coding section 406 according to the present embodiment. As shown in FIG. 23, change adaptive coding section 406 in the present embodiment
Comprises an encoding partial image data acquisition unit 421, a block encoding unit 422, an encoding data synthesis unit 423, and an encoded data storage unit 424.

The encoding partial image data acquisition unit 421
This is a unit for acquiring an encoding block from the encoding partial image data storage unit 405. The encoding block encoding unit 422 is a unit that encodes the encoding block.
The post-encoding data synthesizing unit 423 outputs the post-encoding data at the corresponding position on a per-encoding block basis on the previously-encoded data stored in the post-encoding data storage unit 424. Data and coding block coding unit 42
2 and the data of the encoding block encoded by
It is a means to synthesize by substitution. The encoded data storage unit 424 is a storage unit that stores encoded data for one frame.

Next, the process of generating a compressed video signal, which is the process of the change adaptive encoding unit 406 (step 400)
The flow of 2) will be described in detail. FIG. 2 shows a flow of a process in which the change adaptive encoding unit 406 generates a compressed video signal.
4 is shown in the flowchart. As shown in FIG. 24, the change adaptive encoding unit 406 generates a compressed video signal according to the following flow.

Step 4201: First, the encoding partial image data acquisition section 421 acquires the encoding block stored in the encoding partial image data storage section 405. Step 4202: Next, the encoding block encoder 4
22 encodes the target encoding block. Step 4203: Next, the encoded data synthesizing section 423
Is, on a previously encoded data stored in the post-encoded data storage unit 424, in units of encoding blocks, the encoded data at the corresponding position and the encoding block. The data of the encoding block encoded by the encoding unit 422 is synthesized by replacing the data. Step 4204: Next, the encoded data synthesizing section 423
It is determined whether there is any remaining coding block. If there is a remaining coding block, the process proceeds to step 4201. If there is no remaining coding block, step 4205 is performed. Move on to the process. Step 4205: Next, the encoded data synthesizing unit 423
Obtains the encoded data stored in the encoded data storage unit 424. Step 4206: Next, the encoded data synthesizing unit 423
However, the encoded data obtained in step 4205 is stored in the output compressed video signal storage unit 106.

As is apparent from the above description, the apparatus according to the present embodiment is different from the image data acquiring section 1 according to the third embodiment.
02, by using the changed area information generated in the processing of step 02, the calculation of the area where there is no change is omitted, the calculation amount of the processing for generating the image for encoding is reduced, and the coding of the area where there is no change is performed. And the conversion can be performed with a reduced amount of calculation of the process of generating the compressed video signal.

Even when no change area information is generated as in the processing of the image data acquisition unit 102 in the second embodiment, the following operation is repeated for a block that has changed by one frame. In addition, it is possible to reduce the calculation amount of the process of generating the image for encoding, and to omit the coding process of the area where there is no change, thereby reducing the calculation amount of the process of generating the compressed video signal. Then, the image data acquisition unit 1
02 converts the resolution, converts the image format, performs encoding, and synthesizes the image data of the target block acquired for the area with the change without acquiring the acquired image data as the image data after acquisition. And the encoded data of the frame. At this time, after the resolution of the block is converted to the
The number of pixels should be a multiple of 16 horizontal pixels.

The image generation unit 40 for change adaptive coding
4 is obtained from the post-acquisition image data storage unit 103, the image data of the area on the post-acquisition image data corresponding to the area of the target encoding block on one frame of the image data for encoding. The processing (step 4102) and the processing performed by the resolution conversion unit 412 to convert the image data acquired in the processing of step 4102 to the resolution of the encoding block (step 4104) can be reversed. In addition, instead of generating a coding block, the change adaptive coding image generation unit 404 according to the present embodiment combines one frame of coding image data generated in the past with one frame of coding image data. Is generated, the process of the change adaptive encoding unit 406 causes the encoding unit 10
5 can generate a compressed video signal only.

Further, the processing of the change adaptive coding section 406 in the present embodiment is effective for a stream of only intra frames in MPEG, but in the case of a frame for which inter-frame prediction is performed, a region in which no change has occurred. Can be generated by adding information indicating that the block is the same as before. Furthermore, when the change area on the personal computer terminal image data is equal to or larger than a predetermined area size, the change adaptive coding image generation unit 40
4 generates all the image data for encoding in one frame, and the change adaptive coding unit 406 performs the coding as an intra frame, thereby performing inter-frame prediction on the change area and increasing useless codes. Without coding, and by coding other frames as frames for inter-frame prediction, without increasing the useless code by coding as an intra block for a region that does not change. Can be

<< Fifth Embodiment >> In the present embodiment,
Encoding image generation unit 104 according to the first embodiment
In the processing of the encoding unit 105, a method of changing the image quality after encoding for each area within the area of the personal computer terminal image data acquired by the image data acquisition unit 102 will be described.

In the present embodiment, the following three processes realize changing the image quality after encoding for each area in the area of the acquired personal computer terminal image data. 1) First, in the area of the personal computer terminal image data acquired by the image data acquiring unit 102, the areas are classified according to media such as characters, line drawings, and natural images. 2) Next, the encoding image generation unit 104 performs a process such as a low-pass filter previously associated with each medium for each of the classified regions for each medium, Perform pre-processing for each area. 3) Next, the encoding unit 105 weights the regions classified by media for each region, adjusts the target bit amount to be allocated at the time of encoding for each region, and encodes the image for encoding. This generates a compressed video signal.

By the above three processes, the image quality after encoding can be changed for each area in the area of the acquired personal computer terminal image data. Hereinafter, the manner of changing the image quality after encoding for each region will be described in detail. In the present embodiment, the media represents image data having features such as characters, line drawings, natural images, monochromatic areas, and texture areas. To classify areas according to media means to any area. On the other hand, it is to determine a medium in which the characteristics of the target area are similar. Also,
The flow of the entire process of this embodiment is the same as that of the flowchart of FIG. 3 of the first embodiment except that the encoding image generator 104 stores the personal computer terminal image data To generate encoding image data which is a signal of 4: 2: 0 format of YUV (Y: luminance, U: color difference, V: color difference) which is a format for encoding to MPEG (step 1004). Processing), and processing for generating a compressed video signal from the encoding image data generated by the encoding unit 105 in step 1003 (step 100).
5) is different. Otherwise, the process is the same as the process in the first embodiment.

First, the configuration of the encoding image generation unit 104 according to the present embodiment will be described. The part surrounded by the dotted line in FIG. 25 is a functional block diagram of the encoding image generation unit 104 according to the present embodiment. As shown in FIG. 25, in the present embodiment, the encoding image generation unit 10
Reference numeral 4 includes a YUV conversion unit 501, an area classification unit 502, an area-specific preprocessing unit 503, a resolution conversion unit 504, and an area classification information storage unit 505.

The YUV conversion section 501 is means for converting the image format of the personal computer terminal image data stored in the acquired image data storage section 103 into the YUV 4: 2: 0 format. The area classification unit 502
A means for judging and classifying the media type of the personal computer terminal image data converted to the 4: 2: 0 format in block units, as shown in FIG. 30 and targeted in block units of vertical N pixels and horizontal N pixels. Area classification information in which the area of the block to be described is described by a value indicating the type of each medium. In the present embodiment, area classification is performed by providing a means for extracting a general character area. Here, the above-mentioned “means for extracting a general character area” means, for example,
This is a means for generating a histogram representing the distribution of luminance values in block units of N pixels and N pixels, and determining whether or not the area is a character or line drawing area based on the distribution. In the present embodiment, it is determined whether the area of the block is a text or line drawing area in units of 8 pixels vertically and 8 pixels horizontally.
Otherwise, it is determined to be a natural picture.

The region-specific preprocessing unit 503 is a means for performing region-specific preprocessing on a block-by-block basis based on the region classification information stored in the region classification information storage unit 505. In the present embodiment, the pre-processing for each area is processing for blurring the area of a target block using a filter generally called a low-pass filter for a specified area. The resolution conversion unit 504 is a unit that converts the resolution of the image data after the preprocessing performed by the region-based preprocessing unit 503. The area classification information storage unit 505 includes:
This is a storage unit that stores the area classification information generated by the area classification unit 502.

First, a basic process in which the encoding image generation unit 104 generates an encoding image in the present embodiment will be described. A flowchart of FIG. 26 illustrates a basic processing flow in which the encoding image generation unit 104 generates an encoding image in the present embodiment. As illustrated in FIG. 26, the encoding image generation unit 104 generates an encoding image according to the following flow. Step 5001: First, the YUV conversion unit 501 converts the image format of the personal computer terminal image data stored in the acquired image data storage unit 103 into YUV 4:
Convert to 2: 0 format. Step 5002: Next, the area classification unit 502
Based on the personal computer terminal image data converted into the 4: 2: 0 format, area classification information is generated using a means for extracting a general character area, and stored in the area classification information storage unit 505. Step 5003: Next, the area-based pre-processing section 503 stores the area classification information storage section 505 in the area classification section 502.
Performs pre-processing for each area on a block-by-block basis based on the area classification information generated by. Step 5004: Next, the resolution conversion unit 504 converts the resolution of the image data after the region-based pre-processing unit 503 performs the pre-processing.

Next, the area classification unit 502 sets the YUV 4:
FIG. 27 is a flowchart showing the flow of a process for generating area classification information using means for extracting a general character area based on the personal computer terminal image data converted into the 2: 0 format. As shown in FIG. 27, the area classification unit 502 generates area classification information according to the following flow. Step 5011: First, the area classification unit 502 determines that the YUV
Based on the personal computer terminal image data converted to the 4: 2: 0 format, using a means for extracting a general character area, the area of the target block is divided into blocks of 8 pixels vertically by 8 pixels horizontally. It is determined whether the area is a character or line drawing area, and other areas are determined to be natural images. Step 5012: Next, the area classification unit 502 describes the value representing the media of the area of the target block in the area classification information. In the present embodiment, 0 is described for a character or line drawing area, and 1 is described for a natural image. Step 5013: Next, the area classification unit 502 determines whether or not there is a remaining block. If there is any remaining block, the process returns to step 5011. Step 5013: If there is no remaining block, the area classification unit 5
02 stores the area classification information in the area classification information storage unit 505, and ends.

Next, the region-specific preprocessing unit 503 uses the region classification information stored in the region classification information storage unit 505 based on the region classification information.
FIG. 2 shows a flow of a process of performing preprocessing for each area in block units.
8 is shown in the flowchart of FIG. As shown in FIG. 28, the region-specific pre-processing unit 503 performs region-specific pre-processing according to the following flow. Step 5021: First, the area-specific preprocessing unit 503 stores the area classification information stored in the area classification information storage unit 505, which is described in the area classification information, with a value indicating the medium in which the type of media indicating the target block is designated Determine if there is. In the present embodiment, 1 representing a natural image is a value representing a designated medium. If the value is 1, step 5022: the region-specific pre-processing unit 503 performs a low-pass filter process on the image data of the target block. Step 5023: Otherwise, the region-specific pre-processing unit 503 determines whether there is any remaining block,
If it remains, the process returns to step 5021; otherwise, the process ends.

FIG. 29 is a flowchart showing the flow of the process performed by the resolution conversion unit 504 to convert the resolution of the image data after the preprocessing by the region-specific preprocessing unit 503 has been performed.
As shown in FIG. 29, the resolution conversion unit 504 converts the resolution of the image data after the region-based pre-processing unit 503 performs the pre-processing in the following flow. Step 5031: First, when the resolution of the personal computer terminal image data stored in the post-acquisition image data storage unit 103 is the same as the compressed video signal to be output, the resolution conversion unit 504 ends the process. The resolution conversion unit 504 converts the resolution by performing enlargement and reduction processing, and ends the processing.

As is clear from the above description, the encoding image generation unit 104 according to the present embodiment operates within the area of the personal computer terminal image data acquired by the image data acquisition unit 102.
By classifying the areas according to the media, it is possible to generate the encoding image while changing the pre-processing for each of the classified media areas.

Next, encoding section 10 according to the present embodiment
5 will be described. A portion surrounded by a dotted line in FIG. 31 is a functional block diagram of encoding section 105 in the present embodiment. As shown in FIG. 31, in the present embodiment, encoding section 105 includes weight information generation section 511
, A region-specific weighting encoding unit 512 and a weighting information storage unit 513.

The weighting information generating section 511 generates weighting information as information having a value indicating the weight of the target bit amount at the time of encoding in units of 16 pixels by 16 pixels as shown in FIG. Means.

In the present embodiment, 15 is assigned as a large number to a block to which a large number of bits are to be assigned, and 3 is assigned to a block that does not. This value is used to make the target bit amount proportional to the number when encoding the target block. For example, if the value of 10 is assigned, 3
A bit amount of 2 / min is a target bit amount. In addition,
The numerical value described in the weighting information is an integer of 1 or more.

In the present embodiment, coding section 105 generates weighting information based on the area classification information, adjusts the bit amount to be allocated at the time of coding based on the weighting information for each area, and performs coding. FIG. 32 is a flowchart showing a basic processing flow for generating a compressed video signal by encoding a use image. As shown in FIG. 32, the encoding unit 105
A compressed video signal is generated according to the following flow. Step 5041: First, the weighting information generation unit 511
Generates weighting information based on the area classification information stored in the area classification information storage unit, and stores the generated weighting information in the weighting information storage unit 513. Step 5042: Next, the region-specific weight coding section 51
2 adjusts the target bit amount to be allocated at the time of encoding based on the weighting information stored in the weighting information storage unit 513, and performs encoding.

Next, the weighting information generation section 511 generates weighting information based on the area classification information stored in the area classification information storage section, and generates the weighting information.
33 is shown in the flowchart of FIG. As shown in FIG. 33, the weighting information generation unit 511
Generates weighting information and stores it in the weighting information storage unit 513 in the following flow. Step 5051: First, the weighting information generation unit 511
Determines the weighting value based on the classification of the media representing the target block described in the region classification information stored in the region classification information storage unit 505. In addition,
In the present embodiment, it is assumed that the weighting value is determined in advance depending on the type of each medium. In this embodiment, the resolution of the block targeted for the area classification information is 8 pixels vertically and 8 pixels horizontally, whereas the resolution of the block targeted for the weighting information is 16 pixels vertically and 16 pixels horizontally. There are four target block values in the weighting information described in the area classification information. In this case, the largest value is set as the media of the block. In addition, when the value of the designated medium is at least one of the four values, by using the medium, a block including even a small area of the designated medium can be processed as the designated medium. .

Step 5052: Next, the weighting information generation section 511 describes the weighting value of the target block determined in step 5051 in the weighting information. Step 5053: The weighting information generation unit 511 determines whether there is any remaining block. If there is any remaining block, the process returns to step 5051. If not, step 5054: Weighting information storage unit 5
05 and end.

Next, the region-specific weighting encoding unit 512
Based on the weighting information stored in the weighting information storage unit 513, a flow of a process of adjusting a target bit amount to be allocated at the time of encoding for each region and encoding a coding image to generate a compressed video signal. Is shown in the flowchart of FIG. As shown in FIG. 34, the region-specific weighting encoding unit 512 generates a compressed video signal from an encoding image in the following flow. Step 5061: First, the region-specific weighting encoding unit 51
2 obtains a weight value indicating a block which is a target of the weight information stored in the weight information storage unit 505. Step 5062: Next, the region-specific weighting encoding section 51
2 determines a target generated bit amount after encoding when encoding the target block according to the obtained weighting value. For example, as one of the determination methods,
The amount of generated bits given to one frame is multiplied by the weight value of the target block, and the result is divided by the sum of all the weight values to become the target for the target block. There is a method of using the generated bit amount after encoding. Step 5063: Next, the region-specific weight coding section 51
2 encodes the target block in accordance with the determined target amount of generated bits after encoding. Step 5064: Next, the region-specific weight coding section 51
2 determines whether there are any remaining blocks. If there are any remaining blocks, the process returns to step 5051. If not, step 5065: If there are no remaining blocks, the compressed video is Generate signal and end.

As is clear from the above description, the encoding image generation unit 104 and the encoding unit 10 according to the present embodiment.
Reference numeral 5 denotes a method of classifying areas in the area of the personal computer terminal image data acquired by the image data acquisition unit 102 according to media such as characters, line drawings, and natural images, and an encoding image based on the information of the classified areas. The generation unit 104 and the encoding unit 105
However, by changing the processing for each region, the image quality after encoding can be changed for each region in the region of the acquired PC terminal image data. As a result, the PC terminal image data is coded so as to reduce the high-frequency components of the image by applying a low-pass filter to an area other than the specified medium area, and to reduce the amount of generated bits after coding. This makes it possible to reduce the amount of generated bits after encoding, and to allocate a sufficient amount of bits to the specified media area. The image quality of the area can be improved.

In this embodiment, the value of the area information indicates the area of two media, ie, the designated medium and the other medium. However, the value of the area information indicates three or more types, and It is also possible to select the processing. Also, if the position is specified on the PC terminal image by a pointing device such as a mouse pointer,
By designating the medium at the designated position as the medium of the area to be clearly coded, the user using the present apparatus can directly and graphically select the medium having high image quality during the conversion.

Further, when generating the weighting information, the value of the weighting information for the block at the position specified by the pointing device on the personal computer terminal image and the blocks around the block is changed as shown in FIG. , The image quality around the designated position can always be sharpened. Also, by setting the value of the area information for the block at the position specified by the pointing device on the personal computer terminal video and its surrounding blocks to the value of the specified medium, the processing of the encoding image generation unit 104 is performed. Is changed with respect to the position specified by the pointing device, thereby making it possible to change the image quality of the region of the encoded compressed video signal that targets the position specified by the pointing device.

The weighting information generation unit 511 determines the weighting value based on the classification of the media representing the target block described in the region classification information stored in the region classification information storage unit 505. Processing (step 5)
051), the image quality of the encoded compressed video signal can be more effectively changed by determining the weight value for the media as shown in the following five examples.

Further, the processing of the area-specific pre-processing unit 503 is performed by changing the value indicating the type of medium of the area information and performing the area-specific pre-processing for an area having a large weighting value. The image quality can be changed more effectively. (1) The weighting information generation unit 511 calculates the weighting value
In the processing of the area classification unit 502, when the area is classified into the area of the designated medium and the other area, which are the media specified in advance, according to the ratio between the area (the number of blocks) of the area of the designated medium and the area other than the area. , The variation in the amount of bits allocated to the specified media area due to the change in the ratio of the specified media area in the input image can be reduced. If the area of the area of the designated medium is larger than the area of other areas, the weighting information generation unit 511 will
In accordance with the size ratio, the value of the weighting information for the area of the designated medium is increased, and the value for the other area is reduced. In this case, the area classification unit 502 classifies the area into two types, that is, the area of the designated medium and the other area. However, the area is classified into three or more types, and the weighting information generation unit 511 sets It is also possible to change the weighting according to the importance of the media and the area ratio. (2) The weighting information generation unit 511 sets the weighting value
In the process of the area classification unit 502, when the area is classified into a character area and another area, the area is changed as follows according to the display position of the character area, so that a predetermined area in the input image is changed. Is a character area, the image quality of that area can be sharpened. For example, in the case of an image used in a presentation, when a title region, a header region, a footer region, and the like are determined in advance by a template or the like, a compressed video signal in which the image quality of each region is clear can be generated. When the predetermined important area is the area of the designated medium (character), the weighting information generation unit 511 increases the value of the area to be weighted information. (3) The weighting information generation unit 511 calculates the weighting value
In the processing of the area classification unit 502, when the area is classified into a character area and other areas, the area is changed as described below in accordance with predetermined characteristics such as the color of the character area, so that the input area of the input image is changed. For a region having a predetermined characteristic in a character region, the image quality of the region can be sharpened. If the area determined to be a character area has the specified characteristics, the value of the area targeted for the weighting information is increased. For example, when the area determined to be a character area includes a red pixel, the weighting information generation unit 511 determines that the area includes important description and increases the value of the area to be weighted information. As a result, the image quality of the area becomes clear. further,
If the area determined to be a character area is the same area in the image data acquired in the past, and the same area is consecutively determined to be a character area at a certain number of acquisitions in the past, weighting is performed. The information generation unit 511
The area is regarded as an important area, and the value of the area to be subjected to the weighting information is increased, thereby sharpening the image quality of the area. (4) The weighting information generation unit 511 calculates the weighting value
In the process of the area classification unit 502, when the area is classified into the area of the designated media and the other area, the area is changed as follows according to the bit amount of the generated compressed video signal (for example, the bit rate in MPEG). The distribution of the bit amount allocated to the area of the designated media can be changed according to the bit amount of the compressed video signal to be generated.
When the bit amount of the compressed video signal to be generated is large, the bits are allocated to the area of the designated medium without increasing the weighting ratio.
However, by reducing the difference between the weighting values for the area of the designated medium and the other area, the image quality of the area other than the area of the designated medium is not significantly impaired. When the bit amount of the generated compressed video signal is small, the weighting ratio is increased so that the bit is allocated to the area of the designated medium.
The weighting information generation unit 511 sharpens the area of the designated medium by increasing the difference between the weighting values for the area of the designated medium and the other areas. As a result, even when the bit amount of the compressed video signal to be generated is small, the image quality of the compressed video signal in the area of the designated medium can be kept clear.

[0127] More simply, 1. In the designated media area, maintain the image quality in the state where the band is wide (large) even if the band becomes narrow (low), or By changing the weighting ratio according to the change in the band, the change in the image quality of the designated medium area is reduced even if the band changes. As an image on mounting, for example, the bit allocation is changed according to the fluctuation of the network bandwidth (frame bit volume). Example 1) Bit allocation ratio with many bands Character; natural image = 6; Example 2) Bit allocation ratio with small bandwidth Character; natural image = 9; 1 Since the bandwidth is reduced, the specific gravity for characters is increased. It is so.

[0128]

As is apparent from the above description, the multimedia information conversion apparatus of the present invention converts an image displayed on a display by a personal computer terminal into one compressed video image conforming to a format such as a moving image and MPEG. It can be converted to a signal. Therefore, after being converted into one compressed video signal, it is transmitted to the receiving terminal by a conventional method, so that display on another terminal is realized.

Further, by incorporating the present invention in a video conference system or the like using a compressed video signal, it is possible to share a video among multiple terminals.

Further, the reproduction terminal requires only the minimum function of decoding and reproducing the video signal or the compressed video signal, so that reproduction at a low-cost terminal is possible.

Further, the multimedia information conversion apparatus of the present invention detects a change in the image data of the personal computer terminal, acquires only the changed area, and reuses the previously acquired image data in the other areas. Accordingly, the generation of image data having the same information amount as that when all the image data is obtained from the video card can be realized with a smaller data amount.

Therefore, when acquiring image data, it is possible to reduce the calculation amount of the present apparatus. In particular, in the case of a personal computer terminal that requires a large amount of processing time when acquiring data from a video memory or the like, reducing the amount of data acquired from the video memory or the like leads to a reduction in the processing time. Is realized. Therefore, it is possible to realize a system with a lower cost transmission terminal.

Further, the multimedia information conversion apparatus of the present invention processes a change detection result, which is a result of detection of a change in image data of a personal computer terminal, without increasing the number of calculations required to detect the change. The accuracy of detection can be improved.

As a result, it is possible to acquire a region where the image data has changed more accurately, and a transmission terminal that transmits a video displayed by a personal computer terminal more accurately is realized.

Further, the multimedia information conversion apparatus of the present invention creates and uses a change history based on a change detection result, so that the change is detected without deteriorating the accuracy of change detection. Operations can be reduced. Therefore, image data having the same information amount as that obtained when all image data is obtained from the video card can be obtained with a smaller calculation amount. Further, the amount of calculation of the present apparatus can be reduced, and a system with a lower cost transmitting terminal can be realized.

Further, the multimedia information conversion apparatus of the present invention reuses the encoded data of the stationary part of the input personal computer terminal image data,
By omitting the process of coding the stationary region, the amount of calculation of the process to be coded can be reduced, so that the amount of calculation of the entire conversion device can be reduced, and a system with a lower cost transmission terminal can be used. Can be realized.

Further, the multimedia information conversion apparatus of the present invention has means for classifying media types by region in an acquired image when encoding into one compressed video signal such as MPEG. , Classify the area,
By changing the pre-processing for each area, and by weighting the image and changing the compression ratio for each area at the time of coding, it is possible to perform coding in which the area of the selected medium is clear. Therefore, it is possible to generate a compressed video signal in which the image quality of the image area of the designated medium in the image displayed by the personal computer terminal is clear.
For example, it is possible to generate a compressed video signal in which an area around a position designated by a pointing device such as a character, a line drawing, or a mouse is sharpened.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a basic configuration of a multimedia information conversion device according to a first embodiment.

FIG. 2 is a flowchart showing a processing flow of a drawing information acquisition unit 100 of the multimedia information conversion device according to the first embodiment.

FIG. 3 is a flowchart showing the overall processing flow of the multimedia information conversion apparatus according to the first embodiment, excluding the processing of the drawing information acquisition unit 100;

FIG. 4 is a flowchart showing the flow of a process of generating image data for encoding (step 1003) according to the first embodiment;

FIG. 5 is a block diagram showing a basic configuration of the multimedia information conversion apparatus when acquiring personal computer terminal image data from the image data storage unit 13 in the multimedia information conversion apparatus according to the first embodiment;

FIG. 6 shows an image data acquisition unit 102 according to the second embodiment.
Block diagram showing the overall configuration of

FIG. 7 is a diagram showing a specific example of a change detection acquisition area.

FIG. 8 shows an image data acquisition unit 102 according to the second embodiment.
Of the overall process flow

FIG. 9 illustrates an image data acquisition unit according to a third embodiment.
Block diagram showing the overall configuration of

FIG. 10 is a diagram showing a specific example of change history information.

FIG. 11 is a diagram showing a specific example of change area information.

FIG. 12 shows an image data acquisition unit 10 according to the second embodiment.
2 is a flowchart showing the overall processing flow

FIG. 13 is a flowchart showing a flow of processing for generating change area information and change history information.

FIG. 14 is a flowchart showing the flow of a process in which the in-screen change detection unit 112 adds information indicating that there is no change to a value indicating a target block of change history information.

FIG. 15 is a flowchart showing the flow of a process in which the in-screen change detection unit 112 adds information indicating that there is a change to a value indicating a block targeted for change history information;

FIG. 16 is a flowchart showing the flow of processing for processing change area information and change history information;

FIG. 17 is a flowchart showing the flow of processing for acquiring and combining image data of a region changed in the changed region information.

FIG. 18 is a block diagram of the entire system according to the fourth embodiment;

FIG. 19 is a flowchart showing the flow of processing of the change adaptive encoding unit 406.

FIG. 20 is a block diagram illustrating a basic configuration of an image generation unit for change adaptive coding 404 according to the fourth embodiment.

FIG. 21 is a flowchart illustrating a flow of a process in which the change adaptive encoding image generation unit 404 generates encoding partial image data.

FIG. 22 is a diagram showing a specific example of encoding partial image data.

FIG. 23 is a change adaptive encoding unit 4 according to the fourth embodiment.
06 is a block diagram showing a basic configuration.

FIG. 24 is a change adaptive coding unit 4 according to the fourth embodiment.
06 is a flowchart showing the flow of processing for generating a compressed video signal

FIG. 25 shows an image generation unit for encoding 1 according to the fifth embodiment.
04 functional block diagram

FIG. 26 is an image generation unit for encoding 1 according to the fifth embodiment.
04 is a flowchart showing a basic processing flow for generating an image for encoding.

FIG. 27 is a flowchart showing the flow of generating area classification information by the area classification unit 502;

FIG. 28 is a flowchart illustrating a flow of a process in which a pre-processing unit for each region performs pre-processing for each region.

FIG. 29 is a diagram illustrating an example in which the resolution conversion unit 504 is configured to execute
3 is a flowchart showing the flow of processing for converting the resolution of image data after performing pre-processing

FIG. 30 is a diagram showing a specific example of area classification information.

FIG. 31 is a functional block diagram of an encoding unit 105 according to the fifth embodiment.

FIG. 32 is a flowchart illustrating a flow of processing in which the encoding unit 105 generates a compressed video signal according to the fifth embodiment.

FIG. 33 is a flowchart showing the flow of a process in which the weighting information generation unit 511 generates weighting information.

FIG. 34 is a flowchart showing the flow of a process in which a region-specific weighting encoding unit 512 generates a compressed video signal from an encoding image.

FIG. 35 is a diagram showing a specific example of weighting information;

FIG. 36 is a diagram illustrating a specific example of weighting information generated based on position information of a pointing device.

[Description of Signs] 11 Drawing information creation unit 12 Image data generation unit 13 Image data storage unit 14 Display unit 100 Drawing information acquisition unit 101 Image data storage unit after duplication 102 Image data acquisition unit 103 Data storage unit after acquisition 104 Encoding Image generation unit 105 Encoding unit 106 Output compressed video signal storage unit 107 Overall operation control unit 111 Change detection acquisition region determination unit 112 In-screen change detection unit 113 Partial image data acquisition unit 114 Acquired image synthesis unit 301 Change region information and change History information storage unit 404 Image generation unit for change adaptive coding 405 Partial image data storage unit for coding 406 Change adaptive coding unit 411 Image data partial acquisition unit after acquisition 412 Resolution conversion unit 413 Image format conversion unit 414 Position information Addition unit 421 Encoding partial image data acquisition unit 422 Block encoding unit 4 3 encoding data synthesizing unit 424 post-encoding data storage unit 501 YUV conversion unit 502 area classification unit 503 area preprocessing unit 504 resolution conversion unit 505 area classification information storage unit 511 weight information generation unit 512 area weight encoding unit 513 Weight information storage

 ──────────────────────────────────────────────────続 き Continuing from the front page (72) Inventor Tsutomu Uenoyama 1006 Kadoma Kadoma, Osaka Prefecture Matsushita Electric Industrial Co., Ltd. 72) Inventor Hiroaki Yoshio 1006 Kazuma Kadoma, Kadoma City, Osaka Prefecture F-term in Matsushita Electric Industrial Co., Ltd. TC44 TD05 TD10 TD12 UA02 5C078 BA21 CA02 CA12 DA00 DA01 DB04 9A001 HH27

Claims (39)

[Claims] 1. A terminal comprising: image data generating means for generating terminal image data for displaying on a display device for displaying image data and the like; and terminal image data storing means for storing the terminal image data. A device that acquires the terminal image data and converts the image data into a compressed video signal, wherein the terminal acquires drawing information for generating terminal image data in a process of generating the terminal image data, Drawing information obtaining means for creating a copy of data, image obtaining means for obtaining the terminal image data copied by the drawing information obtaining means, and resolution and format of the terminal image data obtained by the image obtaining means. Sometimes
A multi-functional image processing apparatus comprising: an encoding image generating unit that performs conversion; and an encoding unit that encodes encoding image data that is image data obtained by converting the terminal image data, which is converted by the encoding image generating unit. Media information conversion device. 2. An apparatus for acquiring image data (terminal image data) displayed by a terminal and converting the image data into a compressed video signal, wherein the image acquiring unit acquires the terminal image data, and the image acquiring unit acquires the image data. From the terminal image data
A multimedia information conversion device comprising: an encoding image generating unit that converts the image data into encoding image data; and an encoding unit that encodes the encoding image data generated by the encoding image generating unit. 3. An apparatus for acquiring image data (terminal image data) displayed by a terminal from image data storage means in which the terminal image data is stored, and detecting an area where the terminal image data has changed. Change detecting means, a change area image obtaining means for obtaining image data of an area where the terminal image data has changed according to a detection result of the change detecting means, and image data obtained by the change area image obtaining means. A high-speed image acquisition device, comprising: a change area combining unit that combines the image data with image data acquired in the past. 4. A multimedia information conversion apparatus according to claim 2, wherein said image acquisition means is the high-speed image acquisition apparatus according to claim 3. 5. The encoding image generating unit according to claim 4, wherein the encoding image generating unit generates an encoding image for a changed area of the terminal image data according to a detection result of the change detecting unit. Multimedia information converter. 6. The encoding device according to claim 5, wherein the encoding means encodes the changed area in accordance with the detection result of the change detection means, and combines the changed area with previously encoded data. The multimedia information conversion device according to the above. 7. The encoding means, when generating a frame for performing inter-frame prediction, encodes a newly encoded area as an intra block, and encodes other areas as unchanged blocks. The multimedia information conversion device according to claim 6, wherein 8. The multi-frame encoding apparatus according to claim 4, wherein the encoding means encodes the data as an intra frame when a change greater than a predetermined threshold value is detected in the detection result of the change detecting means. Media information conversion device. 9. The coding means, when a change larger than a predetermined threshold value is detected in the detection result of the change detection means, codes the data as an intra frame, and codes the other frames by inter-frame prediction. The multimedia information conversion device according to claim 4, wherein the conversion is performed. 10. A method for generating change area information, comprising:
4. The high-speed image acquisition apparatus according to claim 3, wherein change area information that is information for separating a block having a change from a block having no change is generated. 11. The high-speed image acquisition apparatus according to claim 10, wherein the change detecting means processes the change area information based on the change area information itself. 12. The high-speed image acquisition method according to claim 3, wherein the change detection means does not perform change detection at a fixed or arbitrary frame interval, and the change area image acquisition means acquires all areas. apparatus. 13. The high-speed image acquisition device according to claim 3, wherein the change area image acquisition means acquires a range that is larger than the detection result of the change detection means. 14. The apparatus according to claim 3, wherein the change detecting means includes a change detecting area determining means for determining an area on the terminal image data for detecting a change in the terminal image data. High-speed image acquisition device. 15. The high-speed image acquisition apparatus according to claim 14, wherein the change detection area determination means changes the frequency of change detection according to the past history of the changed area. 16. The high-speed image according to claim 14, wherein the change detection area determining means dynamically changes an area on the terminal image data for detecting a change in the terminal image data. Acquisition device. 17. A method according to claim 17, wherein the change detection area determining means changes an area on the terminal image data for detecting a change in the terminal image data according to a past history of the changed area. Item 15. A high-speed image acquisition device according to item 14. 18. The method according to claim 17, wherein the change detection area determining means generates change history information having, as information, the number of times of continuous change in the past and the number of times of no continuous change in the past. High-speed image acquisition device. 19. When the value indicating the number of continuous changes in the past in the change history information is within a predetermined range, it is determined that there is a change without performing change detection, and image acquisition is performed. 19. The high-speed image acquisition device according to claim 18, wherein: 20. If the value indicating the number of times the change history information has not changed continuously in the past is within a predetermined range, it is determined that there is no change without performing change detection.
19. The high-speed image acquisition device according to claim 18, wherein no image is acquired. 21. When the value indicating the number of times of continuous change in the past and the value indicating the number of times of no continuous change in the past of the change history information exceed a predetermined range,
2. The method according to claim 1, wherein it is determined that there is a change, and an image is obtained.
21. The high-speed image acquisition device according to 9 or 20. 22. When a value indicating the number of times of continuous change in the past and a value indicating the number of times of no continuous change in the past of the change history information exceed a predetermined range,
It is determined that there is a change, an image is acquired, and a value indicating the number of continuous changes in the past of the change history information and a value indicating the number of times that have not continuously changed in the past are returned to 0. 21. The high-speed image acquisition device according to 19 or 20. 23. An apparatus according to claim 13, further comprising an area classifying means for classifying the area of the terminal image data among a plurality of types of predetermined characteristics for each of the characteristics, and changing a process for each of the classified areas. The multimedia information conversion device according to claim 1. 24. The multimedia information conversion apparatus according to claim 23, wherein said encoding image generating means comprises an area-specific preprocessing section for changing processing according to the area classification determined by the area classification means. 25. An encoding apparatus according to claim 23, wherein the encoding means weights the image quality of the encoding for each area according to the area classification determined by the area classification means, and changes the encoding for each area. A multimedia information conversion device according to claim 1. 26. A weighting means for weighting the image quality of coding for each area according to the classification of the area determined by the area classification means, and a coding means for coding each area based on the weighting information determined by said weighting means. 26. An encoding device for each region, characterized in that the encoding process is changed by using the value of the weighting information with respect to a target bit amount to be allocated in the case of (1). A multimedia information conversion device according to claim 1. 27. An area classification means divides an area of terminal image data into area blocks each of which is a rectangular area of N pixels vertically and M pixels horizontally (N and M are arbitrary integers of 1 or more). For each of the area blocks, an area is information that classifies each of a plurality of types of predetermined characteristics and describes a value indicating a predetermined characteristic determined for the target area block. The multimedia information conversion device according to claim 23, wherein classification information is generated. 28. An area-based pre-processing means for selecting a predetermined process in accordance with an area of a predetermined feature in units of area blocks according to area classification information and performing the processing. 28. The multimedia information conversion device according to claim 27, comprising: 29. The encoding means according to the area classification information,
Weighting means for weighting the image quality of the coding for each area; and a value of the weighting information for a target bit amount to be allocated at the time of coding by the weighting information determined by the weighting means. 28. The multimedia information conversion apparatus according to claim 27, further comprising: a region-specific encoding unit that changes processing. 30. A weighting unit divides a terminal image data area into weighted blocks each of which is a rectangular area of N pixels vertically and M pixels horizontally (N and M are arbitrary integers of 1 or more). For each weighting block, a value of the area classification information for an area of the terminal image data targeted by the weighting block is obtained, and weighting is performed based on the obtained value. Item 30. The multimedia information conversion device according to item 29. 31. A rectangular area of N vertical pixels and M horizontal pixels in a terminal image data area defines a resolution of the terminal image data.
31. The multimedia information conversion apparatus according to claim 30, wherein when converted to the resolution of the encoding image, the area is a block-based resolution area in the encoding process. 32. The multimedia information conversion apparatus according to claim 28, wherein the predetermined feature is divided into at least an area having a character and an area other than the area. 33. The multimedia information according to claim 28, wherein one of the predetermined features is set around a position designated by a pointing device in the terminal image data. Conversion device 34. A weighting means, comprising: a position designated by a pointing device in the terminal image data;
31. The multimedia information conversion apparatus according to claim 30, wherein an area having the same characteristic as a peripheral area is weighted so as to have high image quality or low image quality. 35. A method according to claim 35, wherein the weighting means changes a weighting value for each of the predetermined features in accordance with a ratio of areas of the plurality of regions having the predetermined features in the terminal image data. 31. The multimedia information conversion apparatus according to claim 30, wherein a weighting value is further changed in addition to the above. 36. A weighting means for further changing a weighting value for a specific region among a plurality of regions having predetermined characteristics in the terminal image data. 31. The multimedia information conversion device according to claim 30, wherein: 37. A weighting means for further changing a weighting value for a region including a pixel of a predetermined color in a region having character characteristics in terminal image data. The multimedia information conversion device according to claim 32, wherein: 38. The method according to claim 38, wherein the weighting means includes, in the terminal image data, an area having character features, and in the same area of the terminal image data acquired in the past, the area classifying means continuously connecting to the area having character features. 33. The multimedia information conversion apparatus according to claim 32, wherein the weighting value is further changed in accordance with the number of times determined by said determining. 39. A weighting means for changing a weighting value in addition to a change in a weighting value for each predetermined feature according to a bit amount of a compressed video signal to be generated. 33. The multimedia information conversion device according to claim 32.