JP2000222164A - Multi-image display system and multi-image display method - Google Patents

Abstract

(57) [Problem] To provide a multi-image display system capable of simultaneously displaying a plurality of image data while responding to a change in a display state due to an external action. In a multi-image display system, a sum of input data transfer rates of image data input by a plurality of input units is detected, and the sum of the detected input data transfer rates is used as a data transfer rate of a frame memory. The display priority is determined and assigned to each of the image data to be displayed on the display device 700 according to the input external action, and the sum of the detected input data transfer rates is determined by the data transfer of the frame memory 600. If the rate is higher than the rate, the input data transfer rate in the input units 301 to 304 is adjusted based on the display priority.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to a multi-image display system and a multi-image display method capable of displaying image data respectively input from a plurality of image sources on one display device.

[0002]

2. Description of the Related Art In recent years, presentations have been increasingly performed by displaying image data supplied from a notebook personal computer or the like on a large display device such as a liquid crystal projector or a plasma display. On the other hand, in a conference, a plurality of attendees often save data on a computer, browse the data, and exchange files on the spot. Further, in the future, a video conference system will be used to display the video images of members participating in remote locations on a part of the display area using PinP (Picture in Picture).
Picture) or the simultaneous display of real-time content distributed by digital television.

[0003] From such a background, there is a demand for an image display system for displaying a plurality of image data inputted asynchronously at a time. As this image display system, the inputted image data is once stored in a frame memory. In some cases, a method of transferring image data to a display unit in synchronization with all input image data is used.

[0004]

However, in the above-described conventional image display system, in order to display a plurality of asynchronously input image data at a time, the input image data is once stored / stored in a frame memory. However, since the image data is transferred to the display means in synchronization with all the input image data, the input image data may have a large resolution, a large data capacity, or a high transfer speed. There are cases where the transfer speed when storing data in the frame memory exceeds the data transfer rate of the frame memory. It is clear that such a situation occurs more remarkably as the number of input sources increases. Also,
Asynchronous input in video conferencing systems where all state changes frequently occur, such as sudden addition of input, change in the display position and size of the multi-display window, sudden transition of the input source computer to power save mode, etc. When a plurality of pieces of image data to be displayed are displayed at one time, it is highly likely that the above-described situation occurs very frequently. Therefore, when the above-described conventional image display system is used for a system such as a video conference system in which such a situation frequently occurs, display data is discarded or lost, and the system hangs up. There is a problem such as being deprived.

It is an object of the present invention to provide a multi-image display system and a multi-image display method capable of simultaneously displaying a plurality of image data while responding to a change in the display state due to an external action.

[0006]

According to the first aspect of the present invention,
A plurality of image data input means for respectively inputting image data from a plurality of image sources, and a plurality of image data respectively input from the plurality of image data input means, which can be simultaneously displayed on one display device; In the image display system, external action input means for inputting an external action, a frame memory for storing the image data respectively input by the plurality of image data input means on a frame basis, and the image data from the frame memory. Transferring means for reading and transferring the read image data to the display device; detecting means for detecting a sum of input data transfer rates of image data input by the plurality of image data input means; Compare the sum of the rates with the data transfer rate of the frame memory Comparison means, priority assigning means for determining and assigning a display priority to each of the image data to be displayed on the display device according to the input external action, and the assigned display priority And control means for controlling display of each of the input image data on the display device according to a result of comparison between the data transfer rate of the frame memory and the sum of the detected input data transfer rates. And

According to a second aspect of the present invention, in the multi-image display system according to the first aspect, when the total sum of the detected input data transfer rates is larger than the data transfer rate of the frame memory, the control means may control the input image data. The ratio of the input data transfer rate of the corresponding image data input means is adjusted based on the display priority of each of the image data, and the input data transfer rate of each of the image data input means is controlled in accordance with the adjusted ratio. It is characterized by the following.

According to a third aspect of the present invention, in the multi-image display system according to the second aspect, when the total sum of the detected input data transfer rates is larger than the data transfer rate of the frame memory, the control means performs the detection. The ratio of the input data transfer rate in each of the image data input means is adjusted according to the order of the respective display priorities so that the total of the input data transfer rates obtained is smaller than the data transfer rate of the frame memory. I do.

According to a fourth aspect of the present invention, in the multi-image display system according to the third aspect, the control means reduces the ratio as the display priority decreases.

[0010] The invention according to claim 5 provides the invention according to claims 1 to 4.
In the multi-image display system according to any one of the above, the transfer unit may read the plurality of image data from the frame memory at a predetermined read data transfer rate, and the comparing unit may output the detected input data. The sum of the data transfer rate and the sum of the read data transfer rate of the transfer means is added, and the sum is compared with the data transfer rate of the frame memory.

[0011] The invention according to claim 6 is the invention according to claims 1 to 5.
In the multi-image display system according to any one of the above, when displaying the plurality of image data on the display device while overlapping the plurality of image data, the priority assigning means sets a display priority for the plurality of image data. The image data displayed on the foreground is assigned the highest order, and is assigned so as to become lower in order toward the image data displayed on the back.

The invention according to claim 7 provides the invention according to claims 1 to 6
In the multi-image display system according to any one of the above, the external action input means, as an external action, an operation instruction regarding image display of the display device issued by a user, the control means, A display processing function for performing processing relating to display of the image data in accordance with an operation instruction relating to the image display.

According to an eighth aspect of the present invention, in the multi-image display system according to the seventh aspect, the operation instruction relating to the image display is performed by selecting one image data from a plurality of image data displayed on the display device. It is an operation instruction for selection.

According to a ninth aspect of the present invention, in the multi-image display system according to the seventh aspect, the operation instruction relating to the image display is performed by changing one image data among a plurality of image data displayed on the display device. An operation instruction for moving the image data, wherein the control unit moves the one image data to a corresponding position based on the operation instruction.

According to a tenth aspect of the present invention, in the multi-image display system according to the seventh aspect, the operation instruction relating to the image display is issued to one of a plurality of image data displayed on the display device. An operation instruction for converting the resolution, wherein the control means converts the resolution of the one image data into a corresponding resolution based on the operation instruction.

According to an eleventh aspect of the present invention, in the multi-image display system according to the seventh aspect, when an operation instruction related to the image display is input, the priority assigning means is configured to output the image data specified by the operation instruction. Is assigned the highest display priority.

According to a twelfth aspect of the present invention, in the multi-image display system according to any one of the first to sixth aspects, the external action input means is adapted to interrupt supply of image data from the plurality of image sources. Migration is detected,
This detection result is input as the external action, and the priority assigning means, when the detection result is input by the external action input means, changes the image data from the image source from which the supply of the image data is interrupted. It is characterized by assigning the lowest display priority.

According to a thirteenth aspect of the present invention, in the multi-image display system according to any one of the first to sixth aspects, the priority assigning means comprises a plurality of image data displayed on the display device. Are overlapped with each other, it is determined whether or not the total number of pixels of the display area of the back side image data is smaller than a predetermined number of pixels, and the display priority of the back side image data is determined according to the determination result. It is characterized by changing.

According to a fourteenth aspect of the present invention, in the multi-image display system according to any one of the first to sixth aspects, the image data motion amount detecting means for detecting each motion amount in the plurality of image data is provided. And when the plurality of image data displayed on the display device do not overlap each other, the priority assigning means assigns the display priority according to the amount of movement of each of the detected image data. It is characterized by the following.

According to a fifteenth aspect of the present invention, there is provided a multi-image display method for simultaneously inputting image data from a plurality of image sources and simultaneously displaying each of the input image data on one display device. An external action inputting step of inputting an action, a storing step of storing each of the image data input from the plurality of image sources in a frame memory on a frame basis, and a detecting step of detecting a sum of input data transfer rates of the image data Reading the image data from the frame memory, transferring the read image data to the display device, and comparing the sum of the detected input data transfer rates with the data transfer rate of the frame memory. Process and the display device according to the input external action. An allocation step of determining and assigning a display priority to each of the image data to be displayed, and a comparison result of the assigned display priority, the data transfer rate of the frame memory, and the sum of the detected input data transfer rates Controlling the display of each of the input image data on the display device in accordance with

According to a sixteenth aspect of the present invention, in the multi-image display method according to the fifteenth aspect, the control step includes the step of: when the sum of the detected input data transfer rates is larger than the data transfer rate of the frame memory. The ratio of the input data transfer rate of the corresponding image data input means is adjusted based on the display priority of each of the image data, and the input data transfer rate of each of the image data input means is controlled in accordance with the adjusted ratio. It is characterized by the following.

According to a seventeenth aspect of the present invention, in the multi-image display method according to the sixteenth aspect, the control step includes the step of: when the sum of the detected input data transfer rates is larger than the data transfer rate of the frame memory. The ratio of the input data transfer rate in each of the image data input means is adjusted according to the order of the respective display priorities so that the total of the input data transfer rates obtained is smaller than the data transfer rate of the frame memory. I do.

According to an eighteenth aspect of the present invention, in the multi-image display method according to the seventeenth aspect, the control step reduces the ratio as the display priority decreases.

According to a nineteenth aspect of the present invention, in the multi-image display method according to any one of the fifteenth to eighteenth aspects, the transferring step includes reading the plurality of image data from the frame memory by a predetermined number. Performed at a read data transfer rate, and the comparing step adds the sum of the detected input data transfer rates and the sum of the read data transfer rates, and compares the added value with the data transfer rate of the frame memory. It is characterized by the following.

According to a twentieth aspect of the present invention, in the multi-image display method according to any one of the fifteenth to nineteenth aspects, the allocating step includes superimposing the plurality of pieces of image data on the display device. When displaying,
The display priority is assigned to the plurality of image data so that the image data displayed on the foreground is the highest and the image data displayed on the back is sequentially lower in order.

According to a twenty-first aspect of the present invention, in the multi-image display method according to any one of the fifteenth to twentieth aspects, a display process for performing a process relating to display of the image data in response to an operation instruction relating to the image display. Process
In the external action input step, an operation instruction issued by a user regarding image display of the display device is input as the external action.

According to a twenty-second aspect of the present invention, in the multi-image display method according to the twenty-first aspect, the operation instruction relating to the image display is performed by selecting one image data from a plurality of image data displayed on the display device. It is an operation instruction for selection.

According to a twenty-third aspect of the present invention, in the multi-image display method according to the twenty-first aspect, the operation instruction relating to the image display is performed by changing one image data among a plurality of image data displayed on the display device. An operation instruction for moving, wherein the one image data is moved to a corresponding position based on the operation instruction in the display processing step.

According to a twenty-fourth aspect of the present invention, in the multi-image display method according to the twenty-first aspect, the operation instruction relating to the image display is transmitted to one of a plurality of image data displayed on the display device. An operation instruction for converting a resolution, wherein in the display processing step, the resolution of the one image data is converted to a corresponding resolution based on the operation instruction.

According to a twenty-fifth aspect of the present invention, in the multi-image display method according to the twenty-first aspect, in the assigning step,
When an operation instruction regarding the image display is input,
The highest display priority is assigned to the image data specified by the operation instruction.

According to a twenty-sixth aspect of the present invention, in the multi-image display method according to any one of the fifteenth to twentieth aspects, in the external action input step, the supply of the image data from the plurality of image sources is interrupted. Migration is detected,
This detection result is input as the external action, and in the assigning step, when the detection result is input by the external action input means, the lowest order is added to the image data from the image source from which the supply of the image data is interrupted. Is assigned.

According to a twenty-seventh aspect of the present invention, in the multi-image display method according to any one of the fifteenth to twentieth aspects, in the assigning step, the plurality of pieces of image data displayed on the display device are mutually exchanged. When overlapping, it is determined whether or not the total number of pixels in the display area of the back side image data is smaller than a predetermined number of pixels, and the display priority of the back side image data is changed according to the determination result. It is characterized by the following.

According to a twenty-eighth aspect of the present invention, in the multi-image display method according to any one of the fifteenth to twentieth aspects, an image data motion amount detecting step of detecting a motion amount of each of the plurality of image data is provided. Wherein, in the assigning step, when the plurality of image data displayed on the display device do not overlap each other, assigning the display priority according to the amount of motion of each of the detected image data. It is characterized by.

[0034]

Embodiments of the present invention will be described below with reference to the drawings.

(First Embodiment) FIG. 1 is a block diagram showing a configuration of a first embodiment of a multi-image display system according to the present invention, and FIG. 2 is a block diagram showing a configuration of an input unit in FIG.

As shown in FIG. 1, the multi-image display system includes image sources 101 to 101 composed of a host computer.
Input units 301 to 304 for inputting image signals from
, And simultaneously displays the images indicated by the image signals respectively input to the input units 301 to 304 on the display device 700. Here, image signals from the image sources 101 to 104 are input to the input units 301 to 30 via signal lines 201 to 204, respectively.
4, the image signal includes R, G, and B analog video signals as image data, a Vsync signal as a vertical synchronization signal, and an Hsync signal as a horizontal synchronization signal. The image sources 101 to 104 are a personal computer,
Not only computers such as workstations, but also digital television, video, DV (digital video), MPE
It may be composed of a data stream such as G2. In this embodiment, four image sources 101 to 101 are used.
Although the configuration in which the image sources 104 can be connected is adopted, the number of these image sources may be more than four or less than four.

Each of the input units 301 to 304 is connected to the image source 1
A / D converter for converting an image signal inputted from signal lines 01 to 104 through signal lines 201 to 204 into a digital signal, and a PLL (Phas) for sampling image data.
e Locked Loop). If the input image signal is a digital signal such as an LVDS (Low Voltage Differential Signal), an input unit having a decoder and a differential buffer is used. If the input image signal is a television or video composite signal, an input unit having an encoder for encoding the composite signal into R, G, and B signals is used.

Each of the input sections 301 to 304 simultaneously receives image data from each of the image sources 101 to 104 and a control signal for receiving the image data. Here, as a control signal, a horizontal synchronization signal for synchronizing one line, a vertical synchronization signal for synchronizing one frame or one field, a clock signal for sampling a pixel, a display enable signal indicating a transfer period of valid image data, and the like. Receive. Each of the input units 301 to 304 receives image data from the image sources 101 to 104 at independent timing.

As shown in FIG. 2, the input units 301 to 304 include an input thinning unit 351 for thinning out the data rate of the input image in the time axis direction, and a display format (the number of horizontal pixels, the number of vertical lines) of the received image data. (The number and the number of colors) according to the control of the control unit 500, which will be described later, and a bus interface unit 353 for inputting four independent image data to one common graphic bus 400. The input thinning unit 351 performs thinning control for each input frame in accordance with the control of the control unit 500 described later. In the present embodiment, for the sake of simplicity, the input thinning unit 351 is used.
Is set to perform thinning-out in units of frames, but instead, thinning-out means in units of bits or blocks may be used. The format conversion unit 352 performs resolution conversion for arbitrarily enlarging / reducing the input image, and the bus interface unit 353 performs an address generation control that determines where the input image data is stored in the frame memory 600 described later. And a function for positioning the display window when moving / changing the size.

The control section 500 is a control section for controlling the entire multi-image display system,
U (not shown), a RAM (not shown) for temporarily storing data, a ROM (not shown) for storing a control program,
It includes a timer (not shown) for measuring time, a peripheral input / output interface (not shown), and the like. Here, the control unit 500 can be configured only with logic logic. Further, a program for determining the operation of the CPU may be incorporated in the ROM, or may be configured to be externally transferred via the peripheral input / output interface.

The control unit 500 controls each of the image sources 101 to 1
The pixel clock signal, the Hsync signal, and the Vsync signal of the output image signal are input from signal lines 205 to 208 from signal lines 204 to 204 via signal lines 201 to 204. If the signal output from the image sources 101 to 104 is a composite signal such as video, the sync signal separator uses Vsync.
The signal and the Hsync signal are separated and connected to the control unit 500.

The control unit 500 uses an internal timer to
The period of each of the input Vsync signal and Hsync signal is configured to be measurable. From these measured periods,
It detects the frequency of one pixel (pixel frequency), the screen size (the number of horizontal pixels, the number of vertical lines), the frame rate (the frame frequency), etc. of the signals on the signal lines 201 to 204. As described above, in the present embodiment, the method of detecting the input data transfer rate by using the signal lines 201 to 204 is adopted. This is because the input data transfer rate is the same up to the bus interface unit 353. Is assumed. If the display format converter 352 performs resolution conversion to change the data rate, the input data transfer rate may be detected after the data rate conversion, that is, by using the output signal of the display format converter 352. I just need.

Further, the control section 500 also serves as means for detecting the total sum of the input data transfer rates.
The transfer rates of the signals input to the input units 301 to 304 are detected from the pixel frequencies, and the transfer rates of the output data at the input units 301 to 304 (substitute for the input data transfer rate) are summed. Thus, the sum of the input data transfer rates is detected. The control unit 500 is also configured to perform various other controls of the system, and details thereof will be described later.

The frame memory 600 stores the display device 7
00 is a memory capable of storing at least one frame of image data to be drawn, and readablely stores image data output from the input units 301 to 304. The image data read from the frame memory 600 is output to the display device 700 through the graphic bus 400 and the display drive controller 900. This display device 70
0 is a flat panel having a matrix electrode structure (liquid crystal, plasma display, etc.), CRT (cathode ray tube)
Etc.

The image data output from the input units 301 to 304 are written to and read from the frame memory 600 by the input units 301 to 304, the display drive controller 900 (transfer unit), and the bus controller 1000 (priority determining unit). It is controlled by inter-bus arbitration control. In this bus arbitration control, a transfer request signal is sent from the input units 301 to 304 and the display drive controller 900 (transfer unit) to the signal lines 1001 to 1005 asynchronously at regular intervals as a transfer request of the graphic bus 400 (transfer unit). To the bus controller 1000 (priority determining means), the bus controller 1000 determines the priority of each request, and transmits a transfer permission signal indicating acquisition of the bus right to the signal line 10.
This is performed by returning to the request source via 01 to 1005. The priority related to the bus arbitration control is determined by scalability management that adjusts the data rate of the system based on the input data transfer rate detected by the control unit 500, and is notified to the bus controller 1000 via the signal line 1006.

A display pointer 702 is used for an operation instruction regarding image display of the display device 700. The display pointer 702 constitutes a user interface prepared for facilitating operation of a display window by a user using the multi-image display system, and displays a display cursor 701 on the display device 700 so as to be movable. It has a function, and the user instructs the action of selecting, moving, and enlarging / reducing the display window using buttons provided on the display cursor 701 or the display pointer 702. This action information is transmitted from an infrared port (not shown) provided in the display pointer 702, and received by the infrared light receiving unit 800. The infrared receiving unit 800 transmits the received action information to the control unit 500.
To the display drive controller 9 so as to perform display processing corresponding to the received action information.
Indicate at 00.

Next, a detailed operation of the present multi-image display system will be described.

First, here, the detected input units 301 to 301
Each of the input data transfer rates 304 (data transfer capability per unit time: also referred to as bandwidth) is represented by BWi1, BWi2,
BWi3 and BWi4, and the total is BWi.

BWi = BWi1 + BWi2 + BWi3 + BWi4 Further, the read data transfer rate of display data when the display drive controller 900 reads from the frame memory 600 is BWd, and the data transfer rate of the frame memory 600 is BWm. BWd and BWm are fixed values determined at the time of designing the system. The actual BWm is determined not only by the data transfer rate of the frame memory 600 but also by the effective transfer rate of the graphic bus 400.
BWi is a variable value dynamically determined by the control unit 500 monitoring the connection with the image sources 101 to 104 and determining the input data transfer rate when an input is connected.

This multi-image display system is used to transfer data from the input units 301 to 304 to the frame memory 600 and data from the frame memory 600 to the display drive controller 900 without losing all input data. The condition is BWm> BWi + BWd (1). If BWm <BWi + BWd (2), scalability management for adjusting the data rate of the system needs to be performed.

The operation including the scalability management will be described.

<Operation at Initialization> FIG. 3 is a state transition diagram of the multi-image display system of FIG. 1, and the transition to each state is controlled by the control unit 500.

First, when the power-on action A100 occurs, the process immediately enters the initialization process ST000.

Next, the initialization process will be described with reference to FIGS. FIG. 4 is a flowchart showing the initialization processing in the multi-image display system of FIG. 1, and FIG. 5 is a diagram showing a display example at the time of initialization in the multi-image display system of FIG. Here, an example in which connection of four inputs is detected is shown. The same applies to the following description of the flowchart.

First, in step S101, the connection states of all input ports are detected. In the following step S102, the state (screen size, input data rate) of all four input connections is determined using the above-described means.
Then, the process proceeds to step S103, and the process proceeds to step S10.
2. Based on the screen size determined in step 2, all inputs can be displayed on the screen of the display device 700 at an equal size.
A display layout for determining a display position and an enlargement / reduction ratio is performed (creation of display position information of four inputs and parameters of resolution conversion information).

Next, the process proceeds to step S104, where priorities (display priorities) are assigned in the order in which the connections are detected, and a history thereof is created. This priority is stored in the RAM of the control unit 500. Since this process is performed at the time of initialization, the priority is arbitrarily determined by the system. In the following description, it is assumed that the priority history at the time of initialization is set as input 1> input 2> input 3> input 4... (3) (creation of a priority parameter). This priority is based on the bus controller 1000 described above.
Is also reflected in the adjustment priority.

Subsequently, the process proceeds to step S105, and scalability is managed based on the input data rate determined in step S102. Here, when the above equation (1) is satisfied, the thinning rate for each input is set to a constant default value (1.0) regardless of its priority. That is, no thinning is performed. On the other hand, when the above equation (2) is satisfied, the thinning rate for each input is determined according to the priority of the above equation (3). In the present embodiment, the input thinning section 351 performs frame thinning for taking in data in frame units by using the Vsync signal input via the signal line 201 as a trigger, and uses a means for reducing the input data rate on a time average basis.

In this method, the purpose is to adjust the scalability such that the above equation (1) is satisfied on a time average, and the above equation (2) may be satisfied momentarily. In this case, it is possible to cope with this by providing a buffer for data standby in the bus interface unit 353. Specifically, scalability management by frame thinning is performed by changing the thinning rate of each input, that is, the frame rate, in a time-average manner (meaning that frames are thinned by ON / OFF in units of several frames in the time direction). The latest input data transfer rate of each input (data transfer capability per unit time: also referred to as bandwidth) BWi1 ', BWi2', BWi3 ', BWi4' according to the expression 4)
It starts by recalculating.

BWi1 ′ = BWi1 × (decimation rate of input 1) Decimation rate of input 1 = frame rate after decimation of input 1 (Hz) / frame rate before decimation of input 1 (Hz) BWi2 ′ = BWi2 × ( Input 2 thinning rate) Input 2 thinning rate = frame rate after input 2 thinning (Hz) / frame rate before input 2 thinning (Hz) BWi3 ′ = BWi3 × (input 3 thinning rate) Decimation rate = frame rate after decimation of input 3 (Hz) / frame rate before decimation of input 3 (Hz) BWi4 ′ = BWi4 × (decimation rate of input 4) Decimation rate of input 4 = after decimation of input 4 Frame rate (Hz) / Frame rate before decimation of input 4 (Hz) (4) Next, according to the priority history shown in the above equation (3), BWi ′ = BWi1 ′ + BWi2 ′ + BWi3 ′ + BWi4 ′ BWm> BWi '+ Wd ...... (5) to become such condition control unit 500 is calculated. Of course, if BWm <BWi '+ BWd (6), the scalability adjustment is repeated until the above equation (5) is satisfied.

As a supplement to this description, a simple example is shown below.

For simplicity, assume that the pixel frequency and data volume of each input are the same:
The input data transfer rate is proportional to the frame rate. Under this assumption, the display data transfer rate BWd @ K * 60 Hz The data transfer rate BWi1 @ K * 60 Hz of input 1 The data transfer rate BWi2 @ K * 50 Hz of input 2 The data transfer rate BWi3 @ K * 70 Hz of input 3 input 4 If the data transfer rate of the frame memory is BWm ≒ K * 200 Hz, then BWm <BWi + BWd = K * 300 Hz, and the above equation (2) holds.

However, if the thinning rate is given in the order of 0.8, 0.6, 0.5, 0.2 according to the priority of the above equation (3), BWi1 '= BWi1 × (thinning of input 1) Rate) = K * 60 Hz * 0.8 = K * 48 Hz BWi2 '= BWi2 × (thinning rate of input 2) = K * 60 Hz * 0.6 = K * 36 Hz BWi3 ′ = BWi3 × (thinning rate of input 3) = K * 70Hz * 0.5 = K * 35Hz BWi4 '= BWi4 × (thinning rate of input 4) = K * 60Hz * 0.2 = K * 12Hz The recalculation result is obtained, BWm>BWi' + BWd = K *
191 Hz, and the relationship of the above equation (5) is established.

In step S106, the control unit 500 uses the signal lines 305 to 308 and 1006 to input the parameters determined in steps S103 and S105 to the input units 301 to 304 and the bus controller 100.
Set to 0 and start displaying.

The display image at this time is as shown in the display example at the time of initialization shown in FIG. For each input, the data rate has been reduced by the above process, but all four inputs can be displayed. Such a method can be a particularly effective means when displaying an image with little motion like a still image (for example, a presentation screen) even in a moving image. After this processing, the control unit 500 returns to the ST of FIG.
It becomes idle state of 100 and waits for the next action.

<Operation at Window Selection> As soon as the selection action A200 is generated by the display pointer 702, the control unit 500 immediately executes the window selection process ST shown in FIG.
Enter 200.

FIGS. 6 and 7 show this window selection process.
This will be described with reference to FIG. FIG. 6 is a flowchart showing a procedure of a window selection process in the multi-image display system of FIG. 1, and FIG. 7 is a diagram showing a display example in the window selection process.

The image displayed on the display device 700 is the button operation of the display pointer 702 or the display cursor 7.
When the selection is made by 01, as shown in FIG. 6, first, the window selected in step S201 is set as the top priority window, the priority is reassigned, and a history in which the previous priority history is updated is created. This priority is stored in the RAM of the control unit 500. Here, for example, if input 3 is selected as in the display example when the window is selected in FIG. 6, the priority history shown in the above equation (3) is input 3> input 1> input 2> input 4 (4) 7) (priority parameter creation). This priority is also reflected on the adjustment priority of the bus controller 1000 described above.

Next, the process proceeds to step S202, where scalability is managed. Here, when the above equation (1) is satisfied, a thinning rate of a default value is set for each input regardless of its priority. On the other hand, when the above equation (2) is satisfied, the thinning rate is determined according to the priority of the above equation (7). The change of the thinning rate is performed by the method described above. Here, if the thinning rate is given in the order of 0.5, 0.6, 0.8, 0.2 in accordance with the priority of the above equation (7), BWi1 ′ = BWi1 × (thinning rate of input 1) = K * 60 Hz * 0.5 = K * 30 Hz BWi2 '= BWi2 × (thinning rate of input 2) = K * 60 Hz * 0.6 = K * 36 Hz BWi3 ′ = BWi3 × (thinning rate of input 3) = K * 70 Hz * 0.8 = K * 56 Hz BWi4 '= BWi4 * (thinning rate of input 4) = K * 60 Hz * 0.2 = K * 12 Hz The recalculation result is obtained, and BWm>BWi' + BWd = K *.
194 Hz, which is the above equation (5).

In the subsequent step S203, the control unit 500 uses the signal lines 305 to 308 and 1006 to input the parameters determined in steps S201 and S202 to the input units 301 to 304 and the bus controller 100.
Set to 0 and start displaying. The display image at this time is a display example as shown in FIG. This state is input 3
Indicates a state of being displayed with the highest priority (a state where the thinning rate is the highest among the four inputs). The control unit 500 transfers frame data to the display drive controller 900 using the signal line 901 in order to form a frame in the selection window, and the display drive controller 900 receives the data and performs superimposed display with normal data. After this processing, the control unit 500 returns to ST1 of FIG.
It becomes idle state of 00 and waits for the next action.

<Operation When Moving Window> When a movement action A300 is generated by operating the display pointer 702, the control section 500 immediately enters the window movement processing ST300 in FIG.

FIGS. 8 and 9 show this window moving process.
This will be described with reference to FIG. FIG. 8 is a flowchart showing the procedure of the window moving display process in the multi-image display system of FIG. 1, and FIG. 9 is a diagram showing a display example at the time of the window moving display process in the multi-image display system of FIG. Here, this description is based on the assumption that there are several window movement actions, and the display state is shown in FIG.
Is shown in

When a movement action is input by operating the button of the display pointer 702, as shown in FIG. 8, first, in step S301, the display position of each input display window changed by the movement action is changed (4).
In step S302, the priority is re-assigned so that the priority is lowered from the foreground to the back when the overlapping state is created by several window movement actions. Create a history with updated priority history. This priority is assigned to the control unit 5
00 is stored in the RAM.

The priority history shown in the above equation (7) is changed as follows: input 4> input 3> input 2> input 1 (8) (creation of priority parameter). This priority is also reflected on the adjustment priority of the bus controller 1000 described above.

Next, the process proceeds to step S303, where scalability is managed. Here, when the above equation (5) is satisfied, a default value is set for each input regardless of its priority. On the other hand, when the above equation (6) is satisfied, the thinning rate is determined by referring to the priority of the above equation (8). The thinning rate is changed according to the method described above. Here, for example, according to the priority shown in the above equation (8),
If the thinning rate is given in the order of 0.3, 0.5, 0.6, 0.8, BWi1 '= BWi1 × (thinning rate of input 1) = K * 60 Hz * 0.3 = K * 18 Hz BWi2' = BWi2 × (thinning rate of input 2) = K * 60 Hz * 0.5 = K * 30 Hz BWi3 ′ = BWi3 × (thinning rate of input 3) = K * 70 Hz * 0.6 = K * 42 Hz BWi4 ′ = BWi4 × (thinning rate of input 4) = K * 60 Hz * 0.8 = K * 48 Hz The recalculation result is obtained, and BWm> BWi ′ + BWd = K *
198 Hz, and the relationship of the above equation (5) is established.

In step S304, the above-described step S3
01, the parameters determined in step S303
00 sets the input units 301 to 304 and the bus controller 1000 using the signal lines 305 to 308 and 1006, and starts displaying. After this processing, the control unit 500 returns to FIG.
It becomes an idle state in ST100 and waits for the next action.

<Operation at Resolution Conversion> Display Pointer 70
When the resolution conversion action A400 occurs by the operation 2, the control unit 500 immediately enters the window resolution conversion processing ST400 in FIG.

The window resolution conversion processing will be described with reference to FIGS. 10 and 11. FIG. 10 is a flowchart showing the procedure of the window resolution conversion process in the multi-image display system of FIG. 1, and FIG. 11 is a diagram showing a display example at the time of the window resolution conversion process in the multi-image display system of FIG. Here, this description is based on the assumption that there is a window enlargement action.

When a resolution conversion action occurs by operating the display pointer 702, as shown in FIG. 10, first, in step S401, the display position and size of the display window changed by the resolution conversion (enlargement) action are changed (enlarged). In step S402, when an overlapping state is created by the window resolution conversion (enlargement) action, the selected enlarged window is given the highest priority, and the other windows are the rearmost side. The priority is re-allocated so that the priority becomes lower as going toward the destination, and a history in which the previous priority history is updated is created. Here, if an input window hidden by the enlargement action appears (see FIG. 11, corresponding to inputs 1 and 2),
Since the window display itself has no meaning, it is added that the window other than the window having the display area has the lowest priority. Then, this priority is stored in the RAM of the control unit 500.

The priority history (c) is changed as follows: input 4> input 3 (> input 2 = input 1) (9) (creation of priority parameters). This priority is also reflected on the adjustment priority of the bus controller 1000 described above.

Next, the process proceeds to step S403, where scalability is managed. Here, when the above equation (1) is satisfied, a default thinning rate is set for each input regardless of its priority. When the above equation (2) is satisfied, the thinning rate is determined with reference to the priority shown in the above equation (9). The change of the thinning rate is performed by the method described above. Here, for example, according to the priority shown in the above equation (9) and the information added thereto, that is, the information of the window hidden by the enlarged window, the thinning rate is set to 0.0, 0.
If given in the order of 0, 1.0, 1.0, BWi1 '= BWi1 × (thinning rate of input 1) = K * 60 Hz * 0.0 = 0 Hz BWi2 ′ = BWi2 × (thinning rate of input 2) = K * 60Hz * 0.0 = 0Hz BWi3 '= BWi3 × (thinning rate of input 3) = K * 70Hz * 1.0 = K * 70Hz BWi4 ′ = BWi4 × (thinning rate of input 4) = K * 60Hz * 1 2.0 = K * 60 Hz, the recalculation result was obtained, and BWm> BWi '+ BWd = K *
130 Hz, and the above equation (5) holds. In this case, all the inputs 3 and 4 can be processed at the actual rate without thinning.

Then, the process proceeds to step S404, where the control unit 500 uses the signal lines 305 to 308 and 1006 to input the parameters determined in steps S401 and S403.
Set to 00 and start displaying. Also, so that the user can check the connection of inputs 1 and 2 even when it is not displayed,
The control unit 500 transmits the signal line 9 to the display drive controller 900.
01 and the icon data (the icon 1 shown in FIG. 11).
1, 12), and the display drive controller 900 receives this and superimposes the display on the normal data. After this processing, control section 500 enters the idle state of ST100 and waits for the next action.

<Operation at Power Save> Finally, the operation at the time of power save will be described. In the present multi-image input / display system, it is assumed that each of the image sources 101 to 104 enters the power save mode. In the power save mode, when the image sources 101 to 104 are computers, the control unit 500 controls the signal lines 205 to 2.
08 Hsync signal and Vsync signal state (VESA DP
(MS specifications, etc.) can be easily detected. When the image sources 101 to 104 output a composite video signal, the input units 301 to 104
304 is the entire surface BL of the image data of the signal lines 201 to 204
The power save mode can be easily detected by monitoring the state that the UE continues for a certain period of time. Such a transition to the power save mode is an action that occurs suddenly irrespective of the above-described action from the display pointer 702 by the user.

The control section 500 sets the power save action A
As soon as 500 is recognized, the power saving process ST
Enter 500.

The power saving process will be described with reference to FIGS. FIG. 12 is a flowchart showing the procedure of the power save process in the multi-image display system of FIG. 1, and FIG. 13 is a diagram showing a display example at the time of the power save process in the multi-image display system of FIG. In this description, it is assumed that the input 4 has entered the power save mode.

When at least one of the image sources shifts to the power save mode, as shown in FIG. 12, first, in step S501, the display which has entered the power save mode is invalidated as an input having no display area. . Here, it is assumed that the input 4 has entered the power save mode, and the input 4 is iconified (the icon 1 shown in FIG. 13).
4) In the other windows (inputs 1, 2, 3), the positions and sizes are the same, and are the same as in the previous action.
No resizing is required.

Then, the flow advances to step S502 to re-assign the priorities so that the priorities become lower from the frontmost to the rearmost, and update the previous priority history, except for the windows whose input has been invalidated by the power save action. Create a history. This priority is stored in the RAM of the control unit 500. In this example, it is added that the input 4 is iconified by the power save, and the inputs 1 and 2 that have been hidden and iconified by the input 4 are displayed again.

The priority history (9) is changed as follows: input 2> input 1> input 3 (> input 4) (10) (creation of a priority parameter). This priority is also reflected on the adjustment priority of the bus controller 1000 described above.

Next, the flow advances to step S503 to manage scalability. Here, when the above equation (1) is satisfied, a default thinning rate is set for each input regardless of its priority. On the other hand, when the above equation (2) is satisfied, the thinning rate is determined with reference to the priority shown in the above equation (10). The change of the thinning rate is performed by the method described above. Here, for example, according to the priority of the above equation (10), the thinning rate is set to 0.9, 1.0, 0.8,
If given in the order of 0.0, BWi1 '= BWi1 × (thinning rate of input 1) = K * 60 Hz * 0.9 = K * 54 Hz BWi2 ′ = BWi2 × (thinning rate of input 2) = K * 60 Hz * 1 0.0 = K * 60 Hz BWi3 '= BWi3 × (thinning rate of input 3) = K * 70 Hz * 0.8 = K * 56 Hz BWi4 ′ = BWi4 × (thinning rate of input 4) = K * 60 Hz * 0.0 = 0 Hz, the recalculation result is obtained, BWm> BWi '+ BWd = K *
170 Hz, and the above equation (5) holds.

Then, the process proceeds to step S504, where the control unit 500 uses the signal lines 305 to 308 and 1006 to input the parameters determined in step S503.
304 and the bus controller 1000, and display is started. Also, the control unit 500 uses the signal line 901 to connect the input 4 to the display drive controller 900 by using the signal line 901 so that the user can confirm that the input 4 is in the power save mode even when the display is not displayed, as shown in FIG. Upon receiving the data (icon 14), the display drive controller 900 receives the data and superimposes and displays the normal data. After this processing, control unit 500 returns to ST100
Becomes idle and waits for the next action.

As described above, in the present multi-image input / display system, the display state is such that the user repeats a number of actions (only one example is shown in the present embodiment).
Alternatively, it dynamically changes due to an action on the image source side (transition to the power save mode). In response to this change, the priority history for managing the scalability is calculated by the equation (3) → (7) → ( 8) Formula → (9) Formula →
Since the data is automatically updated as shown in Expression (10), the user operates the multi-image display system with the user interface of only the display cursor 701 or the display pointer 702 without considering the scalability management at all. Will be. Accordingly, it is possible to provide the user with an environment in which scalability management is automatically performed in conjunction with an action by the user or an action (power save mode) on the image source side. That is, a multi-image display system with excellent operability can be constructed.

(Second Embodiment) Next, a second embodiment of the present invention will be described with reference to the drawings.

First, a problem in the method of judging the state of superposition and simply determining the priority as in the first embodiment will be described with reference to FIG. FIG.
FIG. 5 is a diagram showing a display example when a problem occurs in the superimposed state according to the first embodiment of the present invention.

When the same scalability management is performed in the display state shown in FIG. 15 as in the first embodiment, the display becomes the same as the display example in FIG. 9 in the first embodiment. The result at this time is: BWi1 '= BWI1 × (thinning rate of input 1) = K * 60 Hz * 0.3 = K * 18 Hz BWi2 ′ = BWi2 × (thinning rate of input 2) = K * 60 Hz * 0.5 = K * 30 Hz BWi3 '= BWi3 × (thinning rate of input 3) = K * 70 Hz * 0.6 = K * 42 Hz BWi4 ′ = BWi4 × (thinning rate of input 4) = K * 60 Hz * 0.8 = K * 48 Hz. However, looking at the display example of FIG.
Is adjusted as a higher priority window than input 1 having a large display area, despite having almost no display area. For the user, updating the input 1 having a larger display area than the inputs 2 and 3 with high priority should be a system that is easier to use.

In order to solve such a problem, in the present embodiment, when the inputs 1 to 4 overlap with each other on the display device 700, the inputs 1 to 3 on the rear side are used.
Is determined whether the total number of pixels in the display area is smaller than a predetermined number of pixels, and the display priority of the inputs 1 to 3 on the rear side is changed according to the determination result.

This method will be described with reference to FIG. FIG. 14 is a flowchart showing a processing procedure in the second embodiment of the multi-image display system of the present invention.

Specifically, as shown in FIG. 14, first, in step S601, the display position of each input display window changed by the movement action is changed (creation of a display position information parameter of four inputs), and then in step S6.
In 02, when an overlapping state is created by several window movement actions, the priorities are re-assigned so that the priority becomes lower from the foreground to the back, and a history in which the previous priority history is updated is created. This priority is determined by the control unit 500
Is stored in the RAM.

Next, in step S603, the area of each area where each input is actually displayed on the display device 700 is calculated as the number of display pixels of each input, and the calculated number of display pixels is compared with the display threshold. . The calculation of the number of display pixels can be easily obtained because the control unit 500 knows in detail the current priority history and the position and size of each display window. Here, the control unit 500
The number of display pixels of each input in the display example of (1) is the number of display pixels of input 1 = DP1 (number of pixels) The number of display pixels of input 2 = DP2 (number of pixels) The number of display pixels of input 3 = DP3 (number of pixels) Input 4 It is assumed that the number of display pixels is calculated as DP4 (number of pixels).

Next, the number of pixels that the user does not consider effective as a display area is determined in advance. This is defined as a display pixel threshold (the number of TH pixels). Subsequently, the calculated number of display pixels is compared with the display pixel threshold, and it is determined whether or not each display input has a display area larger than the threshold. That is, it is determined whether the display area is meaningful for the user. In the display example of FIG. 15, when the comparison result of DP1> TH DP2 <TH DP3 <TH DP4> TH is obtained, the previous priority history is described in (8) above.
As shown in the equation, input 4> input 3> input 2> input 1 is satisfied. However, in the processing according to the present embodiment, when the display pixel is smaller than the threshold (the number of TH pixels) based on the comparison result, Returning to step S602, the priority of the input is reduced. The priority history is changed as follows: input 4> input 1> input 2 = input 3 (creating a priority parameter) (11) This priority is also reflected in the adjustment priority of the bus controller 1000 described above. Is done.

Next, the flow advances to step S604 to manage scalability. Here, when the relationship of the above equation (1) is established, a default thinning rate is set for each input regardless of its priority. On the other hand, when the relationship of the above equation (2) is established, the thinning rate is determined by referring to the priority of the above equation (11). The change of the thinning rate is performed by the following method. For example, according to the priority shown in the above equation (11), the thinning rate is set to 0.9, 0.2, 0.
If given in the order of 2, 1.0, BWi1 ′ = BWi1 × (thinning rate of input 1) = K * 60 Hz * 0.9 = K * 54 Hz BWi2 ′ = BWi2 × (thinning rate of input 2) = K * 60 Hz * 0.2 = K * 12 Hz BWi3 '= BWi3 × (thinning rate of input 3) = K * 70 Hz * 0.2 = K * 14 Hz BWi4 ′ = BWi4 × (thinning rate of input 4) = K * 60 Hz * 1 2.0 = K * 60 Hz, the recalculation result was obtained, and BWm> BWi '+ BWd = K *
140 Hz, and the above equation (5) holds.

Then, the process proceeds to step S605, where the control unit 500 uses the signal lines 305 to 308 and 1006 to input the parameters determined in step S604.
304 and the bus controller 1000, and display is started. After this processing, control section 500 enters the idle state of ST100 and waits for the next action.

By this processing, input 1 is changed from 18 Hz to 5
4 Hz, input 2 goes from 42 Hz to 12 Hz, input 3 goes to 4
The frame rate of input 4 is changed from 2 Hz to 14 Hz, and the input 4 is changed from 48 Hz to 60 Hz. That is, the frame rate is increased for the inputs 1 and 4 meaningful to the user, and the frame rate is reduced for the inputs 2 and 3 that are not.

As described above, scalability management in the present embodiment, which is conscious of the display area meaningful to the user, can not only solve the above-described problem, but also promote a significant improvement in the user interface. This is also a unique effect of the second embodiment.

(Third Embodiment) Next, a third embodiment of the present invention will be described with reference to the drawings.

In each of the above-described embodiments, there is no overlapping state as described above, and the display area is effective for the user (at least two or more display areas have the threshold values shown in the second embodiment). ), The priority cannot be automatically determined.

Therefore, in the present embodiment, a motion detection unit is added to all of the input units 301 to 304 in FIG. 1 so that there is no superimposed state as shown in each of them and a display area effective for the user (at least If all of the two or more display areas exceed the threshold value shown in the second embodiment, the priority is automatically set using the motion amount detected by the motion detection unit. It is constituted so that it may be determined. Here, the motion detection amount of the motion detection unit is used as a guide when determining the priority of the multi-window display, and it is not required to accurately determine which pixel of the display window is moving and how much. Therefore, the motion detection unit used here can be a simple motion detection circuit that calculates the total number of pixels in one frame as a difference amount between frames.

First, the configuration of the motion detecting section will be described with reference to FIG. FIG. 16 is a block diagram showing a configuration of the motion detection unit in the third embodiment of the multi-image display system of the present invention.

Each of the input units 301 to 304 is provided with a motion detecting unit 360 as shown in FIG. The motion detection unit 360 has an adder 361 and a latch 362,
The adder 361 and the latch 362 cooperate with each other to constitute a circuit for determining the total number of pixels in one frame for each pixel clock. The result of this addition is input to a selector 363, which toggles every frame (Vsync signal) to input the data to a frame data sum buffer 364.
And to the frame data sum buffer 365 alternately. The outputs A and B of the frame data buffers 364 and 365 are passed to the absolute value difference unit 366, where the frame difference data | AB | is calculated. The frame difference data is input to the latch 367 and is output for one frame (Vsync
Signal), and is latched by the control unit 5 via the signal line 309.
Delivered to 00.

Next, the processing using the motion detecting section 360 will be described with reference to FIGS. FIG. 17 is a flowchart showing a procedure of processing using motion detection according to the third embodiment in the multi-image display system of the present invention, FIG. 18 is a diagram showing a display example when motion detection is required, and FIG. It is a timing chart at the time of the motion detection processing in the third embodiment in the multi-image display system of the present invention.

In this example, as shown in FIG.
It is assumed that 3 is a window with a small movement represented by presentation data or the like and input 4 is a window with a large movement such as video data at the same time.

Referring to FIG. 19, at input 1, there is an update (UPDATE) of the frame difference data at the N-1 timing of the Vsync signal, and at the subsequent timing, the same frame is displayed. (SA80
0). At input 2, there is an update (UPDATE) of the frame difference data at the timing of N of the Vsync signal,
At the subsequent timing, the same frame is displayed (SA801). At input 3, N of the Vsync signal
Update of frame difference data at the timing of +3 (UPD
ATE), and at the subsequent timing, the same frame is displayed (SA802). At the input 4, the frame difference data is N-1 to N + 4 of the Vsync signal.
Until the data is updated (UP
DATE) (SA803). The frame period (Vsync signal) of each of the inputs 1 to 4 is completely asynchronous, so that the signal lines 30
9 is passed to the control unit 500 completely asynchronously.

In response to this, the control section 500 receives the cycle TS
The frame difference data of the inputs 1 to 4 are compared at the sampling points (SA804, 806).

When the comparison result of the frame difference data is obtained, the control unit 500 starts the processing shown in FIG. First, in step S701, the priority history is updated in descending order of the difference amount, scalability management is performed in step S702, and parameter setting is performed in step S703. Since the details of these processes are the same as those described in the above embodiments, detailed descriptions thereof are omitted here.

The display update timing based on the new parameter is synchronized with the Vsync signal of input 1 as a representative. As a result, the priority history (SA805) is: 2 frame periods (SA807): input 4> input 1> input 3> input 2 3 frame periods (SA808): input 4> input 2> input 1> input 3 1 Frame period (SA809)... Automatically updated like input 4> input 3> input 2> input 1.

That is, since the input 4 has a lot of movement, the scalability management has been performed so that the priority is raised at the time when the window having the little movement of the inputs 1 to 3 always has the highest priority.

As described above, in the present embodiment, a simple motion detection function is provided, the display priority is determined based on the result of the motion detection, and the priority is reflected in the scalability management. Even in a state without matching, automatic scalability management by the system can be realized, and a drastic improvement of the user interface can be promoted. This is also a unique effect of the present embodiment.

[0116]

As described above, according to the multi-image display system of the first aspect, external action input means for inputting an external action, and image data input by the plurality of image data input means, respectively. A frame memory for storing image data in frame units, a transfer unit for reading image data from the frame memory, and transferring the read image data to a display device, and an input data transfer rate of image data input by a plurality of image data input units. Detecting means for detecting the sum, comparing means for comparing the sum of the detected input data transfer rates with the data transfer rate of the frame memory, and image data to be displayed on the display device in response to an input external action. Priority assigning means for determining and assigning display priorities to
Control means for controlling the display of the input image data on each display device in accordance with the result of comparison between the assigned display priority and the data transfer rate of the frame memory and the sum of the detected input data transfer rates. In addition, a plurality of image data can be simultaneously displayed while responding to a change in the display state due to an external action.

According to the multi-image display system of the present invention, when the sum of the detected input data transfer rates is greater than the data transfer rate of the frame memory, the control means sets the display priority for each input image data. Adjust the ratio of the input data transfer rate of the corresponding image data input means based on the degree, and control the input data transfer rate in each of the image data input means according to the adjusted ratio, without user awareness, Scalability management according to the display priority can be performed.

According to the multi-image display system of the present invention, when the plurality of image data are displayed on the display device while overlapping each other, the priority assigning means determines the display priority for the plurality of image data. Since the image data displayed on the foreground is assigned as the highest rank, the image data is allocated so as to become lower in order toward the image data displayed on the back.
When displaying a plurality of image data overlapping each other on a display device, the highest display priority can be given to the foreground image data, that is, the image data of interest.

According to the multi-image display system of the present invention, the external action input means inputs an operation instruction concerning image display of the display device issued by the user as an external action, and the control means outputs the image display. Display processing function for performing processing relating to the display of image data in response to an operation instruction relating to image display, a user can give an operation instruction relating to image display via an external action input unit, and excellent operability can be obtained. .

According to the multi-image display system of the present invention, the operation instruction relating to image display is an operation instruction for selecting one image data from a plurality of image data displayed on the display device. Thus, one image data can be easily selected from a plurality of image data.

According to the multi-image display system of the ninth aspect, the operation instruction relating to the image display is an operation instruction for moving one of a plurality of image data displayed on the display device. Since the control means moves one image data to the corresponding position based on the operation instruction, one image data among the plurality of image data can be easily moved.

According to the multi-image display system of the tenth aspect, the operation instruction relating to the image display is an operation instruction for converting the resolution of one of the plurality of image data displayed on the display device. Since the resolution of one image data is converted into the corresponding resolution by the control unit based on the operation instruction, the resolution for one image data among the plurality of image data displayed on the display device can be easily converted. can do.

According to the multi-image display system of the present invention, when an operation instruction related to image display is input, the priority assigning means assigns the highest display priority to the image data specified by the operation instruction. Since the assignment is performed, the display priority of the image data specified by the operation instruction, that is, the image data of interest can be set high.

According to the twelfth aspect of the present invention, the external action input means detects a shift to interruption of supply of image data from a plurality of image sources, and inputs the detection result as an external action. When the detection result is input by the external action input unit, the priority assigning unit assigns the lowest display priority to the image data from the image source that has stopped supplying the image data. Image data from an interrupted image source can be invalidated.

In the multi-image display system according to the thirteenth aspect, when the plurality of image data displayed on the display device overlap each other, the priority assigning means sets the total of the display area of the image data on the rear side. It is determined whether or not the number of pixels is smaller than a predetermined number of pixels, and the display priority of the image data on the rear side is changed according to the determination result. Therefore, the display priority of image data having no effective area is reduced. be able to.

According to a fourteenth aspect of the present invention, there is provided a multi-image display system comprising image data motion amount detection means for detecting each motion amount in a plurality of image data, and the priority assignment means is displayed on a display device. When a plurality of image data do not overlap each other, a display priority is assigned according to the amount of motion of each detected image data.
The display priority can be appropriately determined even in a state where the respective image data do not overlap, and scalability management according to the display priority can be performed.

According to the multi-image display method of the present invention, an external action inputting step of inputting an external action and storing each of image data input from a plurality of image sources in a frame memory in frame units. A storage step, a detection step of detecting a sum of input data transfer rates of image data, a transfer step of reading the image data from a frame memory, and transferring the read image data to the display device, and a detected input data transfer A comparing step of comparing the sum of the rates with the data transfer rate of the frame memory; and an allocating step of determining and assigning a display priority to each of the image data to be displayed on the display device according to the input external action. Detects the assigned display priority and data transfer rate of the frame memory A control step of controlling the display of the input image data on the respective display devices in accordance with the result of comparison with the sum of the input data transfer rates. Image data can be displayed simultaneously.

According to the multi-image display method of the present invention, when the total sum of the detected input data transfer rates is larger than the data transfer rate of the frame memory, each of the input image data is handled based on its display priority. The input data transfer rate of the image data input means to be adjusted is adjusted, and the input data transfer rate in each of the image data input means is controlled in accordance with the adjusted ratio. Scalability management can be performed accordingly.

According to the twentieth aspect of the present invention, when a plurality of image data are displayed on a display device so as to overlap each other, the display priority for the plurality of image data is displayed in the foreground. Since the image data is assigned such that the image data becomes the highest order and becomes lower in order toward the image data displayed on the rearmost side, when displaying a plurality of image data overlapping with each other on the display device, the image data on the foreground, that is, attention is paid. The highest display priority can be given to image data to be displayed.

[0130] According to the multi-image display method of the present invention, an operation instruction from the user relating to image display of the display device is input as an external action, and display of image data according to the operation instruction relating to image display is performed. Therefore, the user can give an operation instruction regarding image display via the external action input means, and excellent operability can be obtained.

According to the multi-image display method of the present invention, the operation instruction relating to the image display is an operation instruction for selecting one image data from a plurality of image data displayed on the display device. Thus, one image data can be easily selected from a plurality of image data.

According to the multi-image display method of the present invention, the operation instruction relating to image display is an operation instruction for moving one image data among a plurality of image data displayed on the display device. Since one image data is moved to a corresponding position based on an operation instruction, one image data among a plurality of image data can be easily moved.

According to the multi-image display method of the twenty-fourth aspect, the operation instruction for image display is an operation instruction for converting the resolution of one image data among a plurality of image data displayed on the display device. Since the resolution of one image data is converted to the corresponding resolution based on the operation instruction, the resolution for one image data among the plurality of image data displayed on the display device can be easily converted. .

According to the multi-image display method of the present invention, when an operation instruction relating to image display is input, the highest display priority is assigned to the image data specified by the operation instruction. The display priority of the designated image data, that is, the image data of interest can be set high.

According to the multi-image display method of the twenty-sixth aspect, the shift to the interruption of the supply of image data from a plurality of image sources is detected, and the detection result is input as an external action, and the detection result is When input, the lowest display priority is assigned to the image data from the image source from which the supply of the image data has been interrupted, so that the image data from the image source from which the supply of the image data has been interrupted can be invalidated.

According to the multi-image display method of claim 27, when a plurality of image data displayed on the display device overlap each other, the total number of pixels of the display area of the image data on the rear side is a predetermined number of pixels. It is determined whether or not the number is smaller than the number, and the display priority of the image data on the rear side is changed according to the determination result. Therefore, the display priority of image data having no effective area can be lowered.

According to the multi-image display method of the twenty-eighth aspect, when a plurality of image data displayed on the display device do not overlap each other, display priority is given according to the amount of motion of each detected image data. Since the degrees are assigned, the display priority can be appropriately determined even in a state where the respective image data are not superimposed, and the scalability management according to the display priority can be performed.

[Brief description of the drawings]

FIG. 1 shows a first embodiment of a multi-image display system according to the present invention.
It is a block diagram showing composition of an embodiment.

FIG. 2 is a block diagram illustrating a configuration of an input unit in FIG. 1;

FIG. 3 is a state transition diagram of the multi-image display system of FIG. 1;

FIG. 4 is a flowchart showing an initialization process in the multi-image display system of FIG.

FIG. 5 is a diagram showing a display example at the time of initialization in the multi-image display system of FIG. 1;

FIG. 6 is a flowchart illustrating a procedure of a window selection process in the multi-image display system of FIG. 1;

FIG. 7 is a diagram illustrating a display example in a window selection process.

FIG. 8 is a flowchart showing a procedure of a window moving display process in the multi-image display system of FIG. 1;

FIG. 9 is a diagram showing a display example at the time of window movement display processing in the multi-image display system of FIG. 1;

FIG. 10 is a flowchart illustrating a procedure of a window resolution conversion process in the multi-image display system of FIG. 1;

11 is a diagram showing a display example at the time of window resolution conversion processing in the multi-image display system of FIG.

FIG. 12 is a flowchart showing a procedure of a power save process in the multi-image display system of FIG. 1;

FIG. 13 is a diagram showing a display example at the time of a power save process in the multi-image display system of FIG. 1;

FIG. 14 is a flowchart showing a processing procedure in the second embodiment of the multi-image display system of the present invention.

FIG. 15 is a diagram illustrating a display example when a problem occurs in the superimposed state according to the first embodiment of the present invention.

FIG. 16 is a block diagram illustrating a configuration of a motion detection unit according to a third embodiment of the multi-image display system of the present invention.

FIG. 17 is a flowchart illustrating a procedure of processing using motion detection according to the third embodiment in the multi-image display system of the present invention.

FIG. 18 is a diagram illustrating a display example when motion detection is required.

FIG. 19 is a timing chart at the time of motion detection processing in the third embodiment in the multi-image display system of the present invention.

[Explanation of symbols]

101, 102, 103, 104 Image source 301, 302, 303, 304 Input unit (image data input unit) 351 Input thinning unit (control unit) 352 Display format conversion unit 353 Bus interface unit 360 Motion detection unit 400 Graphic bus (Transfer) Means) 500 control unit (comparing means, input data transfer rate detecting means) 600 frame memory 700 display device (display device) 701 display cursor (external action input means) 702 display pointer (external action input means) 800 infrared light receiving unit 900 display Drive controller (transfer means) 1000 Bus controller (priority determining means)

 ──────────────────────────────────────────────────続 き Continuing on the front page (72) Inventor Shuntaro Araya 3-30-2 Shimomaruko, Ota-ku, Tokyo Inside Canon Inc. (72) Inventor Katsuhiro Miyamoto 3-30-2 Shimomaruko, Ota-ku, Tokyo Canon Incorporated F term (reference) 5B069 BA01 BA04 BC02 CA06 CA14 CA16 CA17 DD11 KA06 LA02 5C082 AA01 AA27 BA41 BB01 CA62 CA63 CB01 DA54 DA87 MM05

Claims (28)

[Claims] 1. A plurality of image data input means for inputting image data from a plurality of image sources, respectively, and a plurality of image data respectively input from the plurality of image data input means are simultaneously displayed on one display device. An external action input unit for inputting an external action, a frame memory for storing image data respectively input by the plurality of image data input units in frame units, and the frame memory. Transfer means for reading out the image data from, and transferring the read out image data to the display device;
Detecting means for detecting the sum of the input data transfer rates of the image data input by the plurality of image data input means, and comparing means for comparing the sum of the detected input data transfer rates with the data transfer rate of the frame memory; Priority assigning means for deciding and assigning a display priority to each of the image data to be displayed on the display device in accordance with the input external action, and for assigning the display priority and the frame memory Control means for controlling display of the input image data on the display device in accordance with a result of comparison between the data transfer rate and the sum of the detected input data transfer rates. system. 2. The image data input device according to claim 2, wherein the control unit is configured to, when the sum of the detected input data transfer rates is greater than the data transfer rate of the frame memory, correspond to each of the input image data based on the display priority. 2. The multi-image display system according to claim 1, wherein the ratio of the input data transfer rate is adjusted, and the input data transfer rate in each of the image data input means is controlled according to the adjusted ratio. 3. The control unit according to claim 2, wherein, when the sum of the detected input data transfer rates is larger than the data transfer rate of the frame memory, the sum of the detected input data transfer rates is smaller than the data transfer rate of the frame memory. 3. The multi-image display system according to claim 2, wherein the ratio of the input data transfer rate in each of the image data input means is adjusted according to the order of display priority. 4. The multi-image display system according to claim 3, wherein said control means reduces the ratio as the display priority decreases. 5. The transfer means performs reading of the plurality of image data from the frame memory at a predetermined read data transfer rate, and the comparing means determines a sum of the detected input data transfer rates and the transfer means. 5. The multi-image display system according to claim 1, wherein the sum is added to the sum of the read data transfer rate and the sum and the data transfer rate of the frame memory is compared. 6. . 6. When the plurality of image data are superimposed on each other and displayed on the display device, the priority assigning means sets a display priority for the plurality of image data to an image displayed in the foreground. The multi-image display system according to any one of claims 1 to 5, wherein the data is assigned such that the highest order is assigned to the image data displayed on the rearmost side in order of decreasing order. 7. The external action input means inputs an operation instruction relating to image display of the display device issued by a user as the action from the outside, and the control means responds to the operation instruction relating to the image display. The multi-image display system according to any one of claims 1 to 6, further comprising a display processing function for performing processing relating to display of the image data. 8. An operation instruction relating to image display is output from one of a plurality of image data displayed on the display device.
8. The multi-image display system according to claim 7, wherein the instruction is an operation instruction for selecting one image data. 9. The operation instruction related to the image display is an operation instruction for moving one image data among a plurality of image data displayed on the display device, and the control unit includes the operation instruction. 8. The multi-image display system according to claim 7, wherein the one image data is moved to a corresponding position on the basis of the following. 10. An operation instruction related to the image display is one of a plurality of image data displayed on the display device.
8. An operation instruction for converting resolution of one image data, wherein the control means converts the resolution of the one image data to a corresponding resolution based on the operation instruction. Image display system. 11. The apparatus according to claim 7, wherein said priority assigning means assigns the highest display priority to the image data specified by said operation instruction when an operation instruction relating to said image display is input. A multi-image display system as described. 12. The external action input means detects a shift to interruption of supply of image data from the plurality of image sources, inputs a result of the detection as the external action, and the priority assigning means includes: 7. The system according to claim 1, wherein when the detection result is input by the external action input means, the lowest display priority is assigned to the image data from the image source from which the supply of the image data is interrupted. A multi-image display system according to any one of the above. 13. When the plurality of image data displayed on the display device overlap with each other, the priority assigning means sets the total number of pixels of the display area of the back side image data to be smaller than a predetermined number of pixels. The multi-image display system according to any one of claims 1 to 6, wherein it is determined whether the image data is small or not, and the display priority of the image data on the back side is changed according to the determination result. 14. An image data motion amount detecting means for detecting an amount of motion in each of the plurality of image data, wherein the priority assigning means is configured to overlap the plurality of image data displayed on the display device with each other. The display priority is assigned according to the amount of motion of each of the detected image data when they do not match.
7. The multi-image display system according to any one of items 6 to 6. 15. A multi-image display method for inputting image data from a plurality of image sources and simultaneously displaying each of the input image data on one display device, wherein an external action for inputting an external action is provided. An input step, a storing step of storing each of the image data input from the plurality of image sources in a frame memory in frame units, a detecting step of detecting a sum of input data transfer rates of the image data, Transferring the read image data to the display device, transferring the read image data to the display device, comparing the sum of the detected input data transfer rate with the data transfer rate of the frame memory, Image data to be displayed on the display device in response to an external action. An assigning step of determining and assigning a display priority to each of them, and according to a result of comparing the assigned display priority and the data transfer rate of the frame memory with the sum of the detected input data transfer rates. Controlling a display of each of the input image data on the display device. 16. The method according to claim 16, wherein, when the sum of the detected input data transfer rates is greater than the data transfer rate of the frame memory, the image data input means corresponding to each of the input image data based on the display priority thereof. 16. The multi-image display method according to claim 15, wherein the ratio of the input data transfer rate is adjusted, and the input data transfer rate in each of the image data input units is controlled according to the adjusted ratio. 17. The method according to claim 17, wherein, when the sum of the detected input data transfer rates is greater than the data transfer rate of the frame memory, the sum of the detected input data transfer rates is smaller than the data transfer rate of the frame memory. 17. The multi-image display method according to claim 16, wherein the ratio of the input data transfer rate in each of the image data input means is adjusted in accordance with the order of the respective display priorities. 18. The multi-image display method according to claim 17, wherein said control step reduces the ratio as the display priority decreases. 19. The transfer step of reading the plurality of image data from the frame memory at a predetermined read data transfer rate, and the comparing step includes the step of summing the detected input data transfer rate and the read data. 19. The multi-image display method according to claim 15, wherein a sum of the transfer rate and the sum is added, and the added value is compared with a data transfer rate of the frame memory. 20. The method according to claim 19, wherein, when displaying the plurality of image data on the display device while overlapping the plurality of image data, the display priority for the plurality of image data is set to the image data displayed on the foreground. 20. The image processing apparatus according to claim 15, wherein the highest order is assigned so as to become lower in order toward the image data displayed on the rearmost side.
The multi-image display method according to any one of the above. 21. A display processing step of performing processing relating to display of the image data in accordance with an operation instruction relating to the image display, wherein an operation relating to image display of the display device issued by a user in the external action input step is provided. 21. The multi-image display method according to claim 15, wherein an instruction is input as the external action. 22. The operation instruction according to claim 21, wherein the operation instruction relating to the image display is an operation instruction for selecting one image data from a plurality of image data displayed on the display device. Multi-image display method. 23. An operation instruction relating to the image display, wherein one of a plurality of image data displayed on the display device is displayed.
22. The multi-image display method according to claim 21, wherein the operation instruction is an operation instruction for moving one image data, and the one image data is moved to a corresponding position based on the operation instruction in the display processing step. . 24. An operation instruction relating to image display, wherein one of a plurality of image data displayed on the display device is output.
22. An operation instruction for converting the resolution of one image data, wherein the display processing step converts the resolution of the one image data into a corresponding resolution based on the operation instruction. Multi-image display method. 25. The image processing apparatus according to claim 21, wherein in the assigning step, when an operation instruction related to the image display is input, the highest display priority is assigned to the image data specified by the operation instruction. Multi-image display method. 26. The external action input step detects a shift to interruption of supply of image data from the plurality of image sources, and inputs a result of the detection as the external action. 21. The display device according to claim 15, wherein when the detection result is input by an external action input unit, the lowest display priority is assigned to image data from an image source that has stopped supplying the image data. A multi-image display method according to one of the above. 27. In the assigning step, when the plurality of image data displayed on the display device overlap each other, whether a total number of pixels in a display area of the back side image data is smaller than a predetermined number of pixels. 21. The multi-image display method according to claim 15, wherein it is determined whether or not the image data is displayed, and the display priority of the image data on the rear side is changed according to the determination result. 28. An image data motion amount detecting step of detecting a motion amount of each of the plurality of image data, wherein in the allocating step, the plurality of image data displayed on the display device overlap each other. 21. The multi-image display method according to claim 15, wherein, when there is no image data, the display priority is assigned according to the amount of motion of each of the detected image data.