JPH11177623A - Data transmission device, data reception device, and data transmission device - Google Patents

Abstract

(57) [Summary] [PROBLEMS] In a system for transmitting a real-time continuous stream to a packet communication network where transmission delay fluctuation is large and packet loss is unavoidable, a configuration capable of suppressing the influence of the packet loss on the reproduction function. It is an object of the present invention to provide a configuration capable of controlling a buffering amount that can cope with a case where packets arriving beyond the maximum allowable delay time are continuously discarded. SOLUTION: A communication state monitoring unit 1 in a receiving terminal 102 is provided.
23, an index such as a packet loss rate or a transmission delay fluctuation is calculated from the monitor information, the data is received by the transmission control unit 113 in the data server 101, and the transmission parameter is determined from the communication state index. In response to this, the transmission parameters are dynamically changed, and the deterioration of the reproduction quality can be reduced.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to data transmission / reception using a packet communication network, and more particularly to real-time data transmission.

[0002]

2. Description of the Related Art Conventionally, there is known a real-time data transmission apparatus as shown in FIG. FIG. 19 shows a configuration of a conventional sound information providing apparatus, and an audio server 1901,
The receiving terminal 1902 is a packet communication network 1903
And provides / receives sound information.

[0003] An audio server 1901 has a network interface 1904a, a data transmission unit 1905a,
In addition to normal data communication means such as a data receiving unit 1906a and a packet generating unit 1907, a sound information managing unit 1910 that manages sound data to be provided, and a packet communication network that uses sound data output from the sound information managing unit. Multi-rate audio encoder 1 for encoding together
911, a buffer 1908 for temporarily storing output results from the encoder, a communication state monitor information management unit 1912 for managing communication state monitor information sent from the receiving terminal,
A transmission control unit 1913 that performs transmission control in response to the communication state monitor information, a terminal request reception unit 1931 that receives a request from the user of the receiving terminal, and a transmission content management unit 1930 that manages transmission content to the receiving terminal user. The transmission control unit 1913 receives the data of the throughput included in the communication state monitor information and determines the transmission data rate from the data rate determination unit 1913 and the transmission parameter notification unit 1916 that notifies the audio encoder 1911 of the transmission parameter determined here. Be composed.

[0004] In addition to the ordinary data communication means similar to the audio server, the receiving terminal 1902 is provided with a receiving buffer 1 for temporarily storing the output from the data receiving section 1906b.
909, a stream restoring unit 1922 for restoring an encoded data stream from received data, a multi-rate audio decoder 1921 for decoding sound information from an encoded stream having a plurality of rates, and an audio output unit for outputting sound from the decoded sound data 1920, a communication state monitoring unit 1923 that analyzes a header of received data to monitor a communication state, and a throughput calculation unit 192 that calculates a throughput in a certain time interval from the communication state monitor data.
4. Reception content setting unit 1932, terminal request generation unit 1933
Is provided.

The operation of the sound information providing apparatus configured as described above for performing transmission control based on the monitor information of the packet communication network will be described below.

[0006] When the receiving terminal 1902 is to receive sound information from the audio server 1901 connected to the packet communication network 1903, the receiving content setting unit 19
At 32, the received content is selected, and the terminal request generation unit 1933 adjusts the format to a predetermined format and transmits the data from the data transmission unit 1905b. Terminal request is network interface 1
The audio server 190 via the 904a and 1904b
1 is received by the data receiving unit 1906a, is received by the terminal request receiving unit 1931, and the transmission content management unit 1930 designates a title ID managed by the sound information management unit 1910.

[0007] The sound information management unit 1910 inputs the data of the designated title to the multi-rate audio encoder 1911 at a constant speed, and the multi-rate audio encoder 1911 encodes at the initial value of the coding rate. The coded data output as a result is accumulated in the temporary buffer 1908, from which the packet generation unit 1
The data transmission unit 190 reads out and adds a packet header including information such as a reception terminal address, a time stamp indicating transmission time, and a stream type.
5a to the packet communication network 1903 via the network interface unit 1904a.

[0008] The receiving terminal 1902 reads the data from the network interface unit 1904b to the data receiving unit 1906b, and separates the packet header there. The payload data is temporarily stored in the reception buffer 1909, restored to the original coded stream by the stream restoration unit 1922, decoded by the multi-rate audio decoder 1921, and passed to the audio output unit 1920. on the other hand,
Information related to the network state from among the separated packet headers is output to the communication state monitor 1923.
The throughput calculation unit 1924 includes a communication state monitor 1923.
, The data necessary for the throughput calculation is acquired, data of a certain length of time is accumulated, the throughput is calculated, and the result is transmitted together with other communication monitor information at appropriate intervals to the communication state monitor information management unit 1912 of the audio server 1901.
Notify.

[0009] The communication status monitor information management unit 1912 of the audio server 1901 passes the throughput information to the data rate determination unit 1914 for transmission control, and the data rate determination unit 1914 checks the necessity of changing the current data rate. . If the throughput is lower than the current transmission data rate, the transmission data rate is decreased, and if there is room, the transmission data rate is increased. The data rate determined here is transmitted from the transmission parameter notification unit 1916 to the audio encoder 1911 as a transmission parameter.

Thus, media data can be transmitted / received by selecting a data rate adapted to the throughput of the packet communication network.

[0011]

The conventional sound information providing apparatus has a mechanism for adjusting the transmission data rate to the throughput of the packet communication network, so that the optimum throughput can be automatically selected.

However, packet loss is unavoidable in a packet communication network, and packet loss is a major cause of deterioration of reproduced data as well as a decrease in throughput.
In the case of sound information, a state occurs in which the sound is interrupted or decoding cannot be continued.

An object of the present invention is to solve this problem and to provide a configuration capable of suppressing the influence of packet loss on a reproduction function even in a packet communication network in which packet loss cannot be avoided. Further, the present invention provides a configuration capable of controlling a buffering amount that can cope with a case where packets arriving beyond the maximum allowable delay time are continuously discarded.

[0014]

Accordingly, in the present invention, in data transmission (particularly, real-time transmission / reception of data), the receiving side monitors information such as dropped packets and arrival delay of received data, and the like. The information is sent to the sending side, and the sending side retransmits the data in various forms based on the information. Thus, interruption of data reproduction on the receiving side in real time can be minimized.

[0015]

BEST MODE FOR CARRYING OUT THE INVENTION In the present invention, first, according to the first, third and sixteenth aspects of the present invention, the receiving side monitors the loss rate of the transmitted packet and transmits the information to the transmitting side. ,
On the transmitting side, when the packet loss rate exceeds a predetermined threshold based on the information, the size of the transmission packet is reduced and the packet is retransmitted. As a result, it is possible to reduce the influence of packet loss on reproduction.

Second, the invention according to claims 2, 3 and 17 is characterized in that the receiving side monitors the loss rate of the packet to be transmitted, sends the information to the sending side, and the sending side uses the information from the information. When the packet loss rate exceeds a predetermined threshold, the same packet is multiplexed. This makes it possible to minimize the probability that all data in a certain time section will be lost when a packet is lost. Further, as in the invention according to claim 4, if the duplicated packets among the multiplexed packets are deleted on the receiving side, useless processing on the receiving side due to multiplex reception can be easily eliminated. .

Third, the invention according to claims 5, 6 and 18 is such that the receiving side monitors the loss rate of the transmitted packet, sends the information to the transmitting side, and the transmitting side uses the information from the information. When the packet loss rate exceeds a predetermined threshold, the packet is divided and retransmitted. Thereby, the receiving side monitors the loss rate of the packet to be transmitted, and sends the information to the transmitting side. If the packet loss rate exceeds the predetermined threshold from the information, the transmitting side divides the packet. And resend it. As a result, it is possible to almost completely restore the reproduction data representing the entire corresponding time section.

Fourth, the invention according to claims 7 and 19 provides:
The receiving side monitors fluctuations due to data delay and data loss called jitter, controls the jitter and the amount of buffering of the received data, and makes adjustments for reproducing the reproduced data. As a result, the buffering amount can be optimally set with respect to the temporal transition of the delay amount, so that smooth communication control can be performed in response to a change in network state in addition to smooth reproduction.

Fifth, the invention according to claims 8 and 20 provides:
In the fifth aspect, in particular, when the target data is a sound or a moving image having a time axis, the time axis adjustment is performed as an adjustment for reproducing the reproduction data, and the same effect can be obtained.

Sixth, according to the ninth, tenth, and twenty-first aspects of the present invention, the receiving side monitors information such as packet loss due to arrival delay and packet loss during transmission, and sends the information to the transmitting side. The transmitting side retransmits the data after performing cross-interleaving if necessary based on the information, and performs deinterleaving on the receiving side to reproduce the data.
As a result, it is possible to improve the error correction performance (as well as to suppress the packet discard / loss) and to improve the data recovery performance against the continuous packet loss as a result.

Seventh, the invention according to claims 11, 12 and 22 is characterized in that the reception side monitors information relating to the reproduction quality at the time of reproduction of the received data, sends the information to the transmission side, and sends the information to the transmission side. On the side, by learning and accumulating the information, learning for improving the reproduction quality is performed, and data transmission control is performed based on the information obtained by the learning. Thereby, the optimal transmission method can be learned. Eighth, the invention according to Claims 13 and 23 monitors the transmission amount and reception amount between devices having the function of bidirectional communication, and controls the buffering amount accordingly. . As a result, for a user terminal that does not require a low delay (high-speed bidirectional communication), the buffering amount is increased (packet discarding due to a transmission delay can be suppressed), so that data transmission quality can be improved. On the other hand, for a user terminal that requires low delay (high-speed bidirectional communication), the data transmission delay can be reduced in order to reduce the buffering amount. In addition, as in the invention according to claim 14, in the eighth invention, in particular, when the target data is a sound or a moving image having a time axis, as an adjustment for reproducing the reproduction data,
The time axis is adjusted, and the same effect can be obtained.

Ninth, the invention according to claims 15 and 24 further improves the data reception quality by performing interleaving (adding error correction processing) at the time of data transmission in the eighth invention. Can be done.

Hereinafter, an embodiment of the present invention will be described with reference to FIG.
This will be described with reference to FIG. It should be noted that the present invention is not limited to these embodiments at all, and can be implemented in various modes without departing from the gist thereof.

(Embodiment 1) FIG. 1 is a diagram showing a system configuration according to a first embodiment of the present invention. In FIG. 1, an audio server 101 and a receiving terminal 102 are connected to a packet communication network 103, Provide and receive information.

The audio server 101 includes, in addition to ordinary data communication means such as a network interface 104a, a data transmission unit 105a, a data reception unit 106a, and a packet generation unit 107, a sound information management unit 110 for managing provided sound data, A multi-rate audio encoder 111 for encoding the data according to the network for transmitting the data, a buffer 108 for temporarily storing output results from the encoder, and a communication state monitor information management for managing communication state monitor information sent from the receiving terminal 102 Part 11
2. A transmission control unit 113 that performs transmission control from the communication state monitor information, and a packet size setting unit 1 that receives a parameter set by the transmission control unit 113 and sets a packet size.
17, a terminal request receiving unit 131 that receives a transmission request from the receiving terminal 102, and a transmission content management unit 130 that manages the content transmitted to the receiving terminal 102.

The transmission control unit 113 receives the data of the throughput included in the communication status monitor information and determines the transmission data rate. The data control unit 113 receives the data of the packet loss rate included in the communication status monitor information and A packet size determining unit 115 for determining a size,
The transmission parameter notification unit 116 notifies the setting contents of the transmission control unit 113 to a part that performs actual transmission control.

The receiving terminal 102 includes a normal data communication unit similar to the audio server 101, a receiving buffer 109 for temporarily storing an output from the data receiving unit 106b, and a stream for restoring an encoded data stream from the received data. A restoring unit 122, a multi-rate audio decoder 121 for decoding sound information from a coded stream having a plurality of rates, an audio output unit 120 for outputting sound from the decoded sound data, and analyzing a header of received data to determine a communication state. It includes a communication status monitor 123 for monitoring, a throughput calculator 124 for calculating a throughput in a certain time interval from a communication status monitor result, a packet loss rate calculator 125, a reception content setting unit 132, and a terminal request generator 133.

The operation of the above-configured sound information providing apparatus for performing packet size control based on network monitor information will be described below.

When the receiving terminal 102 intends to receive sound information from the audio server 101 connected to the packet communication network 103, the receiving content setting unit 132 selects the received content, and the terminal request generating unit 133 converts the received content into a predetermined format. At the same time, the data is transmitted from the data transmission unit 105b.
The terminal request is sent to the network interface 104a, 10
4b, the data receiving unit 1 of the audio server 101
06a, received by the terminal request receiving unit 131, and the transmission content management unit 130 designates a title ID in the sound information management unit 110.

The sound information management unit 110 inputs the data of the specified title to the multi-rate audio encoder 111 at a constant speed, and the multi-rate audio encoder 111 performs encoding at the initial value of the coding rate. The encoded data output as a result is stored in buffer 10
8, the packet generating unit 107 reads out the packet, adds a packet header including information such as a receiving terminal address, a time stamp indicating a transmission time, and a stream type, and sends the packet from the data transmitting unit 105a to the network interface unit. The packet is transmitted to the packet communication network 103 via the communication network 104a.

The receiving terminal 102 reads the data from the network interface unit 104b to the data receiving unit 106b, and separates the packet header there. The payload data is temporarily stored in the reception buffer 109, restored to the original coded stream by the stream restoration unit 122, decoded by the multi-rate audio decoder 121, and passed to the audio output unit 120. On the other hand, information relating to the network state from among the separated packet headers is output to the communication state monitor 123. The throughput calculation unit 124 acquires data necessary for the throughput calculation from the communication state monitor unit 123, and accumulates data of a certain length of time to calculate the throughput. Packet loss rate calculator 1
Reference numeral 25 acquires data necessary for calculating the packet loss rate from the communication state monitor 123, and accumulates data for a certain length of time to calculate the packet loss rate. These calculation results are notified to the communication status monitor information management unit 112 of the audio server 101 at appropriate intervals together with other communication monitor information.

The communication status monitor information management section 112 of the audio server 101 passes the throughput information to the data rate determination section 114 for transmission control, and the data rate determination section 114 checks the necessity of changing the current data rate. . If the throughput is lower than the current transmission data rate, the transmission data rate is decreased, and if there is room, the transmission data rate is increased. Similarly, the communication state monitor information management unit 112 passes the packet loss rate information to the packet size determination unit 115 for transmission control, and the packet size determination unit 115
Check for the need to change the current packet size and determine an appropriate packet size for the current packet loss rate. The data rate determined by the data rate determination unit 114 and the packet size determined by the packet size determination unit 115 are notified from the transmission parameter notification unit 116 to the multi-rate audio encoder 111 and the packet size setting unit 117, respectively.
The transmission according to the transmission method determined in the step is performed.

FIG. 10 shows an example of the packet size determined for the packet loss rate. If the packet loss rate is large, packets are divided and transmitted so as to suppress the effects of packet loss even if the load on the audio server increases slightly. If the effect on the reproduction quality is small even if the frame unit packet is transmitted as it is, the packet is transmitted as it is.

Thus, even when a packet is lost, the time section of the data affected by the reproducing side can be reduced, and it is possible to prevent the reproducing quality from being significantly degraded.

As described above, in the present embodiment, the receiving terminal is provided with the packet loss rate calculating means, and the audio server receives the packet loss rate data from the receiving terminal, and sets the transmission packet size based on this value. However, by retransmitting at the size, the time section in which the reproduction quality deteriorates when a packet is lost can be reduced, and the practical effect is large.

(Embodiment 2) FIG. 2 shows a system configuration diagram in a second embodiment of the present invention. In FIG. 2, 1
Reference numerals 01 to 114, 116, and 120 to 133 have the same configuration as in the first embodiment. In addition, the audio server 101 has a transmission multiplexing number determination unit 201, a transmission number counter 202, and a duplicate packet deletion unit in the receiving terminal 102. 203
Is provided. Transmission multiplex count determination section 201
Is provided inside the transmission control unit 103.

The operation of controlling the number of multiplex transmissions based on the monitor information of the network in the sound information providing apparatus configured as described above will be described below.

The receiving terminal 102 transmits a request for transmitting sound information to the audio server 101, and the audio server 101
The audio data of the corresponding title is encoded and transmitted, and the payload data is decoded by the receiving terminal 102, output from the audio output unit 120, and the throughput and the packet loss rate are calculated from monitoring the separated packet header. The operation of notifying the communication status monitor information management unit 112 of the audio server 101 is the same as in the first embodiment.

The data rate determination unit 114 of the audio server 101 checks the necessity of changing the current data rate, and the communication state monitor information management unit 112 similarly transmits packet loss rate information for packet multiplex transmission for transmission control. The packet multiplexing transmission determining unit 201 checks the necessity of changing the current number of packet transmissions, and determines an appropriate number of packet transmissions for the current packet loss rate. FIG. 11 shows an example of the same packet transmission count determined for the packet loss rate.

When the packet loss rate is large, the same packet is transmitted a plurality of times so as to suppress the influence of the packet loss even if the load on the audio server side slightly increases. Sends the same packet only once. According to the number of transmissions determined by the packet multiplexing transmission determining unit 201, the transmission number counter 202 counts the number of packet transmissions so that the same packet is transmitted the number of times. On the receiving terminal side that receives the same packet redundantly, the duplicate packet deletion unit 203 checks the packet ID of the header and deletes the duplicate data. As a result, even when a packet is lost, the probability that the reproduction side is affected is reduced, and it is possible to prevent the reproduction quality from being significantly degraded.

By such processing, the bandwidth required for communication is increased by the number of times of transmission, but when it is necessary to avoid this, in conjunction with the data rate determination unit 114, the multiplex transmission is performed. It is possible to keep the data rate within the usable communication band.

As described above, in the present embodiment, the receiving terminal is provided with the packet loss rate calculating means, and the audio server receives the packet loss rate data from the receiving terminal, and determines the same packet transmission count based on this value. By setting, even if a packet is lost, the probability that the reproduction quality deteriorates can be extremely suppressed, and the practical effect is large.

(Embodiment 3) FIG. 3 shows a system configuration diagram in a third embodiment of the present invention. In FIG.
Reference numerals 01 to 114, 116, and 120 to 133 have the same configuration as in the first embodiment. In addition, the audio server 101 has a frame division determining unit 301, a frame division processing unit 302, a transmission stream management unit 303, and a reception unit. The configuration is such that a frame synthesizing unit 304 is provided in the terminal 102. The frame division determining unit 301
It is provided within.

The operation of the above-configured sound information providing apparatus for performing frame division control based on the monitor information of the packet communication network will be described below.

The receiving terminal 102 transmits a request for transmitting sound information to the audio server 101, and the audio server 101
The audio data of the corresponding title is encoded and transmitted, and the payload data is decoded by the receiving terminal 102, output from the audio output unit 120, and the throughput and the packet loss rate are calculated from monitoring the separated packet header. The operation of notifying the communication status monitor information management unit 112 of the audio server 101 is the same as in the first embodiment.

The communication status monitor information management section 112 of the audio server 101 passes the throughput information to the data rate determination section 114 for transmission control, and the data rate determination section 114 checks the necessity of changing the current data rate. Similarly, the communication state monitor information management unit 112 passes the packet loss rate information to the frame division determination unit 301 for transmission control, and
In step 1, the necessity of changing the current frame processing method is checked, and an appropriate frame processing method is determined for the current packet loss rate. FIG. 12 shows an example of a frame division method determined for the packet loss rate.

When the packet loss rate is large, one time section is divided into data in which the resolution of the entire time section is reduced so that the influence of the packet loss is suppressed even if the load on the audio server side slightly increases. In such a case, the data is divided into a plurality of frames, respectively encoded, and transmitted as a plurality of streams. When the packet loss rate is small, one time section is transmitted as one frame. The frame division processing unit 302 divides one original frame according to the number of divisions determined by the frame division determination unit 301, and separates each of the original frames as a separate stream.
b, and each buffer 108a,
08b. The transmission stream management unit 303 passes a packet header including an ID capable of identifying each stream to the packet generation unit 107. The packet generation unit 107 adds a packet header to each stream and transmits the packet header from the data transmission unit 105a. I do.

In the receiving terminal 102, the data receiving section 106b
Check the header of the packet, and for each stream ID indicated in the header, a separate reception buffer 109a, 10b
Start with 9b. The data allocated to the receiving buffers 109a and 109b are stored in the stream restoring units 122a and 122a.
At 22b, the stream is restored to an encoded stream, and is input to the multi-rate audio decoders 121a and 121b, respectively. Multi-rate audio decoders 121a, 121
The output from b is combined by the frame combining unit 304 to restore the original data.

Here, when one packet of a plurality of packets transmitted by dividing one frame is lost,
The corresponding time interval is reproduced as data having a low sampling rate. In this part, the sound quality is deteriorated as compared with other time sections, but there is no occurrence of a time section where data is not passed, and the degree of loss of information amount can be reduced.

Thus, even if one packet is lost, the reproducing side can reproduce the sound without interruption by slightly lowering the sound quality in a short time interval, and it is possible to suppress the remarkable deterioration of the reproducing quality. it can. Further, compared to a method of increasing the degree of redundancy to add packet loss tolerance, there is almost no redundancy and the band can be used effectively.

In this embodiment, a similar effect can be obtained even when audio data is video data. With respect to the video data transmission method, a method is known in which data in one time section is divided into a low-resolution component and a high-resolution component, and each of the data is transmitted as a stream. If the data is lost, the video cannot be restored even if only the high resolution component can be received. According to the method of the present invention, when the packet loss rate exceeds a certain threshold, the data in one time section is divided into two packets having the same amount of information and transmitted, so that the data is reproduced by the packet loss. The degree of influence can be reduced.

The number of divisions may be considered to be "2" in consideration of the time required for the division processing, the synthesis processing after reception, the overall efficiency, and the like. Even if the number of divisions is set to 3, 4, or any other number, it does not matter at all, and even in this case, there is no effect on the effects unique to the present embodiment, such as restoration of reproduction data close to 100%, suppression of reproduction interruption, and the like. is not.

As described above, in the present embodiment, the receiving terminal is provided with the packet loss rate calculating means, and the audio server receives the packet loss rate data from the receiving terminal and, based on this value, determines the number of divisions of one frame. By setting
Even if a packet is lost without providing data redundancy, the probability of interruption of sound reproduction can be reduced, and deterioration in reproduction quality can be suppressed.

More specifically, for example, MPEG
In the case of, when a specific i-frame is dropped, all the images are lost. On the other hand, as in the present embodiment, when the packet loss rate exceeds a specific threshold, each of the media data frames is reduced in resolution. The data can be divided into a plurality of pieces so as to reduce the data, and control can be performed so that data per frame is transmitted in a plurality of packets. When one packet is lost, another one packet is used even if the resolution is lowered. The reproduced data representing the entire time section can be restored, and the practical effect is great.

(Embodiment 4) FIG. 4 shows a system configuration diagram in a fourth embodiment of the present invention. In FIG.
Reference numerals 01 to 114, 116, and 120 to 133 have the same configuration as that of the first embodiment. In addition, the discarded packet counting unit 401, the transmission jitter monitor unit 402, the reception buffer control unit 403, the reproduced data The configuration includes an adjustment unit 404. The reception buffer control unit 403 includes a buffering amount storage unit 405 for temporarily storing the current value of the reception buffer parameter, and a maximum allowable delay time storage unit 406.

The operation of the above-configured sound information providing apparatus for performing buffering control based on the monitor information of the packet communication network will be described below.

The receiving terminal 102 transmits a request for transmitting sound information to the audio server 101, and the audio server 101
, The audio data of the corresponding title is encoded and transmitted, the payload data is decoded by the receiving terminal 102, output from the audio output unit 120, the throughput is calculated from monitoring the separated packet header, and the audio server 101 The operation of notifying the communication state monitor information management unit 112 of the third embodiment is the same as that of the first embodiment.

The discarded packet counting section 401 of the receiving terminal 102 arrives beyond the maximum allowable delay time during reception from the communication state monitoring section 123, obtains the number of discarded packets, and stores the packet discarding occurrence state. I do. The transmission jitter monitor 402 acquires data of the fluctuation of the actual arrival time with respect to the arrival reference time from the communication state monitor 123,
A statistic in a certain time section is calculated, and its time change is monitored. From these monitoring results, the reception buffer control unit 403 controls the amount of buffering as described below.

Normally, the receiving buffering amount is determined from the average jitter value output from the transmission jitter monitoring unit 402, and the maximum allowable delay time of the arrival of the received packet is determined from the allowable packet loss rate when the distribution is followed. To perform buffering. FIG. 13 shows an example of the buffering amount control based on the average jitter value. Here, the hysteresis is given to the control amount so that the buffering amount does not oscillate when performing control to follow the average jitter value.

The discarded packet count unit 401 raises an alarm when the packet discard state suddenly deteriorates, and stores the buffering amount and the current value of the maximum allowable delay time in the buffering amount storage unit 405 and the maximum allowable delay time storage unit 406, respectively. To increase the amount of buffering,
Relax the maximum allowable delay time. When the short-term delay deterioration state is resolved, the buffering amount and the maximum allowable delay time are returned to the stored values. When the buffering amount is increased, the reproduction data is lengthened, and when the buffering amount is reduced, the reproduction data needs to be flushed. For this reason, the reproduction data adjustment unit 404 that adjusts the time axis of the reproduction data extends the data in the silent section,
Alternatively, data is deleted, and the time axis of the reproduced data is adjusted.

Here, if the transmission delay amount increases and there is a sign that data starvation occurs in the buffer, the buffering amount is increased so as to alleviate the data starvation and the silent section is set so that the reproduced data is not interrupted. Detect and adjust the playback time there. Also, if the amount of delay decreases and there is a sign that the buffer overflows, reduce the amount of buffering to mitigate it, adjust the time axis of the data released thereby, and shrink the silence section for playback. .

Thus, the buffering amount on the receiving side is gradually changed with respect to the output from the transmission jitter calculating section indicating the long-term change in the network state, and the buffering amount on the receiving side is changed with respect to the short-term change. The amount of ring is rapidly changed, and when the short-term deterioration state is resolved, the buffering amount is returned to the original buffering amount determined based on the output value from the transmission jitter calculating unit. The buffering amount can be controlled so as to be set optimally, and smooth communication control can be performed in response to a change in the network state.

As described above, in the present embodiment, the transmission jitter monitoring unit and the short-term transmission deterioration detecting unit, the buffer control unit for controlling the buffering amount based on the monitoring information from them, and the time axis of the reproduced data are provided in the receiving terminal. By providing a processing unit for performing adjustment, it is possible to perform stable reception control with respect to the difference pattern of the deterioration when the deterioration of the transmission delay is detected in the receiving terminal, and to reduce the probability that sound reproduction is interrupted. In addition, it is possible to suppress the deterioration of the reproduction quality.

More specifically, the conventional technique of reducing the buffering amount for receiving a transmission packet as much as possible to speed up cueing and increasing the buffering amount in order to eliminate the interruption of reproduction has been known. In order to solve the contradictory issues, the output from the transmission jitter calculator, which indicates long-term changes in the network state, such as cutting silence in conversations and discarding data that can be discarded, is the receiving side. The buffering amount of the transmission side is gradually changed, and the buffering amount on the receiving side is changed rapidly for a short-term change, and when the short-term deterioration state is eliminated, the output value from the transmission jitter calculator is also changed. The buffering amount can be returned to the original buffering amount determined as above, and the buffering amount can be controlled so as to be optimally set with respect to the time change of the delay amount. Can smooth communication control with respect to changes in the over click status, its practical effect is large.

(Embodiment 5) FIG. 5 shows a system configuration diagram according to a fifth embodiment of the present invention. In FIG. 5, 1
01-114, 120-133, 401, 403, 40
Reference numerals 5 and 406 have the same configuration as in the fourth embodiment, except that sound data is audio data and a speech speed conversion unit 501 is provided instead of the reproduction data adjustment unit 404 in the receiving terminal 102.

The operation of the above-configured sound information providing apparatus for performing buffering control based on the monitor information of the packet communication network will be described below.

The receiving terminal 102 transmits a request for transmitting sound information to the audio server 101, and the audio server 101
The audio data of the corresponding title is encoded and transmitted, and the payload data is decoded at the receiving terminal 102, output from the audio output unit 120, and the throughput is calculated from monitoring the separated packet header, The operation of notifying the communication state monitor information management unit 112 of the communication terminal 101, the discarded packet count unit 401 of the receiving terminal 102, the transmission jitter monitor unit 40
2 monitors the communication state, and based on the monitoring results, the operation of the receiving buffer control unit 403 changing the buffering size is the same as in the fourth embodiment.

In this configuration, when the buffering amount is increased, the reproduction data is lengthened, and when the buffering amount is reduced, the reproduction data must be flushed. For this purpose, the time axis of the reproduced data is adjusted by changing the data reproduction time by the speech speed converter 501. Figure 1 shows how this playback time is controlled.
It is shown in FIG.

Here, if the amount of transmission delay increases and there is a sign of causing data starvation in the buffer, the buffering amount is increased so as to alleviate the occurrence, and the speech speed is increased so that the reproduced data is not interrupted. Lower to adjust the playback time. Also, the amount of delay has decreased,
If there is a sign that the buffer overflows, reduce the amount of buffering to mitigate it, adjust the time axis of the released data (mainly shorten the playback time), and increase the speech speed only for the necessary time section. Raise and play.

As a result, the time axis can be adjusted arbitrarily even in reproduced data having a small number of silent sections, and data discarding or buffer data starvation due to buffer overflow can occur regardless of increase or decrease in network delay. As a result, the occurrence of discard / depletion of audio data can be suppressed.

As described above, in the present embodiment, in the receiving terminal, the transmission jitter monitoring unit, the short-term transmission deterioration detecting unit, the buffer control unit for controlling the buffering amount based on the monitoring information from them, and the time axis of the reproduced data. And a speech speed conversion processing unit for performing adjustment. When detecting deterioration of transmission delay at the receiving terminal, it is possible to perform stable reception control (smooth reproduction) with respect to a difference pattern of the deterioration, The reliability of data reproduction can be improved, and the practical effect is great.

(Embodiment 6) FIG. 6 shows a system configuration diagram in a sixth embodiment of the present invention. In FIG. 6, 1
Reference numerals 01 to 113 and 120 to 133 are the same as those in the first embodiment. In addition, the transmission mode determination unit 601 and the cross interleave unit 60 are provided in the audio server 101.
3, stream management section 605, continuous packet discard detection section 602 in receiving terminal 102, deinterleave section 604,
The configuration includes a reception control unit 606. The transmission mode determination unit 601 is provided in the transmission control unit 113.

The operation of the sound information providing apparatus configured as described above for performing error correction control based on the monitor information of the packet communication network will be described below.

The receiving terminal 102 transmits a request for transmitting sound information to the audio server 101, and the audio server 101
The audio data of the corresponding title is encoded and transmitted, the payload data is decoded by the receiving terminal 102, and the operation output from the audio output unit 120 is the same as in the first embodiment.

The packet continuation discard detection unit 602 of the receiving terminal 102 detects that packets that have arrived beyond the maximum allowable delay time during reception from the communication state monitoring unit 123 and have been discarded continuously occur, and the audio server 101 The communication status monitor information management unit 112 is notified.

The communication state monitor information management section 112 of the audio server 101 passes the packet continuous discard information to the transmission mode determination section 601 for transmission control, and the transmission mode determination section 601 considers the necessity of changing the current transmission mode. To check. The current frequency of continuous packet discarding is increasing, and if the need for error correction increases, cross-interleave processing is performed on transmitted data.If the frequency of continuous packet discarding decreases, an error occurs. In the case of the transmission mode in which the correction is turned on, it is considered that the necessity is reduced, and the cross interleave processing is stopped. FIG. 15 shows an example of the cross interleave processing. Here, processing for buffering six frames and transmitting the data in eight packets is performed. The transmission stream management unit 605 passes the packet header indicating the transmission mode to the packet generation unit 107, and the packet generation unit 107 adds the packet header to the data stream and transmits the data stream from the data transmission unit 105a.

In the receiving terminal 102, the data receiving section 106b
Then, the header of the packet is separated and passed to the reception control unit 606. The reception control unit 606 checks the content of the header, determines whether or not the deinterleaving process is necessary according to the transmission mode indicated in the header, and Therefore, the deinterleave processing ON / OFF switch is switched.

As a result, the processing delay increases and the data size increases. However, when packets are continuously discarded or lost, the lost data can be recovered by the reproducing side according to the added redundancy. And the deterioration of the reproduction quality can be suppressed.

As described above, in the present embodiment, the receiving terminal is provided with the packet continuous discard detecting unit, and the audio server adds the cross-interleave processing according to the alarm from the detecting unit, so that the receiving terminal can continuously lose or lose the packet. Even when discarded, it is possible to recover lost data, and it is possible to suppress interruption of sound reproduction and deterioration of reproduction quality.

More specifically, when it is detected that continuous discarding of packets has occurred, cross-interleave processing is performed on transmission data, de-interleave processing is performed on the reception side, and transmission delay is performed. , The error recovery performance can be improved, and the data recovery capability against continuous packet loss can be improved, which has a great practical effect.

(Embodiment 7) FIG. 7 shows a system configuration diagram according to a seventh embodiment of the present invention. In FIG. 7, 1
Reference numerals 01 to 113, 116, and 120 to 133 have the same configuration as that of the first embodiment. In addition, the transmission control unit of the audio server 101 has an index analysis unit 701 that analyzes a plurality of indexes of the communication state monitoring result. A test execution unit 702 for executing a test for determining parameters, an index similar pattern search unit 703 for searching a vector pattern of a plurality of indexes, an index pattern and optimum transmission parameter distribution storage unit 704, a test execution flag storage unit 705, and transmission control It is configured to include a processing unit 706 and a reproduction quality change detection unit 710 in the receiving terminal 102.

For the sound information providing apparatus configured as described above, a test is performed to obtain an optimum transmission parameter when the monitor information of the packet communication network includes a plurality of indexes, and the optimum transmission parameter is determined from the result. The operation to be determined will be described.

The receiving terminal 102 transmits a request for transmitting sound information to the audio server 101,
The audio data of the corresponding title is encoded and transmitted, the payload data is decoded by the receiving terminal 102, and the operation output from the audio output unit 120 is the same as in the first embodiment.

Communication status monitor 123 of receiving terminal 102
, A plurality of indexes indicating the communication state such as the throughput and the packet loss rate are calculated. Data indicating the reproduction quality is output from the multi-rate audio decoder 121, and the reproduction quality change detection unit 710 calculates a time change of the reproduction quality data. The receiving terminal 102 notifies the communication status monitor information management unit 112 of the audio server 101 of the plurality of indexes.

In the communication status monitor information management unit 112 of the audio server 101, the index analysis unit 701 analyzes a plurality of communication status monitor indexes received from the receiving terminal 102, and determines whether it is necessary to change the current transmission method. If it is determined that the current transmission method needs to be changed, the index set is passed to a test execution unit 702 that performs a test in order to determine an optimal transmission parameter, and a test execution is instructed.

FIG. 16 shows the procedure for performing the test, and the procedure for performing the test will be described below with reference to FIG.

The test execution unit 702 requests the index similar pattern search unit 703 to search for an index pattern closest to the current index set from past index patterns. The index pattern and optimal transmission parameter distribution storage unit 704 classifies and stores data by clustering past index patterns as vectors, and stores the index similar pattern search unit 7.
03 calculates the distance between the current index set and the representative vector to quickly search for the closest index pattern. In addition, the index similar pattern search unit 703 also outputs, as data, the distribution of the optimal transmission parameters obtained by performing the test in the past in the corresponding index pattern.

The test execution unit 702 generates a plurality of sets of test parameters based on the distribution of the optimal transmission parameters, and sequentially passes the test parameters to the transmission parameter notification unit one set at a time to execute the test. At the start of the test, the data in the test execution flag storage unit 705 is set to ON.

In the receiving terminal 102, a plurality of indices are sent to the data server 10 as a result of the communication state monitoring as in the normal case.
1 and the index analysis unit 70 of the data server 101
In 1, when the test execution flag 705 is set, the index data is unconditionally passed to the test execution unit 702. The test execution unit 702 continues the test with the same parameter set until the index indicating the reproduction quality returned from the receiving terminal 102 converges, and when a representative index set is determined for one parameter set, the test in the next parameter set is performed. Conduct a test.

The test execution unit 702 executes the prepared test, prepares a parameter set again if necessary, repeats the test, and obtains a parameter set that maximizes an index indicating the reproduction quality. When it is determined that the parameter set optimization has converged, the transmission parameter is passed to the transmission parameter notification unit 116, and the data in the test execution flag storage unit 705 is cleared. Also, the index pattern and the optimal transmission parameter distribution storage unit 704 receive the optimal transmission parameter data for the index via the index similar pattern search unit 703.
Update the optimal transmission parameter distribution of.

Thus, when the reproduction quality is degraded,
A test parameter set is created based on the past test results, a new optimal transmission parameter is determined from the test result, and the parameter is passed to the transmission control processing unit to execute a test, and a new test is performed. By reflecting the test results again on the index pattern and the optimal transmission parameter distribution, it is possible to learn the optimal transmission method for the index pattern in the characteristic communication state.

As described above, in the present embodiment, a test is performed for transmission quality parameters and patterns of a plurality of communication state monitoring indexes notified from the receiving terminal to the data server by setting transmission parameters within a certain range. By repeatedly determining the optimal transmission parameters,
It can provide a communication mechanism that performs optimal communication according to the environment, and its practical effect is great.

(Eighth Embodiment) FIG. 8 is a diagram showing a system configuration according to an eighth embodiment of the present invention. 8, the two-way communication terminal 801 is a packet communication network 103.
And transmits and receives audio information between similar terminals.

The two-way communication terminal 801 includes, in addition to ordinary data communication means such as the network interface 104, the data transmission unit 105, the data reception unit 106, and the packet generation unit 107, an application processing unit 802, a user data transmission / reception unit 803, A voice input section 804 such as a microphone for inputting voice, a voiced section detection section 805 for detecting a voiced section from voice input from the voice input section, and a code adapted to a packet communication network for transmitting voice data of a voiced section. Audio CODEC 806 for encoding / decoding, buffer 108 for temporarily storing the encoded output result from the audio CODEC, and reception buffer 10 for temporarily storing the output from data receiving unit 106
9. Stream restoration unit 122 for restoring the encoded data stream from the received data, adjusting the time axis of the reproduction data decoded by the CODEC, and reproducing the silence section into a natural reproduction sound by a comfortable noise generation function or the like. Adjusting section 807, audio output section 120 for outputting data resulting from adjustment by reproduction data adjusting section 807, input state from packet generating section 107 to data transmitting section 105, and input state from data receiving section 106 to receiving buffer 109 , A communication pattern determination unit 808 that grasps the voice transmission status between the plurality of terminals, and obtains determination results from the communication pattern determination unit 808,
The configuration includes a reception buffer control unit 809 that controls the reception buffer.

The operation of monitoring the transmission status between the two terminals and controlling the reception buffer for the two-way communication terminal configured as described above will be described below.

When the two-way communication terminal 801 is performing voice communication with another similar two-way terminal connected to the packet communication network 103, voice is input from the voice input unit 804, and A voiced section is detected, the sound data of the detected voiced section is compressed by the voice CODEC 806, and the encoded data output as a result is stored in the buffer 108.
The packet generation unit 107 reads out the packet, and adds a packet header including information such as a receiving terminal address, a time stamp indicating a transmission time, and a stream type. Via the packet communication network 103. When it is detected that the voiced section ends and the silent section is entered, a control packet for ending the voiced section is transmitted.

On the other hand, the received data is read from the network interface unit 104 to the data receiving unit 106,
Therefore, the packet header is separated. The payload data is temporarily stored in the reception buffer 109, and is restored to the original coded stream by the stream restoration unit 122.
The data is decoded at 806 and passed to the reproduction data adjustment unit 807. In the reproduction data adjustment unit 807, the audio CODEC 806
Adjust the time axis of the playback data decoded by
The silent section is corrected to a natural reproduced sound by processing such as generating comfortable noise and adding data. Data resulting from the adjustment by the reproduction data adjustment unit 807 is output from the audio output unit 120.

In this operation, the input data size from the packet generation unit 107 to the data transmission unit 105 and the input data size from the data reception unit 106 to the reception buffer 109 are input to the communication pattern determination unit 808.
The communication pattern determination unit 808 calculates the difference between the sums in a certain time interval of the data size transmitted by both,
The time change is monitored. This is shown in FIG.
The actual determination is performed as follows. The difference in the amount of transmission data between the terminals is obtained, the time change of the value is monitored, and the frequency of zero crossing is counted. As shown in FIG. 17a, when the frequency of zero crossing is large, it indicates a situation where speakers frequently change between terminal users. In such a situation, low delay is required to provide a smooth communication environment. You. Also, as shown in FIG.
When the frequency of the zero crossing is low, it indicates that the pattern is a one-sided utterance pattern from the user of one terminal. In such a scene, a very low delay is not required. In such a case, it is possible to increase the buffering amount and improve the quality of data reproduction.

Communication pattern determination unit 808
In step 9, the reception buffering amount is determined from the average jitter value and the like, and the maximum allowable delay time of the arrival of the reception packet is determined from the allowable packet loss rate when the distribution is followed, and buffering is performed. When the communication pattern determination unit 808 determines that the message is in a one-way state, the reception buffer control unit 809 increases the buffering amount from a normal value, thereby relaxing the maximum allowable delay time.

If the communication pattern determination unit 808 detects that the pattern has changed, the reception buffer control unit 809 changes the amount of buffering. Adjustment of the reproduction data is easy by increasing or decreasing the silent section length.
If buffer overflow and data starvation still occur, the reproduction data adjustment unit 807 adjusts the time axis of the reproduction data using a known speech speed conversion technique or the like. Further, in a silent section, a comfortable noise is generated and natural reproduction data is output.

Thus, when it is determined that the data transmission amount from the partner terminal is unilaterally larger than the data transmission amount from the own terminal in a plurality of points related to the same communication, the own terminal determines Can be controlled so as to suppress the packet loss due to the transmission delay, and the quality of media data transmission can be improved in a user terminal that does not require a low delay.

This embodiment can be applied to a case where communication is performed between a plurality of terminals. In the communication pattern determining unit 808, the data receiving unit 1
It is possible to know from the source address of the packet received at 06 how many terminals other than the own terminal are involved in the current communication. Assuming that the number of terminals transmitting packets other than the own terminal is N, the packet generation unit 107
The input data size to the data transmission unit 105 and the input data size to the reception buffer 109 from the data reception unit 106 are input to the communication pattern determination unit 808. The difference between the doubling time and the sum in a certain time interval of the data size transmitted by the terminal other than the own terminal is obtained, and the time change is monitored. As in the case between two terminals,
By monitoring the time change of this difference value, the frequency of zero crossing is counted, and when the frequency of zero crossing is large, it indicates a state in which speakers frequently change between terminals including the own terminal user, and such a state is shown. In the scene, the amount of buffering is reduced to provide a smooth communication environment, and when the frequency of zero crossing is small, it indicates that the user of the own terminal is turning to the listening side in communication, and in such a scene, the amount of buffering is And the quality of data reproduction can be improved.

As described above, in the present embodiment, the communication pattern is determined by monitoring the transmission state of each terminal in each bidirectional communication terminal, and it is determined that communication is not hindered even if a delay occurs. In the terminal, to provide a configuration that makes it possible to relax the delay conditions and improve the quality of data reception,
Its practical effect is great.

(Embodiment 9) FIG. 9 shows a system configuration of a ninth embodiment of the present invention. In FIG. 9, 104 to 109, 120, and 1 in the two-way communication terminal
Reference numerals 22, 802 to 808 have the same configuration as that of the eighth embodiment, and further include a cross interleave unit 901 and a deinterleave unit 902.

The operation of monitoring the transmission status between a plurality of terminals and performing data error control for the two-way communication terminal configured as described above will be described below.

The flow of data processing when the two-way communication terminal 801 performs voice communication with another similar two-way terminal is the same as in the eighth embodiment. Communication pattern judgment unit 8
In 08, the sum of the data size transmitted by each terminal in a certain time section is calculated, and the time change is monitored. FIG. 18 shows an example of the monitor data. Here, when the transmission amount from each terminal does not depend on a specific terminal as shown in FIG.
This indicates a situation where speakers frequently change between terminal users. In such a situation, low delay is required to provide a smooth communication environment.

In this case, even if some packet loss occurs, it is important to perform transmission and reception with as low a delay as possible, so that no cross interleave processing is performed. FIG.
When it is detected that only the transmission amount from one specific terminal protrudes and is large over a certain time period as shown in b, it indicates that the pattern is a unilateral utterance pattern from the user of one terminal. In such situations, low delay is not required. In such a case, cross interleave processing is performed.

If the communication pattern determination unit 808 detects that the pattern has changed, the cross interleave processing is turned on / off. As a result, the buffering amount is changed, but the buffering is increased. In such a case, the extension of the reproduction data requires flushing of the reproduction data to reduce the buffering amount. Therefore, in order to adjust the time axis of the reproduction data, the reproduction data adjustment unit 807 extends or deletes the data in the silent section and adjusts the time axis of the reproduction data. Alternatively, it is also possible to adjust the time axis of the reproduced data using a known speech speed conversion technique. In a silent section, a comfortable noise is generated, and reproduced data natural for human perception is output.

Thus, when it is determined that the amount of data transmission from a specific point is prominent among a plurality of points related to the same communication, the maximum allowable delay amount of reception at another point is increased. In addition, cross-interleave processing is performed on transmitted data, de-interleave processing is performed on the receiving side, packet discard due to transmission delay is suppressed, error correction performance is improved, and data recovery ability against continuous packet loss can be improved. In communication patterns where low delay is not required as the first priority, the quality of media data transmission can be improved, and even if continuous packet discarding or continuous packet loss occurs, it can be recovered to the extent corresponding to the added redundancy. it can.

As described above, in the present embodiment, when voice communication is performed between a plurality of terminals in each two-way communication terminal, the communication pattern is determined by monitoring the transmission state of each terminal, and the delay is determined. If it is determined that communication is not hindered even when the error occurs, an error correction process is added to provide a configuration that can improve the quality of data reception.

Although the first to ninth embodiments have been described with reference to the case where the media data is sound information, the present invention can be similarly applied to other media such as video information. In the present invention, 1
Although the embodiment in the case of transmitting media data of various types has been described, the present invention can be similarly applied to the case of transmitting a plurality of media simultaneously. The first to seventh
Although the first embodiment has been described in connection with the case where one-way media data is transmitted from the audio server to the receiving terminal, the same configuration can be applied to a two-way communication terminal. Also, in the first to seventh embodiments, a configuration has been described in which the audio server performs compression encoding when a request is made from a terminal. It is also possible to adopt a configuration in which the data is stored, predetermined data is taken out according to a request, packetized, and transmitted.

[0112]

As described above, according to the present invention, first,
On the receiving side, the loss rate of the transmitted packet is monitored, and the information is sent to the transmitting side. If the packet loss rate exceeds the predetermined threshold from the information, the transmitting side reduces the size of the transmitted packet. And resend it.
As a result, it is possible to reduce the influence of packet loss on reproduction.

Second, the receiving side monitors the loss rate of the packet to be transmitted, and sends the information to the transmitting side. On the transmitting side, if the packet loss rate exceeds the predetermined threshold from the information, The same packet is multiplexed. This makes it possible to minimize the probability that all data in a certain time section will be lost when a packet is lost. In addition, if the duplicated packets among the multiplexed packets are deleted on the receiving side, useless processing on the receiving side due to the multiplex reception can be easily eliminated.

Third, on the receiving side, the loss rate of the transmitted packet is monitored, and the information is sent to the transmitting side. On the transmitting side, if the packet loss rate exceeds the predetermined threshold from the information, The packet is divided and retransmitted. Accordingly, the receiving side monitors the loss rate of the transmitted packet and sends the information to the transmitting side. If the packet loss rate exceeds a predetermined threshold from the information, the transmitting side divides the packet. And resend it. As a result, it is possible to almost completely restore the reproduction data representing the entire corresponding time section.

Fourth, on the receiving side, fluctuations due to data delay and data loss called jitter are monitored, and the jitter and the buffering amount of the received data are controlled to make adjustments for reproducing the reproduced data. Things. As a result, the buffering amount can be set optimally with respect to the time-based transition of the delay amount, so that smooth communication control can be performed in response to a change in network state in addition to smooth reproduction.

Fifth, in the fifth aspect of the present invention, when the target data is a sound or a moving image having a time axis, the time axis is adjusted as an adjustment for reproducing the reproduction data. Is obtained.

Sixth, the receiving side monitors information such as packet discard due to arrival delay, packet loss during transmission, etc., and sends the information to the sending side. The sending side performs cross-interleaving from the information if necessary. And resend it,
The receiving side performs deinterleaving to reproduce data. As a result, packet discard / loss can be suppressed, error correction performance can be improved, and as a result, data recovery performance against continuous packet loss can be improved.

Seventh, on the receiving side, when reproducing the received data, the information on the reproduction quality is monitored, and the information is transmitted to the transmitting side, and the transmitting side stores and tests the information. Learning for improving reproduction quality is performed, and data transmission control is performed based on information obtained by the learning. Thereby, the optimal transmission method can be learned.

Eighth, between devices having a function of two-way communication, the respective transmission and reception amounts are monitored, and the respective buffering amounts are controlled accordingly. As a result, for a user terminal that does not require a low delay (high-speed bidirectional communication), the buffering amount is increased (packet discarding due to a transmission delay can be suppressed), so that data transmission quality can be improved. On the other hand, for a user terminal that requires a low delay (high-speed bidirectional communication), the data transmission speed can be improved in order to reduce the buffering amount. Further, in the eighth aspect, particularly when the target data is a sound or a moving image having a time axis, the time axis adjustment is performed as an adjustment for reproducing the reproduction data, and the same effect can be obtained.

Ninth, in the eighth invention, by performing interleaving (adding an error correction process) at the time of data transmission, data reception quality can be further improved.

[Brief description of the drawings]

FIG. 1 is a system configuration diagram showing a sound information providing apparatus according to a first embodiment of the present invention.

FIG. 2 is a system configuration diagram showing a sound information providing device according to a second embodiment of the present invention.

FIG. 3 is a system configuration diagram showing a sound information providing device according to a third embodiment of the present invention.

FIG. 4 is a system configuration diagram showing a sound information providing apparatus according to a fourth embodiment of the present invention.

FIG. 5 is a system configuration diagram showing a sound information providing device according to a fifth embodiment of the present invention.

FIG. 6 is a system configuration diagram showing a sound information providing device according to a sixth embodiment of the present invention.

FIG. 7 is a system configuration diagram showing a sound information providing apparatus according to a seventh embodiment of the present invention.

FIG. 8 is a system configuration diagram showing a bidirectional communication terminal according to an eighth embodiment of the present invention.

FIG. 9 is a system configuration diagram showing a bidirectional communication terminal according to a ninth embodiment of the present invention.

FIG. 10 is an example diagram of packet size control in the first embodiment of the present invention.

FIG. 11 is a diagram illustrating an example of the same packet transmission count control in the second embodiment of the present invention.

FIG. 12 is an example diagram of frame division control according to the third embodiment of the present invention.

FIG. 13 is a diagram illustrating buffering amount control according to a fourth embodiment of the present invention.

FIG. 14 is an example diagram of reproduction time control according to a fifth embodiment of the present invention.

FIG. 15 is a diagram illustrating an example of a cross interleave process according to the sixth embodiment of the present invention;

FIG. 16 is a flowchart showing a test execution procedure according to the seventh embodiment of the present invention.

FIG. 17 is an example diagram of communication pattern determination according to the eighth embodiment of the present invention.

FIG. 18 is an example of monitor data according to a ninth embodiment of the present invention.

FIG. 19 is a system configuration diagram showing a conventional sound information providing apparatus.

[Explanation of symbols]

Reference Signs List 101 audio server 102 receiving terminal 103 packet communication network 104, 104a, 104b network interface unit 105, 105a, 105b data transmitting unit 106, 106a, 106b data receiving unit 107 packet generating unit 108 buffer 109 receiving buffer 110 sound information managing unit 111 , 111a, 111b Multi-rate audio encoder 112 Communication status monitor information management unit 113 Transmission control unit 114 Data rate determination unit 115 Packet size determination unit 116 Transmission parameter notification unit 117 Packet size setting unit 120 Audio output unit 121, 121a, 121b Multi-rate Audio decoder 122 Stream restoration unit 123 Communication status monitoring unit 124 Throughput calculation unit 125 Packet loss rate Output unit 130 Transmission content management unit 131 Terminal request reception unit 132 Reception content setting unit 133 Terminal request generation unit 201 Transmission multiplex count determination unit 202 Transmission count counter 203 Duplicate packet deletion unit 301 Frame division determination unit 302 Frame division processing unit 303 Transmission stream Management unit 304 Frame combining unit 401 Discarded packet counting unit 402 Transmission jitter monitoring unit 403 Receive buffer control unit 404 Playback data adjustment unit 405 Buffering amount storage unit 406 Maximum allowable delay time storage unit 501 Speech speed conversion unit 601 Transmission mode determination unit 602 Continuous packet discard detection unit 603 Cross interleave unit 604 Deinterleave unit 605 Stream management unit 606 Reception control unit 701 Index analysis unit 702 Test execution unit 703 Index similar pattern search unit 7 4 Index pattern and optimal transmission parameter distribution storage unit 705 Test execution flag storage unit 706 Transmission control processing unit 710 Playback quality change detection unit 801 Two-way communication terminal 802 Application processing unit 803 User data transmission / reception unit 804 Microphone 805 Sound section detection unit 806 Audio CODEC 807 Playback data adjustment unit 808 Communication pattern determination unit 809 Reception buffer control unit 901 Cross interleave unit 902 Deinterleave unit 1901 Audio server 1902 Receiving terminal 1903 Packet communication network 1904a, 1904b Network interface 1905a, 1905b Data transmission unit 1906a, 1906b Data receiving unit 1907 Packet generating unit 1908 Buffer 1909 Receiving buffer 1910 Sound information Control section 1911 multi-rate audio encoder 1912 communication state monitor information management section 1913 transmission control section 1914 data rate determination section 1916 transmission parameter notification section 1920 audio output section 1921 multi-rate audio decoder 1922 stream restoration section 1923 communication state monitoring section 1924 throughput calculation section 1930 Transmission content management unit 1931 Terminal request reception unit 1932 Reception content setting unit 1933 Terminal request generation unit

Claims (24)

[Claims] 1. A medium that can communicate with a receiving terminal that receives and reproduces data via a network or the like, and that inputs and stores media data, and a medium that compresses data input and stored from the media input means. A data compression unit; a packet transmission unit that transmits the media data compressed by the media data compression unit as a packet; a reception unit that receives a packet loss rate; and a transmission size of the packet is determined according to the packet loss rate. A packet size determining unit, which receives a packet loss rate, which is one of the arrival states of packets transmitted from the packet transmitting unit, from the receiving terminal, and when the packet loss ratio exceeds a predetermined threshold,
A data transmitting apparatus characterized in that a packet size is divided into small pieces and transmitted again. 2. Media receiving means for receiving and reproducing data, capable of communicating via a network or the like, media input means for inputting and storing media data, and media for compressing data input and stored from the media input means. A data compression unit, a packet transmission unit that transmits the media data compressed by the media data compression unit as a packet, a reception unit that receives a packet loss rate, and a multiplex transmission number of the same packet according to the packet loss rate. Determining means for receiving, from the receiving terminal, a packet loss rate which is one of the arrival states of the packet transmitted from the packet transmitting means, and when the packet loss rate exceeds a specific threshold value, A data transmission device which controls so as to perform multiplex transmission. 3. A packet receiving means communicable with a data server for transmitting data via a network or the like, for receiving a data packet transmitted from the data server, and receiving data from the packet received by the packet receiving means. Decompression means, media data decompression means for receiving and decompressing the compressed data output from the decompression means, a packet loss rate calculation means for calculating a packet loss rate of packets received by the packet reception means,
A data receiving device comprising: a transmitting unit configured to transmit the packet loss rate to a data server. 4. The apparatus according to claim 3, further comprising a duplicate packet deleting unit for detecting and deleting a duplicate packet from the received packet.
A data receiving device according to claim 1. 5. A medium that can communicate with a receiving terminal that receives and reproduces data via a network or the like, and that inputs and stores media data, and a medium that compresses data input and stored from the media input means. A data compression unit; a packet transmission unit that transmits the media data compressed by the media data compression unit as a packet; a reception unit that receives a packet loss rate; Dividing means for dividing the stored data, receiving a packet loss rate, which is one of the arrival states of packets transmitted from the packet transmitting means, from the receiving terminal, and when the packet loss rate exceeds a predetermined threshold value. And a data transmitting apparatus for compressing and transmitting data divided by the dividing means. 6. The data receiving apparatus according to claim 3, further comprising data combining means for combining a plurality of pieces of decompressed data from the media data decompressing means into a state before division. 7. A jitter monitoring means for monitoring a jitter which is a time or a time change of a packet arrival delay, and a discarded packet information acquisition means for acquiring information on a packet which arrived beyond a permissible delay time and was discarded. Receiving buffer control means for controlling a buffering amount and an allowable value of delay time from the jitter and the discarded packet information, and performing an adjustment for smoothly reproducing data according to the controlled buffering amount 4. The data receiving apparatus according to claim 3, further comprising a reproduction data adjusting unit. 8. When the target data is audio or video,
8. The data receiving apparatus according to claim 7, wherein the reproduction data adjustment unit adjusts the reproduction time when changing the buffering amount according to an instruction from the reception buffer control unit. 9. A medium that can communicate with a receiving terminal that receives and reproduces data via a network or the like, and that inputs and stores media data, and a medium that compresses data input and stored from the media input means. A data compression unit; a packet transmission unit that transmits the media data compressed by the media data compression unit as a packet; a reception unit that receives packet discard information; A transmission mode determining means for determining whether or not the transmission data is transmitted, and a cross interleaving means for performing a cross interleaving process on the transmission data. The packet discard information, which is one of the arrival states, is It receives from Shin terminal, if it is determined that for cross interleaving, data transmission apparatus characterized by sending packets of data subjected to cross-interleaving processing. 10. A data receiving apparatus which can communicate with a data server for transmitting data via a network or the like, receives a data packet sent from the data server, and outputs data from the packet received by the packet receiving means. Restoration means for restoring the compressed data, media data decompression means for receiving and decompressing the compressed data output from the decompression means, and information on packets that arrived beyond the permissible delay time and were discarded, or during transmission. Packet discard information management means for managing packet discard information which is information relating to packet loss, transmitting means for transmitting the packet discard information to a data server, and, when received data is subjected to cross-interleave processing, A data receiving apparatus comprising: deinterleaving means for performing deinterleaving. 11. A medium that can communicate with a receiving terminal that receives and reproduces data via a network or the like, and that inputs and stores media data, and a medium that compresses data that is input and stored from the media input means. Data compression means, packet transmission means for transmitting the media data compressed by the media data compression means into packets, reception means for receiving reproduction quality information, and transmission method for controlling a transmission method based on the reproduction quality information Control means, and packet data sent from the packet sending means,
Receiving reproduction quality information, which is information relating to reproduction quality in the case of reproduction on the receiving terminal side, from the receiving terminal, performing test transmission for determining transmission parameters based on the reproduction quality information, and storing the result. A data transmission device that analyzes and learns an optimal transmission method for a communication state. 12. A data receiving device that can communicate with a data server that transmits data via a network or the like, receives a data packet transmitted from the data server, and converts a packet received from the packet receiving device into data. , A media data decompression means for receiving and decompressing the compressed data output from the decompression means, and a reproduction quality or reproduction quality receiving information on the quality of the reproduction data output from the media data decompression means. A data receiving apparatus comprising: reproduction quality information generating means for generating reproduction quality information indicating a change in the reproduction quality; and transmission means for transmitting reproduction quality change information to a data server. 13. A medium input means for inputting / accumulating media data, which is capable of two-way communication via a packet communication network or the like, and detecting that a state of a medium has changed from said media input means. Media state change detecting means for performing data compression from media input means only when a change is detected by the media data state change detecting means, and receiving and expanding compressed data; Packet sending means for sending the media data compressed by the data compressing / decompressing means into packets, packet receiving means for receiving an arbitrary packet, and recovering the data from the packet received by the packet receiving means to obtain the media data Restoration means for inputting to the compression / decompression means and transmission from the packet sending means A communication pattern determining unit that monitors a transmission state of the received packet and a reception state of a packet from another point received by the packet receiving unit, and receives the packet based on a determination result of the communication pattern determining unit. A reception buffer control unit for controlling a buffering amount; and a reproduction data adjustment unit for performing adjustment for smoothly reproducing data in accordance with the controlled buffering amount, among a plurality of terminals involved in the same communication. When it is determined that the data transmission amount from the partner terminal is larger than a predetermined reference compared to the data transmission amount from the own terminal, the reception maximum allowable delay amount at the own terminal is increased, and A data transmission apparatus characterized in that control is performed to suppress packet discard. 14. When the target data is audio or video, the reproduction data adjustment means adjusts the reproduction time of the reproduction data when changing the buffering amount in accordance with an instruction from the reception buffer control means. The data transmission device according to claim 13. 15. A specific terminal among a plurality of points related to the same communication, comprising: a cross interleave means for performing cross interleave processing on transmission data; and a deinterleave means for performing deinterleave processing on reception data. When it is determined that the amount of data transmission from the terminal is prominent, the maximum reception delay amount is increased in all terminals, and cross-interleave processing is performed on transmission data, and de-interleave processing is performed in reception processing. 14. The method according to claim 13, wherein when it is determined that increasing the delay does not hinder communication, packet discarding due to transmission delay is suppressed, error correction performance is improved, and data recovery capability against continuous packet loss is improved. A data transmission device according to claim 1. 16. Connecting to a packet communication network,
A data server having a network interface and transmitting real-time data to a predetermined address;
A receiving terminal connected to the packet communication network to receive data from the data server and immediately reproduce the data; the data server compresses data from the media data supplying unit and the data from the media data supplying unit Media data compression means, a packet transmission means for packetizing the media data compressed by the media data compression means and transmitting the packet to a packet communication network, and receiving the arrival status report data of the packet transmitted from the packet transmission means from a receiving side. A communication status monitor information managing means for receiving and managing communication status monitor information, wherein the receiving terminal receives a data packet sent from the data server from a packet communication network, and receives the packet. Packet received by A stream restoring unit for restoring a data stream, media data decompressing means for receiving and decompressing compressed data output from the stream restoring unit, and checking a communication state by analyzing a header of a packet received by the packet receiving means. Communication state monitoring means for performing communication, and a communication state monitor data transmitting means for returning a monitoring result of the receiving side communication state monitoring means to a data server, wherein the communication state monitoring means of the receiving terminal calculates a packet loss rate. A packet loss rate calculator, notifying the data server of the packet loss rate by the communication state monitor data transmitting means, and the data server further comprising a means of determining a transmission packet size based on the notified packet loss rate data; , When the packet loss rate exceeds a predetermined threshold, Real-time data transmission apparatus and controls to transmit and fence dividing. 17. Connect to a packet communication network,
A data server having a network interface and transmitting real-time data to a predetermined address;
A receiving terminal connected to the packet communication network, receiving data from the data server and immediately reproducing the data, wherein the data server transmits each media data supply unit and data from the media data supply unit. Media data compression means for compressing, packet transmission means for packetizing the media data compressed by the media data compression means and transmitting the packet to a packet communication network, and receiving state report data of the packet transmitted from the packet transmission means on the receiving side And a communication state monitor information managing unit for managing communication state monitor information, wherein the receiving terminal receives a data packet sent from the data server from a packet communication network, and the packet Packet received by receiving means A stream restorer for restoring the data stream from the preparative, a media data expanding means for expanding by receiving the compressed data output from the stream restorer,
A receiving-side communication state monitoring unit that analyzes a header of a packet received by the packet receiving unit and checks a communication state; and a data transmission unit that returns a result of monitoring by the communication state monitoring unit to a data server. In the communication state monitoring means of the receiving terminal, a packet loss rate calculation unit for calculating a packet loss rate is provided, and the packet loss rate is notified to the data server by the communication state monitor data transmission means, and the data server side is notified. Means for determining the number of multiplex transmissions of the same packet according to the packet loss rate data, and the receiving terminal further comprising: a duplicate packet removing means for detecting and removing a duplicate packet from the received packet, wherein the packet loss rate is a specific threshold value. Characterized in that the same packet is multiplexed when it exceeds Real-time data transmission device. 18. Connecting to a packet communication network,
A data server having a network interface and transmitting real-time data to a predetermined address;
A receiving terminal connected to the packet communication network, receiving data from the data server and immediately reproducing the data, wherein the data server transmits each media data supply unit and data from the media data supply unit. A multi-stage output rate compatible media data compression means for inputting and compressing according to a multi-stage output rate, and transmitting the media data compressed by the multi-stage output rate compatible media data compression means to a packet communication network as a packet A packet sending unit, a communication status monitor information managing unit that receives a packet arrival status report data transmitted from the packet sending unit from a receiving side, and manages communication status monitor information; and a throughput data from the communication status monitor information managing unit. Media data sent by Transmission control means for determining the data rate of
The compression ratio in the media data compression unit can be switched to a plurality of stages by control data from the transmission control unit. The receiving terminal receives a data packet sent from the data server from a packet communication network. Means, a stream restoring unit for restoring a data stream from a packet received by the packet receiving unit, a media data decompressing unit for receiving and decompressing compressed data output from the stream restoring unit, and a stream received by the packet receiving unit. A receiving-side communication state monitoring unit that analyzes a packet header to check a communication state; and a data transmitting unit that returns a monitoring result of the receiving-side communication state monitoring unit to a data server. The monitor calculates a packet loss rate. A packet loss rate calculation unit, which notifies the data server of the packet loss rate by the communication state monitor data transmitting means, and instructs the data server whether to divide the media data into frames based on the notified packet loss rate data. The transmission mode determination unit to be determined and the switching method from the transmission mode determination unit are used to switch the processing method of the frame so that each of the divided partial frames has a reduced data resolution over the entire time section of the frame. Further comprising: a transmission frame processing unit having a function of dividing the data into transmission data, and a transmission stream management unit for receiving a switching data from the transmission mode determination unit and adding a packet header for compressed data transmission that differs depending on a frame transmission mode switched. The terminal checks the header of the received packet, A stream recovery unit for determining a system transmission mode, a function of the stream restoring unit to input a plurality of divided streams to a plurality of the media data decompression units, and a process of combining decompressed data from the plurality of media data decompression units. A receiving frame processing unit, wherein when the packet loss rate exceeds a predetermined threshold, the frame of the media data is divided into a plurality of pieces so that each piece of data reduces the overall resolution, and data per frame is divided. A real-time data transmission device that controls transmission in a plurality of packets. 19. Connecting to a packet communication network,
A data server having a network interface and transmitting real-time data to a predetermined address;
A receiving terminal connected to the packet communication network, receiving data from the data server and immediately reproducing the data, wherein the data server transmits each media data supply unit and data from the media data supply unit. Media data compression means for compressing, packet transmission means for packetizing the media data compressed by the media data compression means and transmitting the packet to a packet communication network, and receiving state report data of the packet transmitted from the packet transmission means on the receiving side And a communication state monitor information managing unit for managing communication state monitor information, wherein the receiving terminal receives a data packet sent from the data server from a packet communication network, and the packet Packet received by receiving means Stream restoring unit for restoring a data stream from the stream, media data decompression means for receiving and decompressing compressed data output from the stream decompression unit, and analyzing the header of the packet received by the packet receiving means to determine the communication state. A receiving-side communication state monitoring unit for checking, a data transmitting unit for returning a monitoring result of the receiving-side communication state monitoring unit to a data server, and a communication state monitoring unit of the receiving terminal; A transmission jitter monitoring unit that calculates a statistic of a jitter amount in a certain time interval and monitors a time change thereof; a short-term transmission deterioration detection unit that detects a packet arriving beyond the maximum allowable delay time and discarded; When the jitter amount information is output from the jitter monitor, the buffering amount and the maximum allowable A receiving buffer control unit having a function of determining the delay time and storing the current value of the buffering amount and the maximum allowable delay time; and Reproduction data adjustment means for adjusting the time axis.In normal times, the buffering amount on the receiving side and the maximum allowable delay time for the arrival of the received packet are determined based on the output value from the transmission jitter monitor section, For the output from the transmission jitter calculator that shows a temporary change, the buffering amount on the receiving side is slowly changed, and for the short-term change, the buffering amount on the receiving side is changed abruptly. When the degraded state is eliminated, the buffering amount is returned to the original buffering amount determined based on the output value from the transmission jitter calculating unit, and the time change of the delay amount is taken into consideration. A real-time data transmission device for controlling the buffering amount to be set optimally. 20. Assume that the sound data is sound data, and that the reproduction data adjustment means for adjusting the time axis of the reproduction data when changing the buffering amount in accordance with an instruction from the reception buffer control means has a speech speed conversion processing unit. 20. The real-time data transmission device according to claim 19, wherein: 21. Connecting to a packet communication network,
A data server having a network interface and transmitting real-time data to a predetermined address;
A receiving terminal connected to the packet communication network, receiving data from the data server and immediately reproducing the data, wherein the data server transmits each media data supply unit and data from the media data supply unit. Media data compression means for compressing, packet transmission means for packetizing the media data compressed by the media data compression means and transmitting the packet to a packet communication network, and receiving state report data of the packet transmitted from the packet transmission means on the receiving side And a communication state monitor information managing unit for managing communication state monitor information, wherein the receiving terminal receives a data packet sent from the data server from a packet communication network, and the packet Packet received by receiving means Stream restoring unit for restoring a data stream from the stream, media data decompression means for receiving and decompressing compressed data output from the stream decompression unit, and analyzing the header of the packet received by the packet receiving means to determine the communication state. Receiving-side communication state monitoring means for checking, data transmission means for returning a monitoring result of the receiving-side communication state monitoring means to a data server, and a maximum allowable delay time exceeding an output value of the communication state monitoring means of the receiving terminal. A continuous packet discard detection unit for detecting that packet discards due to the arrival of the packet or packet loss on the transmission path have occurred continuously. Transmission mode determination unit that determines whether or not to perform cross interleaving A cross-interleave section for performing cross-interleave processing on transmission data in response to switching data from the transmission mode determination section, a transmission stream management section for adding a header indicating that the data has been subjected to interleave processing, A deinterleaving unit that performs deinterleaving processing on the received data, and a reproduction data adjustment unit that adjusts the time axis of the reproduction data when turning on / off the interleaving processing according to an instruction from the reception buffer control unit. When it is detected that continuous discarding of packets has occurred, cross-interleave processing is performed on transmission data, deinterleaving processing is performed on the receiving side, packet loss due to transmission delay is suppressed, and error correction is performed. Data recovery for continuous packet loss with improved performance Real-time data transmission apparatus characterized by improving the capability. 22. Connecting to a packet communication network,
A data server having a network interface and transmitting real-time data to a predetermined address;
A receiving terminal connected to the packet communication network, receiving data from the data server and immediately reproducing the data, wherein the data server transmits each media data supply unit and data from the media data supply unit. Media data compression means for compressing, packet transmission means for packetizing the media data compressed by the media data compression means and transmitting the packet to a packet communication network, and receiving state report data of the packet transmitted from the packet transmission means on the receiving side Communication status monitor information management means for managing communication status monitor information, and a transmission control processing unit for controlling a transmission method, wherein the reception terminal performs packet communication of a data packet transmitted from the data server. Packet receiving means for receiving from the network; A stream restoration unit for restoring a data stream from the packet received by the packet reception unit; a media data decompression unit for receiving and decompressing compressed data output from the stream decompression unit; and a header of the packet received by the packet reception unit Receiving communication state monitoring means for analyzing communication status and checking communication state, data transmitting means for returning a plurality of indices obtained from the result of monitoring by the communication state monitoring means to a data server, and output from the media data decompression means. Playback quality change detection means for receiving the media playback data quality information and detecting the playback quality change, and returns the playback quality change information to the data server together with the plurality of indexes. Multiple indexes of received communication status monitoring results An index analysis unit that analyzes and determines the need to change the current transmission parameters, a test execution unit that performs tests to determine the optimal transmission parameters, and index patterns that were registered during past tests An index pattern and an optimum parameter distribution storage unit for recording the distribution of the optimal parameters obtained as a result of the test, and a test execution flag storage unit for storing a test execution flag indicating whether or not the test is currently being executed. When the reproduction quality is degraded, a test parameter set is created based on the past test results, the parameters are passed to the transmission control processing unit, a test is performed, and optimal transmission is performed based on the test results. After determining the parameters, the new test results are additionally recorded in the index pattern and the optimal parameter distribution storage unit again, The index pattern and optimal parameter distribution storage unit classify the index patterns into clusters based on vector distances, accumulate the optimal parameter distribution for each index pattern with a characteristic communication state, and learn the optimal transmission method for the communication state. Characteristic real-time data transmission device. 23. Connecting to a packet communication network,
A network interface, having a function as a two-way communication terminal for transmitting and receiving real-time data to and from one or more locations via a packet communication network, each media data supply means, and data from the media data supply means From, media state change detecting means for detecting that the state of the media has changed, and compressing data from the media data supply means only when a change is detected by the media data state change detecting means,
Media data compression / expansion receiving and expanding compressed data
Decompression means, packet transmission means for packetizing the media data compressed by the media data compression / decompression means and transmitting the packet to the packet communication network, packet reception means for receiving a data packet from the packet communication network, and packet reception Means for restoring a data stream from a packet received by said means and inputting said data stream to said media data compression / decompression means, comprising: a transmission state of a packet transmitted from said packet transmission means; A communication pattern determining unit that monitors a reception state of a packet from another point received by the packet receiving unit and determines a communication pattern, and a reception that determines a reception buffering amount based on a determination result of the communication pattern determining unit. Buffer system Means,
Reproduction data adjustment means for adjusting the time axis of the reproduction data when the buffering amount is changed according to an instruction from the reception buffer control means, among a plurality of terminals involved in the same communication, data from the own terminal. When it is determined that the data transmission amount from the partner terminal is unilaterally larger than the transmission amount, the maximum reception delay amount at the own terminal is increased to suppress packet discarding at the receiving terminal due to transmission delay. Real-time data transmission device characterized in that it is controlled in such a manner. 24. A cross interleave section for performing cross interleave processing on transmission data, a transmission stream management section for adding a header indicating that the data has been subjected to interleave processing, and a deinterleave processing for receiving data. If it is determined that the amount of data transmission from a specific terminal is prominent among multiple points involved in the same communication, the maximum allowable delay for reception is increased for all terminals. Also, only when it is determined that cross-interleave processing is performed on transmitted data and de-interleave processing is performed on received data, and that increasing delay does not hinder communication, packet discard due to transmission delay is suppressed and error correction is performed. Improves performance and data for continuous packet loss Real-time data transmission apparatus according to claim 23, characterized in that to improve recoverability.