CN117411891A - A data transmission method, data transmission device and data transmission system - Google Patents

Abstract

A data transmission method, the sender divides the data of the object file into a plurality of data segments. And a plurality of data transmission links are established between the transmitting end and the receiving end. When data transmission is carried out between a transmitting end and a receiving end, the transmitting end utilizes a stream compression technology, and each data transmission link compresses the read data in real time while reading the data segment to obtain a compressed data stream. The transmitting end transmits the compressed data stream in real time. The receiving end decompresses the compressed data stream to obtain the original data while receiving the compressed data stream by using a stream decompression technology. And the receiving end writes the original data in each data transmission link into the memory in real time according to the storage position of each data segment in the memory. The transmitting end and the receiving end can detect the running environment in the data transmission process, and the number of the data transmission links is adjusted in real time according to the factors of communication bandwidth, calculation power of a processor and the like, so that the optimal transmission efficiency is achieved between the transmitting end and the receiving end.

Description

A data transmission method, data transmission device and data transmission system

Technical field

The present invention relates to the technical field of data transmission, and in particular, to a data transmission method, a data transmission device and a data transmission system.

Background technique

With the development of computer technology, the data that computers need to process is getting larger and larger. During the data transmission process, if the data transmitted by the computer is relatively large, the data transmission time will be longer. Therefore, how to improve the data transmission speed of computers is an urgent problem that needs to be solved.

Contents of the invention

In order to solve the above problems, embodiments of the present application provide a data transmission method. The sending end divides the data of the target file into multiple data segments. Multiple data transmission links are established between the sender and the receiver. When data is transmitted between the sender and the receiver, the sender uses streaming compression technology. Each data transmission link reads the data segment while compressing the read data in real time to obtain a compressed data stream. The sender sends the compressed data stream in real time. The receiving end uses streaming decompression technology to receive the compressed data stream and decompress the compressed data stream to obtain the original data. The receiving end writes the original data in each data transmission link into the memory in real time according to the storage location of the data segment transmitted by each data transmission link in the memory. This application segments the data of the target file and transmits each data segment in parallel, which can improve data transmission efficiency. In addition, each data transmission link compresses and transmits the read data in real time, which not only reduces the total amount of data transmission, but also does not require additional memory to store compressed packages, reducing hardware costs.

To this end, the following technical solutions are adopted in the embodiments of this application:

In the first aspect, this application provides a data transmission method. The method is executed by the sending end, including: receiving a data transmission request instruction sent by the receiving end. The data transmission request instruction carries the identification of the target file; and determining based on the identification of the target file. target file, and divide the data of the target file into N data segments, where N is a positive integer greater than 1; send segmentation information to the receiving end. The segmentation information includes the data length of the target file, the number of data segments, and each The data length of the data segment and the offset of each data segment. The data length of the target file instructs the receiving end to determine the target storage space of the target file. The number of data segments instructs the receiving end to establish N data transmission links. The data of each data segment The length and offset of each data segment instruct the receiving end to determine the storage location of the data segment in the target storage space; receive data segment storage request instructions sent by N data transmission links; use the streaming compression algorithm to read each data segment And compress the read data to obtain a compressed data stream; send the compressed data stream to the receiving end through N data transmission links.

In one implementation, before receiving the data segment storage request instructions sent by N data transmission links, it also includes: the sending end configures N sub-processors according to the number of data segments, the sending end includes a processor, and the processor configures N sub-processors, the sub-processors are used to read the data of the data segment and compress the read data.

In one implementation, dividing the target file into N data segments specifically includes: determining the data length of the target file, and dividing the data of the target file into N data segments.

In one implementation, dividing the target file into N data segments specifically includes: determining the number of communication channels between the sending end and the receiving end according to the communication bandwidth between the sending end and the receiving end; Quantity, determines the number of data segments that the target file's data is divided into.

In one implementation, a streaming compression algorithm is used to read each data segment and compress the read data to obtain a compressed data stream, which specifically includes: after the first sub-processor receives the first data segment storage request instruction, Read the data of the first data segment, the N sub-processors include the first sub-processor, the N data segments include the first data segment, the N data transmission links include the first data transmission link, and the first data segment stores The request instruction is a storage request instruction for the data segment transmitted by the first data transmission link; when the first sub-processor reads out the data of the set length of the first data segment, it compresses the data of the set length to obtain the first To compress the data stream, the set length is smaller than the length of the first data segment.

In one implementation, it also includes: detecting the communication bandwidth between the sending end and the receiving end; when the communication bandwidth is less than a first set threshold, reducing the number of sub-processors on the sending end, and the first set threshold is to allocate N The communication bandwidth of a communication channel.

In one implementation, the method further includes: sending a first feedback instruction to the receiving end, where the first feedback instruction instructs the receiving end to reduce the number of established data transmission links.

In one implementation, the method further includes: detecting the computing power of the processor in the sending end; when the computing power of the processor is greater than a second set threshold, reducing the number of sub-processors at the sending end, and the second set threshold is The computing power of the processor cannot be configured with the minimum computing power of N sub-processors.

In one implementation, the method further includes: sending a second feedback instruction to the receiving end, where the second feedback instruction instructs the receiving end to reduce the number of configured sub-processors.

In the second aspect, this application provides a data transmission method. The method is executed by the receiving end and includes: sending a data transmission request instruction to the sending end. The data transmission request instruction instructs the sending end to divide the data of the target file into N data segments. N is a positive integer greater than 1; receive the segmentation information sent by the sending end. The segmentation information includes the data length of the target file, the number of data segments, the data length of each data segment and the offset of each data segment; according to the data segment number, establish N data transmission links; send a data segment storage request instruction to the sending end through N data transmission links, and the data segment storage request instruction instructs the sending end to read and send N data segments; the receiving sending end passes N The compressed data stream sent by each data transmission link; the stream decompression algorithm is used to decompress the compressed data stream, and the data transmitted by each data transmission link is stored in the set storage location in the target storage space.

In one implementation, N data transmission links are established according to the number of data segments, which specifically includes: the receiving end configures N sub-processors according to the number of data segments, the receiving end includes a processor, and the processor configures N sub-processors The sub-processor is used to decompress the compressed data stream and write the decompressed data to the set storage location in the target storage space.

In one implementation, a streaming decompression algorithm is used to decompress the compressed data stream, and the data transmitted by each data transmission link is stored in a set storage location, specifically including: according to the data length of the target file, Determine the target storage space for storing the target file in the memory; determine the storage location of each data segment in the target storage space according to the data length of each data segment and the offset of each data segment; when the second sub-processor passes the second After receiving the second compressed data stream of the second data segment, the data transmission link decompresses the second compressed data stream to obtain the original data; the N sub-processors include the second sub-processor, and the N data transmission links include the Two data transmission links, N data segments include the second data segment, the second compressed data stream is the compressed data stream obtained by the sending end compressing the second data segment of the set length read, and the original data is read by the sending end Set the length of the second data segment, the length of the original data is less than the length of the second data segment; the second sub-processor writes the original data into the storage space of the second data segment, and the target storage space includes the storage of the second data segment space.

In one implementation, it also includes: detecting the communication bandwidth between the receiving end and the sending end; when the communication bandwidth is less than a first set threshold, reducing the number of sub-processors on the receiving end, and the first set threshold is to allocate N The communication bandwidth of a communication channel.

In one implementation, the method further includes: sending a first feedback instruction to the sending end, where the first feedback instruction instructs the sending end to reduce the number of configured sub-processors.

In one embodiment, the method further includes: detecting the computing power of the processor in the receiving end; when the computing power of the processor is greater than a second set threshold, reducing the number of sub-processors at the receiving end, and the second set threshold is The computing power of the processor cannot be configured with the minimum computing power of N sub-processors.

In one implementation, the method further includes: sending a second feedback instruction to the sending end, where the second feedback instruction instructs the sending end to reduce the number of configured sub-processors.

In a third aspect, the present application provides a data transmission device, including: a transceiver, configured to receive a data transmission request instruction sent by the receiving end, where the data transmission request instruction carries an identification of the target file; a processor, configured to perform the data transmission according to the target file identification, determine the target file, and divide the data of the target file into N data segments, where N is a positive integer greater than 1; the transceiver is also used to send segmented information to the receiving end, and the segmented information includes the target file. The data length, the number of data segments, the data length of each data segment and the offset of each data segment. The data length of the target file instructs the receiving end to determine the target storage space of the target file. The number of data segments instructs the receiving end to establish N Data transmission link, the data length of each data segment and the offset of each data segment instruct the receiving end to determine the storage location of the data segment in the target storage space; receive data segment storage request instructions sent by N data transmission links; process The transceiver is also used to use the streaming compression algorithm to read each data segment and compress the read data to obtain a compressed data stream; the transceiver is also used to send the compressed data stream to the receiving end through N data transmission links.

In one implementation, the processor is further configured to configure N sub-processors according to the number of data segments, and the sub-processors are configured to read the data of the data segments and compress the read data.

In one implementation, the processor is specifically configured to determine the data length of the target file, and divide the data of the target file into N data segments.

In one implementation, the processor is specifically configured to determine the number of communication channels between the sending end and the receiving end according to the communication bandwidth between the sending end and the receiving end; and determine the number of the target file according to the number of communication channels. The number of data segments the data is divided into.

In one implementation, the processor is specifically configured to read the data of the first data segment after the first sub-processor receives the first data segment storage request instruction. The N sub-processors include the first sub-processor, The N data segments include the first data segment, the N data transmission links include the first data transmission link, the first data segment storage request instruction is a data segment storage request instruction transmitted by the first data transmission link; and when the first After reading the data of the set length of the first data segment, the sub-processor compresses the data of the set length to obtain a first compressed data stream, and the set length is smaller than the length of the first data segment.

In one embodiment, the processor is also configured to detect the communication bandwidth between the sending end and the receiving end; and when the communication bandwidth is less than a first set threshold, reduce the number of sub-processors on the sending end, the first setting The threshold is the communication bandwidth allocated to N communication channels.

In one implementation, the transceiver is further configured to send a first feedback instruction to the receiving end, where the first feedback instruction instructs the receiving end to reduce the number of data transmission links established.

In one embodiment, the processor is also configured to detect the computing power of the processor in the sending end; and when the computing power of the processor is greater than the second set threshold, reduce the number of sub-processors in the sending end, and the second Set the threshold to the minimum computing power of N sub-processors that cannot be configured by the processor's computing power.

In one implementation, the transceiver is further configured to send a second feedback instruction to the receiving end, and the second feedback instruction instructs the receiving end to reduce the number of configured sub-processors.

In the fourth aspect, the present application provides a data transmission device, including: a transceiver for sending a data transmission request instruction to the sending end. The data transmission request instruction instructs the sending end to divide the data of the target file into N data segments, N is a positive integer greater than 1; receives the segmentation information sent by the sending end. The segmentation information includes the data length of the target file, the number of data segments, the data length of each data segment and the offset of each data segment; the processor uses It is used to establish N data transmission links according to the number of data segments; the transceiver is also used to send data segment storage request instructions to the sending end through N data transmission links, and the data segment storage request instructions instruct the sending end to read and send N data segments; receive the compressed data stream sent by the sending end through N data transmission links; the processor is also used to use the streaming decompression algorithm to decompress the compressed data stream and transfer the data transmitted by each data transmission link. The data is stored in the set storage location in the target storage space.

In one implementation, the processor is further configured to configure N sub-processors according to the number of data segments. The sub-processors are used to decompress the compressed data stream and write the decompressed data into the device in the target storage space. in the specified storage location.

In one implementation, the processor is specifically configured to determine a target storage space for storing the target file in the memory according to the data length of the target file; and determine each data segment based on the data length of each data segment and the offset of each data segment. The storage location of the data segment in the target storage space; when the second sub-processor receives the second compressed data stream of the second data segment through the second data transmission link, it decompresses the second compressed data stream to obtain the original data ; N sub-processors include second sub-processors, N data transmission links include second data transmission links, N data segments include second data segments, and the second compressed data stream sets a length for the sending end to read The compressed data stream obtained by compressing the second data segment. The original data is the second data segment with a set length read by the sending end. The length of the original data is smaller than the length of the second data segment; the second sub-processor writes the original data. into the storage space of the second data segment, and the target storage space includes the storage space of the second data segment.

In one embodiment, the processor is also configured to detect the communication bandwidth between the receiving end and the sending end; and when the communication bandwidth is less than a first set threshold, reduce the number of sub-processors on the receiving end, the first setting The threshold is the communication bandwidth allocated to N communication channels.

In one implementation, the transceiver is further configured to send a first feedback instruction to the sending end, where the first feedback instruction instructs the sending end to reduce the number of configured sub-processors.

In one embodiment, the processor is also configured to detect the computing power of the processor in the receiving end; and when the computing power of the processor is greater than the second set threshold, reduce the number of sub-processors at the receiving end, and the second Set the threshold to the minimum computing power of N sub-processors that cannot be configured by the processor's computing power.

In one implementation, the transceiver is further configured to send a second feedback instruction to the sending end, and the second feedback instruction instructs the sending end to reduce the number of configured sub-processors.

In a fifth aspect, this application provides a data transmission system, including: at least one sending end that may be implemented as in the first aspect and at least one receiving end that may be implemented in the first aspect; wherein, at least one sending end and at least A communication connection is established between receivers.

In a sixth aspect, the present application provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed in a computer, the computer is caused to perform the methods that may be implemented in each of the first and second aspects.

In a seventh aspect, the present application provides a computer program product, which is characterized in that the computer program product stores instructions, and when the instructions are executed by a computer, the instructions enable the computer to implement the methods that may be implemented in each of the first and second aspects.

Description of the drawings

The drawings needed to be used in the description of the embodiments or the prior art are briefly introduced below.

Figure 1 is a schematic architectural diagram of a data transmission system provided in an embodiment of the present application;

Figure 2 is a schematic diagram of the hardware architecture of the data transmission system provided in the embodiment of the present application for data transmission;

Figure 3 is a schematic flow chart of data transmission between the sending end and the receiving end provided in the embodiment of the present application.

Figure 4 is an architectural schematic diagram of a data transmission device provided in an embodiment of the present application;

FIG. 5 is an architectural schematic diagram of a data transmission device provided in an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application.

The term "and/or" in this article is an association relationship that describes related objects, indicating that there can be three relationships. For example, A and/or B can mean: A exists alone, A and B exist simultaneously, and B exists alone. these three situations. The symbol "/" in this article indicates that the associated object is or, for example, A/B means A or B.

The terms "first", "second", etc. in the description and claims herein are used to distinguish different objects, rather than to describe a specific order of objects. For example, the first response message and the second response message are used to distinguish different response messages, but are not used to describe a specific sequence of response messages.

In the embodiments of this application, words such as "exemplary" or "for example" are used to represent examples, illustrations or explanations. Any embodiment or design described as "exemplary" or "such as" in the embodiments of the present application is not to be construed as preferred or advantageous over other embodiments or designs. Rather, use of the words "exemplary" or "such as" is intended to present the concept in a concrete manner.

In the description of the embodiments of this application, unless otherwise stated, the meaning of “multiple” refers to two or more, for example, multiple processing units refers to two or more processing units, etc.; multiple Component refers to two or more components, etc.

In order to improve the data transmission efficiency of the computer, a solution has been proposed in the prior art: the computer can divide the data to be transmitted into multiple data packets. The computer calls up multiple communication channels and transmits different data packets respectively. During the data transmission process, although the computer shortens the data transmission time, the total amount of data transmitted remains unchanged. It only uses more communication channels and does not improve the computer's data transmission efficiency.

A second solution has been proposed in the prior art, which is: the computer can divide the data transmitted with the belt into multiple data packets. The computer compresses multiple data packets separately to obtain multiple compressed packets. The computer calls up multiple communication channels and transmits different data packets respectively. After existing computers compress multiple divided data packets into multiple compressed packets, they simultaneously transmit the compressed packets through multiple communication channels, which can improve data transmission efficiency. However, after the computer compresses the data packet into a compressed package, it requires additional memory to store the compressed package, which increases the hardware cost. In addition, during the data transmission process, the computer compresses the data package and decompresses the compressed package, which requires a lot of time. Therefore, the data transmission process of the computer may take longer, and the data transmission efficiency of the computer cannot be improved.

In order to solve the deficiencies in the existing technology, this application provides a data transmission method, device and system. In this embodiment of the present application, after the computer determines the data packet to be transmitted, it divides the data packet to be transmitted into multiple sub-data packets. The computer calls up a corresponding number of communication channels according to the number of sub-data packets. When the computer reads the data in the sub-packet, after the computer reads out the data of the set size, it transmits the data of the set size to the streaming compressor. After receiving the data of the set size, the streaming compressor compresses the data of the set size to obtain a compressed package. The streaming compressor sends the compressed packet to the receiving end through the communication channel.

After receiving the compressed packet, the receiving end transmits the compressed packet to the streaming compressor. After receiving the compressed packet, the streaming compressor decompresses the compressed packet and obtains the original data. The streaming compressor sends the raw data to the memory in the receiving end, where it is stored.

In this application, during the data transmission process, the computer uses a streaming compressor to compress the data to be transmitted while reading it. The computer compresses data of a set size and does not need to compress data packets, which can reduce the use of additional memory. In addition, during the data transmission process of this application, the computer does not need to compress and decompress the data packets, which can save a lot of time and improve the data transmission efficiency of the computer.

Figure 1 is a schematic architectural diagram of a data transmission system provided in an embodiment of the present application. The data transmission system shown in Figure 1 includes a sending end 100 and a receiving end 200. Among them, the sending end 100 and the receiving end 200 can be cloud servers, laptops, tablets, smart phones, smart watches, smart bracelets, music players, audio players, U disks, removable hard drives, cars, base stations, outdoor Cabinets, drones and other equipment.

The sending end 100 includes a processor 110, a memory 120, a transceiver 130 and a bus 140. Among them, the processor 110, the memory 120 and the transceiver 130 establish communication connections through the bus 140 to realize data transmission between the processor 110, the memory 120 and the transceiver 130.

The receiving end 200 includes a processor 210, a memory 220, a transceiver 230 and a bus 240. Among them, the processor 210, the memory 220 and the transceiver 230 establish communication connections through the bus 240 to realize data transmission between the processor 210, the memory 220 and the transceiver 230.

The processor may be a general purpose processor or a special purpose processor. For example, the processor may include a central processing unit (CPU) and/or a baseband processor. For example, the processor 110 of the sending end 100 can read the data stored in the memory 120 of the sending end 100 according to the received request instruction. For example, the processor 210 of the receiving end 200 can send a request instruction to a specific device according to requirements to obtain the set data.

The memory may store a program (which may also be an instruction or code), and the program may be run by the processor, causing the processor to perform the data transmission method described in this solution. Optionally, data may also be stored in the memory. For example, the processor 110 can read data stored in the memory 120, and the data can be stored at the same storage address as the program, or the data can be stored at a different storage address from the program. In this application, the processor 110 and the memory 120 in the sending end 100 can be set up separately or integrated together. For example, integrated on a single board or system on chip (SOC). Similarly, the processor 210 and the memory 220 in the receiving end 200 can be set separately or integrated together.

The transceiver can realize the input (reception) and output (transmission) of signals. For example, the transceiver may include a transceiver or a radio frequency chip. The transceiver may also include a communication interface. For example, the sending end 100 can receive the request instruction sent by the receiving end 200 through the transceiver 130 . The sending end 100 can send the data stored in the memory 120 to the receiving end 200 through the transceiver 130 . For example, the receiving end 200 may send a request instruction to the sending end 100 through the transceiver 230 . The receiving end 200 can receive the data stored in the memory 120 sent by the transmitting end 100 through the transceiver 230 .

In this application, a communication connection is established between the transceiver 130 in the sending end 100 and the transceiver 230 in the receiving end 200 to realize data transmission between the sending end 100 and the receiving end 200. The communication method between the transceiver 130 and the transceiver 230 may be 2G/3G/4G/5G, wireless fidelity (Wi-Fi), Bluetooth (bluetooth, BT), ZigBee, etc.

It can be understood that the structure illustrated in the embodiment of the present application does not limit the specific structures of the sending end 100 and the receiving end 200 . In other embodiments of the present application, the sending end 100 and the receiving end 200 may include more or less components than shown in the figures, or combine some components, or split some components, or arrange different components. The components illustrated may be implemented in hardware, software, or a combination of software and hardware.

Figure 2 is a schematic diagram of the hardware architecture of the data transmission system provided in the embodiment of the present application for data transmission. The data transmission system shown in Figure 2 includes a sending end 100 and a receiving end 200. The processor 110 in the sending end 100 includes N sub-processors 111. The processor 210 in the receiving end 200 includes N sub-processors 211. N communication channels are established between the sending end 100 and the receiving end 200. Each sub-processor 111 in the sending end 100 is connected to a communication channel respectively. Each sub-processor 211 in the receiving end 200 is connected to a communication channel respectively.

In this application, a sub-processor 111 in the processor 110, a communication channel between the processor 110 and the processor 210, and a sub-processor 211 in the processor 210 may form a data transmission link. The N data transmission links in the data transmission system do not affect each other. Among them, the sub-processor 111 stores a streaming compression algorithm, which is used to compress the data while reading the data. The sub-processor 211 stores a streaming decompression algorithm, which is used to decompress data while receiving data.

After the sending end 100 divides the data of the target file into N data segments, it sends segmentation information such as the data length of the target file, the number of data segments, the data length of each data segment, and the offset of each data segment to the receiving end. 200. The receiving end 200 allocates storage space in its own memory 220 based on the segmentation information, and determines the storage location of each data segment.

During the data transmission process, the sub-processor 111 in the sending end 100 reads the data segments in the target file stored in the memory 130 . A sub-processor 111 reads the data of a data segment. During the data reading process of the data segment, the sub-processor 111 uses a streaming compression algorithm to compress the read data in real time to obtain a compressed data stream. The sub-processor 111 sends the compressed data stream to the receiving end 200 through the communication channel. After receiving the compressed data stream, the sub-processor 211 in the receiving end 200 decompresses the compressed data stream in real time using a streaming decompression algorithm to obtain the original data stream. The sub-processor 211 writes the original data stream into the storage space of the corresponding data segment in the memory 220.

Figure 3 is a schematic flowchart of data transmission between the sending end and the receiving end provided in the embodiment of the present application. As shown in Figure 3, the data transmission process between the sending end 100 and the receiving end 200 is as follows:

Step S301, the receiving end 200 sends a data transmission request instruction to the sending end 100. The data transfer request instruction is used to obtain the target file stored in the sending end 100 .

In this application, the application scenarios for data transmission between the receiving end 200 and the sending end 100 may include application (APP) downloading, data file transmission, picture transmission, transaction information transmission, etc. Among them, the sending end 100 can be a cloud server, a base station, a smartphone, or other device that stores data. The receiving end 200 may be a terminal device such as a user's smartphone, tablet computer, or smart watch.

Take data file transfer as an example. When the receiving end 200 needs a target file, the receiving end 200 can automatically find a device storing the target file and establish a communication connection with the device. The receiving end 200 can also passively establish a communication connection with the device storing the target file under the manual operation of the user. After the communication connection is established between the receiving end 200 and the sending end 100, a request instruction for obtaining the target file is sent to the sending end 100, allowing the sending end 100 to determine the target file to be sent.

Step S302: The sending end 100 determines the target file according to the data transmission request instruction, and divides the target file into N data segments according to the set rules. Among them, N is a positive integer greater than 1.

Specifically, the data transfer request instruction generally carries the identifier of the target file. After receiving the data transmission request instruction, the sending end 100 detects whether the target file is stored in its own memory 120 according to the identification of the target file. If the sending end 100 detects that there is no target file in the memory 120, the sending end 100 will send a termination request to the receiving end 200 to terminate data transmission between the receiving end 200 and the sending end 100. If the sending end 100 detects that the target file is stored in the memory 120, the sending end 100 will preprocess the target file. In this application, the sending end 100 divides the data of the target file into multiple data segments during the preprocessing process. After the sending end 100 divides the data of the target file into multiple data segments, the multiple data segments are transmitted in parallel through multiple communication channels, thereby improving the data transmission efficiency between the sending end 100 and the receiving end 200.

The number of data segments the sending end 100 divides into data segments is related to the communication bandwidth between the sending end 100 and the receiving end 200 . Among them, the greater the communication bandwidth between the sending end 100 and the receiving end 200, the greater the number of data segments the sending end 100 divides the target file into. The smaller the communication bandwidth between the sending end 100 and the receiving end 200, the smaller the number of data segments the sending end 100 divides the target file into.

In one embodiment, if the target file is composed of N independent data segments, the sending end 100 may not need to split the target file. During the data transmission process, the sending end 100 can send N independent data segments to the receiving end 200 through N communication channels. In other embodiments, if the number n of data segments in the target file is smaller than the set number N, the sending end 100 can divide the data segments with relatively long data lengths, and divide the data of the target file into the set number N. data segment.

In one embodiment, after determining the target file, the sending end 100 obtains the data length of the target file. The sending end 100 divides the data of the target file into N pieces according to the number N of data segments and the data length of the target file. For example, the data length of the target file is Nx kb. Among them, the data length of 1kb-x kb is used as the first data segment, the data length of (x+1)kb-2x kb is used as the second data segment, and the data length of (2x+1)kb-3x kb is used as the second data segment. data segment, and so on for other data segments.

Step S303: The sending end 100 sends the segmentation information of the target file to the receiving end 200. Among them, the segmentation information carries information such as the data length of the target file, the number of data segments, the data length of each data segment, and the offset of each data segment.

Specifically, after the sending end 100 divides the data of the target file into N data segments, it records the data length of the target file, the number of data segments, the memory of each data segment, the offset of each data segment, and other parameters. The sending end 100 sends parameters such as the data length of the target file, the number of data segments, the data length of each data segment, and the offset of each data segment to the receiving end 200 as segmentation information.

The offset of the data segment refers to the position of the data length of the data segment in the data length of the target file. For example, the data length of the target file is Nx kb. The data length of the first data segment is x kb, then the offset of the first data segment is 1kb. The data length of the second data segment is x kb, then the offset of the second data segment is (x+1)kb. The data length of the third data segment is x kb, then the offset of the third data segment is (2x+1)kb. The offsets of other data segments are similarly deduced.

The sending end 100 may configure a corresponding number of sub-processors 111 in the processor 110 according to the number of data segments. The sub-processor 111 stores a streaming compression algorithm for compressing data while reading the data. A sub-processor 111 reads and compresses data for a data segment.

Step S304: The receiving end 200 establishes multiple data transmission links according to the segmentation information.

Specifically, after receiving the segmentation information, the receiving end 200 parses out parameters such as the data length of the target file, the number of data segments, the data length of each data segment, and the offset of each data segment in the segmentation information. The receiving end 200 configures a corresponding number of sub-processors 211 in the processor 210 according to the number of data segments. The sub-processor 211 stores a streaming decompression algorithm, which is used to decompress data while receiving data. A sub-processor 211 decompresses a file and writes it to memory.

The receiving end 200 establishes N communication channels between the receiving end 200 and the sending end 100 according to the number of data segments and the communication bandwidth between the receiving end 200 and the sending end 100. A sub-processor 111, a communication channel and a sub-processor 211 form a data transmission link. A data transmission link transmits one data segment. Optionally, during data transmission, if the communication bandwidth between the sending end 100 and the receiving end 200 is reduced, the number of communication channels is reduced. At this time, the sending end 100 can reduce the number of sub-processors 111 and reduce the running memory of the processor 110. The receiving end 200 can reduce the number of sub-processors 211 and reduce the running memory of the processor 210. In other embodiments, other communication connections can be established between the receiving end 200 and the sending end 100 to increase the communication bandwidth between the sending end 100 and the receiving end 200 and ensure that there are N communication channels between the sending end 100 and the receiving end 200.

According to the data length of the target file, the receiving end 200 occupies part of the blank memory in its own memory 220 in advance as a target storage space for storing the target file. The receiving end 200 determines the storage location of each data segment in the target storage space based on the memory of each data segment and the offset of each data segment. For example, the target storage space has Nx kb of memory. When the data length of the first data segment is x kb, the offset of the first data segment is 1kb, and the storage location of the first data segment is 1kb-x kb. The data length of the second data segment is x kb, the offset of the second data segment is (x+1)kb, and the storage location of the second data segment is (x+1)kb-2x kb. The data length of the third data segment is x kb, the offset of the third data segment is (2x+1)kb, and the storage location of the third data segment is (2x+1)kb-3x kb. The storage locations of other data segments are similarly deduced.

Step S305: The receiving end 200 sends a data segment storage request instruction to the sending end 100 through each data transmission link.

Specifically, when data is transmitted between the sending end 100 and the receiving end 200, the sending end 100 transmits data in parallel through multiple data transmission links. Therefore, when the receiving end 200 requests the sending end 100 to send a data segment, it needs to send a data segment storage request instruction through each data transmission link to trigger each data transmission link to perform data transmission.

In this application, multiple data transmission links between the sending end 100 and the receiving end 200 can transmit data simultaneously, which can improve the data transmission efficiency of the data transmission system. When the sending end 100 and the receiving end 200 transmit data, the data transmission links are not affected by each other, which can improve the stability of data transmission between the sending end 100 and the receiving end 200. Optionally, when the sending end 100 does not receive a data segment storage request instruction from a data transmission link, the data transmission link is in a disconnected state.

Step S306: The sending end 100 allows the sub-processors 111 in each data transmission link to read the data segments according to the storage request instructions of each data segment.

Specifically, the data segment storage request instruction carries the identifier of the data segment. When the sending end 100 receives the data segment storage request instruction sent by the data transmission link, the sub-processor 111 in the data transmission link parses the data segment storage request instruction to obtain the data segment identification. The sub-processor 111 determines the data segment in the target file according to the data segment identifier, and reads the data segment in the memory 120 . After the sending end 100 receives the data segment storage request instructions sent by multiple data transmission links, the sub-processor 111 in each data transmission link will read the corresponding data segment in the memory 120 to implement multiple Data segments are transferred in parallel.

In step S307, the sub-processor in the sending end 100 uses a streaming compression algorithm to compress the data of the read data segment to obtain a compressed data stream.

Streaming compression algorithm refers to the technology of real-time compression of read data. The streaming compression algorithm processes the original data stream into a compressed stream to obtain a compressed data stream. Compressed data streams are different from common compressed packages. The compressed data stream compresses the data read by the sub-processor 111 in real time, removes redundant information in the data, and uses the shortest encoding to save the most complete data information. Compared with existing compressed packets, the compressed data stream has smaller memory and can be transmitted in the communication channel. In other embodiments, the streaming compression technology may be deflate compression algorithm, LZ77 compression algorithm, huffman compression algorithm, etc., which is not limited in this application.

In this application, when the sub-processor 111 reads the data of the data segment in the memory, the sub-processor 111 reads the data and uses a stream compression algorithm to compress the read data to obtain a compressed data stream. For example, take a sub-processor 111 as an example. When the sub-processor 111 reads the data of the data segment, after the sub-processor 111 reads the data of the set length, the stream compressor in the sub-processor 111 compresses the data of the set length to obtain compression. data flow. The sub-processor 111 transmits the compressed data stream to the transceiver 130 and sends it to the receiving end 200 through the communication channel. In other embodiments, the set length may be tens of kb, hundreds of kb, or other values, which is not limited in this application.

Step S308: The sending end 100 sends the compressed data stream to the receiving end 200 through each data transmission link.

Normally, when data is transmitted between the sending end 100 and the receiving end 200, the data of the data segment or the data of the data segment compressed packet are transmitted. In this application, the sub-processor 111 compresses the data of the data segment in real time to obtain multiple compressed data streams. When transmitting data through the data transmission link, the sending end 100 sends the compressed data stream to the receiving end 200 in time sequence. The data length of multiple compressed data streams is smaller than the data length of the data segment and the data length of the data segment compression package. When the sending end 100 transmits a compressed data stream, it can reduce the amount of transmitted data and improve data transmission efficiency.

In step S309, after the receiving end 200 receives the compressed data packet through each data transmission link, the sub-processor 211 in each data transmission link decompresses the compressed data packet using a streaming decompression algorithm to obtain original data.

The receiving end 200 receives the compressed data stream sent by the sending end 100 in real time through various data transmission links. Compressed data stream is a data packet obtained by compressing the data of the data segment. The receiving end 200 cannot directly store the compressed data stream in its own memory 220 . Each sub-processor 211 in the receiving end 200 is provided with a streaming decompressor. After receiving the compressed data stream, the stream compressor in the sub-processor 211 decompresses the compressed data stream and restores the compressed data stream to original data.

The streaming decompression algorithm refers to the technology of real-time decompression of the read compressed data stream. The streaming decompression algorithm processes the compressed packet into the original data stream to obtain the original data. In other embodiments, the streaming compression technology may be deflate decompression algorithm, LZ77 decompression algorithm, huffman decompression algorithm, etc., which is not limited in this application.

In step S310, each sub-processor 211 in the receiving end 200 writes the original data into the storage location of the corresponding data segment in the target storage space.

In the receiving end 200, after the sub-processors 211 in each data transmission link decompress the original data in real time, each sub-processor 211 determines the storage location of the data segment in the memory 211 according to the identification of the data segment. The sub-processor 211 stores the original data segments in the storage space of the memory 211 in real time, thereby transferring the data in the sending end 100 to the receiving end 200.

In this embodiment of the present application, the sending end 100 divides the target file into multiple data segments. Multiple data transmission links are established between the sending end 100 and the receiving end 200. When data is transmitted between the sending end 100 and the receiving end 200, the sending end 100 uses streaming compression technology. Each data transmission link reads the data segment while compressing the read data in real time to obtain a compressed data stream. The sending end 100 sends the compressed data stream in real time. The receiving end 200 uses streaming decompression technology to receive the compressed data stream and decompress the compressed data stream to obtain original data. The receiving end 200 writes the original data in each data transmission link into the memory in real time according to the storage location of the data segment transmitted by each data transmission link in the memory. This application segments the data of the target file and transmits each data segment in parallel, which can improve data transmission efficiency. In addition, each data transmission link compresses and transmits the read data in real time, which not only reduces the total amount of data transmission, but also does not require additional memory to store compressed packages, reducing hardware costs.

In this application, during the data transmission process of the sending end 100, the processor 110 will detect its own operating environment, such as detecting the communication bandwidth, the computing power of the processor, etc. In one case, the processor 110 detects that the communication bandwidth between the transceiver 130 and the transceiver 230 of the receiving end 200 is reduced, the communication bandwidth is less than a set threshold, and the number of communication channels between the sending end 100 and the receiving end 200 reduce. At this time, the processor 110 can reduce the number of sub-processors 111 according to the current number of communication channels to prevent the processor 110 from being in a high running memory state. Optionally, the processor 110 sends feedback information to the receiving end 200 to allow the processor 210 in the receiving end 200 to reduce the number of sub-processors 211 to prevent the processor 210 from being in a high computing power state.

In one case, the processor 110 detects that it is in a high computing power state, the processor's computing power is greater than a set threshold, and the processor 110 cannot maintain a sufficient number of sub-processors 111 running. If the number of communication channels between the sending end 100 and the receiving end 200 remains unchanged, the processor 110 can allow part of the data segments to be transmitted through the data transmission link, and let the other part of the data segments be sent to the receiving end 200 through normal data transmission. middle. Optionally, the processor 110 sends feedback information to the receiving end 200 so that the processor 210 in the receiving end 200 also maintains the same number of sub-processors 211 running to avoid wasting the computing power of the processor 210. In other embodiments, the processor 110 detects that its own computing power is reduced, and the processor's computing power is less than a set threshold. The processor 110 increases the number of sub-processors 111 to allow each data segment to transmit data through the data transmission link.

In this application, during the data transmission process of the receiving end 200, the processor 210 will detect its own operating environment, such as detecting the communication bandwidth, the computing power of the processor, etc. In one case, the processor 210 detects that the communication bandwidth between the transceiver 230 and the transceiver 130 of the sending end 100 is reduced, the communication bandwidth is less than a set threshold, and the number of communication channels between the sending end 100 and the receiving end 200 reduce. At this time, the processor 110 can reduce the number of sub-processors 111 according to the current number of communication channels to prevent the processor 210 from being in a high running memory state. Optionally, the processor 210 sends feedback information to the sending end 100 to allow the processor 110 in the sending end 100 to reduce the number of sub-processors 111 to prevent the processor 110 from being in a high computing power state.

In one case, the processor 210 detects that it is in a high computing power state, the processor's computing power is greater than a set threshold, and the processor 210 cannot maintain a sufficient number of sub-processors 211 running. If the number of communication channels between the sending end 100 and the receiving end 200 remains unchanged, the processor 210 can allow part of the data segments in the sending end 100 to be transmitted through the data transmission link, and allow other part of the data segments to be transmitted through the normal data transmission method. sent to the receiving end 200. Optionally, the processor 210 sends feedback information to the sending end 100 so that the processor 110 in the sending end 100 also maintains the same number of sub-processors 111 running to avoid wasting the computing power of the processor 110. In other embodiments, the processor 210 detects that its own computing power is reduced, and the processor's computing power is less than a set threshold. The processor 210 increases the number of sub-processors 211 to allow each data segment to transmit data through the data transmission link.

In the embodiment of this application, during the data transmission process, the sending end 100 and the receiving end 200 will detect their own operating environments, such as detecting communication bandwidth, processor computing power, etc., based on factors such as communication bandwidth and processor computing power. , adjust the number of data transmission links between the sending end 100 and the receiving end 200 in real time to achieve optimal transmission efficiency between the sending end 100 and the receiving end 200.

FIG. 4 is a schematic structural diagram of a data transmission device provided in an embodiment of the present application. The data transmission device 400 shown in Figure 4 includes a processor 401 and a transceiver 402. Among them, the specific working processes of the processor 401 and the transceiver 402 are as follows:

In one implementation, the transceiver 402 is configured to receive a data transfer request instruction sent by the receiving end, and the data transfer request instruction carries an identifier of the target file; the processor 401 is configured to determine the target file according to the identifier of the target file, and transfer the The data of the target file is divided into N data segments, where N is a positive integer greater than 1; the transceiver 401 is also used to send segmentation information to the receiving end. The segmentation information includes the data length of the target file, the number of data segments, The data length of each data segment and the offset of each data segment. The data length of the target file instructs the receiving end to determine the target storage space of the target file. The number of data segments instructs the receiving end to establish N data transmission links. The data length of each data segment The data length and the offset of each data segment instruct the receiving end to determine the storage location of the data segment in the target storage space; receive data segment storage request instructions sent by N data transmission links; the processor 401 is also used to utilize streaming compression The algorithm reads each data segment and compresses the read data to obtain a compressed data stream; the transceiver is also used to send the compressed data stream to the receiving end through N data transmission links.

In one implementation, the processor 401 is also configured to configure N sub-processors according to the number of data segments, and the sub-processors are configured to read the data of the data segments and compress the read data.

In one implementation, the processor 401 is specifically configured to determine the data length of the target file and divide the data of the target file into N data segments.

In one implementation, the processor 401 is specifically configured to determine the number of communication channels between the sending end and the receiving end according to the communication bandwidth between the sending end and the receiving end; and determine the number of the target file according to the number of communication channels. The number of data segments the data is divided into.

In one implementation, the processor 401 is specifically configured to read the data of the first data segment after the first sub-processor receives the first data segment storage request instruction. The N sub-processors include the first sub-processor, The N data segments include the first data segment, the N data transmission links include the first data transmission link, the first data segment storage request instruction is a data segment storage request instruction transmitted by the first data transmission link; and when the first After reading the data of the set length of the first data segment, the sub-processor compresses the data of the set length to obtain a first compressed data stream, and the set length is smaller than the length of the first data segment.

In one implementation, the processor 401 is also used to detect the communication bandwidth between the sending end and the receiving end; and when the communication bandwidth is less than a first set threshold, reduce the number of sub-processors on the sending end, the first setting The threshold is the communication bandwidth allocated to N communication channels.

In one implementation, the transceiver 402 is also configured to send a first feedback instruction to the receiving end, and the first feedback instruction instructs the receiving end to reduce the number of established data transmission links.

In one implementation, the processor 401 is also configured to detect the computing power of the processor in the sending end; and when the computing power of the processor is greater than the second set threshold, reduce the number of sub-processors in the sending end, and the second Set the threshold to the minimum computing power of N sub-processors that cannot be configured by the processor's computing power.

In one implementation, the transceiver 402 is also configured to send a second feedback instruction to the receiving end, and the second feedback instruction instructs the receiving end to reduce the number of configured sub-processors.

FIG. 5 is an architectural schematic diagram of a data transmission device provided in an embodiment of the present application. The data transmission device 500 shown in Figure 5 includes a processor 501 and a transceiver 502. Among them, the specific working processes of the processor 501 and the transceiver 502 are as follows:

In one implementation, the transceiver 502 is used to send a data transfer request instruction to the sending end. The data transfer request instruction instructs the sending end to divide the data of the target file into N data segments, where N is a positive integer greater than 1; receive and send The segmentation information sent by the client, the segmentation information includes the data length of the target file, the number of data segments, the data length of each data segment and the offset of each data segment; the processor 501 is used to establish N according to the number of data segments. data transmission links; the transceiver 502 is also used to send a data segment storage request instruction to the sending end through N data transmission links, and the data segment storage request instruction instructs the sending end to read and send N data segments; the receiving sending end passes The compressed data stream sent by N data transmission links; the processor 501 is also used to decompress the compressed data stream using a streaming decompression algorithm, and store the data transmitted by each data transmission link in the device in the target storage space. in the specified storage location.

In one implementation, the processor 501 is also configured to configure N sub-processors according to the number of data segments. The sub-processors are used to decompress the compressed data stream and write the decompressed data into the device in the target storage space. in the specified storage location.

In one implementation, the processor 501 is specifically configured to determine a target storage space for storing the target file in the memory based on the data length of the target file; determine each data segment based on the data length and offset of each data segment. The storage location of the data segment in the target storage space; when the second sub-processor receives the second compressed data stream of the second data segment through the second data transmission link, it decompresses the second compressed data stream to obtain the original data ; N sub-processors include second sub-processors, N data transmission links include second data transmission links, N data segments include second data segments, and the second compressed data stream sets a length for the sending end to read The compressed data stream obtained by compressing the second data segment. The original data is the second data segment with a set length read by the sending end. The length of the original data is smaller than the length of the second data segment; the second sub-processor writes the original data. into the storage space of the second data segment, and the target storage space includes the storage space of the second data segment.

In one implementation, the processor 501 is also used to detect the communication bandwidth between the receiving end and the sending end; and when the communication bandwidth is less than a first set threshold, reduce the number of sub-processors on the receiving end, the first setting The threshold is the communication bandwidth allocated to N communication channels.

In one implementation, the transceiver 502 is also configured to send a first feedback instruction to the sending end, and the first feedback instruction instructs the sending end to reduce the number of configured sub-processors.

In one implementation, the processor 501 is also configured to detect the computing power of the processor in the receiving end; and when the computing power of the processor is greater than the second set threshold, reduce the number of sub-processors at the receiving end, and the second Set the threshold to the minimum computing power of N sub-processors that cannot be configured by the processor's computing power.

In one implementation, the transceiver 502 is also configured to send a second feedback instruction to the sending end, and the second feedback instruction instructs the sending end to reduce the number of configured sub-processors.

Embodiments of the present application also provide a computer-readable storage medium on which a computer program is stored. When the computer program is executed in a computer, the computer is caused to execute the steps described in the above-mentioned Figures 2 to 4 and corresponding descriptions. Either method.

The embodiments of the present application also provide a computer program product. The computer program product stores instructions. When the instructions are executed by a computer, the computer implements the instructions described in the above-mentioned Figures 2 to 4 and corresponding descriptions. Either method.

Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented with electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of the embodiments of the present application.

Additionally, various aspects or features of embodiments of the present application may be implemented as methods, apparatus, or articles of manufacture using standard programming and/or engineering techniques. The term "article of manufacture" as used in this application encompasses a computer program accessible from any computer-readable device, carrier or medium. For example, computer-readable media may include, but are not limited to: magnetic storage devices (eg, hard disks, floppy disks, tapes, etc.), optical disks (eg, compact discs (CD), digital versatile discs (DVD), etc. ), smart cards, and flash memory devices (e.g., erasable programmable read-only memory (EPROM), cards, sticks, or key drives, etc.). Additionally, the various storage media described herein may represent one or more devices and/or other machine-readable media for storing information. The term "machine-readable medium" may include, but is not limited to, wireless channels and various other media capable of storing, containing and/or carrying instructions and/or data.

In the above embodiments, the data transmission 400 in Figure 4 and the data transmission 500 in Figure 5 can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another, e.g., the computer instructions may be transferred from a website, computer, server, or data center Transmission to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line) or wireless (such as infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains one or more available media integrated. The usable media may be magnetic media (eg, floppy disks, hard disks, magnetic tapes), optical media (eg, high-density digital video discs (DVD)), or semiconductor media (eg, solid state disks). ,SSD)), etc.

It should be understood that in the various embodiments of the present application, the size of the sequence numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its functions and internal logic, and should not be The implementation process of the embodiments of this application does not constitute any limitations.

Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, devices and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be described again here.

In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or can be integrated into another system, or some features can be ignored, or not implemented. On the other hand, the coupling or direct coupling or communication connection between each other shown or discussed may be through some interfaces, and the indirect coupling or communication connection of the devices or units may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or they may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of the present application are essentially or the part that contributes to the existing technology or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium , including several instructions to cause a computer device (which may be a personal computer, a server, or an access network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the embodiments of this application. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk and other media that can store program code. .

The above are only specific implementation modes of the embodiments of the present application, but the protection scope of the embodiments of the present application is not limited thereto. Any person familiar with the technical field can easily implement the implementation within the technical scope disclosed in the embodiments of the present application. Any changes or substitutions that come to mind should be included in the protection scope of the embodiments of this application.

Claims (35)

1. A data transmission method, wherein the method is performed by a transmitting end, and comprises:

receiving a data transmission request instruction sent by a receiving end, wherein the data transmission request instruction carries an identifier of a target file;

determining the target file according to the identification of the target file, and dividing the data of the target file into N data segments, wherein N is a positive integer greater than 1;

transmitting segmentation information to the receiving end, wherein the segmentation information comprises the data length of the target file, the number of the data segments, the data length of each data segment and the offset of each data segment, the data length of the target file indicates the receiving end to determine the target storage space of the target file, the number of the data segments indicates the receiving end to establish N data transmission links, and the data length of each data segment and the offset of each data segment indicate the receiving end to determine the storage position of the data segment in the target storage space;

Receiving data segment storage request instructions sent by N data transmission links;

reading each data segment by using a stream compression algorithm and compressing the read data to obtain a compressed data stream;

and transmitting the compressed data stream to the receiving end through the N data transmission links.

2. The method of claim 1, further comprising, prior to said receiving the data segment storage request instruction sent by the N data transmission links:

the transmitting end configures N sub-processors according to the number of the data segments, the transmitting end comprises a processor, the processor configures the N sub-processors, and the sub-processors are used for reading the data of the data segments and compressing the read data.

3. The method according to any one of claims 1 or 2, wherein the dividing the target file into N data segments specifically comprises:

and determining the data length of the target file, and uniformly dividing the data of the target file into the N data segments.

4. The method according to any one of claims 1 or 2, wherein the dividing the target file into N data segments specifically comprises:

Determining the number of communication channels between the sending end and the receiving end according to the communication bandwidth between the sending end and the receiving end;

and determining the number of data segments of the target file according to the number of the communication channels.

5. The method according to any one of claims 2-4, wherein said reading said respective data segments and compressing the read data using a stream compression algorithm to obtain a compressed data stream, in particular comprising:

after receiving a first data segment storage request instruction, a first sub-processor reads data of a first data segment, wherein the N sub-processors comprise the first sub-processor, the N data segments comprise the first data segment, the N data transmission links comprise a first data transmission link, and the first data segment storage request instruction is a data segment storage request instruction transmitted by the first data transmission link;

and after the first sub-processor reads the data with the set length of the first data segment, compressing the data with the set length to obtain a first compressed data stream, wherein the set length is smaller than the length of the first data segment.

6. The method of any one of claims 1-5, further comprising:

detecting the communication bandwidth between the sending end and the receiving end;

and when the communication bandwidth is smaller than a first set threshold, reducing the number of the sub-processors of the sending end, wherein the first set threshold is used for distributing the communication bandwidths of N communication channels.

7. The method as recited in claim 6, further comprising:

and sending a first feedback instruction to the receiving end, wherein the first feedback instruction instructs the receiving end to reduce the number of established data transmission links.

8. The method of any one of claims 1-7, further comprising:

detecting the computing power of a processor in the transmitting end;

and when the computing power of the processor is larger than a second set threshold, reducing the number of the sub-processors of the sending end, wherein the second set threshold is the minimum computing power of the processor, and the computing power of the processor cannot be configured for N sub-processors.

9. The method as recited in claim 8, further comprising:

and sending a second feedback instruction to the receiving end, wherein the second feedback instruction instructs the receiving end to reduce the number of configuration sub-processors.

10. A method for data transmission, the method being performed by a receiving end and comprising:

transmitting a data transmission request instruction to a transmitting end, wherein the data transmission request instruction instructs the transmitting end to divide the data of the target file into N data segments, and N is a positive integer greater than 1;

receiving segmentation information sent by the sending end, wherein the segmentation information comprises the data length of the target file, the number of the data segments, the data length of each data segment and the offset of each data segment;

establishing N data transmission links according to the number of the data segments;

transmitting a data segment storage request instruction to the transmitting end through the N data transmission links, wherein the data segment storage request instruction indicates the transmitting end to read and transmit the N data segments;

receiving compressed data streams sent by the sending end through the N data transmission links;

and decompressing the compressed data stream by using a stream decompression algorithm, and storing the data transmitted by each data transmission link in a set storage position in a target storage space.

11. The method according to claim 9, wherein said establishing N data transmission links according to the number of data segments, comprises:

The receiving end configures N sub-processors according to the number of the data segments, the receiving end comprises a processor, the processor configures the N sub-processors, and the sub-processors are used for decompressing the compressed data stream and writing the decompressed data into a set storage position in the target storage space.

12. The method according to any one of claims 10 or 11, wherein said decompressing the compressed data stream using a streaming decompression algorithm and storing the data transmitted by each data transmission link in a set storage location, specifically comprises:

determining the target storage space for storing the target file in a memory according to the data length of the target file;

determining storage positions of the data segments in the target storage space according to the data length of the data segments and the offset of the data segments;

after receiving a second compressed data stream of a second data segment through a second data transmission link, the second sub-processor decompresses the second compressed data stream to obtain original data; the N sub-processors include the second sub-processor, the N data transmission links include the second data transmission link, the N data segments include the second data segment, the second compressed data stream is a compressed data stream obtained by compressing the second data segment with the set length read by the transmitting end, the original data is the second data segment with the set length read by the transmitting end, and the length of the original data is smaller than the length of the second data segment;

The second sub-processor writes the original data into a storage space of the second data segment, the target storage space including the storage space of the second data segment.

13. The method according to any one of claims 10-12, further comprising:

detecting the communication bandwidth between the receiving end and the transmitting end;

and when the communication bandwidth is smaller than a first set threshold, reducing the number of the sub-processors of the receiving end, wherein the first set threshold is used for distributing the communication bandwidths of N communication channels.

14. The method as recited in claim 13, further comprising:

and sending a first feedback instruction to the sending end, wherein the first feedback instruction instructs the sending end to reduce the number of configuration sub-processors.

15. The method according to any one of claims 10-14, further comprising:

detecting an computational power of a processor in the receiving end;

and when the computing power of the processor is larger than a second set threshold, reducing the number of the sub-processors of the receiving end, wherein the second set threshold is the minimum computing power of the processor, and the computing power of the processor cannot be configured for N sub-processors.

16. The method as recited in claim 15, further comprising:

And sending a second feedback instruction to the sending end, wherein the second feedback instruction instructs the sending end to reduce the number of configuration sub-processors.

17. A data transmission apparatus, comprising:

the transceiver is used for receiving a data transmission request instruction sent by the receiving end, wherein the data transmission request instruction carries an identifier of a target file;

the processor is used for determining the target file according to the identification of the target file, dividing the data of the target file into N data segments, wherein N is a positive integer greater than 1;

the transceiver is further configured to send segmentation information to the receiving end, where the segmentation information includes a data length of the target file, a number of the data segments, a data length of each data segment, and an offset of each data segment, the data length of the target file indicates the receiving end to determine a target storage space of the target file, the number of the data segments indicates the receiving end to establish N data transmission links, and the data length of each data segment and the offset of each data segment indicate the receiving end to determine a storage position of the data segment in the target storage space;

Receiving data segment storage request instructions sent by N data transmission links;

the processor is further used for reading each data segment by utilizing a stream compression algorithm and compressing the read data to obtain a compressed data stream;

the transceiver is further configured to send a compressed data stream to the receiving end through the N data transmission links.

18. The apparatus of claim 17, wherein the processor is further configured to configure N sub-processors for reading data of a data segment and compressing the read data according to the number of data segments.

19. The apparatus according to any one of claims 17 or 18, wherein the processor is configured to determine a data length of the target file, and to divide the data of the target file into the N data segments.

20. The apparatus according to any one of claims 17 or 18, wherein the processor is configured to determine the number of communication channels between the transmitting end and the receiving end based on a communication bandwidth between the transmitting end and the receiving end; and

and determining the number of data segments of the target file according to the number of the communication channels.

21. The apparatus according to any one of claims 18-20, wherein the processor is specifically configured to read data of a first data segment after receiving a first data segment storage request instruction by a first sub-processor, where the N sub-processors include the first sub-processor, the N data segments include the first data segment, the N data transmission links include a first data transmission link, and the first data segment storage request instruction is a data segment storage request instruction transmitted by the first data transmission link; and

and after the first sub-processor reads the data with the set length of the first data segment, compressing the data with the set length to obtain a first compressed data stream, wherein the set length is smaller than the length of the first data segment.

22. The apparatus according to any one of claims 17-21, wherein the processor is further configured to detect a communication bandwidth between the transmitting end and the receiving end; and

and when the communication bandwidth is smaller than a first set threshold, reducing the number of the sub-processors of the sending end, wherein the first set threshold is used for distributing the communication bandwidths of N communication channels.

23. The apparatus of claim 22, wherein the transceiver is further configured to send a first feedback instruction to the receiving end, the first feedback instruction instructing the receiving end to reduce the number of established data transmission links.

24. The apparatus of any of claims 17-23, wherein the processor is further configured to detect a computing power of a processor in the sender; and

and when the computing power of the processor is larger than a second set threshold, reducing the number of the sub-processors of the sending end, wherein the second set threshold is the minimum computing power of the processor, and the computing power of the processor cannot be configured for N sub-processors.

25. The apparatus of claim 24, wherein the transceiver is further configured to send a second feedback instruction to the receiving end, the second feedback instruction instructing the receiving end to reduce the number of configuration sub-processors.

26. A data transmission apparatus, comprising:

the transceiver is used for sending a data transmission request instruction to the sending end, wherein the data transmission request instruction instructs the sending end to divide the data of the target file into N data segments, and N is a positive integer greater than 1;

Receiving segmentation information sent by the sending end, wherein the segmentation information comprises the data length of the target file, the number of the data segments, the data length of each data segment and the offset of each data segment;

the processor is used for establishing N data transmission links according to the number of the data segments;

the transceiver is further configured to send a data segment storage request instruction to the sending end through the N data transmission links, where the data segment storage request instruction indicates the sending end to read and send the N data segments;

receiving compressed data streams sent by the sending end through the N data transmission links;

the processor is further configured to decompress the compressed data stream using a streaming decompression algorithm, and store the data transmitted by each data transmission link in a set storage location in the target storage space.

27. The apparatus of claim 26, wherein the processor is further configured to configure N sub-processors according to the number of data segments, the sub-processors configured to decompress a compressed data stream and write the decompressed data into set storage locations in the target storage space.

28. The apparatus according to any one of claims 26 or 27, wherein the processor is configured to determine the target storage space storing the target file in a memory based on a data length of the target file;

determining storage positions of the data segments in the target storage space according to the data length of the data segments and the offset of the data segments;

after receiving a second compressed data stream of a second data segment through a second data transmission link, the second sub-processor decompresses the second compressed data stream to obtain original data; the N sub-processors include the second sub-processor, the N data transmission links include the second data transmission link, the N data segments include the second data segment, the second compressed data stream is a compressed data stream obtained by compressing the second data segment with the set length read by the transmitting end, the original data is the second data segment with the set length read by the transmitting end, and the length of the original data is smaller than the length of the second data segment;

the second sub-processor writes the original data into a storage space of the second data segment, the target storage space including the storage space of the second data segment.

29. The apparatus according to any of claims 26-28, wherein the processor is further configured to detect a communication bandwidth between the receiving end and the transmitting end; and

and when the communication bandwidth is smaller than a first set threshold, reducing the number of the sub-processors of the receiving end, wherein the first set threshold is used for distributing the communication bandwidths of N communication channels.

30. The apparatus of claim 29, wherein the transceiver is further configured to send a first feedback instruction to the sender, the first feedback instruction instructing the sender to reduce the number of configuration sub-processors.

31. The apparatus of any of claims 26-30, wherein the processor is further configured to detect a computing power of a processor in the receiving end; and

and when the computing power of the processor is larger than a second set threshold, reducing the number of the sub-processors of the receiving end, wherein the second set threshold is the minimum computing power of the processor, and the computing power of the processor cannot be configured for N sub-processors.

32. The apparatus of claim 31, wherein the transceiver is further configured to send a second feedback instruction to the sender, the second feedback instruction instructing the sender to reduce the number of configuration sub-processors.

33. A data transmission system, comprising:

at least one transmitting end as claimed in any of the claims 1-9 and at least one receiving end as claimed in any of the claims 10-16; wherein, the communication connection is established between the at least one sending end and the at least one receiving end.

34. A computer readable storage medium having stored thereon a computer program which, when executed in a computer, causes the computer to perform the method of any of claims 1-16.

35. A computer program product, characterized in that it stores instructions that, when executed by a computer, cause the computer to implement the method of any one of claims 1-16.