CN118642883A - Data processing method, device, equipment and readable storage medium based on network card - Google Patents

Abstract

The application discloses a data processing method, a device, equipment and a readable storage medium based on a network card, wherein the method comprises the following steps: dividing an initial block storage message into N unit storage messages with target storage lengths based on the unit storage lengths, generating redundancy check values of each unit storage message, generating N access block storage messages according to the N unit storage messages and the N redundancy check values, and converting the message format of the unit storage messages in the N access block storage messages through a remote direct access protocol to obtain N unit network messages, and generating N transmission network messages according to the N unit network messages and the N redundancy check values; the storage length of the N unit network messages is equal to the target storage length; and sending the N transmission network messages to the second equipment so that the second equipment writes the N transmission network messages into the node storage component. By adopting the application, the processing efficiency of the network card storage service can be improved, and the consumption of computing resources can be reduced.

Description

Data processing method, device, equipment and readable storage medium based on network card

Technical Field

The present application relates to the field of computer technology, and in particular to a data processing method, device, equipment and readable storage medium based on a network card.

Background Art

Currently, in the storage business of the network card, multiple components each ensure data correctness. The network card needs to unload the block storage (block) message of the front-end device in the block storage device to the logic circuit component of the network card. After receiving the block storage message, the logic circuit component will generate a correctness check value for the block storage message, and forward the block storage message and its check value to the storage development component (Storage Performance Development Kit, SPDK) in the system on chip component (System On Chip, SOC). The storage development component verifies the correctness of the block storage message and its correctness check value. If correct, it will continue to pass the block storage message and its correctness check value to the kernel processing component (Kernel Transmission Control Protocol, Kernel TCP).

In the kernel processing component, the block storage message and its correctness check value are also checked. If correct, the block storage message is converted into a network (net) message through the TCP protocol. Since the maximum transmission unit (Maximum Transmission Unit, MTU) of the network message supported by the TCP protocol is much smaller than the block storage message, the block storage message needs to be divided into multiple message fragments, and an integrity check value (checksum in the TCP protocol) and correctness check value are generated for each message fragment, and all message fragments and the corresponding integrity check value and correctness check value are sent to the message sending component. The message sending component verifies the integrity check value to determine the integrity of multiple message fragments, and verifies the correctness check value to determine the correctness of multiple message fragments. When all message fragments pass the verification, the multiple message fragments and the integrity check values and correctness check values corresponding to the multiple message fragments are sent to the back-end device. Each component needs to consume resources to perform data consistency checks, and each component needs to implement exception handling after errors, resulting in high overall link processing latency, low processing efficiency, and high computing resource consumption for the storage service of the network card.

Summary of the invention

The embodiments of the present application provide a data processing method, device, equipment and readable storage medium based on a network card, which can improve the processing efficiency of the network card storage service and reduce the consumption of computing resources.

On the one hand, an embodiment of the present application provides a data processing method based on a network card, the method is applied to the network card, the network card includes a logic circuit component and a system-on-chip component; the method includes:

Obtaining an initial block storage message transmitted by a block storage component in the first device through a logic circuit component, dividing the initial block storage message into N unit storage messages with a target storage length based on a unit storage length of a node storage component of the second device, generating a redundant check value for each unit storage message, generating N access block storage messages according to the N unit storage messages and the N redundant check values, and transmitting the N access block storage messages to the on-chip system component; N is a positive integer; an access block storage message includes a unit storage message and a corresponding redundant check value; the target storage length is less than the unit storage length;

In the system-on-chip component, the unit storage messages in the N access block storage messages are converted into message formats through a remote direct access protocol to obtain N unit network messages, and N transmission network messages are generated according to the N unit network messages and N redundant check values; one transmission network message includes one unit network message and an associated redundant check value; the storage lengths of the N unit network messages are all equal to the target storage length; the storage lengths of the N transmission network messages are all less than or equal to the unit storage length;

N transmission network messages are sent to the second device so that the second device verifies each transmission network message through the node storage component, and when all N transmission network messages pass the verification, the N transmission network messages are written into the node storage component.

Among them, it also includes:

Obtaining a unit storage length of a node storage component of the second device, determining a storage length of message header information associated with a remote direct access protocol, and obtaining a preset storage length for a redundancy check value;

The target storage length is determined based on the unit storage length, the storage length of the message header information, and the preset storage length.

The redundant check value of each unit storage message is generated, including:

Based on the bit length P of the business operation value, the unit storage message is post-padded to obtain the padded storage message, and based on the business operation value, the padded storage message is XOR-divided to obtain the business redundancy value; P is a positive integer; the business operation value is determined based on the preset storage length for the redundancy check value;

If the bit length of the service redundancy value meets the front-filling condition, the service redundancy value is front-filled, and the service redundancy value after the front-filling is determined as the redundancy check value;

If the bit length of the service redundancy value does not meet the leading bit filling condition, the service redundancy value is determined as the redundancy check value.

The padding storage message includes M unit bit values, where M is a positive integer. Based on the service operation value, an XOR division operation is performed on the padding storage message to obtain a service redundancy value, including:

Based on the bit length P of the service operation value, among the M unit bit values of the padding storage message, the continuous P unit bit values starting from the target bit are determined as unit operation data, and based on the service operation value, the unit operation data are XOR-operated to obtain a unit redundancy value; the target bit refers to the highest bit among the bits whose unit bit value is 1 in the padding storage message;

The unit bit values other than P unit bit values among the M unit bit values are determined as Q to-be-filled bit values, and post-fill processing is performed on the unit redundant value based on the bit length P of the business operation value and the Q to-be-filled bit values to obtain new unit operation data with the same bit length as the business operation value, and XOR operation is continued to be performed on the new unit operation data based on the business operation value to obtain a new unit redundant value, until the Q to-be-filled bit values are all used to complete the post-fill processing, and the finally obtained unit redundant value is determined as the business redundant value; Q is the difference between M and P.

Among them, in the system-on-chip component, the unit storage messages in the N access block storage messages are converted into message formats through the remote direct access protocol to obtain N unit network messages, and N transmission network messages are generated according to the N unit network messages and N redundant check values, including:

Obtain unit storage messages from N access block storage messages through the on-chip system component, convert the message formats of the N unit storage messages through the remote direct access protocol to obtain N unit network messages, and generate message header information corresponding to each unit network message;

N unit network messages, the message header information corresponding to the N unit network messages, and N redundant check values are respectively encapsulated into N transmission network messages; a transmission network message is obtained by encapsulating a unit network message, the corresponding message header information, and the associated redundant check value.

The system-on-chip component includes a protocol stack component; sending N transmission network messages to the second device includes:

Send N transport network messages to the protocol stack component;

In the protocol stack component, a unit network message in each transmission network message and a redundant check value associated with the unit network message are obtained;

Based on N unit network messages, N target check values are generated. If the target check value corresponding to each unit network message is the same as the associated redundant check value, the N transmission network messages are sent to the second device through the protocol stack component.

Among them, it also includes:

Obtaining, through the system-on-chip component, N transmission network messages sent by the second device through the node storage component, verifying each transmission network message, and sending the N transmission network messages to the logic circuit component when all the N transmission network messages pass the verification;

In the logic circuit component, a unit network message in the N transmission network messages is converted into a message format through a remote direct access protocol to obtain N unit storage messages, and based on message header information respectively associated with the N unit storage messages, the N unit storage messages are reassembled into an initial block storage message;

The initial block storage message is transmitted to the first device.

On the one hand, an embodiment of the present application provides another data processing method based on a network card, the method is applied to the network card, the network card includes a logic circuit component and a system-on-chip component; the method includes:

Obtain N transmission network messages sent by the second device through the node storage component through the on-chip system component, verify each transmission network message, and send the N transmission network messages to the logic circuit component when all the N transmission network messages pass the verification; the storage lengths of the N transmission network messages are less than or equal to the unit storage length of the node storage component in the second device; a transmission network message includes a unit network message and a corresponding redundancy check value;

In the logic circuit component, the unit network messages in the N transmission network messages are converted into message formats through a remote direct access protocol to obtain N unit storage messages, and the N unit storage messages are reorganized into initial block storage messages based on message header information respectively associated with the N unit storage messages; the storage lengths of the N unit network messages and the storage lengths of the N unit storage messages are both equal to the target storage length, and the target storage length is less than the unit storage length;

The initial block storage message is transmitted to the first device.

On the one hand, an embodiment of the present application provides another data processing method based on a network card, which is applied to a second device, and the second device interacts with a first device through the network card; the method includes:

Obtain N transmission network messages sent by the first device through the network card; N is a positive integer, and the storage lengths of the N transmission network messages are all less than or equal to the unit storage length of the node storage component in the second device; a transmission network message includes a unit network message and an associated redundant check value; the N unit network messages are obtained by the network card performing message format conversion on the unit storage messages in the N access block storage messages through the remote direct access protocol, and an access block storage message includes a unit storage message and a redundant check value generated for the unit storage message; the N unit storage messages are obtained by dividing the initial block storage message based on the unit storage length; the storage lengths of the N unit network messages and the storage lengths of the N unit storage messages are both equal to the target storage length, and the target storage length is less than the unit storage length;

Obtain the unit network message in each transmission network message and the redundant check value associated with the unit network message, generate N target check values based on N unit network messages, and if the target check value corresponding to each unit network message is the same as the associated redundant check value, store each transmission network message separately in the node storage component.

Wherein, each transmission network message is stored in a node storage component respectively, including:

Obtain message header information from N transmission network messages, determine storage addresses corresponding to the N transmission network messages based on the division and sorting values in the N message header information, and based on the N storage addresses, store each transmission network message in the storage unit corresponding to the associated storage address in the node storage component.

Among them, it also includes:

When a storage service read request sent by the first device is obtained, N stored transmission network messages are obtained from the node storage component, each transmission network message is verified, and when all N transmission network messages are verified, the N transmission network messages are sent to the network card, so that the network card obtains N unit storage messages through the N transmission network messages, and generates an initial block storage message based on the N unit storage messages; the initial block storage message is used for transmission to the first device.

An embodiment of the present application provides a network card, including:

A logic circuit component, configured to obtain an initial block storage message transmitted by a block storage component in a first device, divide the initial block storage message into N unit storage messages with a target storage length based on a unit storage length of a node storage component of a second device, generate a redundant check value for each unit storage message, generate N access block storage messages according to the N unit storage messages and the N redundant check values, and transmit the N access block storage messages to an on-chip system component; N is a positive integer; an access block storage message includes a unit storage message and a corresponding redundant check value; the target storage length is less than the unit storage length;

A system-on-chip component is used to convert the message format of the unit storage messages in the N access block storage messages through a remote direct access protocol to obtain N unit network messages, and generate N transmission network messages according to the N unit network messages and N redundant check values; one transmission network message includes one unit network message and an associated redundant check value; the storage lengths of the N unit network messages are all equal to the target storage length; the storage lengths of the N transmission network messages are all less than or equal to the unit storage length;

The system-on-chip component is also used to send N transmission network messages to the second device, so that the second device verifies each transmission network message through the node storage component, and when the N transmission network messages are all verified, the N transmission network messages are written into the node storage component.

The logic circuit component is further used to obtain the unit storage length of the node storage component of the second device, determine the storage length of the message header information associated with the remote direct access protocol, and obtain the preset storage length for the redundancy check value;

The logic circuit component is also used to determine the target storage length based on the unit storage length, the storage length of the message header information and the preset storage length.

The logic circuit component is specifically used to perform post-placement processing on the unit storage message based on the bit length P of the business operation value to obtain the place-padded storage message, and perform XOR division operation on the place-padded storage message based on the business operation value to obtain the business redundancy value; P is a positive integer; the business operation value is determined based on the preset storage length for the redundancy check value;

The logic circuit component is specifically used for performing a front-filling process on the service redundancy value if the bit length of the service redundancy value meets the front-filling condition, and determining the service redundancy value after the front-filling process as the redundancy check value;

The logic circuit component is specifically used to determine the service redundancy value as the redundancy check value if the bit length of the service redundancy value does not meet the leading bit filling condition.

The padding storage message includes M unit bit values, where M is a positive integer;

The logic circuit component is specifically used to determine, based on the bit length P of the business operation value, the continuous P unit bit values starting from the target bit in the M unit bit values of the bit-padded storage message as unit operation data, and perform an XOR operation on the unit operation data based on the business operation value to obtain a unit redundancy value; the target bit refers to the highest bit among the bits whose unit bit values are 1 in the bit-padded storage message;

A logic circuit component is specifically used to determine the unit bit values other than P unit bit values among M unit bit values as Q to-be-filled bit values, perform post-fill processing on the unit redundant value based on the bit length P of the business operation value and the Q to-be-filled bit values, obtain new unit operation data with the same bit length as the business operation value, continue to perform XOR operation on the new unit operation data based on the business operation value to obtain a new unit redundant value, until the Q to-be-filled bit values are all used to complete the post-fill processing, and the finally obtained unit redundant value is determined as the business redundant value; Q is the difference between M and P.

The system-on-chip component is specifically used to obtain unit storage messages from N access block storage messages, convert the message formats of the N unit storage messages through a remote direct access protocol to obtain N unit network messages, and generate message header information corresponding to each unit network message;

The system-on-chip component is specifically used to encapsulate N unit network messages, the message header information corresponding to the N unit network messages, and N redundant check values into N transmission network messages; a transmission network message is obtained by encapsulating a unit network message, the corresponding message header information, and the associated redundant check value.

Among them, the system-on-chip components include storage development components and protocol stack components;

A storage development component is used to send N transmission network messages to the protocol stack component;

A protocol stack component, used for obtaining a unit network message in each transmitted network message and a redundant check value associated with the unit network message;

The protocol stack component is also used to generate N target check values based on N unit network messages. If the target check value corresponding to each unit network message is the same as the associated redundant check value, the N transmission network messages are sent to the second device through the protocol stack component.

in,

The system-on-chip component is further used to obtain N transmission network messages sent by the second device through the node storage component, verify each transmission network message, and send the N transmission network messages to the logic circuit component when all the N transmission network messages pass the verification;

The logic circuit component is further used to convert the message format of the unit network message in the N transmission network messages through the remote direct access protocol to obtain N unit storage messages, and reorganize the N unit storage messages into an initial block storage message based on the message header information respectively associated with the N unit storage messages;

The logic circuit component is further used to transmit the initial block storage message to the first device.

On the one hand, an embodiment of the present application provides another network card, including:

The system-on-chip component is used to obtain N transmission network messages sent by the second device through the node storage component, verify the redundant check value in each transmission network message, and send the N transmission network messages to the logic circuit component when the N transmission network messages are all verified; the storage lengths of the N transmission network messages are all less than or equal to the unit storage length of the node storage component; and one transmission network message includes one unit network message and the corresponding redundant check value;

A logic circuit component is used to convert the message format of the unit network messages in the N transmission network messages through a remote direct access protocol to obtain N unit storage messages, and reorganize the N unit storage messages into an initial block storage message based on message header information respectively associated with the N unit storage messages; the storage length of the N unit network messages and the storage length of the N unit storage messages are both equal to the target storage length, and the target storage length is less than the unit storage length;

The logic circuit component is further used to transmit the initial block storage message to the first device.

An embodiment of the present application provides a data processing device, including:

A message acquisition module, used for acquiring N transmission network messages sent by the first device through the network card; N is a positive integer, and the storage lengths of the N transmission network messages are all less than or equal to the unit storage length of the node storage component in the second device; a transmission network message includes a unit network message and an associated redundant check value; the N unit network messages are obtained by the network card performing message format conversion on the unit storage messages in the N access block storage messages through the remote direct access protocol, and an access block storage message includes a unit storage message and a redundant check value generated for the unit storage message; the N unit storage messages are obtained by dividing the initial block storage message based on the unit storage length; the storage lengths of the N unit network messages and the storage lengths of the N unit storage messages are both equal to the target storage length, and the target storage length is less than the unit storage length;

The message verification module is used to obtain the unit network message in each transmission network message and the redundant check value associated with the unit network message, and generate N target check values based on N unit network messages. If the target check value corresponding to each unit network message is the same as the associated redundant check value, each transmission network message is stored in the node storage component respectively.

In a possible implementation, when the message verification module is used to store each transmission network message in the node storage component, it is specifically used to perform the following operations:

Obtain message header information from N transmission network messages, determine storage addresses corresponding to the N transmission network messages based on the division and sorting values in the N message header information, and based on the N storage addresses, store each transmission network message in the storage unit corresponding to the associated storage address in the node storage component.

In a possible implementation, the message checking module is further configured to perform the following operations:

When a storage service read request sent by the first device is obtained, N stored transmission network messages are obtained from the node storage component, each transmission network message is verified, and when all N transmission network messages are verified, the N transmission network messages are sent to the network card, so that the network card obtains N unit storage messages through the N transmission network messages, and generates an initial block storage message based on the N unit storage messages; the initial block storage message is used to be transmitted to the first device.

On the one hand, an embodiment of the present application provides a computer device, including: a processor, a memory, and a network interface;

The processor is connected to the memory and the network interface, wherein the network interface is used to provide data communication function, and the memory is used to store computer programs. When the computer program is executed by the processor, the computer device executes the method provided in the embodiment of the present application.

On the one hand, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. The computer program is suitable for being loaded and executed by a processor so that a computer device having the processor executes the method provided by the embodiment of the present application.

In one aspect, an embodiment of the present application provides a computer program product, which includes a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the computer device executes the method provided in the embodiment of the present application.

The embodiment of the present application combines the characteristics of the unit storage length of the node storage component, the maximum transmission unit of the block storage message, and the maximum transmission unit of the network message, and divides the initial block storage message transmitted by the block storage component in the logic circuit component by a target storage length that is less than or equal to the unit storage length, obtains N unit storage messages with the target storage length, generates a redundant check value for each unit storage message, generates N access block storage messages according to the N unit storage messages and the N redundant check values, and transmits the N access block storage messages to the on-chip system component. The access block storage message can be a network message obtained by adding message information such as a check value, and the unit storage message can be a message fragment in the block storage message format. The target storage length can be less than or equal to the maximum transmission unit of the unit network message in the remote direct access protocol, so that after the unit storage message is converted into a message format in the on-chip system component through the remote direct access protocol, the storage length of the unit network message obtained is the same as the storage length of the unit storage message. It allows the unit storage message to be transmitted between the logic circuit component and the system-on-chip component without the need to fragment the message due to the difference between the maximum transmission unit of the block storage message and the maximum transmission unit of the network message, nor is it necessary to perform multiple correctness checks due to the different redundant check values of the block storage message and the network message, and there is no need to perform integrity checks on the message fragments. N transmission network messages are generated through N unit network messages and N redundant check values, one transmission network message includes a unit network message and the associated redundant check value, the storage length of the N unit network messages is equal to the target storage length, the storage length of the N transmission network messages is less than or equal to the unit storage length, and the transmission network message can be a network message whose message format conforms to the remote direct access protocol. The N transmission network messages are sent to the second device, and the second device verifies each transmission network message through the node storage component. When the N transmission network messages are all verified, the N transmission network messages are written into the node storage component. Since the storage length of a unit network message is the same as the storage length of a unit storage message, the redundant check value generated by the unit network message is also the same as the redundant check value generated by the unit storage message. The redundant check value generated by the logic circuit component can be carried to the end of the network card, reducing the computing resource consumption of the on-chip system components, reducing the overall link latency, and improving the processing efficiency of the network card storage service.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

FIG1 is a schematic diagram of a network architecture provided in an embodiment of the present application;

FIG2 is a schematic diagram of a data processing scenario provided in an embodiment of the present application;

FIG3 is a flow chart of a data processing method based on a network card provided in an embodiment of the present application;

FIG4 is a second flow chart of a data processing method based on a network card provided in an embodiment of the present application;

FIG5 is a second schematic diagram of a data processing scenario provided in an embodiment of the present application;

6 is a flowchart diagram of a data processing method based on a network card provided in an embodiment of the present application;

7 is a flowchart of a data processing method based on a network card according to an embodiment of the present application;

FIG8 is a structural diagram 1 of a network card provided in an embodiment of the present application;

FIG9 is a second structural diagram of a network card provided in an embodiment of the present application;

FIG10 is a schematic diagram of the structure of a data processing device provided in an embodiment of the present application;

FIG. 11 is a schematic diagram of the structure of a computer device provided in an embodiment of the present application.

DETAILED DESCRIPTION

The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

Please refer to Figure 1, which is a schematic diagram of a network architecture provided by an embodiment of the present application. As shown in Figure 1, the network architecture may include a first device 100, a network card 200, and a second device 300, wherein there is a communication connection between the first device 100 and the network card 200, and there is a communication connection between the network card 200 and the second device 300. Among them, the above-mentioned communication connection does not limit the connection method, and can be directly or indirectly connected through a wired communication method, or directly or indirectly connected through a wireless communication method, or through other methods, and the present application does not limit it here.

It should be understood that the first device 100 can exchange data with the second device 300 through the network card 200, the network card 200 can be integrated with the first device 100, or the network card 200 can also be an independent device, which is not limited in the embodiment of the present application. When the network card 200 is an independent device, the network card 200 can be connected to the first device 100 through a peripheral component interconnect standard (Peripheral Component Interconnect Express, PCIE) bus. The second device 300 can be configured with a network access device corresponding to the network card 200, and the second device 300 can exchange data with the first device 100 through the network access device. Optionally, the network architecture may include one or more first devices, and the one or more first devices are connected to the network card 200, and exchange data with the second device 300 through the network card 200.

Among them, the first device 100 may include: smart phones, tablet computers, laptops, desktop computers, intelligent voice interaction devices, smart home appliances (for example, smart TVs), wearable devices, vehicle terminals, aircraft and other smart terminals with data processing functions. Among them, the vehicle terminal can be a terminal device in a smart traffic scenario and an assisted driving scenario. It should be understood that each terminal device in the terminal device cluster shown in Figure 1 can be installed with an application client with data processing functions. When the application client runs in each terminal device, it can interact with the second device 300 shown in Figure 1 above.

The application client may specifically include: a vehicle client, a smart home client, an entertainment client (e.g., a game client), a multimedia client (e.g., a video client), a social client, and an information client (e.g., a news client), etc. The application client in the embodiment of the present application may be integrated in a client (e.g., a social client) or may be an independent client (e.g., a news client). The embodiment of the present application does not limit the type of the application client.

Among them, the second device 300 can be a server corresponding to the application client. The second device 300 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), as well as big data and artificial intelligence platforms.

Among them, cloud technology refers to a hosting technology that unifies hardware, software, network and other resources in a wide area network or local area network to realize data computing, storage, processing and sharing. Cloud technology is a general term for network technology, information technology, integration technology, management platform technology, application technology, etc. based on the cloud computing business model. It can form a resource pool, which is used on demand and flexible and convenient. Cloud computing technology will become an important support. The background services of technical network systems require a large amount of computing and storage resources, such as video websites, picture websites and more portal websites. With the high development and application of the Internet industry, in the future, each item may have its own identification mark, which needs to be transmitted to the background system for logical processing. Data of different levels will be processed separately. All kinds of industry data need strong system backing support to be realized through cloud computing.

As shown in FIG1 , the first device 100 may include a block storage component, the network card 200 may include a logic circuit component and a system-on-chip component (SOC), and the second device 300 may include a node storage component (Cell). Among them, the first device 100 may be a front-end device, which may also be referred to as a host, and the network card 200 may be a Smart Network Interface Card (SNIC), a high-performance network access card with a network processor as the core, which can be used to process various network protocols or storage protocols from the host. The block storage component may be a device component such as a hard disk drive (HDD, Hard Disk Drive), a solid state drive (SSD, Solid State Drive), etc., which uses a fixed-size data block (Block) as the basic storage unit. The logic circuit component may be implemented by an ASIC (Application-Specific Integrated Circuit) or an FPGA (Field-Programmable Gate Array, Programmable Logic Array). The second device 300 may be a back-end device that provides storage business services.

It can be understood that the first device 100 can be provided with multiple virtual machines (VM), each of which can run one or more virtual functions (VF), each of which can correspond to one or more protocol queues (queues), and the multiple protocol queues include a storage service read queue and a storage service write queue. The storage service read queue can be used to execute the read task of the storage service, and the storage service write queue can be used to execute the write task of the storage service. The multiple protocol queues can be stored in the block storage component.

For ease of understanding, taking the writing task of the storage service as an example, the first device 100 can offload the initial block storage message in the block storage component to the network card 200, and the network card 200 encapsulates the initial block storage message into a transmission network message based on the remote direct access protocol (RDMA) and sends it to the second device 300 to complete the writing of the storage service. Among them, the initial block storage message can be an IO (Input/Output Access Message) access message for the block storage (block) component in the operating system, that is, a BLK message. The transmission network message can be a network (net) message whose message format conforms to the RDMA protocol, that is, a NET-RDMA message. Offloading can include offloading for the RDMA protocol, offloading for the User Datagram Protocol (UDP), offloading for Erasure Coding (EC), etc., which is not limited in the embodiments of the present application. Offloading can refer to separating various network protocols and storage protocols in the host and executing them by the network card.

The embodiment of the present application processes various network protocols and storage protocols separated by the first device through a network card. The network card has an independent computing unit, such as a logic circuit component, an on-chip system component, etc., which can complete the execution of a storage service read queue and a storage service write queue, etc., which can greatly reduce the computing resource consumption of the first device, and can execute intensive storage services through a dedicated processor, thereby increasing the speed of message transmission and improving the network performance and efficiency of storage services.

Please refer to Figure 2, which is a schematic diagram of a data processing scenario provided by an embodiment of the present application. As shown in Figure 2, the first device may include a block storage component, the first device may be the first device 100 as shown in Figure 1, the network card may include a logic circuit component and a system-on-chip component, the network card may be the network card 200 as shown in Figure 1, wherein the system-on-chip component may include a storage development component (SPDK) and a protocol stack component (Protocol Stack), and the second device 300 may include a node storage component (Cell).

For ease of understanding, taking the write task of the storage service as an example, the first device can unload the initial block storage message in the block storage component to the network card, and the network card obtains the initial block storage message transmitted by the first device through the logic circuit component. Among them, the initial block storage message can be an IO access message for the block storage component in the operating system, that is, a BLK message.

The logic circuit component of the network card can obtain the unit storage length of the node storage component of the second device, and the unit storage length can be the maximum storage length of the storage unit in the node storage component. The logic circuit component can divide the initial block storage message into N unit storage messages with a target storage length based on the unit storage length, wherein the unit storage message can be a message fragment in a BLK format, and the target storage length can be less than the unit storage length. The logic circuit component can generate a redundant check value (Cyclic Redundancy Check, CRC, also known as cyclic redundancy check) for each unit storage message. The logic circuit component can generate N access block storage messages through N unit storage messages and N redundant check values, and transmit the N access block storage messages to the on-chip system component of the network card. Among them, the access block storage message can be a BLK message obtained by adding message information such as a check value, and an access block storage message can include a unit storage message and a corresponding redundant check value. The target storage length can be less than or equal to the maximum transmission unit of the unit network message in the remote direct access protocol (RDMA).

In the system-on-chip component, the storage development component can convert the message format of the unit storage message in the N access block storage messages through the remote direct access protocol to obtain N unit network messages, wherein the remote direct access protocol is a message-based data transmission protocol, a technology used for high-performance network communication. The unit network message can be a message of a network device in the operating system, that is, a NET message. The storage development component can generate N transmission network messages through N unit network messages and N redundant check values, and the storage development component can pass the N transmission network messages to the protocol stack component in the system-on-chip component. Among them, the transmission network message can be a NET message whose message format conforms to the RDMA protocol, that is, a NET-RDMA message. A transmission network message can include a unit network message and an associated redundant check value, the storage lengths of the N unit network messages are all equal to the target storage length, and the storage lengths of the N transmission network messages are all less than or equal to the unit storage length.

The protocol stack component can use the RDMA API (Application Programming Interface), for example, to create an RDMA connection with the second device through the libibverbs library, wherein the libibverbs library is a standard library that provides a series of RDMA interfaces for performing RDMA operations, including but not limited to memory registration, sending and receiving data, performing RDMA read and write, etc. The protocol stack component can register the storage area in the node storage component of the second device, send N transmission network messages to the second device, and the second device verifies the redundant check value in each transmission network message through the node storage component, and writes the N transmission network messages into the node storage component when all the N transmission network messages are verified.

The reading task of the storage service and the writing task of the storage service can be a reverse process. In the reading task of the storage service, the second device can obtain the N stored transmission network messages from the node storage component, and check the redundant check value in each transmission network message. When the N transmission network messages are all checked and passed, the N transmission network messages are sent to the network card. The network card can receive the N transmission network messages through the protocol stack component, and pass the N transmission network messages to the storage development component. The storage development component converts the message format of the unit network message in the N transmission network messages to obtain N unit storage messages, and passes the N unit storage messages to the logic circuit component. The logic circuit component reorganizes the N unit storage messages into an initial block storage message, and sends the initial block storage message to the block storage component in the first device for storage.

It can be understood that the network card proposed in the embodiment of the present application can be applied to a cloud hard disk (Cloud Block Store, CBS), allowing users to create and manage persistent block storage volumes in a cloud environment, and provide a persistent, highly reliable data block-level storage service for a cloud server (Cloud Virtual Machine, CVM). The network card can provide correctness storage verification and RDMA network function offloading, improve the storage performance of the second device, and provide safe and reliable storage services. The network card can also be used to execute firewall rules, security detection and prevention systems (Intrusion Detection and Prevention System, IDPS) and other security protocols to improve the security of storage services. Support software-defined networking (Software-Defined Networking, SDN) and network function virtualization (Network Functions Virtualization, NFV) to increase data transmission rates.

The embodiment of the present application combines the characteristics of the unit storage length of the node storage component, the maximum transmission unit of the block storage message, and the maximum transmission unit of the network message, and divides the initial block storage message transmitted by the block storage component in the logic circuit component through the target storage length that is less than or equal to the unit storage length, and obtains N unit storage messages with the target storage length, and the unit storage message can be a message fragmentation in the block storage message format. The target storage length can be less than or equal to the maximum transmission unit of the unit network message in the remote direct access protocol, so that after the unit storage message is converted into a message format through the remote direct access protocol in the on-chip system component, the storage length of the unit network message obtained is the same as the storage length of the unit storage message. Since the storage length of the unit network message is the same as the storage length of the unit storage message, the redundant check value generated by the unit network message is also the same as the redundant check value generated by the unit storage message, and the redundant check value generated by the logic circuit component can be carried to the end of the network card, reducing the computing resource consumption of the on-chip system component, reducing the overall link delay, and improving the processing efficiency of the network card storage service.

Please refer to FIG. 3, which is a flowchart of a data processing method based on a network card provided in an embodiment of the present application. The data processing method can be executed by a network card, and the network card can be the network card 200 shown in FIG. 1. The following will take the data processing method executed by the network card as an example for description. The data processing method may include at least the following steps S101-S103:

Step S101, obtaining an initial block storage message transmitted by a block storage component in a first device through a logic circuit component, dividing the initial block storage message into N unit storage messages with a target storage length based on a unit storage length of a node storage component of a second device, generating a redundant check value for each unit storage message, generating N access block storage messages according to the N unit storage messages and the N redundant check values, and transmitting the N access block storage messages to an on-chip system component; N is a positive integer; an access block storage message includes a unit storage message and a corresponding redundant check value; the target storage length is less than the unit storage length;

Specifically, the network card may include a logic circuit component and a system-on-chip component, and the first device may establish a communication connection with the second device through the network card to exchange data. Among them, the first device may be a front-end device, which may also be referred to as a host. The first device may include a block storage device. The network card may be an intelligent network card, a high-performance network access card with a network processor as the core, which can be used to process various network protocols or storage protocols from the host. The block storage component may be a device component such as a hard disk drive, a solid-state drive, etc. that uses a fixed-size data block as the basic storage unit. The logic circuit component may be implemented by an ASIC or an FPGA. The second device may be a back-end device that provides storage business services.

The first device may run multiple protocol queues, including a storage service read queue and a storage service write queue. The storage service read queue may be used to execute storage service read tasks, and the storage service write queue may be used to execute storage service write tasks. The multiple protocol queues may be stored in a block storage component.

In the write task of the storage service, the first device can unload the initial block storage message in the block storage component to the network card, and the network card obtains the initial block storage message transmitted by the first device through the logic circuit component. Among them, the initial block storage message can be an IO access message for the block storage component in the operating system, that is, a BLK message. The logic circuit component of the network card can obtain the unit storage length of the node storage component of the second device, and the unit storage length can be the maximum storage length of the storage unit in the node storage component. Unloading can include unloading for RDMA protocol, UDP protocol unloading, code deletion unloading, etc., which is not limited in the embodiments of the present application. Unloading can refer to separating various network protocols and storage protocols in the host and executing them by the network card.

The logic circuit component can divide the initial block storage message into N unit storage messages with target storage lengths based on the unit storage length, wherein the target storage length can be less than the unit storage length, and the target storage length can be less than or equal to the maximum transmission unit of the unit network message in the remote direct access protocol (RDMA). The unit storage message can be a message fragment in BLK format. The logic circuit component can generate a redundant check value (CRC) for each unit storage message. The logic circuit component can generate N access block storage messages through N unit storage messages and N redundant check values, and transmit the N access block storage messages to the on-chip system component of the network card. Among them, the access block storage message can be a BLK message obtained by adding message information such as a check value, and an access block storage message can include a unit storage message and a corresponding redundant check value. The redundant check value can also be called a cyclic redundancy check, which is a commonly used check code in the communication field, including a series of data encoding rules such as shift and division. Commonly used encoding methods include CRC-32, CRC-16, etc.

Step S102, in the system-on-chip component, the unit storage messages in the N access block storage messages are converted into message formats through the remote direct access protocol to obtain N unit network messages, and N transmission network messages are generated according to the N unit network messages and the N redundant check values; one transmission network message includes one unit network message and the associated redundant check value; the storage lengths of the N unit network messages are all equal to the target storage length; the storage lengths of the N transmission network messages are all less than or equal to the unit storage length;

Specifically, the system-on-chip component may convert the message format of the unit storage message in the N access block storage messages through the remote direct access protocol (RDMA) to obtain N unit network messages, wherein the remote direct access protocol is a message-based data transmission protocol and a technology for high-performance network communication. The unit network message may be a message of a network device in an operating system, namely, a NET message. The system-on-chip component may generate N transmission network messages through the N unit network messages and the N redundant check values, and the system-on-chip component may transmit the N transmission network messages to the protocol stack component in the system-on-chip component. Among them, the transmission network message may be a NET message whose message format complies with the RDMA protocol, namely, a NET-RDMA message. A transmission network message may include a unit network message and an associated redundant check value, the storage lengths of the N unit network messages are all equal to the target storage length, and the storage lengths of the N transmission network messages are all less than or equal to the unit storage length.

Step S103, sending N transmission network messages to the second device, so that the second device verifies each transmission network message through the node storage component, and when the N transmission network messages are all verified, the N transmission network messages are written into the node storage component.

Specifically, the on-chip system component can send N transmission network messages to the second device, and the second device verifies the redundant check value in each transmission network message through the node storage component. When all N transmission network messages are verified, the N transmission network messages are written into the node storage component.

The embodiment of the present application combines the characteristics of the unit storage length of the node storage component, the maximum transmission unit of the block storage message, and the maximum transmission unit of the network message, and divides the initial block storage message transmitted by the block storage component in the logic circuit component by a target storage length that is less than or equal to the unit storage length, obtains N unit storage messages with the target storage length, generates a redundant check value for each unit storage message, generates N access block storage messages according to the N unit storage messages and the N redundant check values, and transmits the N access block storage messages to the on-chip system component. The access block storage message can be a network message obtained by adding message information such as a check value, and the unit storage message can be a message fragment in the block storage message format. The target storage length can be less than or equal to the maximum transmission unit of the unit network message in the remote direct access protocol, so that after the unit storage message is converted into a message format in the on-chip system component through the remote direct access protocol, the storage length of the unit network message obtained is the same as the storage length of the unit storage message. It allows the unit storage message to be transmitted between the logic circuit component and the system-on-chip component without the need to fragment the message due to the difference between the maximum transmission unit of the block storage message and the maximum transmission unit of the network message, nor is it necessary to perform multiple correctness checks due to the different redundant check values of the block storage message and the network message, and there is no need to perform integrity checks on the message fragments. N transmission network messages are generated through N unit network messages and N redundant check values, one transmission network message includes a unit network message and the associated redundant check value, the storage length of the N unit network messages is equal to the target storage length, the storage length of the N transmission network messages is less than or equal to the unit storage length, and the transmission network message can be a network message whose message format conforms to the remote direct access protocol. The N transmission network messages are sent to the second device, and the second device verifies each transmission network message through the node storage component. When the N transmission network messages are all verified, the N transmission network messages are written into the node storage component. Since the storage length of a unit network message is the same as the storage length of a unit storage message, the redundant check value generated by the unit network message is also the same as the redundant check value generated by the unit storage message. The redundant check value generated by the logic circuit component can be carried to the end of the network card, reducing the computing resource consumption of the on-chip system components, reducing the overall link latency, and improving the processing efficiency of the network card storage service.

Please refer to FIG. 4, which is a flow chart of a data processing method based on a network card provided in an embodiment of the present application. The data processing method can be executed by a network card, and the network card can be the network card 200 shown in FIG. 1. The following will take the data processing method executed by the network card as an example for description. The data processing method may include at least the following steps S201-S207:

Step S201, obtaining an initial block storage message transmitted by a block storage component in a first device through a logic circuit component;

For details, please refer to the detailed description of step S101 in the embodiment corresponding to FIG. 3 above, which will not be described in detail in the embodiment of the present application.

Step S202, obtain the unit storage length of the node storage component of the second device, determine the storage length of the message header information associated with the remote direct access protocol, and obtain the preset storage length for the redundant check value; determine the target storage length based on the unit storage length, the storage length of the message header information and the preset storage length.

Specifically, the network card can obtain the unit storage length of the node storage component of the second device, and the unit storage length can be the maximum storage length of the storage unit in the node storage component. The network card can determine the storage length of the message header information associated with the remote direct access protocol (RDMA), and obtain the preset storage length for the redundant check value. The storage length of the redundant check value is determined based on the business operation value for generating the redundant check value, and the storage length of the redundant check value can be the bit length P-1 of the business operation value.

It can be understood that the unit storage length, the storage length of the message header information and the preset storage length are all preset numerical information, and the network card can determine the difference between the unit storage length and the storage length of the message header information and the preset storage length as the storable length, and determine the target storage length based on the storable length. Among them, the target storage length can be less than or equal to the target storage length.

Step S203, based on the unit storage length of the node storage component of the second device, dividing the initial block storage message into N unit storage messages with a target storage length;

Specifically, please refer to Figure 5, which is a second schematic diagram of a data processing scenario proposed in an embodiment of the present application. As shown in Figure 5, the logic circuit component can divide the initial block storage message into N unit storage messages with a target storage length based on the unit storage length. The target storage length may be less than or equal to the maximum transmission unit of the unit network message in the remote direct access protocol (RDMA). The unit storage message may be a message fragment in BLK format, and the N unit storage messages may include unit storage message 1, unit storage message 2, ..., unit storage message N. Among them, the target storage length may be less than the unit storage length. For example, the storage length of the initial block storage message may be 12KB (Kilo Byte), and the target storage length may be 4KB. Then, the logic circuit component may divide the initial block storage message into 3 unit storage messages with a storage length of 4KB.

When the storage length of the initial block storage message is not aligned with the target storage length, for example, the storage length of the initial block storage message is 15KB, and the target storage length is 4KB, the logic circuit component can divide the initial block storage message into the target storage message and the message to be divided that is aligned with the target storage length, wherein the storage length of the message to be divided can be 12KB, and the target storage message can be 3KB, then, the logic circuit component can divide the message to be divided into 3 unit storage messages with a storage length of 4KB, determine the target storage message as a message that is not aligned with the target storage length, or fill the target storage length to the target storage length, and in the subsequent process, the target storage message is processed in the same way as the unit storage message.

Step S204, based on the bit length P of the business operation value, post-padded processing is performed on the unit storage message to obtain a padded storage message, and based on the business operation value, an XOR division operation is performed on the padded storage message to obtain a business redundancy value; P is a positive integer; the business operation value is determined based on a preset storage length for the redundant check value; if the bit length of the business redundancy value meets the front-padded condition, the business redundancy value is front-padded, and the business redundancy value after the front-padded processing is determined as the redundant check value; if the bit length of the business redundancy value does not meet the front-padded condition, the business redundancy value is determined as the redundant check value.

Specifically, please refer to Figure 5. As shown in Figure 5, the logic circuit component can generate a redundant check value (CRC) for each unit storage message. For example, it can generate a redundant check value CRC_1 corresponding to unit storage message 1, a redundant check value CRC_2 corresponding to unit storage message 2, ..., a redundant check value CRC_N corresponding to unit storage message N. The redundant check value generation method may include CRC-8, CRC-16 and CRC-32, etc. CRC-8, CRC-16 and CRC-32 have different check value lengths and complexities. For ease of understanding, CRC-8 is used as an example for explanation. The business operation value of CRC-8 can be a fixed value determined based on a preset storage length for the redundant check value. The business operation value can be a binary number 11001, and the bit length P of the business operation value is 5.

Taking the example that one of the N unit storage messages is the binary number 11010011, the logic circuit component can, based on the bit length P of the business operation value, fill in P-1 bits of value 0 after the unit storage message to obtain a filled storage message. The filled storage message is the binary number 110100110000. The logic circuit component can perform an XOR division operation on the filled storage message based on the business operation value to obtain a business redundancy value.

The process of performing an XOR division operation on the padding storage message may be: based on the bit length P of the business operation value, among the M unit bit values of the padding storage message, P consecutive unit bit values starting at the target bit are determined as unit operation data, and based on the business operation value, an XOR operation is performed on the unit operation data to obtain a unit redundancy value; the target bit refers to the highest bit among the bits whose unit bit values in the padding storage message are 1; the unit bit values other than the P unit bit values among the M unit bit values are determined as Q to-be-padded bit values, and based on the bit length P of the business operation value and the Q to-be-padded bit values, the unit redundancy value is post-padded to obtain new unit operation data having the same bit length as the business operation value, and the new unit operation data is continued to be XORed based on the business operation value to obtain a new unit redundancy value, until the Q to-be-padded bit values are all used to complete the post-padded processing, and the finally obtained unit redundancy value is determined as the business redundancy value; Q is the difference between M and P.

Specifically, the logic circuit component can determine the consecutive P unit bit values starting from the target bit as unit operation data among the M unit bit values of the padding storage message based on the bit length P of the business operation value, wherein the target bit refers to the highest bit among the bits with unit bit value of 1 in the padding storage message, and the bit length of the unit operation data is the same as the bit length of the business operation value, and the unit operation data can be the binary number 10110.

The logic circuit component can perform an exclusive OR (XOR) operation on the unit operation data 10110 and the business operation value 11001 to obtain a unit redundant value, and the unit redundant value is a binary number 1111 (that is, the operation result of the exclusive OR operation is 01111). The logic circuit component can determine the unit bit values except P unit bit values among the M unit bit values as Q to-be-filled bit values, where Q is the difference between M and P, and the Q to-be-filled bit values can refer to the binary number 0110000. The logic circuit component can perform post-fill processing on the unit redundant value based on the bit length P of the business operation value and the Q to-be-filled bit values to obtain a new unit operation data with the same bit length as the business operation value. The new unit operation data can be a binary number 11110. The logic circuit component can continue to perform an exclusive OR operation on the unit operation data 11110 based on the business operation value 11001 to obtain a unit redundant value 111. Based on the Q to-be-filled bit ratio Based on the bit values in the Q bit values to be filled that are not used for the filling process, the unit redundant value 111 is post-filled to obtain unit operation data 11111 having the same bit length as the business operation value, the unit operation data 11111 and the business operation value 11001 are XORed to obtain unit redundant value 110, based on the bit values in the Q bit values to be filled that are not used for the filling process, the unit redundant value 110 is post-filled to obtain unit operation data 11000 having the same bit length as the business operation value, the unit operation data 11000 is XORed to obtain unit redundant value 1, based on the bit values in the Q bit values to be filled that are not used for the filling process, the unit redundant value 1 is post-filled to obtain unit redundant value 100, at this time, the Q bit values to be filled are all used to complete the post-filling process, and the logic circuit component can determine the finally obtained unit redundant value 100 as the business redundant value 100.

If the bit length of the service redundancy value meets the front-filling condition, the logic circuit component can perform front-filling processing on the service redundancy value, and determine the service redundancy value 0100 after the front-filling processing as the redundancy check value 0100, wherein the front-filling condition means that the bit length of the service redundancy value is less than P-1. If the bit length of the service redundancy value does not meet the front-filling condition, the logic circuit component can directly determine the service redundancy value as the redundancy check value. By judging the service redundancy value based on the front-filling condition, the bit length of the redundancy check value can be unified to P-1.

Step S205, generating N access block storage messages according to the N unit storage messages and the N redundant check values, and transmitting the N access block storage messages to the on-chip system component;

Specifically, please refer to Figure 5. As shown in Figure 5, the logic circuit component can add the message header information of the BLK message to each unit storage message, and generate an access block storage message together with the associated redundant check value, and transmit N access block storage messages to the on-chip system component of the network card. Among them, the access block storage message can be a BLK message obtained by adding message information such as a check value, and an access block storage message can include the message header information of a BLK message, a unit storage message and the corresponding redundant check value. The specific content of the message header information of the BLK message is not limited in the embodiments of the present application.

Step S206, obtain the unit storage messages in N access block storage messages through the on-chip system component, convert the message format of the N unit storage messages through the remote direct access protocol to obtain N unit network messages, and generate message header information corresponding to each unit network message; encapsulate the N unit network messages, the message header information corresponding to the N unit network messages, and the N redundant check values into N transmission network messages respectively; a transmission network message is obtained by encapsulating a unit network message, the corresponding message header information and the associated redundant check value.

Specifically, the system-on-chip component may include a storage development component and a protocol stack component. The storage development component may obtain a unit storage message in N access block storage messages, convert the message format of the unit storage message in the N access block storage messages through a remote direct access protocol (RDMA), obtain N unit network messages, and generate RDMA protocol message header information corresponding to each unit network message, wherein the RDMA protocol message header information may include routing information (Route Header, RH), base transport information (Base Transport Header, BTH) and RDMA protocol information, the routing information may include the source address and destination address of the message in each system, the base transport information includes the type and sequence number of the message, etc., and the RDMA protocol information may include detailed information of the RDMA operation, such as RDMA read or write operation, backend storage address, frontend storage address, etc. The remote direct access protocol is a message-based data transmission protocol, a technology used for high-performance network communication. The unit network message may be a message of a network device in an operating system, namely, a NET message.

The storage development component can encapsulate N unit network messages, the message header information corresponding to the N unit network messages, and N redundant check values into N transmission network messages. Among them, the transmission network message can be a NET message whose message format conforms to the RDMA protocol, that is, a NET-RDMA message. A transmission network message can include a message header information of an RDMA protocol, a unit network message, and the associated redundant check value. The storage length of the N unit network messages is equal to the target storage length, and the storage length of the N transmission network messages is less than or equal to the unit storage length.

Step S207, sending N transmission network messages to the protocol stack component; in the protocol stack component, obtaining the unit network message in each transmission network message and the redundant check value associated with the unit network message; based on the N unit network messages, generating N target check values, if the target check value corresponding to each unit network message is the same as the associated redundant check value, then sending the N transmission network messages to the second device through the protocol stack component.

Specifically, the storage development component may transmit N transmission network messages to the protocol stack component in the system-on-chip component, and the protocol stack component may verify the correctness of the unit network message in the transmission network message obtained by message format conversion.

The protocol stack component can obtain the unit network message in each transmission network message and the redundant check value associated with the unit network message, perform XOR division operation on the business operation value and N unit network messages, and obtain N target check values. If the target check value corresponding to each unit network message is the same as the associated redundant check value, the protocol stack component can perform kernel bypass processing (Bypass Kernel) on each transmission network message through the integrated user-mode communication component (RDMA). Kernel bypass processing allows the message to bypass the kernel processing component (Kernel) and directly access the node storage of the second device remotely through the RDMA protocol.

It can be understood that through kernel bypass processing, the transmission of network messages does not need to be processed by kernel processing components, which can reduce the overhead of copying and verifying messages between different system components and reduce processing delays.

The protocol stack component can use the RDMA API (Application Programming Interface), for example, to create an RDMA connection with the second device through the libibverbs library, wherein the libibverbs library is a standard library that provides a series of RDMA interfaces for performing RDMA operations, including but not limited to memory registration, sending and receiving data, performing RDMA read and write, etc. The protocol stack component can register the storage area in the node storage component of the second device, send N transmission network messages to the second device, and the second device verifies the redundant check value in each transmission network message through the node storage component, and writes the N transmission network messages into the node storage component when all the N transmission network messages are verified.

It can be understood that the reading task of the storage service and the writing task of the storage service can be a reverse process. The process of the reading task of the storage service can be: through the on-chip system component, obtain N transmission network messages sent by the second device through the node storage component, and verify the redundant check value in each transmission network message. When the N transmission network messages are all verified, the N transmission network messages are sent to the logic circuit component; in the logic circuit component, the unit network messages in the N transmission network messages are converted into message formats through the remote direct access protocol to obtain N unit storage messages, and based on the message header information respectively associated with the N unit storage messages, the N unit storage messages are reorganized into an initial block storage message; the initial block storage message is transmitted to the first device.

Specifically, the second device can obtain the N stored transmission network messages from the node storage component, check the redundant check value in each transmission network message, and when the N transmission network messages are all checked and passed, send the N transmission network messages to the network card. The network card can receive the N transmission network messages through the protocol stack component, pass the N transmission network messages to the storage development component, and the storage development component converts the message format of the unit network message in the N transmission network messages to obtain N unit storage messages. The N unit storage messages are passed to the logic circuit component, and the logic circuit component reorganizes the N unit storage messages into an initial block storage message, and sends the initial block storage message to the block storage component in the first device for storage.

The embodiment of the present application combines the characteristics of the unit storage length of the node storage component, the maximum transmission unit of the BLK message, and the maximum transmission unit of the NET message. At the source of the network card, the initial block storage message transmitted by the block storage component is divided by a target storage length that is less than or equal to the unit storage length, and N unit storage messages with the target storage length are obtained, so that the storage length of the unit storage message can be less than or equal to the maximum transmission unit of the unit network message in the remote direct access protocol. After the unit storage message is converted into a message format through the remote direct access protocol in the on-chip system component, the storage length of the obtained unit network message is the same as the storage length of the unit storage message, so that the correctness of the unit network message can be verified by the redundant check value of the unit storage message, so that when the unit storage message is transmitted between the logic circuit component and the on-chip system component, there is no need to perform message fragmentation due to the difference in the maximum transmission unit of the BLK message and the maximum transmission unit of the NET message, nor is there any need to perform multiple correctness checks due to the different redundant check values of the BLK message and the NET message, and there is no need to perform integrity checks on the message fragments. N transmission network messages are generated through N unit network messages and N redundant check values, one transmission network message includes a unit network message and an associated redundant check value, the storage length of the N unit network messages is equal to the target storage length, the storage length of the N transmission network messages is less than or equal to the unit storage length, and the transmission network message can be a NET message whose message format conforms to the RDMA protocol, that is, a NET-RDMA message. The N transmission network messages are sent to the second device, and the second device verifies each transmission network message through the node storage component. When the N transmission network messages are all verified, the N transmission network messages are written into the node storage component. Since the storage length of the unit network message is the same as the storage length of the unit storage message, the redundant check value generated by the unit network message is also the same as the redundant check value generated by the unit storage message. The redundant check value generated by the logic circuit component can be carried to the end of the network card, which reduces the computing resource consumption of the on-chip system component, reduces the overall link delay, and improves the processing efficiency of the network card storage service.

On the other hand, there is no need to perform multiple redundant check value checks between different components of the network card, so each component does not need to implement exception handling after an error, reducing the processing cost of exception handling. At the same time, due to the unified redundant check value, the verification algorithm between the first device and the second device can be consistent, which is convenient for code reuse, implementation and verification. By implementing kernel bypass processing of messages by the protocol stack component that can implement the RDMA protocol, data errors caused by factors such as kernel processing component failure and link errors can be avoided. In addition, through kernel bypass processing, the transmission of network messages does not require kernel processing components, which can reduce the overhead of copying and verifying messages between different system components and reduce processing delays.

Please refer to FIG. 6, which is a flow chart of a data processing method based on a network card provided in an embodiment of the present application. The data processing method can be executed by a network card, and the network card can be the network card 200 shown in FIG. 1. The following will take the data processing method executed by the network card as an example for description. The data processing method may include at least the following steps S301-S303:

Step S301, obtaining N transmission network messages sent by the second device through the node storage component through the on-chip system component, verifying each transmission network message, and sending the N transmission network messages to the logic circuit component when all the N transmission network messages are verified; the storage lengths of the N transmission network messages are less than or equal to the unit storage length of the node storage component in the second device; a transmission network message includes a unit network message and a corresponding redundancy check value;

Specifically, the network card may include a logic circuit component and a system-on-chip component, and the first device may establish a communication connection with the second device through the network card to exchange data. The first device may be a front-end device, also known as a host, and the network card may be an intelligent network card, a high-performance network access card with a network processor as the core, which may be used to process various network protocols or storage protocols from the host. The logic circuit component may be implemented by ASIC or FPGA. The second device may be a back-end device that provides storage business services.

The first device may run multiple protocol queues, including a storage service read queue and a storage service write queue. The storage service read queue may be used to execute storage service read tasks, and the storage service write queue may be used to execute storage service write tasks. The multiple protocol queues may be stored in a block storage component.

In the reading task of the storage service, the network card can obtain N transmission network messages sent by the second device through the node storage component through the on-chip system component, wherein the storage lengths of the N transmission network messages are all less than or equal to the unit storage length of the node storage component in the second device, the unit storage length can be the maximum storage length of the storage unit in the node storage component, and the transmission network message can be a NET message whose message format conforms to the RDMA protocol, i.e., a NET-RDMA message. A transmission network message may include a unit network message and an associated redundant check value. The unit network message may be a message of a network device in an operating system, i.e., a NET message. The specific generation process of the N transmission network messages can refer to the description of steps S201 to S207 of the embodiment corresponding to FIG. 4 above, and the embodiments of the present application will not be repeated here.

The system-on-chip component can obtain the unit network message in each transmission network message and the redundant check value associated with the unit network message, perform XOR division operation on the service operation value and N unit network messages, and obtain N target check values. If the target check value corresponding to each unit network message is the same as the associated redundant check value, the N transmission network messages are sent to the logic circuit component.

Step S302, in the logic circuit component, the unit network messages in the N transmission network messages are converted into message formats through the remote direct access protocol to obtain N unit storage messages, and based on the message header information respectively associated with the N unit storage messages, the N unit storage messages are reorganized into initial block storage messages; the storage length of the N unit network messages and the storage length of the N unit storage messages are both equal to the target storage length, and the target storage length is less than the unit storage length;

Specifically, the logic circuit component can convert the message format of the unit network message in the N transmission network messages through the remote direct access protocol to obtain N unit storage messages. Among them, the remote direct access protocol is a message-based data transmission protocol, a technology used for high-performance network communication. The unit storage message can be a message fragment in BLK format, and the logic circuit component can reorganize the N unit storage messages into an initial block storage message based on the message header information respectively associated with the N unit storage messages. The initial block storage message can be an IO access message for the block storage component in the operating system, that is, a BLK message.

Step S303: Transmit the initial block storage message to the first device.

Specifically, the logic circuit component can transmit the initial block storage message to the first device, and the first device can store the initial block storage message through the block storage component. The block storage component can be a device component such as a hard disk drive, a solid state drive, etc. that uses a fixed-size data block as a basic storage unit.

The embodiment of the present application obtains N transmission network messages sent by the second device through the node storage component through the on-chip system component, verifies each transmission network message, and sends the N transmission network messages to the logic circuit component when the N transmission network messages are all verified. Among them, the storage lengths of the N transmission network messages are all less than or equal to the unit storage length of the node storage component in the second device; a transmission network message includes a unit network message and a corresponding redundant check value. In the logic circuit component, the message format of the unit network message in the N transmission network messages is converted through the remote direct access protocol to obtain N unit storage messages, and based on the message header information respectively associated with the N unit storage messages, the N unit storage messages are reorganized into an initial block storage message. Among them, the storage length of the N unit network messages and the storage length of the N unit storage messages are both equal to the target storage length, and the target storage length is less than the unit storage length, and the initial block storage message is transmitted to the first device. By combining the characteristics of the unit storage length of the node storage component, the maximum transmission unit of the BLK message, and the maximum transmission unit of the NET message, the initial block storage message is divided by a target storage length that is less than or equal to the unit storage length, and N unit storage messages with the target storage length are obtained, so that the storage length of the unit storage message can be less than or equal to the maximum transmission unit of the unit network message in the remote direct access protocol. After the unit storage message is converted into a message format through the remote direct access protocol in the on-chip system component, the storage length of the obtained unit network message is the same as the storage length of the unit storage message, and the redundant check value generated by the unit network message is also the same as the redundant check value generated by the unit storage message, thereby realizing a unified check algorithm for the logic circuit component, realizing end-to-end data protection from the front end of the first device to the back end of the second device, and improving the security of the storage business.

Please refer to FIG. 7, which is a flowchart diagram 4 of a data processing method based on a network card provided in an embodiment of the present application. The data processing method can be executed by a second device, and the second device can be the second device 300 shown in FIG. 1. The following will take the data processing method executed by the network card as an example for description. The data processing method can at least include the following steps S401-S403:

Step S401, obtaining N transmission network messages sent by the first device through the network card; N is a positive integer, and the storage lengths of the N transmission network messages are all less than or equal to the unit storage length of the node storage component in the second device; a transmission network message includes a unit network message and an associated redundant check value; the N unit network messages are obtained by the network card performing message format conversion on the unit storage messages in the N access block storage messages through the remote direct access protocol, and an access block storage message includes a unit storage message and a redundant check value generated for the unit storage message; the N unit storage messages are obtained by dividing the initial block storage message based on the unit storage length; the storage lengths of the N unit network messages and the storage lengths of the N unit storage messages are both equal to the target storage length, and the target storage length is less than the unit storage length;

Specifically, the first device can establish a communication connection with the second device through a network card to exchange data. The first device can be a front-end device, which can also be called a host. The network card can be an intelligent network card, a high-performance network access card with a network processor as the core, which can be used to process various network protocols or storage protocols from the host. The second device can be a back-end device that provides storage business services.

The second device can obtain N transmission network messages sent by the first device through the network card through the node storage component. Among them, the transmission network message can be a NET message whose message format conforms to the RDMA protocol, that is, a NET-RDMA message. The storage lengths of the N transmission network messages are all less than or equal to the unit storage length of the node storage component in the second device. A transmission network message can include a unit network message and an associated redundant check value. The N unit network messages can be obtained by the network card through the remote direct access protocol (RDMA) to convert the message format of the unit storage message in the N access block storage messages. An access block storage message includes a unit storage message and a redundant check value (CRC) generated for the unit storage message. The unit storage message can be a message fragment in the BLK format. The redundant check value can also be called a cyclic redundancy check, which is a commonly used check code in the communication field, including a series of data encoding rules such as shift and division. Common encoding methods include CRC-32, CRC-16, etc.

The N unit storage messages are obtained by dividing the initial block storage message based on the unit storage length. The initial block storage message can be an IO access message for the block storage component in the operating system, that is, a BLK message. The storage length of the N unit network messages and the storage length of the N unit storage messages are both equal to the target storage length, and the target storage length is less than the unit storage length. The specific generation process of the N transmission network messages can refer to the description of steps S201 to S207 of the embodiment corresponding to Figure 4 above, and the embodiment of the present application will not be repeated here.

Step S402, obtain the unit network message in each transmission network message and the redundant check value associated with the unit network message, generate N target check values based on N unit network messages, if the target check value corresponding to each unit network message is the same as the associated redundant check value, then store each transmission network message separately in the node storage component.

Specifically, the node storage component can obtain the unit network message in each transmission network message and the redundant check value associated with the unit network message, perform XOR division operation on the business operation value and N unit network messages, and obtain N target check values. If the target check value corresponding to each unit network message is the same as the associated redundant check value, each transmission network message is stored in the node storage component respectively.

It can be understood that N unit network messages may be out of order during the transmission process between the network card and the second device. The node storage component can sort each transmission network message before storing it. The process can be: obtaining the message header information in the N transmission network messages, and based on the division and sorting values in the N message header information, determining the storage addresses corresponding to the N transmission network messages respectively, and based on the N storage addresses, storing each transmission network message in the storage unit corresponding to the associated storage address in the node storage component.

Specifically, the node storage component can obtain the message header information in N transmission network messages, wherein the message header information of the RDMA protocol may include routing information (Route Header, RH), basic transmission information (Base Transport Header, BTH), RDMA protocol information and partition sorting value, routing information may include the source address and destination address of the message in each system, basic transmission information includes the type and sequence number of the message, etc., and RDMA protocol information may include detailed information of RDMA operations, such as RDMA read or write operations, backend storage address, frontend storage address, etc. The partition sorting value may determine the arrangement order between N unit storage messages when the initial block storage message is divided to obtain N unit storage messages.

The node storage component can determine the storage addresses corresponding to N transmission network messages based on the division and sorting values in the N message header information, and based on the N storage addresses, store each transmission network message in the storage unit corresponding to the associated storage address in the node storage component.

Step S403, when a storage service read request sent by the first device is obtained, the N stored transmission network messages are obtained from the node storage component, each transmission network message is verified, and when the N transmission network messages are all verified, the N transmission network messages are sent to the network card, so that the network card obtains N unit storage messages through the N transmission network messages, and generates an initial block storage message based on the N unit storage messages; the initial block storage message is used for transmission to the first device.

Specifically, when the network card obtains the storage service read request sent by the first device, the second device can obtain the N stored transmission network messages from the node storage component, and check the redundant check value in each transmission network message. When the N transmission network messages are all checked, the N transmission network messages are sent to the network card. The network card can receive the N transmission network messages through the protocol stack component, and pass the N transmission network messages to the storage development component. The storage development component converts the message format of the unit network message in the N transmission network messages to obtain N unit storage messages. The N unit storage messages are passed to the logic circuit component. The logic circuit component reorganizes the N unit storage messages into an initial block storage message, and sends the initial block storage message to the block storage component in the first device for storage.

The embodiment of the present application obtains N transmission network messages sent by the first device through the network card; N is a positive integer, and the storage lengths of the N transmission network messages are all less than or equal to the unit storage length of the node storage component in the second device; a transmission network message includes a unit network message and an associated redundant check value; the N unit network messages are obtained by the network card through the remote direct access protocol to convert the message format of the unit storage message in the N access block storage messages, and an access block storage message includes a unit storage message and a redundant check value generated for the unit storage message ; N unit storage messages are obtained by dividing the initial block storage message based on the unit storage length; the storage length of the N unit network messages and the storage length of the N unit storage messages are both equal to the target storage length, and the target storage length is less than the unit storage length; obtain the unit network message in each transmission network message and the redundant check value associated with the unit network message, and generate N target check values based on the N unit network messages. If the target check value corresponding to each unit network message is the same as the associated redundant check value, each transmission network message is stored in the node storage component respectively. By combining the characteristics of the unit storage length of the node storage component, the maximum transmission unit of the BLK message, and the maximum transmission unit of the NET message, the initial block storage message is divided by the target storage length that is less than or equal to the unit storage length, and N unit storage messages with the target storage length are obtained, so that the storage length of the unit storage message can be less than or equal to the maximum transmission unit of the unit network message in the remote direct access protocol. After the unit storage message is converted into a message format by the remote direct access protocol in the on-chip system component, the storage length of the unit network message obtained is the same as the storage length of the unit storage message, and the redundant check value generated by the unit network message is also the same as the redundant check value generated by the unit storage message, thereby realizing a unified check algorithm for the logic circuit component, realizing end-to-end data protection from the front end of the first device to the back end of the second device, and improving the security of the storage business. In addition, the transmission process of the N unit network messages in the network card and the second device is sorted by the message header information, thereby improving the feasibility of the storage business.

Please refer to Figure 8, which is a schematic diagram of the structure of a network card provided by an embodiment of the present application. As shown in Figure 8, the network card 1 includes a logic circuit component 510 and a system-on-chip component 520, wherein the system-on-chip component 520 includes a storage development component 521 and a protocol stack component 522.

The logic circuit component 510 is used to obtain an initial block storage message transmitted by the block storage component in the first device, divide the initial block storage message into N unit storage messages with a target storage length based on the unit storage length of the node storage component of the second device, generate a redundant check value for each unit storage message, generate N access block storage messages according to the N unit storage messages and the N redundant check values, and transmit the N access block storage messages to the on-chip system component 520; N is a positive integer; an access block storage message includes a unit storage message and a corresponding redundant check value; the target storage length is less than the unit storage length;

The system-on-chip component 520 is used to convert the message format of the unit storage messages in the N access block storage messages through the remote direct access protocol to obtain N unit network messages, and generate N transmission network messages according to the N unit network messages and the N redundant check values; one transmission network message includes one unit network message and the associated redundant check value; the storage lengths of the N unit network messages are all equal to the target storage length; the storage lengths of the N transmission network messages are all less than or equal to the unit storage length;

The on-chip system component 520 is also used to send N transmission network messages to the second device, so that the second device verifies each transmission network message through the node storage component, and when the N transmission network messages are all verified, the N transmission network messages are written into the node storage component.

in,

The logic circuit component 510 is further used to obtain a unit storage length of a node storage component of the second device, determine a storage length of message header information associated with a remote direct access protocol, and obtain a preset storage length for a redundancy check value;

The logic circuit component 510 is further used to determine the target storage length based on the unit storage length, the storage length of the message header information and the preset storage length.

in,

The logic circuit component 510 is specifically used to perform post-padding processing on the unit storage message based on the bit length P of the business operation value to obtain the padded storage message, and perform an XOR division operation on the padded storage message based on the business operation value to obtain the business redundancy value; P is a positive integer; the business operation value is determined based on the preset storage length for the redundancy check value;

The logic circuit component 510 is specifically used to perform a front-filling process on the service redundancy value if the bit length of the service redundancy value meets the front-filling condition, and determine the service redundancy value after the front-filling process as the redundancy check value;

The logic circuit component 510 is specifically configured to determine the service redundancy value as a redundancy check value if the bit length of the service redundancy value does not meet the leading bit padding condition.

The padding storage message includes M unit bit values, where M is a positive integer;

The logic circuit component 510 is specifically used to determine, based on the bit length P of the service operation value, among the M unit bit values of the bit-padded storage message, the continuous P unit bit values starting at the target bit as the unit operation data, and perform an XOR operation on the unit operation data based on the service operation value to obtain a unit redundancy value; the target bit refers to the highest bit among the bits whose unit bit values are 1 in the bit-padded storage message;

The logic circuit component 510 is specifically used to determine the unit bit values other than P unit bit values among the M unit bit values as Q to-be-filled bit values, perform post-fill processing on the unit redundant value based on the bit length P of the business operation value and the Q to-be-filled bit values, and obtain new unit operation data with the same bit length as the business operation value, and continue to perform XOR operation on the new unit operation data based on the business operation value to obtain a new unit redundant value, until the Q to-be-filled bit values are all used to complete the post-fill processing, and the finally obtained unit redundant value is determined as the business redundant value; Q is the difference between M and P.

in,

The system-on-chip component 520 is specifically used to obtain the unit storage message in the N access block storage messages, convert the message format of the N unit storage messages through the remote direct access protocol to obtain N unit network messages, and generate message header information corresponding to each unit network message;

The on-chip system component 520 is specifically used to encapsulate N unit network messages, the message header information corresponding to the N unit network messages, and N redundant check values into N transmission network messages; a transmission network message is obtained by encapsulating a unit network message, the corresponding message header information and the associated redundant check value.

Among them, the system-on-chip component 520 includes a storage development component 521 and a protocol stack component 522;

The storage development component 521 is used to send N transmission network messages to the protocol stack component 522;

The protocol stack component 522 is used to obtain the unit network message in each transmission network message and the redundant check value associated with the unit network message;

The protocol stack component 522 is also used to generate N target check values based on N unit network messages. If the target check value corresponding to each unit network message is the same as the associated redundant check value, the N transmission network messages are sent to the second device through the protocol stack component 522.

in,

The system-on-chip component 520 is further used to obtain N transmission network messages sent by the second device through the node storage component, verify each transmission network message, and send the N transmission network messages to the logic circuit component 510 when all the N transmission network messages pass the verification;

The logic circuit component 510 is further used to convert the message format of the unit network message in the N transmission network messages through the remote direct access protocol to obtain N unit storage messages, and reorganize the N unit storage messages into an initial block storage message based on the message header information respectively associated with the N unit storage messages;

The logic circuit component 510 is further configured to transmit the initial block storage message to the first device.

Please refer to Fig. 9, which is a second structural diagram of a network card provided by an embodiment of the present application. As shown in Fig. 9, the network card 2 includes a system-on-chip component 610 and a logic circuit component 620.

The system-on-chip component 610 is used to obtain N transmission network messages sent by the second device through the node storage component, verify the redundant check value in each transmission network message, and send the N transmission network messages to the logic circuit component 620 when the N transmission network messages are all verified; the storage lengths of the N transmission network messages are all less than or equal to the unit storage length of the node storage component; a transmission network message includes a unit network message and a corresponding redundant check value;

The logic circuit component 620 is used to convert the message format of the unit network message in the N transmission network messages through the remote direct access protocol to obtain N unit storage messages, and reorganize the N unit storage messages into an initial block storage message based on the message header information respectively associated with the N unit storage messages; the storage length of the N unit network messages and the storage length of the N unit storage messages are both equal to the target storage length, and the target storage length is less than the unit storage length;

The logic circuit component 620 is further configured to transmit the initial block storage message to the first device.

Please refer to Fig. 10, which is a schematic diagram of the structure of a data processing device provided in an embodiment of the present application. As shown in Fig. 10, the data processing device 1 includes a message acquisition module 710 and a message verification module 720.

The message acquisition module 710 is used to acquire N transmission network messages sent by the first device through the network card; N is a positive integer, and the storage lengths of the N transmission network messages are all less than or equal to the unit storage length of the node storage component in the second device; a transmission network message includes a unit network message and an associated redundant check value; the N unit network messages are obtained by the network card performing message format conversion on the unit storage messages in the N access block storage messages through the remote direct access protocol, and an access block storage message includes a unit storage message and a redundant check value generated for the unit storage message; the N unit storage messages are obtained by dividing the initial block storage message based on the unit storage length; the storage lengths of the N unit network messages and the storage lengths of the N unit storage messages are both equal to the target storage length, and the target storage length is less than the unit storage length;

The message verification module 720 is used to obtain the unit network message in each transmission network message and the redundant verification value associated with the unit network message, and generate N target verification values based on N unit network messages. If the target verification value corresponding to each unit network message is the same as the associated redundant verification value, each transmission network message is stored in the node storage component respectively.

In a possible implementation, when the message checking module 720 is used to store each transmission network message in the node storage component, it is specifically used to perform the following operations:

Obtain message header information from N transmission network messages, determine storage addresses corresponding to the N transmission network messages based on the division and sorting values in the N message header information, and based on the N storage addresses, store each transmission network message in the storage unit corresponding to the associated storage address in the node storage component.

In a possible implementation, the message checking module 720 is further configured to perform the following operations:

When a storage service read request sent by the first device is obtained, N stored transmission network messages are obtained from the node storage component, each transmission network message is verified, and when all N transmission network messages are verified, the N transmission network messages are sent to the network card, so that the network card obtains N unit storage messages through the N transmission network messages, and generates an initial block storage message based on the N unit storage messages; the initial block storage message is used for transmission to the first device.

The embodiment of the present application obtains N transmission network messages sent by the first device through the network card; N is a positive integer, and the storage lengths of the N transmission network messages are all less than or equal to the unit storage length of the node storage component in the second device; a transmission network message includes a unit network message and an associated redundant check value; the N unit network messages are obtained by the network card through the remote direct access protocol to convert the message format of the unit storage message in the N access block storage messages, and an access block storage message includes a unit storage message and a redundant check value generated for the unit storage message ; N unit storage messages are obtained by dividing the initial block storage message based on the unit storage length; the storage length of the N unit network messages and the storage length of the N unit storage messages are both equal to the target storage length, and the target storage length is less than the unit storage length; obtain the unit network message in each transmission network message and the redundant check value associated with the unit network message, and generate N target check values based on the N unit network messages. If the target check value corresponding to each unit network message is the same as the associated redundant check value, each transmission network message is stored in the node storage component respectively. By combining the characteristics of the unit storage length of the node storage component, the maximum transmission unit of the BLK message, and the maximum transmission unit of the NET message, the initial block storage message is divided by the target storage length that is less than or equal to the unit storage length, and N unit storage messages with the target storage length are obtained, so that the storage length of the unit storage message can be less than or equal to the maximum transmission unit of the unit network message in the remote direct access protocol. After the unit storage message is converted into a message format by the remote direct access protocol in the on-chip system component, the storage length of the unit network message obtained is the same as the storage length of the unit storage message, and the redundant check value generated by the unit network message is also the same as the redundant check value generated by the unit storage message, thereby realizing a unified check algorithm for the logic circuit component, realizing end-to-end data protection from the front end of the first device to the back end of the second device, and improving the security of the storage business. In addition, the transmission process of the N unit network messages in the network card and the second device is sorted by the message header information, thereby improving the feasibility of the storage business.

In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program that has a predetermined function and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories), or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

Please refer to Figure 11, which is a structural diagram of a computer device provided in an embodiment of the present application. As shown in Figure 11, the computer device 1000 may include: a processor 1001, a network interface 1004 and a memory 1005. In addition, the above-mentioned computer device 1000 may also include: a user interface 1003, and at least one communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. Among them, the user interface 1003 may include a display screen (Display), a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory or a non-volatile memory (non-volatile memory), such as at least one disk memory. The memory 1005 may also be at least one storage device located away from the aforementioned processor 1001. As shown in Figure 11, the memory 1005 as a computer-readable storage medium may include an operating system, a network communication module, a user interface module, and a device control application.

In the computer device 1000 shown in FIG. 11 , the network interface 1004 can provide a network communication element; the user interface 1003 is mainly used to provide an input interface for the user; and the processor 1001 can be used to call the device control application stored in the memory 1005 to achieve:

Obtain N transmission network messages sent by the first device through the network card, and transmit the N transmission network messages to the node storage component; N is a positive integer, and the storage lengths of the N transmission network messages are all less than or equal to the unit storage length of the node storage component in the second device; a transmission network message includes a unit network message and an associated redundant check value; the N unit network messages are obtained by the network card performing message format conversion on the unit storage messages in the N access block storage messages through the remote direct access protocol, and an access block storage message includes a unit storage message and a redundant check value generated for the unit storage message; the N unit storage messages are obtained by dividing the initial block storage message based on the unit storage length; the storage length of the N unit network messages and the storage length of the N unit storage messages are both equal to the target storage length, and the target storage length is less than the unit storage length;

Obtain the unit network message in each transmission network message and the redundant check value associated with the unit network message, generate N target check values based on N unit network messages, and if the target check value corresponding to each unit network message is the same as the associated redundant check value, store each transmission network message separately in the node storage component.

It should be understood that the computer device 1000 described in the embodiment of the present application can execute the description of the data processing method in any of the embodiments corresponding to FIG. 7 above, which will not be repeated here. In addition, the description of the beneficial effects of adopting the same method will not be repeated.

In addition, it should be pointed out here that: the embodiment of the present application also provides a computer-readable storage medium, and the above-mentioned computer-readable storage medium stores a computer program. When the above-mentioned processor executes the above-mentioned computer program, it can execute the description of the above-mentioned data processing method in any of the embodiments corresponding to Figure 7 above, so it will not be repeated here. In addition, the description of the beneficial effects of using the same method will not be repeated. For technical details not disclosed in the computer-readable storage medium embodiment involved in this application, please refer to the description of the method embodiment of this application.

The computer-readable storage medium may be the data processing device provided in any of the aforementioned embodiments or the internal storage unit of the computer device, such as a hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the computer device. Further, the computer-readable storage medium may also include both the internal storage unit of the computer device and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer-readable storage medium may also be used to temporarily store data that has been displayed or is to be displayed.

In addition, it should be pointed out here that: the embodiment of the present application also provides a computer program product, which includes a computer program, and the computer program is stored in a computer-readable storage medium. The processor of the computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the computer device executes the method provided by any of the embodiments corresponding to Figure 7 above.

The terms "first", "second", etc. in the description, claims, and drawings of the embodiments of the present application are used to distinguish different objects, rather than to describe a specific order. In addition, the term "comprising" and any variation thereof are intended to cover non-exclusive inclusions. For example, a process, method, device, product, or equipment comprising a series of steps or units is not limited to the listed steps or modules, but may optionally include steps or modules that are not listed, or may optionally include other step units inherent to these processes, methods, devices, products, or equipment.

Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of network elements in the above description. Whether these network elements are executed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described network elements for each specific application, but such implementation should not be considered to be beyond the scope of this application.

The method and related device provided by the embodiment of the present application are described with reference to the method flow chart and/or structural diagram provided by the embodiment of the present application, and can be specifically implemented by computer program instructions for each process and/or box of the method flow chart and/or structural diagram, and the combination of the process and/or box in the flow chart and/or block diagram. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable device to produce a machine, so that the instructions executed by the processor of the computer or other programmable device produce a device for realizing the function specified in one process or multiple processes of the flow chart and/or one box or multiple boxes of the structural diagram. These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, and the instruction device realizes the function specified in one process or multiple processes of the flow chart and/or one box or multiple boxes of the structural diagram. These computer program instructions may also be loaded onto a computer or other programmable device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and/or one or more boxes in the structure diagram.

The steps in the method of the embodiment of the present application can be adjusted in order, combined and deleted according to actual needs.

The modules in the device of the embodiment of the present application can be merged, divided and deleted according to actual needs.

The above disclosure is only the preferred embodiment of the present application, which certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims (17)

1. The data processing method based on the network card is characterized in that the method is applied to the network card, and the network card comprises a logic circuit component and a system-on-chip component; the method comprises the following steps:

The method comprises the steps that an initial block storage message transmitted by a block storage component in first equipment is obtained through a logic circuit component, the initial block storage message is divided into N unit storage messages with target storage lengths based on unit storage lengths of a node storage component of second equipment, redundancy check values of each unit storage message are generated, N access block storage messages are generated according to the N unit storage messages and the N redundancy check values, and the N access block storage messages are transmitted to a system-on-chip component; n is a positive integer; an access block storage message comprises a unit storage message and a corresponding redundancy check value; the target storage length is smaller than the unit storage length;

In the system-on-chip component, converting a message format of a unit storage message in the N access block storage messages through a remote direct access protocol to obtain N unit network messages, and generating N transmission network messages according to the N unit network messages and the N redundancy check values; a transport network message comprising a unit network message and an associated redundancy check value; the storage length of the N unit network messages is equal to the target storage length; the storage length of the N transmission network messages is smaller than or equal to the unit storage length;

And sending the N transmission network messages to the second equipment so that the second equipment checks each transmission network message through the node storage component, and writing the N transmission network messages into the node storage component when all the N transmission network messages pass the check.

2. The method as recited in claim 1, further comprising:

Acquiring unit storage length of a node storage component of the second equipment, determining storage length of message header information associated with a remote direct access protocol, and acquiring preset storage length aiming at a redundancy check value;

and determining a target storage length based on the unit storage length, the storage length of the message header information and the preset storage length.

3. The method of claim 1, wherein generating the redundancy check value for each unit storage message comprises:

Performing post-bit filling processing on the unit storage message based on the bit length P of the service operation value to obtain a bit filling storage message, and performing exclusive OR division operation on the bit filling storage message based on the service operation value to obtain a service redundancy value; p is a positive integer; the service operation value is determined based on a preset storage length for the redundancy check value;

If the bit growth of the service redundancy value meets the pre-bit-supplementing condition, performing pre-bit-supplementing processing on the service redundancy value, and determining the service redundancy value after the pre-bit-supplementing processing as the redundancy check value;

And if the bit length of the service redundancy value does not meet the pre-bit supplementing condition, determining the service redundancy value as the redundancy check value.

4. The method of claim 3, wherein the bit-fill memory message comprises M unit bit values, M being a positive integer; the exclusive or division operation is performed on the bit-filling storage message based on the service operation value to obtain a service redundancy value, which comprises the following steps:

Based on the bit length P of the service operation value, determining continuous P unit bit values taking a target bit as a starting position in M unit bit values of the bit-supplementing storage message as unit operation data, and performing exclusive OR operation on the unit operation data based on the service operation value to obtain a unit redundancy value; the target bit refers to the highest bit in the bit with the unit bit value of 1 in the bit-filling storage message;

Determining the unit bit values except the P unit bit values in the M unit bit values as Q bit values to be complemented, performing post-complementation processing on the unit redundancy values based on the bit length P of a service operation value and the Q bit values to be complemented to obtain new unit operation data with the same bit length as the service operation value, and continuously performing exclusive OR operation on the new unit operation data based on the service operation value to obtain a new unit redundancy value until the Q bit values to be complemented are all used for finishing the post-complementation processing, and determining the finally obtained unit redundancy value as a service redundancy value; q is the difference between M and P.

5. The method of claim 1, wherein in the system-on-chip component, performing message format conversion on unit storage messages in the N access block storage messages by using a remote direct access protocol to obtain N unit network messages, and generating N transport network messages according to the N unit network messages and the N redundancy check values includes:

Obtaining unit storage messages in the N access block storage messages through the system-on-chip component, converting the message format of the N unit storage messages through a remote direct access protocol to obtain N unit network messages, and generating message header information corresponding to each unit network message respectively;

Respectively encapsulating the N unit network messages, the message header information corresponding to the N unit network messages and the N redundancy check values into N transmission network messages; a transport network message is encapsulated by a unit network message, corresponding header information and associated redundancy check values.

6. The method of claim 1, wherein the system on chip component comprises a protocol stack component; the sending the N transport network packets to the second device includes:

transmitting N transmission network messages to the protocol stack assembly;

In the protocol stack assembly, obtaining a unit network message in each transmission network message and a redundancy check value associated with the unit network message;

generating N target check values based on N unit network messages, and if the target check value corresponding to each unit network message is the same as the associated redundancy check value, transmitting the N transmission network messages to the second equipment through the protocol stack assembly.

7. The method as recited in claim 1, further comprising:

acquiring N transmission network messages sent by the second equipment through the node storage component through the system-on-chip component, checking each transmission network message, and sending the N transmission network messages to the logic circuit component when all the N transmission network messages pass the check;

In the logic circuit assembly, converting a message format of a unit network message in the N transmission network messages through a remote direct access protocol to obtain N unit storage messages, and recombining the N unit storage messages into the initial block storage message based on message header information respectively associated with the N unit storage messages;

and transmitting the initial block storage message to the first device.

8. The data processing method based on the network card is characterized in that the method is applied to the network card, and the network card comprises a logic circuit component and a system-on-chip component; the method comprises the following steps:

acquiring N transmission network messages sent by a second device through a node storage component through the system-on-chip component, checking redundancy check values in each transmission network message, and sending the N transmission network messages to the logic circuit component when all the N transmission network messages pass the check; the storage length of the N transmission network messages is smaller than or equal to the unit storage length of the node storage component; a transmission network message comprises a unit network message and a corresponding redundancy check value;

In the logic circuit assembly, converting a message format of a unit network message in the N transmission network messages through a remote direct access protocol to obtain N unit storage messages, and recombining the N unit storage messages into an initial block storage message based on message header information respectively associated with the N unit storage messages; the storage length of the N unit network messages and the storage length of the N unit storage messages are equal to a target storage length, and the target storage length is smaller than the unit storage length;

and transmitting the initial block storage message to the first device.

9. The data processing method based on the network card is characterized in that the method is applied to second equipment, and the second equipment performs data interaction with first equipment through the network card; the method comprises the following steps:

Acquiring N transmission network messages sent by the first equipment through the network card; n is a positive integer, and the storage lengths of the N transmission network messages are smaller than or equal to the unit storage length of the node storage component in the second equipment; a transport network message comprising a unit network message and an associated redundancy check value; the N unit network messages are obtained by converting the message format of unit storage messages in N access block storage messages through a remote direct access protocol by the network card, wherein one access block storage message comprises one unit storage message and a redundancy check value generated for the unit storage message; the N unit storage messages are obtained by dividing the initial block storage messages based on the unit storage length; the storage length of the N unit network messages and the storage length of the N unit storage messages are equal to a target storage length, and the target storage length is smaller than the unit storage length;

and acquiring unit network messages in each transmission network message and redundancy check values associated with the unit network messages, generating N target check values based on the N unit network messages, and if the target check values corresponding to each unit network message are the same as the associated redundancy check values, respectively storing each transmission network message into the node storage component.

10. The method of claim 9, wherein storing each transport network message to the node storage component comprises:

And acquiring message header information in N transmission network messages, determining storage addresses respectively corresponding to the N transmission network messages based on the dividing and sorting values in the N message header information, and storing each transmission network message to a storage unit corresponding to the associated storage address in the node storage component based on the N storage addresses.

11. The method as recited in claim 9, further comprising:

When a storage service reading request sent by the first device is obtained, obtaining N stored transmission network messages from the node storage component, checking each transmission network message, and when all the N transmission network messages pass the check, sending the N transmission network messages to the network card so that the network card obtains the N unit storage messages through the N transmission network messages, and generating the initial block storage message based on the N unit storage messages; the initial block storage message is used for being transmitted to the first device.

12. A network card, comprising:

The logic circuit component is used for acquiring an initial block storage message transmitted by the block storage component in the first equipment, dividing the initial block storage message into N unit storage messages with target storage lengths based on the unit storage length of the node storage component of the second equipment, generating a redundancy check value of each unit storage message, generating N access block storage messages according to the N unit storage messages and the N redundancy check values, and transmitting the N access block storage messages to the system-on-chip component; n is a positive integer; an access block storage message comprises a unit storage message and a corresponding redundancy check value; the target storage length is smaller than the unit storage length;

The system-on-chip component is configured to perform message format conversion on unit storage messages in the N access block storage messages through a remote direct access protocol to obtain N unit network messages, and generate N transport network messages according to the N unit network messages and the N redundancy check values; a transport network message comprising a unit network message and an associated redundancy check value; the storage length of the N unit network messages is equal to the target storage length; the storage length of the N transmission network messages is smaller than or equal to the unit storage length;

the system-on-chip component is further configured to send the N transport network messages to the second device, so that the second device checks each transport network message through the node storage component, and when all of the N transport network messages pass the check, writes the N transport network messages into the node storage component.

13. A network card, comprising:

The system-on-chip component is used for acquiring N transmission network messages sent by the second equipment through the node storage component, checking the redundancy check value in each transmission network message, and sending the N transmission network messages to the logic circuit component when all the N transmission network messages pass the check; the storage length of the N transmission network messages is smaller than or equal to the unit storage length of the node storage component; a transmission network message comprises a unit network message and a corresponding redundancy check value;

The logic circuit component is configured to perform message format conversion on unit network messages in the N transport network messages through a remote direct access protocol to obtain N unit storage messages, and recombine the N unit storage messages into the initial block storage message based on header information respectively associated with the N unit storage messages; the storage length of the N unit network messages and the storage length of the N unit storage messages are equal to a target storage length, and the target storage length is smaller than the unit storage length;

the logic circuit component is further configured to transmit the initial block storage packet to the first device.

14. A data processing apparatus, the data processing apparatus being applied to a second device, the data processing apparatus comprising:

The message acquisition module is used for acquiring N transmission network messages sent by the first equipment through the network card; n is a positive integer, and the storage lengths of the N transmission network messages are smaller than or equal to the unit storage length of the node storage component in the second equipment; a transport network message comprising a unit network message and an associated redundancy check value; the N unit network messages are obtained by converting the message format of unit storage messages in N access block storage messages through a remote direct access protocol by the network card, wherein one access block storage message comprises one unit storage message and a redundancy check value generated for the unit storage message; the N unit storage messages are obtained by dividing the initial block storage messages based on the unit storage length; the storage length of the N unit network messages and the storage length of the N unit storage messages are equal to a target storage length, and the target storage length is smaller than the unit storage length;

and the message check module is used for acquiring the unit network messages in each transmission network message and the redundancy check values associated with the unit network messages, generating N target check values based on the N unit network messages, and respectively storing each transmission network message into the node storage component if the target check value corresponding to each unit network message is the same as the associated redundancy check value.

15. A computer device, comprising: a processor, a memory, and a network interface;

The processor is connected to the memory and the network interface, wherein the network interface is configured to provide a data communication function, the memory is configured to store a computer program, and the processor is configured to invoke the computer program to cause the computer device to perform the method of any of claims 9-11.

16. A computer readable storage medium, characterized in that the computer readable storage medium has stored therein a computer program adapted to be loaded and executed by a processor to cause a computer device having the processor to perform the method of any of claims 9-11.

17. A computer program product, characterized in that the computer program product comprises a computer program stored in a computer readable storage medium and adapted to be read and executed by a processor to cause a computer device with the processor to perform the method of any of claims 9-11.