CN118648275A - Method and apparatus for network interface card (NIC) object consistency (NOC) messages - Google Patents

Abstract

A method and apparatus for checking network interface card (NIC) consistency in remote direct memory access (RDMA) transactions are provided. The method includes: receiving a request to read an object from an address memory from a sender by an RDMA NIC at a receiver, checking when a header version is unlocked and extracting a version number Vobj from each cache line, and verifying whether the Vobj matches the least significant byte (LSB) of the version field ObV in the header of the object. When the Vobj in each cache line matches the ObV: deleting the Vobj from each cache line of the object, reading the data of the object, and sending the object to the sender. If there is no match, retrying a predefined number of times to verify whether each Vobj matches the ObV in the header of the object, and if there is no match, sending a failure response.

Description

Method and apparatus for network interface card (NIC) object consistency (NOC) messages

Technical Field

In some embodiments of the present invention, the present invention relates to communication systems and more particularly, but not limited to, methods and apparatus for network interface card (NIC) object coherency (NOC) messages.

Background Art

Remote direct memory access (RDMA) is direct memory access from one computer's memory to another computer's memory without involving the operating system of either computer. RDMA is a technology widely used for low-latency and high-bandwidth networking in modern data centers and computer clusters. RDMA offloads memory operations from the central processing unit (CPU) to the RDMANIC (RNIC), which directly accesses the memory. This offload saves CPU time, so the CPU is free to perform other tasks. The RDMA software layer uses instructions called "verbs" to perform RDMA operations, which are then converted into work requests that are written to the RNIC queue. Each work request is called a work queue element (WQE). WQEs are used for marked (one-sided write/read/atomic) operations and for unmarked (two-sided send/receive) operations. RDMA peers communicate through queue-pairs (QPs) that provide various transport services. Common QP types (published by IB specifications) are reliable connection (RC), reliable datagram (RD), unreliable datagram (UD), extended reliable connection (XRC), and unreliable connection (UC). Some companies have proprietary QP types, such as Amazon SRD and Mellanox DCT.

Summary of the invention

An object of the present invention is to provide an apparatus and method for testing object consistency and reducing computing resources and network bandwidth and shortening latency of large data packets by transferring object consistency checks and operations to RNIC and by optimizing object consistency flows.

Another object of the present invention is to provide a method for performing a read before write (RBW) request, which reduces computing resources and saves time.

Another object of the present invention is to provide a method for performing RDMA protocol opcodes of compare and swap (CAS) and read as a single operation, wherein the RDMA behavior understands the packet structure and performs the required operations on the packet using CAS and read opcodes, thereby saving time and computing resources.

Other systems, methods, features and advantages of the present invention will be or become apparent to those skilled in the art after studying the following drawings and detailed description. It is intended that all such other systems, methods, features and advantages be included in this description, be within the scope of the present invention, and be protected by the accompanying claims.

In one aspect, the present invention relates to a device for receiving a plurality of transactions, comprising a remote direct memory access (RDMA) network interface card (RNIC), the device being configured to:

receiving a request from a sender to read an object from a memory address;

Check when the header version is unlocked and extract the version number Vobj from each cache line and verify that the Vobj matches the least significant byte (LSB) of the version field of the header of the object;

When the Vobj in each cache line matches the value in the version field:

remove the Vobj from each cache line of the object,

Reading data of the object;

sending the object to the sender;

When the header version is locked, or when the Vobj in each cache line does not match the value in the version field, retry a predefined number of times to verify whether the header version is unlocked and verify whether each Vobj matches the value in the version field in the header of the object, and send a failure response when there is no match.

The RNIC performs the check and sends the object to the sender without Vobj, saving network and computing resources and reducing latency.

In another implementation of the first aspect, the RNIC is further used to:

receiving a request from a sender to write an object to the memory address;

extracting the value in the version field of the header of the object, writing the value into each cache line of the object, and writing the object to the memory address;

A success response is sent to the sender.

In a second aspect, the present invention relates to a device for receiving a plurality of transactions, comprising a remote direct memory access (RDMA) network interface card (RNIC), the device being configured to:

receiving a request from a sender to read an object from a memory address;

extracting data from each data packet of the object;

Calculating a hash function value for the data in each data packet of the object, and updating the calculated hash function value of the current data packet at the temporary RNIC memory address;

At the last data packet, verify whether the updated hash function value matches the value in the hash value field in the header of the object;

When the hash value matches the value in the hash value field in the header of the object, sending the object to the sender;

When the calculated hash value does not match the value in the hash value field of the header of the object, retry a predefined number of times to:

Calculate the hash function value for the data in each data packet, and update the calculated hash function value of the current data packet in the temporary RNIC memory. At the last data packet, verify whether the updated hash function value matches the value in the hash value field in the header of the object. When there is no match, send a failure response.

In another implementation of the second aspect, the RNIC is further used to:

receiving a request from a sender to write an object to the memory address;

calculating the hash function value for the data in each packet of the object, and updating the calculated hash function value of the current packet at a temporary RNIC memory address, and at a last packet, updating the calculated hash function value in the hash value field in the header of the object, and updating the calculated hash function value of the current packet at a temporary sender RNIC memory address, and wherein, at the last packet, the updated hash function value is written to the hash value field in the header of the object;

Writing the object to the memory address;

A success response is sent to the sender.

In another implementation of the second aspect, the RNIC is further used to:

receiving from the sender the request to write the object to the address memory; wherein the hash function value is calculated by the sender RNIC for the data in each packet of the object, and the calculated hash function value of the current packet is updated at a temporary sender RNIC memory address, and wherein, at the last packet, the updated hash function value is written to the hash value field in the header of the object;

Writing the object into the address memory;

A success response is sent to the sender.

In another implementation of the second aspect, the RNIC is further used to:

receiving a read before write (RBW) request from one or more transmitters to read an object from the memory address, wherein a packet of the RBW request includes a bit indicating an expectation to receive a request from the one or more transmitters to write another object to the memory address;

allocating a row in a table, the row having an identification (ID) of the one or more senders, a timestamp of the received RBW request, and the address memory of each sender from which the object was read;

Upon receiving a request to write an object to the memory at the address from another sender, wherein the other sender has a row in the table for a RBW request to read the object from the memory address and a timestamp, wherein the timestamp is less than the timestamp requested by the one or more senders:

sending a notification to the one or more transmitters based on the transmitter ID in the table to avoid sending a request to write to the memory address;

The row for the other transmitter and the row for the one or more transmitter IDs and timestamps are deleted from the table.

In another implementation of the second aspect, the RNIC is further used to:

One or more transmitters transmit an RBW request to read the object from the memory address, wherein a data packet of the RBW request includes a bit indicating that a request to write another object to the memory address is expected to be received from the one or more transmitters.

In another implementation of the second aspect, the RNIC is further used to:

A zero read request is sent, indicating that no request is expected from the one or more senders to write the object to the address memory.

In another implementation of the second aspect, the RNIC is further used to:

receiving a zero read request from the one or more senders, the zero read request indicating that a request to write an object to the address memory is not expected to be sent from the one or more senders;

The row of the one or more transmitter IDs and timestamps is deleted from the table.

In a third aspect, the present invention relates to a device for sending a plurality of transactions, comprising a remote direct memory access (RDMA) network interface card (RNIC), the device being configured to:

Sending a request to a receiver to read an object from a memory address, wherein a version number Vobj is extracted by the receiver from each cache line of the object and the Vobj of each cache line is verified to match an LSB value in a version field of a header of the object;

receiving the object from the receiver when the Vobj in each cache line matches the LSB value in the version field, wherein the receiver deletes the Vobj from each cache line of the object; or

A failure response is received from the receiver when the Vobj in each cache line does not match the LSB value in the version field.

In another implementation of the third aspect, the RNIC is further used to:

sending a request to a receiver to write an object to the memory address; wherein the LSB value in the version field of the header of the object is extracted by the receiver and written into each cache line of the object, and the object is written to the memory address;

A success response is received from the receiver.

In a fourth aspect, the present invention relates to a device for sending a plurality of transactions, comprising a remote direct memory access (RDMA) network interface card (RNIC), the device being configured to:

Sending a request to a receiver to read an object from a memory address; wherein data is extracted by the receiver from each data packet of the object, and a hash function value is calculated for the data in each data packet of the object, and at the last data packet, the updated hash function value is verified by the receiver to match the value in the hash value field in the header of the object;

receiving the object from the receiver when the hash value matches the value in the hash value field in the header of the object;

A failure response is received when the calculated hash value does not match the value in the hash value field of the header of the object.

In another implementation of the fourth aspect, the RNIC is further used to:

Sending a request to a receiver to write an object to a memory address; wherein the hash function value is calculated by the receiver for the data in each data packet of the object, and the calculated hash function value of the current data packet is updated by the receiver at a temporary receiver RNIC memory address, and at the last data packet, the calculated hash function value is updated in the hash value field of the header of the object;

After the object is written to the address memory by the receiver, a success response is received from the receiver.

In another implementation of the fourth aspect, the RNIC is further used to:

Calculating the hash function value for the data in each data packet of the object, and updating the calculated hash function value of the current data packet at the temporary RNIC memory address;

At the last data packet, writing the updated hash function value into the hash value field of the header of the object;

sending the request to write the object to the receiver;

After the object is written to the address memory by the receiver, a success response is received from the receiver.

In a fifth aspect, the present invention relates to a device for optimizing a compare and swap operation and a stream operation of a read request into a single operation in a remote direct memory access (RDMA) transaction, comprising an RDMA network interface card (RNIC), the device being configured to:

Receive a request from the sender for an optimized compare and swap and read operation:

comparing the contents of the first memory address to the first value;

When the content of the first memory address is equal to the first value:

replacing the content of the first memory address with a second value;

Read the contents of the second memory address;

Sending a success response and the content read from the second memory address to the transmitter;

When the content of the first memory address is not equal to the first value:

A failure response is sent to the sender.

In a sixth aspect, the present invention relates to a method for receiving a plurality of transactions, comprising:

At the remote direct memory access (RDMA) network interface card (RNIC):

receiving a request from a sender to read an object from a memory address;

Extracting a data version number Vobj from each cache line of the object, and verifying whether the Vobj of each cache line matches a least significant byte (LSB) value in a version field of a header of the object;

When the Vobj in each cache line matches the value in the version field:

remove the Vobj from each cache line of the object,

Reading data of the object;

sending the object to the sender;

When the Vobj in each cache line does not match the LSB value in the version field, retry a predefined number of times to verify whether each Vobj matches the value in the LSB version field in the header of the object, and when there is no match,

Send a failure response.

In another implementation of the sixth aspect, the method further includes:

receiving a request from a sender to write an object to the memory address;

extracting the LSB value in the version field of the header of the object, writing the value into each cache line of the object, and writing the object to the memory address;

A success response is sent to the sender.

In a seventh aspect, the present invention relates to a method for receiving a plurality of transactions, comprising:

At the remote direct memory access (RDMA) network interface card (RNIC):

receiving a request from a sender to read an object from a memory address;

extracting data from each data packet of the object;

Calculating a hash function value for the data in each data packet of the object, and updating the calculated hash function value of the current data packet at a temporary RNIC memory address;

At the last data packet, verify whether the updated hash function value matches the value in the hash value field in the header of the object;

When the hash value matches the value in the hash value field in the header of the object, sending the object to the sender;

When the calculated hash value does not match the value in the hash value field of the header of the object, retry a predefined number of times to:

Calculate the hash function value for the data in each data packet, and update the calculated hash function value of the current data packet in the temporary RNIC memory. At the last data packet, verify whether the updated hash function value matches the value in the hash value field in the header of the object. When there is no match, send a failure response.

In another implementation of the seventh aspect, the method further includes:

receiving a request from a sender to write an object to the memory address;

Calculating the hash function value for the data in each data packet of the object, and updating the calculated hash function value of the current data packet at a temporary RNIC memory address, and at a last data packet, updating the calculated hash function value in the hash value field in the header of the object;

Write the object to the memory address and send a success response.

In another implementation of the seventh aspect, the method further includes:

Calculating the hash function value for the data in each data packet of the object at the transmitter RNIC, and updating the calculated hash function value of the current data packet at a temporary transmitter RNIC memory address;

At the last data packet, writing the updated hash function value into the hash value field of the header of the object;

sending the request to write the object to the receiver;

receiving, at the receiver RNIC, the request to write the object;

A success response is sent to the sender.

In an eighth aspect, the present invention relates to a method for sending a plurality of transactions, comprising:

At the remote direct memory access (RDMA) network interface card (RNIC):

Send a request to the receiver to read the object from the memory address, wherein a version number Vobj is extracted by the receiver RNIC from each cache line of the object, and the Vobj of each cache line is verified by the receiver RNIC to be consistent with the version field in the header of the object

Whether the LSB value matches;

receiving the object from the receiver RNIC when the Vobj in each cache line matches the LSB value in the version field, wherein the receiver deletes the Vobj from each cache line of the object; or

When the Vobj in each cache line does not match the LSB value in the version field, a failure response is received from the receiver RNIC.

In a ninth aspect, the present invention relates to a method for sending a plurality of transactions, comprising:

At the remote direct memory access (RDMA) network interface card (RNIC):

Sending a request to a receiver RNIC to read an object from a memory address; wherein data is extracted by the receiver RNIC from each data packet of the object, and a hash function value is calculated for the data in each data packet of the object, and at the last data packet, the updated hash function value is verified by the receiver RNIC to match the value in the hash value field in the header of the object;

receiving the object from the receiver when the hash value matches the value in the hash value field of the header of the object;

A failure response is received when the calculated hash value does not match the value in the hash value field of the header of the object.

In a tenth aspect, the present invention relates to a method for optimizing a compare and swap operation and a stream operation of a read request into a single operation in a remote direct memory access (RDMA) transaction, comprising:

At the receiver, when a request for an optimized compare and swap and read operation is received from the sender:

comparing the contents of the first memory address to the first value;

When the content of the first memory address is equal to the first value:

replacing the content of the first memory address with a second value;

Read the contents of the second memory address;

Sending a success response and the content read from the second memory address to the transmitter;

When the content is not equal to the first value:

A failure response is sent to the sender.

Unless otherwise defined, all technical and/or scientific terms used herein have the same meanings as those generally understood by those of ordinary skill in the art to which the embodiments belong. Although methods and materials similar or equivalent to the methods and materials described herein can be used in the practice or testing of the embodiments, exemplary methods and/or materials are described below. In the event of a conflict, this patent specification (including definitions) shall prevail. In addition, these materials, methods and examples are illustrative only and are not necessarily restrictive.

BRIEF DESCRIPTION OF THE DRAWINGS

Some embodiments of the present invention will be described herein by way of example only in conjunction with the accompanying drawings. In particular, the details shown are described in detail in conjunction with the accompanying drawings, and it should be emphasized that the details shown are shown by way of example and for the purpose of illustrative discussion of the embodiments of the present invention. Thus, it will be apparent to those skilled in the art how to implement the embodiments of the present invention based on the description of the accompanying drawings.

In the attached picture:

FIG1 schematically shows a block diagram of an apparatus for checking consistency of NIC objects in an RDMA transaction provided by some embodiments of the present invention;

FIG2 schematically illustrates the layout of an object memory provided by some embodiments of the present invention, wherein the object version field ObV is 64 bits, at the beginning of the header of the object (i.e., in the first cache line), and the LSB of ObV is 16 bits, denoted as Vobj, at the beginning of each of the subsequent cache lines;

FIG3 a schematically illustrates an object memory layout before deleting Vobj from a cache line, according to some embodiments of the present invention;

FIG3 b schematically illustrates an object memory layout after Vobj is deleted from a cache line according to some embodiments of the present invention;

4 is a schematic sequence diagram of an example of checking NIC object consistency in an RDMA transaction by verifying whether Vobj in each cache line is equal to the LSB of the object version number ObV in the header of the object when receiving a read request provided by some embodiments of the present invention;

5 is a schematic sequence diagram of an example of checking NIC object consistency in an RDMA transaction by verifying whether Vobj in each cache line is equal to the LSB of the object version number ObV in the header of the object when receiving a write request provided by some embodiments of the present invention;

6 schematically shows a flowchart of a method for checking NIC object consistency in an RDMA transaction by verifying whether Vobj in each cache line is equal to the LSB of ObV in the header of the object when receiving a read request, provided by some embodiments of the present invention;

7 schematically shows a flowchart of a method for checking NIC object consistency in an RDMA transaction by inserting the LSB of the ObV in the header of the object into each cache line of the object when sending a read request, provided by some embodiments of the present invention;

FIG8 schematically illustrates a memory layout provided by some embodiments of the present invention, wherein an object is stored in memory and has a header field, the value of which is calculated by a hash function of the object;

FIG9 schematically shows a flow chart of a method for checking NIC consistency in an RDMA transaction by a hash function when receiving a read request provided by some embodiments of the present invention;

FIG10 schematically shows a sequence diagram of an example of checking the consistency of a NIC object in an RDMA transaction by a hash function without first locking the object when receiving a read request, according to some embodiments of the present invention;

FIG. 11 schematically shows a flowchart of an example of an RDMA write request provided by some embodiments of the present invention, wherein the hash function is calculated in the receiver RNIC 122;

FIG. 12 schematically shows a sequence diagram of an example of writing an object in an RDMA transaction using a hash function calculated at a receiver RNIC, provided by some embodiments of the present invention;

FIG13 schematically shows a sequence diagram of an example of writing an object in an RDMA transaction using a hash function calculated at a sender RNIC, provided by some embodiments of the present invention;

FIG. 14 schematically shows a flowchart of an example of an RDMA write request provided by some embodiments of the present invention, wherein the hash function is calculated in the sender RNIC 112;

FIG15 schematically shows a flow chart of a method for checking NIC consistency in an RDMA transaction by a hash function when sending a read request provided by some embodiments of the present invention;

FIG16 schematically shows a flow chart of a method for sending a read before write (RBW) request provided by some embodiments of the present invention, wherein the RBW request is a read request with a notification about a write request, and the write request is expected to be sent after the read request;

17a to 17g schematically illustrate examples of methods for read before write (RBW) requests provided by some embodiments of the present invention;

FIG. 18 schematically shows a flow chart of a method for optimizing a compare and swap (CAS) operation and a stream operation of a read request into a single operation in an RDMA transaction, provided by some embodiments of the present invention;

FIG. 19 schematically illustrates a sequence diagram of optimized compare and swap (CAS) and read operations provided by some embodiments of the present invention.

DETAILED DESCRIPTION

In some embodiments of the present invention, the present invention relates to communication systems and more particularly, but not limited to, methods and apparatus for network interface card (NIC) object coherency (NOC) messages.

In multithreaded applications running on multicore systems or running on distributed computing platforms, you often encounter certain data structures that are frequently read but relatively infrequently change. An example is a database server that has a database list that rarely changes but needs to be consulted for every query that accesses the database, and another example is a main memory distributed object store that leverages RDMA. In this case, it is necessary to guarantee extremely fast read access as well as prevent inconsistencies. The way to achieve this is to use lock-free read-only operations.

In a distributed shared memory system using RDMA, the lock-free read method will impose a heavy load on computing resources and network resources.

There are some methods that use object consistency by inserting the least significant byte (LSB) of the version number on each cache line and then checking if the version number has changed. However, this wastes a lot of computing resources and network resources when reading remote objects. The way to implement remote reads in these methods is to issue RDMA read requests through RC QP, and then test the LSB of the version number on each cache line. If the test fails, wait for the timeout value and issue the request again, which wastes CPU, bandwidth and has high latency.

The way these methods implement remote writes is to add the LSB to each cache line on the remote side and then send the object back to its originating machine, wasting computational CPU and incurring network latency.

Currently not supported by RDMA, this is a way to test object consistency by stripping the object's least significant byte (LSB) from each cache line when reading or adding the LSB to each cache line on the remote or local NIC.

Therefore, it is necessary to provide a device and method for testing object consistency, which can check whether the memory area is strictly serializable and consistent on the one hand, and reduce computing resources and network bandwidth and shorten the latency of large data packets on the other hand.

According to some embodiments of the present invention, the present invention provides an apparatus and method for testing object consistency and reducing computing resources and network bandwidth and shortening latency of large data packets by transferring object consistency operations to a NIC and by optimizing object consistency flows.

Before explaining at least one embodiment of the present invention in detail, it should be understood that the present invention is not necessarily limited to the details of the structure and arrangement of the components and/or methods set forth in the following description and/or illustrated in the drawings and/or examples. The present invention may have other embodiments or may be practiced or implemented in various ways.

The present invention may be a system, method and/or computer program product. The computer program product may include one or more computer-readable storage media having computer-readable program instructions that cause a processor to perform various aspects of the present invention.

The computer readable storage medium may be a tangible device capable of retaining and storing instructions for use by an instruction execution device. The computer readable storage medium may be, for example but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the above devices.

The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to a corresponding computing/processing device, or downloaded to an external computer or external storage device via a network such as the Internet, a local area network, a wide area network, and/or a wireless network.

The computer-readable program instructions may be executed entirely on the user's computer and/or computerized device, partially on the user's computer and/or computerized device, as a stand-alone software package, partially on the user's computer (and/or computerized device), partially on a remote computer or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer and/or computerized device via any type of network, including a local area network (LAN) or a wide area network (WAN), and may also be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, an electronic circuit including a programmable logic circuit, a field-programmable gate array (FPGA), or a programmable logic array (PLA), etc. may be executed by utilizing the state information of the computer-readable program instructions to personalize the electronic circuit and execute the computer-readable program instructions to perform various aspects of the present invention.

Here, various aspects of the present invention are described in conjunction with the flowcharts and/or block diagrams of the methods, devices (systems) and computer program products of the embodiments of the present invention. It should be understood that each box in the flowchart illustration and/or block diagram and the combination of boxes in the flowchart illustration and/or block diagram can be implemented by computer-readable program instructions.

The flowchart and block diagram in the figure show the architecture, function and operation of the possible implementation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a section or a partial instruction, including one or more executable instructions for implementing one or more specified logical functions. In some alternative implementations, the functions described in the box may not be implemented in the order described in the figure. For example, in fact, the two boxes shown in succession can be executed almost simultaneously, or sometimes in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and/or flowchart and the combination of boxes in the block diagram and/or flowchart can be implemented by a system based on special-purpose hardware, which performs specific functions or actions, or performs a combination of special-purpose hardware and computer instructions.

Now referring to FIG. 1, a block diagram of an apparatus for checking NIC object consistency in an RDMA transaction provided by some embodiments of the present invention is schematically shown. The apparatus 100 includes a sender 110 and a receiver 120. The sender includes a memory 111, an RDMA NIC (RNIC) 112 further including a processor 113, and one or more applications 114. The receiver 120 includes a memory 121, an RNIC 122 including a processor 123, and one or more applications 124. The sender 110 sends a lock-free read request to the receiver 120 to read an object from the memory 121, which is a remote memory of the sender 110. According to some embodiments of the present invention, the RNIC 122 verifies the object consistency by using the least significant byte (LSB) header version field in each cache line. When the sender 110 requests to read an object from the memory 121, the object has the LSB of the header version number appended to a fixed position on each object cache line. Then, RNIC 122 checks, through processor 123, whether the header version is unlocked, and in the case that it is unlocked, RNIC 122 extracts the version number Vobj from each cache line and verifies whether it matches the LSB of the object version field ObV of the header of the object. The verification is done by comparing the LSB of ObV with Vobj of each cache line.

FIG2 schematically illustrates the layout of an object memory provided by some embodiments of the present invention, wherein the object version field ObV is 64 bits, at the beginning of the header of the object (i.e., in the first cache line), and the LSB of ObV is 16 bits, denoted as Vobj, at the beginning of each cache line in subsequent cache lines. When a new object is created or updated, ObV in the first cache line is incremented and the LSB of ObV is added to a fixed position across each cache line of the object, and then the memory 121 is updated. According to some embodiments of the present invention, when a thread on a remote sender 110 or a local receiver 120 reads an object, verification is performed by looping over all cache lines of the object and checking whether Vobj in each cache line is the same as the LSB of ObV. When the same, the object is read and sent to the sender 110 without Vobj in each cache line.

When the sender 110 reads an object from the receiver 120, it allocates a buffer and writes the object into the buffer without Vobj in each cache line. When the RNIC writes the object into the buffer, it skips the LSB field and does not write it into the buffer. Therefore, the object is received in the buffer without the LSB field, i.e., without Vobj in each cache line. In the case where the header version of the object is locked, or in the case where the Vobj of one or more cache lines is not equal to the LSB of ObV, the receiver retries a predefined number of times to verify that the header version is unlocked and that each Vobj matches the value in the version field in the header of the object. When there is no match, the receiver 120 sends a failure response to the sender 110.

Figure 3a schematically illustrates the object memory layout before deleting Vobj from the cache line provided by some embodiments of the present invention. Figure 3b schematically illustrates the object memory layout after deleting Vobj from the cache line provided by some embodiments of the present invention.

According to some embodiments of the present invention, when the sender 110 sends a write request to the receiver 120 to write an object to the memory 121, the RNIC 122 of the receiver 120 extracts the value ObV in the version field of the header of the object and writes the value to each cache line of the object, and then writes the object to the memory address in the memory 121. When the object is successfully written to the memory 121, the RNIC 122 sends a success response to the sender 110.

4 is a schematic sequence diagram of an example of checking NIC object consistency in an RDMA transaction by verifying whether Vobj in each cache line is equal to the LSB of ObV in the header of the object when a read request is received, provided by some embodiments of the present invention. At 401, a QP is created on the application 114 and sent to the RNIC 112 of the sender. At 402, a read request is sent from the application 114 to the RNIC 112 to read the address A of length L in the memory 121. In RDMA, when a QP is created at the sender, a QP is created at the receiver in parallel, and the receiver RNIC 122 waits to receive a read request from the sender at 403. At 404, a read request is sent from the RNIC 112 at the sender to the RNIC 122 at the receiver, and the read request is received. At 405, when the read request is received, the counter is set to zero, and at 406, the RNIC 122 checks whether the counter is equal to a predefined number X. At 407, when the counter is different from X, the RNIC 122 extracts all values Vobj from each cache line in the requested address and verifies whether they are equal to the value of the LSB of the object version ObV in the header of the object. When not equal, at 408, the RNIC 122 waits for a predefined time interval T and adds 1 to the counter. Then, the RNIC 122 returns to 406 to try the verification again. After the receiver RNIC 122 tries and fails a predefined number of times X, that is, when the counter is equal to X, at 409, a failure response is sent to the sender, and the RNIC 122 returns to 403 and waits for a new read request. In the case where the value of Vobj extracted from each cache line is equal to the LSB of ObV in the header of the object, at 411, the Vobj value is deleted from each cache line of the object, and the object is read and sent to the sender without Vobj in each cache line. Then the RNIC 122 returns to 403 and waits for a new read request.

5 is a schematic sequence diagram of an example of checking NIC object consistency in an RDMA transaction by verifying whether Vobj in each cache line is equal to the LSB of the object version number ObV in the header of the object when a write request is received, provided by some embodiments of the present invention. At 501, a QP is created on the application 114 and sent to the RNIC 112 of the sender. At 502, a write request is sent from the application 114 to the RNIC 112 to write the object to address A of length L in the memory 121. The data packet sent from the sender in the write request does not have a value of Vobj in the cache line. In RDMA, when a QP is created at the sender, a QP is created at the receiver in parallel, and the receiver RNIC 122 waits to receive a write request from the sender at 503. At 504, a write request is sent from the RNIC 112 at the sender to the RNIC 122 at the receiver, and the write request is received. When a write request is received, the RNIC 122 extracts the LSB value of the ObV in the header of the object and inserts the value at the beginning of each cache line at 505. After inserting the LSB value into each cache line, at 506, the object is written to the memory 121, and at 507, the RNIC 122 sends a success response to the sender 110.

6 schematically illustrates a flow chart of a method for checking NIC object consistency in an RDMA transaction by verifying whether Vobj in each cache line is equal to the LSB of ObV in the header of an object when a read request is received, provided by some embodiments of the present invention. At 601, a request to read an object from a memory address on a memory 121 is received at the RNIC 122 of the receiver 120. At 602, the RNIC 122 extracts the data version number Vobj from each cache line of the object and verifies whether the Vobj of each cache line matches the LSB value of the version field ObV in the header of the object. At 603, when the Vobj in each cache line matches the value in the version field, Vobj is deleted from each cache line of the object by the RNIC 122, the data of the object is read, and the object is sent to the sender 110. And at 604, when the Vobj in each cache line does not match the LSB value in the version field, the RNIC 122 retries a predefined number of times to verify whether each Vobj matches the value in the LSB of ObV in the header of the object. When there is no match, a failure response is sent to the sender 110 .

FIG7 schematically shows a flowchart of a method for checking NIC object consistency in an RDMA transaction by inserting the LSB of ObV in the header of an object into each cache line of the object when sending a read request provided by some embodiments of the present invention. At 701, a request to read an object from a memory address is sent by RNIC 112 from transmitter 110 to receiver 120. When the read request is received at receiver 120, version number Vobj is extracted by the receiver from each cache line of the object, and Vobj of each cache line is verified to see if it matches the LSB value of the object version number ObV field of the header of the object. At 702, when Vobj in each cache line matches the LSB value of ObV, transmitter 110 receives the object from receiver 120, wherein Vobj is deleted from each cache line of the object by the receiver, i.e., the object is received in transmitter 110 without Vobj in each cache line. Otherwise, at 703, when Vobj in each cache line does not match the LSB value of ObV, transmitter 110 receives a failure response from the receiver. In the case where a write request requesting to write an object to an address in memory 121 is sent from sender 110 to receiver 120, the LSB value of ObV is extracted and written to each cache line of the object by receiver 120. The object is written to the memory address in memory 121, and sender 110 receives a success response from receiver 120. According to some embodiments of the present invention, the LSB value of the version field in the header of the object is written to the beginning of each cache line.

According to some other embodiments of the present invention, when a read or write request is received, the NIC consistency in the RDMA transaction may be checked by a hash function.

Fig. 8 schematically illustrates a memory layout provided by some embodiments of the present invention, wherein an object is stored in memory and has a header field, the value of which is calculated by a hash function of the object. In some embodiments of the present invention, when an object is created or updated, a hash value of the object is calculated and the hash value is stored in a hash value header field. When an object is read by a thread on a sender (or any remote node) or a local node, it is tested by calculating the hash value of the object and comparing the result with the hash value header. If they are the same, the object is serializable. If they are not the same, the object is being processed by an application and should be reread or discarded.

According to some embodiments of the present invention, any hash function with good distribution can be used, including cyclic redundancy check-16 (CRC-16), cyclic redundancy check-32 (CRC-32), cyclic redundancy check-64 (CRC-64), message digest algorithm 5 (MD5), etc.

The hash value header field is selected according to the hash function used, for example, MD5 uses a 128-bit header, CRC-64 uses a 64-bit header, CRC-32 uses a 32-bit header, and so on.

The hash function covers the entire contents of the object and is calculated by the RNIC that wrote the object to the memory address. If the hash function calculated by the reader matches the value in the hash field in the object's header, the object is consistent.

FIG9 schematically illustrates a flow chart of a method for checking NIC consistency in an RDMA transaction by a hash function when receiving a read request provided by some embodiments of the present invention. At 901, a read request is received in a receiver 120 to read from a memory address in a memory 121. At 902, an RNIC 122 at the receiver 120 extracts data from each data packet of an object. At 903, the RNIC 122 calculates a hash function value for the data in each data packet of the object, and updates the calculated hash function value of the current data packet at a temporary memory address in the RNIC 122. At 904, when the RNIC reaches the last data packet of the object, the RNIC 122 verifies whether the updated hash function value matches the value in the hash value field in the header of the object. At 905, when the hash value matches the value in the hash value field in the header of the object, at 906, the RNIC 122 sends the object to the sender 110 in response to the request to read the object. Alternatively, at 907, when the calculated hash value does not match the value in the hash value field of the header of the object, the RNIC 122 retries a predefined number of times to calculate the hash function value for the data in each data packet. The RNIC 122 updates the calculated hash function value of the current data packet at the temporary RNIC memory. At the last data packet, it is verified whether the updated hash function value matches the value in the hash value field of the header of the object. At 908, when there is no match, the RNIC 122 sends a failure response to the transmitter 110.

FIG10 schematically shows a sequence diagram of an example of checking the consistency of a NIC object in an RDMA transaction by a hash function without first locking the object when receiving a read request, provided by some embodiments of the present invention. At 1001, a QP is created on an application 114 and sent to the RNIC 112 of the sender 110. At 1002, a read request is sent from the application 114 to the RDMA NIC 112 to read an address A of length L in the memory 121. When the QP is created at the sender, the QP is created in parallel at the receiver 120, and the QP in the receiver is created using a hash function calculated on an object stored in the memory 121. The result of the hash function is stored at the hash value field in the header of the object together with the header length. The receiver RDMA NIC 122 waits to receive a read request from the sender 110 at 1003. At 1004, a read request is sent from the RDMA NIC 112 at the sender 110 to the RDMA NIC 122 at the receiver 120, and the read request is received. At 1005, when a read request is received, the counter is set to zero, and at 1006, the RDMA NIC 122 checks whether the counter is equal to a predefined number X. At 1007, when the counter is different from X, the RDMA NIC 122 extracts data from each packet of the object and calculates a hash function for each packet, and updates the calculated hash function value of the current packet at a temporary memory address in the RNIC 122. When the RNIC reaches the last packet of the object, the RNIC 122 verifies whether the updated hash function value matches the value in the hash value field in the header of the object. When the updated hash function value stored in the temporary memory of the RNIC 122 is not equal to the value of the hash value field in the header of the object, at 1008, the RDMA NIC 122 waits for a predefined time interval T and increments the counter by 1. The RDMA NIC 122 then returns to 1006 and attempts verification again. After the receiver RDMA NIC 122 tries and fails a predefined number of times X, i.e., when the counter is equal to X, at 1009, a failure response is sent to the sender 110, and at 1010, the RDMA NIC 122 returns to 1003 and waits for a new read request. In the case where the updated hash function value stored in the temporary memory of the RNIC 122 is equal to the value of the hash value field in the header of the object, at 1011, the RNIC 122 sends the object to the sender 110 in response to the request to read the object. The object is then sent to the application 114 at 1012. Then, at 1013, the RDMA NIC 122 returns to 1003, waiting for a new read request.

According to some other embodiments of the present invention, in an RDMA write request, a hash function is calculated by RNIC 122 in receiver 120. FIG. 11 schematically shows a flowchart of an example of an RDMA write request provided by some embodiments of the present invention, wherein the hash function is calculated in receiver RNIC 122. At 1101, an RDMA write request is sent from sender 110 to receiver 120. In the write request of the example of FIG. 11, data packets are divided into first write, middle write, last write, and immediate data (for example, in InfiniBand (IB)/RDMA over converged Ethernet (RoCE)/RoCE version 2 (RoCEv2)). The write request is received at receiver RNIC 122, and at 1102, the receiver RNIC 122 extracts data from all data packets of an object in memory 121 and calculates a hash function for each data packet of the object. The result of each calculation is stored in a temporary memory of the RNIC 122, such as in a QP context (QPC), and is updated with each calculation of a packet. When the hash function of the last packet is calculated, at 1103, the result is updated, for example, at the QPC, and the updated hash function is then updated at the hash value field in the header of the object. At 1104, the RNIC 122 writes the object in the memory 121, and a success response is sent to the sender 110.

FIG12 schematically shows a sequence diagram of an example of writing an object in an RDMA transaction using a hash function calculated at a receiver RNIC, provided by some embodiments of the present invention. At 1201, a QP is created on an application 114 and sent to the RNIC 112 of the sender 110. At 1202, a write request is sent from the application 114 to the RNIC 112 to write to an address A of length L in the memory 121. When the QP is created at the sender, the QP is created in parallel at the receiver 120, and the QP in the receiver is created using a hash function calculated for an object stored in the memory 121. The result of the hash function is stored at the hash value field in the header of the object together with the header length. The receiver RNIC 122 waits to receive a write request from the sender 110 at 1203. At 1204, a write request is sent from the RNIC 112 at the sender 110 to the RNIC 122 at the receiver 120, and the write request is received. At 1205, upon receiving the write request, the RNIC 122 extracts data from all packets of the object in the memory 121 and calculates a hash function for each packet of the object. The result of each calculation is stored on a temporary memory of the RNIC 122, such as in a QP context (QPC), and is updated with each calculation of the packet. When the hash function of the last packet is calculated, the result is updated, for example, at the QPC. At 1206, the RNIC 122 writes the object in the memory 121, and at 1207, a success response is sent to the sender 110.

According to some other embodiments of the present invention, in the case of an RDMA write request, the hash function may be calculated at the sender RNIC 112 instead of at the receiver RNIC 122. In this case, the receiver 120 receives a request from the sender 110 to write an object to an address memory in the memory 121, and a hash function value calculated by the sender RNIC 112 for the data in each packet of the object. The receiver RNIC 122 writes the object to the memory 121 and sends a success response to the sender 110.

According to some embodiments of the present invention, the hash function is calculated in the sender 110, rather than in the receiver 120. FIG. 13 schematically illustrates a sequence diagram of an example of writing an object in an RDMA transaction using a hash function calculated at the sender RNIC, provided by some embodiments of the present invention. At 1301, a QP is created on the application 114 and sent to the RNIC 112 of the sender 110. The QP in the sender is created using a hash function calculated on an object stored in the memory 111. At 1302, the result of the final value of the hash function is stored at the hash offset field of the header of the object together with the header length. At 1303, a write request is sent from the application 114 to the RNIC 112 to write to the address A of length L in the memory 121. At 1304, the hash function is calculated for the entire object, and at 1305, the write request is sent from the sender RNIC 112 to the receiver RNIC 122. When the QP is created at the sender, the QP is created in parallel at the receiver 120. The receiver RNIC 122 waits to receive a write request from the sender 110 at 1306. At 1307, upon receiving the write request, the RNIC 122 writes the object to the memory 121 and inserts the hash function value calculated at the sender RNIC 112 into the hash field in the header of the object written to the memory 121. At 1307, a success response is sent to the sender 110.

14 schematically illustrates a flowchart of an example of an RDMA write request through RoCEv2 provided by some embodiments of the present invention, wherein the hash function is calculated in the sender RNIC 112. At 1401, an opcode for write and checksum is created. In this opcode, the destination virtual address (VA) of the last SGE to which the data must be written is determined, rather than the source from which the data is obtained. This is because in this case, the data (i.e., the hash function) is stored at the QPC, which is calculated by the sender RNIC 112 and is therefore known to the RNIC 112. At 1402, the RNIC 112 reads data from the memory 111, which is divided into scatter gather entries (SGE). At 1403, the RNIC 112 extracts data from each SGE, calculates the hash function and stores the resulting data in a temporary memory within the QPC of the RNIC 112. At 1404, the RNIC 112 sends the data packet read by the RNIC 112 to the network as a write request, that is, divided into first write, middle write, and last write. In the last SGE read by the RNIC 112, the destination VA is written, so the RNIC knows the destination, that is, the address of the write object. In addition, at 1405, the last SGE is read by the RNIC 112, and at 1406, the final value of the hash function at the QPC stored in the RNIC 112 is written to the additional data packet "write only", which is sent to the network after the first write, middle write, and last write data packets at 1407. At 1408, when there is an immediate data packet, the immediate data packet is sent after the "write only" packet.

Now referring to FIG. 15 , a flowchart of a method for checking NIC consistency in an RDMA transaction by a hash function when sending a read request provided by some embodiments of the present invention is schematically shown. At 1501, the RNIC 112 of the sender 110 sends a read request to the receiver 120 to read an object from a memory address in the memory 121. At 1502, the RNIC 122 of the receiver 120 extracts data from each data packet of the object and calculates a hash function value for the data in each data packet of the object. The calculated hash function value is stored in the QPC using the temporary memory of the RNIC 122, wherein for each data packet, the hash function value is updated at the QPC and sent at the last data packet, and the RNIC 122 verifies whether the updated hash function value matches the value in the hash value field in the header of the object. At 1503, when the hash value matches the value in the hash value field in the header of the object, at 1504, the sender 110 receives the object (i.e., the object successfully read) from the receiver. However, at 1505, when the calculated hash value does not match the value in the hash value field of the object's header, sender 110 receives a failure response at 1506. According to some embodiments of the present invention, the hash function may be calculated by RNIC 112 of sender 110.

According to some embodiments of the present invention, since the RDMA protocol defines specific read behaviors, the RNIC enables the receiver to return a special read response (without returning a payload) for fault detection. The acknowledgment extended transport header (AETH) field can be used. Therefore, a sender that obtains a special read response or an AETH syndrome field skips the packet sequence number (PSN) gap reserved for that particular read response, just as it does when it obtains all packets (PSN) from the receiver. Even if a completion generation signal is not issued for the read operation, the sender uses the completion queue to report an error of the fault to the application.

Now referring to FIG. 16, a flowchart of a method for sending a read before write (RBW) request provided by some embodiments of the present invention is schematically shown, wherein the RBW request is a read request with a notification about a write request, which is expected to be sent after the read request. At 1601, the RNIC 122 receives a read before write (RBW) request from one or more senders to read an object from a memory address in the memory 121, wherein the data packet of the RBW request includes a bit indicating that a request to write the object to the memory address is expected to be received from one or more senders. At 1602, the RNIC 122 allocates a row in a table, which has an identification ID of one or more senders and a timestamp of the exact time of receiving the request to read the object from the memory address of each sender. At 1603, a request to write the object to the address memory is received from another sender, wherein the other sender has a row in the table for the request to read the object from the memory address and a timestamp, wherein the timestamp is less than the timestamp of the one or more sender requests. In this case, at 1604, according to some embodiments of the present invention, the RNIC 122 sends a notification to one or more senders based on the sender ID in the table to avoid sending a request to write to the memory address because the content of the memory address has changed. Then, at 1605, the RNIC 122 deletes the row of the other sender and the row of the one or more sender IDs and timestamp from the table because the RBW request of the other sender is completed and the write request of the one or more senders is avoided. In some embodiments of the present invention, the one or more senders send a RBW request to read an object from a memory, wherein the packet of the RBW request includes a bit indicating that a request to write another object to the memory address is expected to be received from the one or more senders.

In some other embodiments of the present invention, after one or more senders send the RBW request, one or more senders may send a zero read request indicating that a request to write an object to the address memory is not expected from the one or more senders. In this case, the receiver RNIC 122 receives the zero read request from the one or more senders indicating that a request to write an object to the address memory is not expected from the one or more senders, and deletes the row of the one or more sender IDs and timestamps from the table.

Figures 17a to 17g schematically illustrate an example of a method for a read before write (RBW) request provided by some embodiments of the present invention. Figure 17a shows a transmitter B 1702, which sends a RBW request to read from address 0x1234 to the RNIC 1705 of the receiver 1704. According to some embodiments of the present invention, the read request also includes a bit indicating that a write request is expected to be sent from the transmitter B 1702, and when the bit is enabled, the read request is a RBW request. The RNIC 1704 receives the request and allocates a row for the transmitter B 1702 in the table 1706. In this row, the ID of the transmitter B 1702 is stored, and the timestamp 1111 of the receipt of the RBW request from the transmitter B 1702 and the address 0x1234 on which the read and write operations will be performed are also stored. FIG17b schematically shows transmitter C 1703 sending a RBW request to read from address 0x1234 to the RNIC 1705 of the receiver 1704. The read request also includes a bit indicating that a write request is expected from transmitter C 1703. RNIC 1704 receives the RBW request and allocates a row for transmitter C 1703 in table 1706. In this row, the ID of transmitter C 1702 is stored, as well as the timestamp 1130 at which the RBW request from transmitter C 1703 was received and the address 0x1234 to which the read and write operations are to be performed. FIG17c schematically shows transmitter A 1701 sending a request to read from address 0x1234 to the RNIC 1705 of the receiver 1704. The read request does not include any indication that a write request is expected to be sent later from transmitter A 1701. RNIC 1704 receives the request, however, it is not assigned to any row in table 1706.

FIG. 17d schematically illustrates transmitter B 1702 sending a write request to address 0x1234, as indicated in the RBW request previously sent by transmitter B 1702 (as shown in FIG. 17a). FIG. 17e schematically illustrates that the row of transmitter B 1702 is deleted because the RBW request is completed, the write request of transmitter B 1702 is completed, and the write operation is performed on the contents of address 0x1234. Since the contents of address 0x1234 are changed, the RNIC 1705 notifies the remaining transmitters in the table (i.e., transmitter C 1703 in this case) that the contents of address 0x1234 are changed, and deletes the row of transmitter C 1703 from table 1706. Due to this notification, the expected write request from transmitter C 1703 is avoided. In FIG. 17f, transmitter A 1701 sends a write request to address 0x1234. However, since the content of the address has changed in the time between the read request from sender A 1701 and the write request from sender A 1701, the write request fails. Figure 17g schematically shows RNIC 1705 sending a failure notification to sender A 1701. From this example, it can be seen that by using a read request with an indication bit of the expected write request to be sent, the sending of a write request, failure, and failure notification is avoided in the case where the content of the address has changed. This saves time and computing resources.

Now referring to FIG. 18, a flowchart of a method for optimizing the stream operation of compare and swap (CAS) operations and read requests into a single operation in an RDMA transaction provided by some embodiments of the present invention is schematically shown. In a two-phase locking (2PL) protocol, an object needs to be locked before its contents are read. When the object is on a remote node, this is currently accomplished through two one-sided RDMA requests: a CAS (lock object) request and then a read request. This requires two round trips from the local node L to the remote node R. The optimized CAS and read operation is a new opcode CAS-and-READ that combines the two operations: that is, if the CAS at address X is successful, N bytes are read from address Y. If unsuccessful, failure is returned. At 1801, the RNIC 122 receives a request for an optimized compare and swap and read (CAS and read) operation from the transmitter 110. At 1802, the RNIC 122 compares the contents of the first memory address with the first value. At 1803, when the content of the first memory address is equal to the first value, the RNIC 122 replaces the content of the first memory address with the second value. At 1804, the RNIC 122 reads the content of the second memory address, and at 1805, the RNIC 122 sends a success response and the content read from the second memory address to the sender 110. However, at 1806, when the content is not equal to the first value, the RNIC 122 sends a failure response to the sender 110.

FIG. 19 schematically illustrates a sequence diagram of optimized compare and swap (CAS) and read operations provided by some embodiments of the present invention. At 1901, a QP is created at the sender 110 and the receiver 120. At 1902, the RNIC 122 in the receiver 120 waits for the CAS and read request from the sender to arrive. At 1903, the CAS and read operation is sent from the application 114 to the RNIC 112 in the sender 110. The CAS and read operation includes comparing and swapping a destination address and a first value compared with the content of the destination address, replacing a second value of the CAS destination address content, reading the destination address, and reading a length. At 1904, the CAS and read request is sent from the RNIC 112 in the sender to the RNIC 122 in the receiver. At 1905, after receiving the CAS and read request, the RNIC 122 goes to the CAS address, and at 1906, the RNIC 122 compares the content of the CAS address with the first value. At 1907, when the content of the CAS address is not equal to the first value, the RNIC sends a failure response to the RNIC 112 in the transmitter 110, and at 1908, the failure response is sent to the application 114. However, at 1905, when the content of the CAS address is equal to the first value, the RNIC 122 replaces the content of the CAS address with the second value. Then, at 1909, the RNIC goes to the second memory address (read address) and reads the content of the second memory address. Then, at 1910, the RNIC 122 checks whether the read operation is successful. When the read fails and the content of the second address memory is not successfully read, a failure response is sent to the RNIC 112 at 1911, and the failure response is sent from the RNIC 112 to the application 114 at 1911. However, when the reading of the content of the second address memory is successful, at 1913, the RNIC 122 sends a success response with the content read from the second memory address to the transmitter 110. Then, at 1914, a success response is sent to the application 114.

Other systems, methods, features and advantages of the present invention will be or become apparent to those skilled in the art after studying the following drawings and detailed description. It is intended that all such other systems, methods, features and advantages be included in this description, be within the scope of the present invention, and be protected by the accompanying claims.

The description of the various embodiments of the present invention is for illustrative purposes only, and these descriptions are not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terms used herein are selected to best explain the principles, practical applications, or technological advances of the embodiments, or to enable other persons skilled in the art to understand the embodiments disclosed herein, compared to the technologies available on the market.

It is expected that during the life of this patent application, many related methods and apparatus will be developed for checking NIC object consistency, reducing computing resources and network bandwidth, and shortening latency for large data packets.

And the scope of the term method and apparatus for checking NIC object consistency, reducing computing resources and network bandwidth, and shortening latency for large data packets is intended to a priori include all such new technologies.

As used herein, the term "about" refers to ± 10%.

The terms "including", "having" and variations thereof mean "including but not limited to". This term encompasses the terms "consisting of" and "consisting essentially of.

The phrase "consisting essentially of" means that the composition or method may include additional ingredients and/or steps, but only if the additional ingredients and/or steps do not materially alter the basic and novel characteristics of the claimed composition or method.

As used herein, the singular forms "a", "an" and "the" include plural references unless the context clearly indicates otherwise. For example, the term "a compound" or "at least one compound" may include a plurality of compounds, including mixtures thereof.

The word “exemplary” is used herein to mean “serving as an example, instance, or illustration.” Any embodiment described as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments and/or to exclude the incorporation of features of other embodiments.

As used herein, the word “optionally” means “provided in some embodiments and not provided in other embodiments.” Any particular embodiment of the present invention may include a number of “optional” features, unless such features are in conflict.

Throughout this application, embodiments of the present invention may be presented in a range format. It should be understood that the description of the range format is only for convenience and brevity, and should not be interpreted as a fixed limitation on the scope of the present invention. Therefore, the description of the range should be considered to have specifically disclosed all possible sub-ranges and individual numerical values within the range. For example, the description of a range such as from 1 to 6 should be considered to have specifically disclosed sub-ranges such as 1 to 3, 1 to 4, 1 to 5, 2 to 4, 2 to 6, 3 to 6, and individual numbers such as 1, 2, 3, 4, 5, and 6 within the range. This applies regardless of how wide the range is.

When a numerical range is indicated herein, any number (fractional or integer) listed within the indicated range is included. The phrases "a range between a first indicated number and a second indicated number" and "a range from a first indicated number to a second indicated number" are used interchangeably herein to include the first indicated number and the second indicated number and all fractions and integers between the two.

It should be understood that, for the sake of clarity, certain features of the present invention described in the context of a separate embodiment may also be provided in combination in a single embodiment. On the contrary, for the sake of brevity, the various features of the present invention described in the context of a single embodiment may also be provided individually or in any suitable sub-combination or as any suitable other embodiment of the present invention. Certain features described in the context of various embodiments should not be considered as essential features of these embodiments, unless the embodiments are inoperable without these elements.

All disclosures, patents and patent applications mentioned in this specification are incorporated herein by reference in their entirety, to the extent that each individual disclosure, patent or patent application is specifically and individually indicated to be incorporated herein by reference. In addition, any reference or identification to the present application shall not be construed as admitting that such references can be used as prior art of the present invention. For section headings, they should not be construed as necessary limitations. In addition, any one or more priority documents of the present application are incorporated herein by reference in their entirety.

Claims (23)

1. An apparatus for receiving a plurality of transactions, comprising a remote direct memory access (remote direct memory access, RDMA) network interface card (RDMA network INTERFACE CARD, RNIC), the apparatus being configured to:

receiving a request from a sender to read an object from a memory address;

checking when the header version is unlocked and extracting a version number Vobj from each cache line and verifying if the Vobj matches the least significant byte (LEAST SIGNIFICANT byte, LSB) of the version field of the header of the object;

When the Vobj in each cache line matches the value in the version field:

deleting the Vobj from each cache line of the object,

Reading data of the object;

Transmitting the object to the transmitter;

Retrying a predefined number of times when the header version is locked or when the Vobj in each cache line does not match the value in the version field to verify whether the header version is unlocked and to verify whether each Vobj matches the value in the version field in the header of the object, and sending a failure response when there is no match.

2. The RNIC of claim 1, characterized in that said RNIC is further to:

receiving a request from a sender to write an object to the memory address;

Extracting the value in the version field of the header of the object, writing the value into each cache line of the object, and writing the object into the memory address;

And sending a success response to the sender.

3. An apparatus for receiving a plurality of transactions, comprising a remote direct memory access (remote direct memory access, RDMA) network interface card (RDMA network INTERFACE CARD, RNIC), the apparatus being configured to:

receiving a request from a sender to read an object from a memory address;

extracting data from each data packet of the object;

calculating a hash function value for the data in each data packet of the object, and updating the calculated hash function value of the current data packet at a temporary RNIC memory address;

Verifying, at the last data packet, whether the updated hash function value matches a value in a hash value field in the header of the object;

transmitting the object to the transmitter when the hash value matches the value in the hash value field in the header of the object;

Retrying a predefined number of times when the calculated hash value does not match the value in the hash value field of the header of the object to:

Calculating the hash function value for the data in each data packet, updating the calculated hash function value of the current data packet at the temporary RNIC memory, verifying whether the updated hash function value matches the value in the hash value field in the header of the object at the last data packet, and sending a failure response when there is no match.

4. The RNIC of claim 3, characterized in that said RNIC is further to:

receiving a request from a sender to write an object to the memory address;

Calculating the hash function value for the data in each data packet of the object and updating the calculated hash function value for the current data packet at a temporary RNIC memory address, the calculated hash function value in the hash value field in the header of the object at the last data packet;

writing the object into the memory address;

and sending a success response.

5. The RNIC according to claim 4, wherein said RNIC is further configured to:

Receiving the request from the sender to write an object into the address memory; wherein the hash function value is calculated by the sender RNIC for data in each data packet of the object and the calculated hash function value of the current data packet is updated at a temporary sender RNIC memory address, and wherein at the last data packet the updated hash function value is written into the hash value field in the header of the object;

writing the object into the address memory;

And sending a success response to the sender.

6. A device according to claim 3, characterized in that the device is further adapted to:

receiving a read-before-write (read before write, RBW) request to read an object from the memory address from one or more senders, wherein a packet of the RBW request includes a bit indicating a desire to receive a request from the one or more senders to write another object to the memory address;

Allocating a row in a table, the row having an Identification (ID) of the one or more senders, a timestamp of the received RBW request, and the address memory of each sender from which the object was read;

when a request to write an object to the address memory is received from another sender, wherein the other sender has a row in the table of RBW requests for reading the object from the memory address with a timestamp that is less than the timestamp of the one or more sender requests:

Sending a notification to the one or more transmitters according to the transmitter ID in the table to avoid sending a request to write the memory address;

the row of the other transmitter and the row of the one or more transmitter IDs and time stamps are deleted from the table.

7. The apparatus of claim 6, wherein the apparatus is further configured to:

one or more senders send RBW requests to read the objects from the memory address, wherein packets of the RBW requests include bits indicating a desire to receive a request from the one or more senders to write another object to the memory address.

8. The apparatus of claim 7, wherein the apparatus is further configured to:

A zero read request is sent indicating that a request to write an object to the address memory is not expected to be sent from the one or more senders.

9. The apparatus of claim 7, wherein the apparatus is further configured to:

receiving a zero read request from the one or more senders, the zero read request indicating that a request to write an object to the address memory is not expected to be sent from the one or more senders;

the row of the one or more sender IDs and timestamps is deleted from the table.

10. An apparatus for transmitting a plurality of transactions, comprising a remote direct memory access (remote direct memory access, RDMA) network interface card (RDMA network INTERFACE CARD, RNIC), the apparatus being configured to:

transmitting a request to a receiver to read an object from a memory address, wherein a version number Vobj is extracted by the receiver from each cache line of the object, and the Vobj of each cache line is verified as matching LSB values in a version field of a header of the object;

Receiving the object from the receiver when the Vobj in each cache line matches the LSB value in the version field, wherein the receiver deletes the Vobj from each cache line of the object; or (b)

A failure response is received from the receiver when the Vobj in each cache line does not match the LSB value in the version field.

11. The RNIC of claim 10, characterized in that the RNIC is further to:

transmitting a request to a receiver to write an object to the memory address; wherein the LSB value in the version field of the header of the object is extracted by the receiver and written into each cache line of the object, and the object is written into the memory address;

A success response is received from the receiver.

12. An apparatus for transmitting a plurality of transactions, comprising a remote direct memory access (remote direct memory access, RDMA) network interface card (RDMA network INTERFACE CARD, RNIC), the apparatus being configured to:

Transmitting a request to the receiver to read the object from the memory address; wherein data is extracted from each data packet of the object by the receiver and a hash function value is calculated for the data in each data packet of the object and at the last data packet, the updated hash function value verifies by the receiver whether it matches a value in a hash value field in a header of the object;

receiving the object from the receiver when the hash value matches the value in the hash value field of the header of the object;

A failure response is received when the calculated hash value does not match the value in the hash value field of the header of the object.

13. The RNIC of claim 12, characterized in that the RNIC is further to:

Transmitting a request to the receiver to write the object to the memory address; wherein the hash function value is calculated by the receiver for the data in each data packet of the object and the calculated hash function value for a current data packet is updated by the receiver at a temporary receiver RNIC memory address and at a last data packet the calculated hash function value is updated in the hash value field of the header of the object;

and after the object is written into the address memory by the receiver, receiving a successful response from the receiver.

14. The RNIC of claim 13, characterized in that the RNIC is further to:

calculating the hash function value for the data in each data packet of the object, and updating the calculated hash function value of the current data packet at a temporary RNIC memory address;

Writing the updated hash function value to the hash value field of the header of the object at the last data packet;

Transmitting the request to write the object to the receiver;

and after the object is written into the address memory by the receiver, receiving a successful response from the receiver.

15. An apparatus for optimizing compare and exchange operations and streaming operations of read requests as a single operation in a remote direct memory access (remote direct memory access, RDMA) transaction, comprising an RDMA Network Interface Card (RNIC) for:

Receiving a request from the sender for an optimized compare and exchange and read operation:

Comparing the content of the first memory address with a first value;

When the content of the first memory address is equal to the first value:

Replacing the content of the first memory address with a second value;

reading the content of the second memory address;

transmitting a success response to the transmitter and the content read from the second memory address;

When the content of the first memory address is not equal to the first value:

and sending a failure response to the sender.

16. A method for receiving a plurality of transactions, comprising:

At a remote direct memory access (remote direct memory access, RDMA) network interface card (RDMA network INTERFACE CARD, RNIC):

receiving a request from a sender to read an object from a memory address;

extracting a data version number Vobj from each cache line of the object and verifying whether the Vobj of each cache line matches a Least Significant Byte (LSB) value in a version field of a header of the object;

When the Vobj in each cache line matches the value in the version field:

deleting the Vobj from each cache line of the object,

Reading data of the object;

Transmitting the object to the transmitter;

Retrying a predefined number of times when the Vobj in each cache line does not match the LSB value in the version field to verify whether each Vobj matches the value in the LSB version field in the header of the object, and sending a failure response when there is no match.

17. The method as recited in claim 16, further comprising:

receiving a request from a sender to write an object to the memory address;

extracting the LSB value in the version field of the header of the object, writing the value in each cache line of the object, and writing the object to the memory address;

And sending a success response to the sender.

18. A method for receiving a plurality of transactions, comprising:

At a remote direct memory access (remote direct memory access, RDMA) network interface card (RDMAnetwork INTERFACE CARD, RNIC):

receiving a request from a sender to read an object from a memory address;

extracting data from each data packet of the object;

Calculating a hash function value for the data in each data packet of the object, and updating the calculated hash function value of the current data packet at a temporary RNIC memory address;

Verifying, at the last data packet, whether the updated hash function value matches a value in a hash value field in the header of the object;

transmitting the object to the transmitter when the hash value matches the value in the hash value field in the header of the object;

Retrying a predefined number of times when the calculated hash value does not match the value in the hash value field of the header of the object to:

Calculating the hash function value for the data in each data packet, updating the calculated hash function value of the current data packet at the temporary RNIC memory, verifying whether the updated hash function value matches the value in the hash value field in the header of the object at the last data packet, and sending a failure response when there is no match.

19. The method as recited in claim 18, further comprising:

receiving a request from a sender to write an object to the memory address;

Calculating the hash function value for the data in each data packet of the object and updating the calculated hash function value for the current data packet at a temporary RNIC memory address, the calculated hash function value in the hash value field in the header of the object at the last data packet;

And writing the object into the memory address and sending a success response.

20. The method as recited in claim 19, further comprising:

calculating said hash function value for said data in each data packet of said object at a sender RNIC and updating said calculated hash function value for a current data packet at a temporary sender RNIC memory address;

Writing the updated hash function value to the hash value field of the header of the object at the last data packet;

Transmitting the request to write the object to the receiver;

Receiving, at the receiver RNIC, the request to write the object to the memory address;

And sending a success response to the sender.

21. A method for transmitting a plurality of transactions, comprising:

At a remote direct memory access (remote direct memory access, RDMA) network interface card (RDMAnetwork INTERFACE CARD, RNIC):

transmitting a request to a receiver to read an object from a memory address, wherein a version number Vobj is extracted from each cache line of the object by a receiver RNIC, and the Vobj of each cache line is verified by the receiver RNIC as matching LSB values in a version field of a header of the object;

receiving the object from the receiver RNIC when the Vobj in each cache line matches the LSB value in the version field, wherein the receiver deletes the Vobj from each cache line of the object; or (b)

A failure response is received from the receiver RNIC when the Vobj in each cache line does not match the LSB value in the version field.

22. A method for transmitting a plurality of transactions, comprising:

At a remote direct memory access (remote direct memory access, RDMA) network interface card (RDMAnetwork INTERFACE CARD, RNIC):

Transmitting a request to the receiver RNIC to read the object from the memory address; wherein data is extracted from each data packet of the object by the receiver RNIC and a hash function value is calculated for the data in each data packet of the object and at the last data packet, the updated hash function value is verified by the receiver RNIC as matching the value in the hash value field in the header of the object;

receiving the object from the receiver when the hash value matches the value in the hash value field of the header of the object;

A failure response is received when the calculated hash value does not match the value in the hash value field of the header of the object.

23. A method for optimizing compare and exchange operations and streaming operations of read requests as a single operation in a remote direct memory access (remote direct memory access, RDMA) transaction, comprising:

at the receiver, when a request for optimized compare and exchange and read operations is received from the transmitter:

Comparing the content of the first memory address with a first value;

When the content of the first memory address is equal to the first value:

Replacing the content of the first memory address with a second value;

reading the content of the second memory address;

transmitting a success response and the content read from the second memory address to the transmitter;

When the content is not equal to the first value:

and sending a failure response to the sender.