CN105683934A - Destination port handling for data transfers - Google Patents

Abstract

A method includes receiving a command in a target port, where the command is provided by an initiator and is associated with a write operation. The method comprises the following steps: the target is used to process a data transfer of the initiator associated with the write operation in response to the command. The processing comprises the following steps: based on the characteristics of the command, selectively using memory pre-allocated by the storage array controller before the command is received at the target port for the transfer or requesting allocation of memory from the storage array controller for the transfer.

Description

Destination port handling for data transfers

background method

Computers can access storage area networks (SANs) to store and retrieve large amounts of data. A typical SAN includes pools of consolidated mass storage devices (tape drives, hard drives, optical drives, etc.), and typically provides relatively high-speed block-level storage, which is useful for backup, archiving, database Category purposes can be beneficial.

Description of drawings

1 is a schematic diagram of a computer system according to an example implementation.

2 is a schematic diagram of the physical machine of FIG. 1 including a host adapter and a storage array controller, according to an example implementation.

3 and 4 are flowcharts describing target port processing of commands, according to example implementations.

detailed description

Referring to FIG. 1 , according to an example implementation, a computer system 100 includes a storage area network (SAN) 110 that includes a collection of consolidated physical media storage devices 130 (tape drives, optical drives, hard drives, combinations of these, etc.). Pool, the pool of physical media storage devices may be used by clients 102 (desktops, laptops, tablets, smartphones, etc.) for data storage and retrieval purposes.

As an example, client 102 may communicate with various servers 170 (database, email server, file server, etc.) Retrieve data from SAN110. For the example of FIG. 1 , client 102 may communicate with server 170 using network fabric 106 (eg, a local area network (LAN)-based fabric, a wide area network (WAN)-based fabric, an Internet-based fabric, etc.).

In general, block-level reads and writes generated by servers 170 are processed by storage array controllers 134 of storage arrays 120 (N storage arrays 120-1 . . . 120-N shown in FIG. 1 ) of SAN 110 in order to Data is stored in and retrieved from physical storage devices 130 of array 120 .

As shown in FIG. 1, the server 170 is coupled to the storage array 120 through a SAN structure 180. As the skilled person can understand, the SAN structure 180 may include (as an example) a Fiber Channel (FC) structure, a structure based on the Internet Protocol (IP) , switches, gateways, FC to SCSI bridges, etc.

According to example implementations disclosed herein, server 170 communicates with storage array 120 using a messaging protocol compliant with Fiber Channel Protocol (FCP) or Internet Small Computer System Interface (iSCSI) protocol; and more specifically, server 170 may communicate with storage array 120 by providing A message targeted to a specific host adapter port to initiate a specific read or write operation.

In general, a particular storage array 120 includes one or more host adapters 136, according to an example implementation. Host adapter 136 provides a front-end interface configured to communicate with server 170 and present storage on drives 130 of storage array 120 as a logical unit. Storage array 120 further includes storage array controller 134 which (among its other functions) performs the conversion of logical storage units to physical storage units and provides a backend interface to communicate with associated drives 130 of storage array 120 .

In the context detailed below, for a particular write or read of data transfer between server 170 and storage array 120, an "initiator" (eg, the master port of server 170) initiates a call to a particular "target" by providing a message. For a write or read operation on a port (e.g., a port of host adapter 136), the cancel identifies the target port, includes a write/read command, and specifies one or more characteristics of the associated write/read operation . The message may be an Information Unit (IU) (of the FCP) or a Protocol Data Unit (PDU) (of the iSCSI protocol).

Certain targets and initiators can go through a login process that sets how data transfers take place between these two entities. For example, as a result of a login process, a particular target port may be designated as capable of receiving unsolicited data (or "direct" data) accompanying a write command in a message from the initiator.

Alternatively, the login process may result in the target port being designated not to receive unsolicited data with write commands. For the latter configuration, the target port controls when the initiator provides the data associated with the write, since the initiator does not provide the write data until the target port responds with a message indicating that the target port is ready to receive the write data . This may alternatively be referred to as the target port providing an XFR_RDY (for FCP) or R2T (for iSCSI protocol) signal.

Turning now to a more specific example, an initiator may generate a message including a write command targeting a specific port of host adapter 136 . For this example, as well as other examples described herein, the target port is configured not to receive unsolicited data with write commands. Alternatively, for a write operation, the initiator waits for the target port to provide an indication that it is ready to receive the write data before the initiator provides the data to the target port.

When a target port receives data as part of a write operation, the target port transfers the data into an area of main memory of storage array controller 134 that storage array controller 134 has allocated for this purpose. At any time, the main memory of storage array controller 134 may be allocated to receive data from multiple ongoing write operations.

One way to handle the processing of write commands received at the target port is for the storage array controller to directly participate in the data transfer phase of the write operation. In this approach, the storage array controller controls when the target port asserts (to the initiator) that it is ready to receive write data. In addition, when a write command is received and memory becomes available, the storage array controller allocates an area of its main memory among the target ports for receiving the write data.

More specifically, in the direct method, in response to receiving a write command, the target port first notifies the storage array controller (eg, via an interrupt) of the command. The storage array controller then allocates a portion of its main memory to receive the associated write data and notifies the target port of the allocation. After receiving the memory allocation, the target port responds to the initiator with a ready indication (i.e., the target port provides a message with an XFR_RDY or R2T signal), and the initiator responds by transferring data to the target port. response.

SCSI write transactions can be handled in a manner that reduces the number of interrupts per transaction on that target and improves CPU utilization and latency. One way to reduce the number of interrupts per transaction on that target (and reduce latency for SCSI write requests) is based on the SCSI standard's "first burst" approach, where the target receives a burst of unsolicited data with a write command . The first burst characteristic is set by the initiator and target which negotiate the first burst characteristic during the login process, so that when the target is configured to receive the first burst, the target uses a pre-allocated buffer. Thus, when the initiator sends a write command, the write command is accompanied by write data, and the target uses a pre-allocated buffer to store the data before interrupting the array controller. However, the initiator may not be built or configured to achieve the first burst.

According to the example method disclosed herein, buffers on the target host bus adapter are pre-allocated for non-first burst write transactions, which also allows for a reduced number of interrupts without the involvement of the initiator and does not rely on the initiator to support the first burst Ability.

In this manner, the systems and methods disclosed herein serve to optimize writes (eg, optimize SCSI writes) by pre-allocating memory for transmission of associated write data between initiator and target ports. In this context, "pre-allocated" memory refers to one or more regions of the storage array controller's memory allocated by the storage array controller for the exclusive use of a particular port for future write operations. This pre-allocation means that storage array controller 134 is not directly involved in the data phase of a particular write operation. In this manner, a particular target port is configured to communicate with an initiator for a particular write operation to transmit write data to that target port and store that data in storage array controller 134, according to example implementations disclosed herein. All of this without the involvement of the storage array controller 134. Thus, overhead can be offloaded from storage array controller 134 to the target ports, and the time associated with the transfer of write data can be reduced, among possible benefits.

More specifically, according to an example implementation, storage array controller 134 programs a particular port of host adapter 136 with one or more parameters that allow the port to handle its data transfers using pre-allocated memory. Write-like operations are characterized. To be used by that port for write operations, array controller 134 pre-allocates one or more memory buffers to a particular port.

According to an example implementation, a particular port has exclusive access to one or more memory buffers it has allocated for a specified type of write until the port releases the allocated memory buffers back to the storage array controller 134. until. If a particular write command does not fall within the specified class, then the storage array controller 134 directly participates in the data phase: the target port alerts the storage array controller 134 to accept the command; and, at the target port, sends an indication to the initiator Before being ready to receive a message to write data, the target port waits for memory array controller 134 to allocate memory for transferring the associated write data.

Thus, referring to FIG. 3 , according to an example implementation, a method 300 includes receiving (block 304 ) a write command from an initiator in a port. Pursuant to block 308 of method 300, based on one or more characteristics of the write operation (described by the write command), the The array controller requests the allocation of memory, and the port handles the data transfer associated with the write operation.

Referring to FIG. 2 in conjunction with FIG. 1 , according to an example implementation, host adapter 136 and storage array controller 134 may be part of the same physical machine 200 . In this context, a physical machine 200 is composed of real hardware (central processing unit (CPU), memory devices, bus interfaces, etc.) and real machine-executable instructions or "software" (operating system instructions, driver instructions, application instructions, etc. ) to form a real machine.

As an example, storage array controller 134 may be formed from a main system board of physical machine 200, and main adapter 136 may be formed from a main adapter card that is inserted into a corresponding bus slot on the mainboard. In a further implementation, storage array controller 134 and host adapter 136 may further be formed on the same motherboard. Accordingly, many variations are contemplated which are within the scope of the appended claims.

As shown in FIG. 2 , host adapter 136 may generally include one or more port processors (Q port processors 210-1 . . . 210-Q shown in FIG. 2 ) that form respective Target ports 204 (eg, Q target ports 204-1 . . . 204-Q shown in FIG. 2). In some example implementations, a specific port handler 210 is configured to process read and write operations for a specified target port 204; and in further example implementations, a specific port handler 210 may process target port 204 for read and write operations. As part of this processing, port handler 210 uses pre-allocated memory buffers to handle data transfers associated with write commands within predefined classes, as described herein.

More specifically, according to an example implementation, storage array controller 134 includes one or more central processing units (CPUs) 214 coupled to storage array controller 134 via bridge 218 220 of the main memory.

In general, main memory 220 may temporarily store instructions associated with the execution of machine-executable instructions, as well as data involved in preliminary, intermediate, and final results associated with the process. According to some implementations, main memory 220 may store machine-executable instructions that, when executed by one or more CPUs 214, cause the one or more CPUs 214 to perform all or a portion of the methods disclosed herein, such as the method 300 and method 400 (described below).

In general, the main memory 220 is a non-transitory storage medium that may be formed from a semiconductor storage device, an optical storage device, a magnetic media-based storage device, a removable media device, etc., depending on the specific implementation.

According to an example implementation, a region of main memory 220 is allocated to receive incoming write data. More specifically, according to an example implementation, memory 220 includes a buffer 221 for receiving incoming write data. The buffer 221 is a designated area of the main memory 220 . Buffers 221 may each have the same length or size; or, depending on the particular implementation, buffers 221 may have different sizes.

When the port processor 210 receives write data for an associated write operation, the port processor 210 performs a direct memory access (DMA) to the main memory 220 to store the write data in one or more allocated buffers 221 middle. After the data has been transferred, one or more CPUs 214 may perform functions such as logical-to-physical data unit conversion and store the data in memory devices 130 via one or more input/output (I/O) processors 230 in one or more. The one or more buffers 221 allocated for a particular write command may be pre-allocated prior to receiving the write command, or may be allocated after receiving the write command, depending on whether the associated write falls within an eligible or specified class. assigned after the command.

According to an example implementation, the one or more CPUs 214 identify the eligible or designated class of write operations to be handled by a particular port processor 210 of a particular port 204 and schedule the port processor 210 accordingly. In this regard, one or more CPUs 214 may program that particular port processor 210 with one or more parameters describing the type of write operation, and with the pre-defined port processor 210 to be used when transferring the write data to main memory 220. The port processor 210 is allocated a pool of one or more buffers 221 . According to an example implementation, the pre-allocated one or more buffers 221 are used exclusively by the designated port 204 .

A given class of write operations to be handled by a particular port 204 is defined by one or more parameters describing one or more characteristics of the write operation to be covered. For example, according to some implementations, a particular write operation class includes a write command associated with a particular block length or size of write data. In this way, the block size may be related to the cache line size of the cache memory 222 of the storage array controller 134 . For example, the particular cache size may be 32 kilobytes (kB).

According to a further example implementation, the port handler 210 may consider considerations other than the block size of the written data. For example, according to some example implementations, a particular write operation may be associated with a data offset. Such a write operation may be a write operation in which the offset does not cause the write data to be stored in more than one cache line.

For example, the block size for a particular write operation can be equal to the cache line size. Thus, according to an example implementation, an offset of 10 may result in two cache lines being used and thus may not be considered part of this write class. Likewise, storage array controller 134 handles the initial phase of such a write operation by first allocating one or more buffers 221 to port processor 210, which then notifies the initiator that it is ready.

Referring to FIG. 4 , as a more specific example, a write command may be processed by storage array 120 in accordance with method 400 , according to an example implementation. Method 400 includes the target port receiving (block 404) a command from the initiator. If the target port determines (decision block 408) that the command is a write command, then the target port determines (decision block 412) whether the write command qualifies the port to handle communication with the initiator to transfer data using pre-allocated memory. If so, the target port handles the transfer using (block 416) a buffer from its pre-allocated pool and, according to block 416, indicates to the initiator that it is ready. However, if the write command is not eligible (decision block 412), the port forwards (block 420) the request to the storage array controller, waits for recognition from the controller's memory buffer, and then indicates that the port is ready to receive data. As shown in FIG. 4, after signaling its readiness to the initiator, the target port waits (block 424) to write data.

As indicated by decision block 428, the transfer of data to memory of the storage array controller depends on whether the write is in a class to be handled using pre-allocated memory. In this way, if the write is a qualifying write, then according to block 432, the target port uses DMA to transfer the data to the storage array controller's pre-allocated memory at the appropriate offset. Otherwise, per block 436, the target port uses a DMA transfer to transfer the data to a memory array buffer allocated after receiving the write command.

When the data transfer is complete, the target port notifies (block 440) the storage array controller that the data phase is over. The target port then waits (block 444 ) for status from the storage array controller and, per block 450 , sends (block 450 ) the status to the initiator.

As shown in FIG. 4 , if the command is not a write command (decision block 408 ), the port uses the storage array controller to condition additional processing stages to the initiator per block 446 ; and control passes to block 444 .

Among potential advantages of the methods and systems disclosed herein, write operation performance can be improved without requiring the initiator to modify its standard behavior. Configuration is under full control of the storage array. The storage array controller can experience a significant reduction in interrupt handling, and the storage array controller can experience a significant reduction in its CPU load. The systems and methods disclosed herein may be particularly beneficial for storage arrays with relatively high target port densities. Other and different advantages are contemplated within the scope of the appended claims.

Although a limited number of examples have been disclosed herein, various modifications and changes therein will be apparent to those skilled in the art having the benefit of this disclosure. It is intended in the appended claims to cover all such modifications and changes.

Claims (15)

1. A method comprising: receiving a command in the target port, the command provided by the initiator and associated with the write operation; and processing, using the target, a data transfer for the initiator associated with the write operation in response to receiving the command, wherein the processing includes selectively targeting the transfer based on a characteristic of the command Using memory pre-allocated by the storage array controller prior to receipt of the command by the target port, or requesting an allocation of memory from the storage array controller for the transfer. 2. The method of claim 1 , wherein using the target port comprises: selectively using the target port to communicate with the target port without requesting an allocation of memory from the storage array controller for the transfer. The initiator provides a ready-to-transmit signal. 3. The method of claim 1 , wherein the command comprises a write command, and using the target port comprises: selectively, based at least in part on a block size associated with the data, without first requesting memory The case of allocation indicates that the target port is ready to receive the data. 4. The method of claim 3 , wherein using the target port comprises: selectively, based at least in part on whether the block size exceeds a line size of cache memory used by the storage array controller, without first Requesting memory allocation indicates that the target port is ready to receive the data. 5. The method of claim 1 , wherein using the target port comprises: based at least in part on whether the offset associated with the data matches more than one line of cache memory used by the storage array controller an association, optionally indicating that the target port is ready to receive the data without first requesting an allocation of memory from the storage array controller. 6. The method of claim 1, further comprising: The storage array controller is used to program the target port with at least one parameter indicative of the characteristic. 7. The method of claim 1, further comprising: Memory is preallocated to the target port for the transfer using the storage array controller. 8. A device comprising: a storage array controller configured to allocate memory to receive data associated with a write operation in response to the port indicating receipt of a command associated with the write operation from the initiator; and a port handler for processing the write operation, the port handler selectively based on characteristics of the write operation: providing an indication to the initiator that the port is ready to receive the data; and The data is transferred to a memory region pre-allocated by the storage array controller prior to the port receiving the command. 9. The apparatus according to claim 8, wherein the port processor is planned with a pre-allocated memory area that is the same as that reserved by the storage array controller for use by the port processor At least one memory buffer association for . 10. The apparatus of claim 8, wherein the port processor performs a direct memory access (DMA) transfer of the data to the memory region. 11. The apparatus of claim 8, wherein the port processor selectively waits for a command from the storage array controller after the port processor receives the command based on the nature of the write operation. memory association. 12. The apparatus of claim 8, further comprising: a media storage drive coupled to the storage array controller, wherein the storage array controller stores the data in at least one of the drives. 13. An article comprising a non-transitory computer-readable storage medium storing instructions that, when executed by a computer, cause the computer to: programming a port processor with at least one parameter describing at least one characteristic of a write command from an initiator, said port processor indicating to said initiator that it is ready to receive a data to process a write operation, and transfer said data to a memory region allocated before said write command was received by said port handler; and After the port processor receives another write command not described by the at least one parameter, memory is allocated for the port processor to transfer data associated with the another write command. 14. The article of claim 13, wherein the characteristic includes a chunk size associated with the data. 15. The article of claim 13, wherein the at least one parameter indicates whether the data is to be associated with one cache line or with multiple cache lines.