CN118672941B - A task execution method, device, equipment and storage medium - Google Patents

Abstract

The application discloses a task execution method, a device, equipment and a storage medium, which relate to the technical field of caching and are applied to a streaming multiprocessor and comprise the following steps: generating a read request aiming at the self first-level cache based on a calculation task generated by an upper computer; the computing task corresponds to instruction data and operand data; if the read request does not hit the data, forwarding the read request to a preset cross matrix through a first-level cache so as to forward the read request to a second-level cache shared by a plurality of streaming multiprocessors; reading instruction data or operand data from the secondary cache via the primary cache based on the read request; and executing the calculation task according to the read instruction data or operand data, and caching the calculation result into a secondary cache so that the upper computer can process the calculation result corresponding to the calculation task. Therefore, the data corresponding to the calculation task can be efficiently read through the hierarchical cache structure of the independent first-level cache and the shared second-level cache, the cache resources are centrally managed, and the delay is reduced.

Description

A task execution method, device, equipment and storage medium

Technical Field

The present invention relates to the field of cache technology, and in particular to a task execution method, device, equipment and storage medium.

Background Art

GPU (Graphic Processing Unit) usually contains multiple stream multiprocessors (SMP), which significantly accelerate the computing process by executing different data streams at the same time. However, as the number of cores increases, how to efficiently manage the memory hierarchy, especially the L2 cache (L2-Cache), has become an important challenge to improve the overall efficiency of the system. In traditional GPU design, each stream multiprocessor may have an independent L2 cache. Although such a design ensures the independence and data access speed of each core, it has problems such as resource waste, data redundancy, and complex cache consistency maintenance. Especially under highly parallel workloads, data is frequently transferred between cores. Independent cache will cause a large amount of data migration and invalid cache line refresh, which seriously affects system performance and energy efficiency.

It can be seen that how to reduce data transmission delay and improve cache hit rate is a problem to be solved in this field.

Summary of the invention

In view of this, the purpose of the present invention is to provide a task execution method, device, equipment and storage medium, which can efficiently read the data corresponding to the computing task through the hierarchical cache structure of independent first-level cache and shared second-level cache, centrally manage cache resources, and reduce latency. The specific scheme is as follows:

In a first aspect, the present application provides a task execution method, applied to a streaming multiprocessor, comprising:

Generate a read request for its own first-level cache based on the computing task generated by the host computer; the streaming multiprocessor corresponds to an independent first-level cache; the computing task corresponds to instruction data and operand data;

If the read request does not hit data in the first-level cache, the read request is forwarded to a preset cross matrix through the first-level cache, so that the read request is forwarded to a second-level cache shared by a plurality of streaming multiprocessors through the preset cross matrix; an arbitrator and a router are integrated in the preset cross matrix, and are used to forward the read request to the second-level cache according to an arbitration mechanism and an address space corresponding to the read request;

Reading instruction data or operand data corresponding to the computing task from the secondary cache via the primary cache based on the read request;

The computing task is executed according to the read instruction data or the operand data, and the corresponding computing result is cached in the secondary cache so that the host computer can process the computing result corresponding to the computing task.

Optionally, the generating of a read request for its own first-level cache based on the computing task generated by the host computer includes:

Activate its own scheduling unit based on the computing tasks generated by the host computer to generate several thread bundles;

The initial instruction address corresponding to the computing task and the name of the thread warp are passed to its own instruction fetch unit, so that the instruction fetch unit parses the initial instruction address and the name of the thread warp, and generates a read request for its own first-level cache according to the parsing result.

Optionally, the method further includes:

If the read request hits data in the first-level cache, instruction data or operand data corresponding to the computing task is read from the first-level cache through the read request, so as to execute the computing task according to the read instruction data or operand data.

Optionally, reading instruction data or operand data corresponding to the computing task from the first-level cache through the read request includes:

After reading the corresponding instruction data or operand data locally from the first-level cache according to the read request, the instruction data or operand data read from the first-level cache is cached in a preset buffer corresponding to the streaming multiprocessor;

The instruction data or operand data corresponding to the computing task is read from the preset buffer.

Optionally, forwarding the read request to a preset cross matrix through the first-level cache, so as to forward the read request to a second-level cache shared by a plurality of streaming multiprocessors through the preset cross matrix, includes:

The read request is forwarded by the first-level cache via its own corresponding main line interface to the first interface coupler of a preset cross matrix, so that the preset cross matrix uses the internally integrated read arbiter and routing to forward the read request to the second interface coupler, and the read request is passed to the second-level cache via the second interface coupler using the slave line interface corresponding to the second-level cache, so as to complete the process of forwarding the read request to the second-level cache shared by several streaming multiprocessors.

Optionally, executing the computing task according to the read instruction data or the operand data includes:

Selecting a target computing unit that matches the computing task through its own execution unit; the target computing unit includes a logic computing unit, a floating-point computing unit, a memory access unit and a function unit;

The target computing unit is used to execute the computing task corresponding to the read instruction data or operand data to obtain a corresponding computing result.

Optionally, caching the corresponding operation result into the secondary cache includes:

The calculation result is forwarded to the third interface coupler of the preset cross matrix through the master line interface of the first-level cache, so that the preset cross matrix forwards the calculation result to the fourth interface coupler using the internally integrated write arbiter and routing, and caches the calculation result to the second-level cache through the fourth interface coupler using the slave line interface corresponding to the second-level cache.

In a second aspect, the present application provides a task execution device, applied to a streaming multiprocessor, comprising:

A read request generation module, used to generate a read request for its own first-level cache based on a computing task generated by a host computer; the streaming multiprocessor corresponds to an independent first-level cache; the computing task corresponds to instruction data and operand data;

A forwarding module, used for forwarding the read request to a preset cross matrix through the first-level cache when the read request does not hit data in the first-level cache, so as to forward the read request to the second-level cache shared by a plurality of streaming multiprocessors through the preset cross matrix; an arbitrator and a router are integrated in the preset cross matrix, and used for forwarding the read request to the second-level cache according to an arbitration mechanism and an address space corresponding to the read request;

A data reading module, configured to read instruction data or operand data corresponding to the computing task from the secondary cache via the primary cache based on the read request;

The task execution module is used to execute the computing task according to the read instruction data or the operand data, and cache the corresponding computing result to the secondary cache so that the host computer can process the computing result corresponding to the computing task.

In a third aspect, the present application provides an electronic device, including:

Memory, used to store computer programs;

The processor is used to execute the computer program to implement the task execution method as described above.

In a fourth aspect, the present application provides a computer-readable storage medium for storing a computer program, wherein the computer program implements the task execution method as described above when executed by a processor.

It can be seen that the streaming multiprocessor in the present application can generate a read request for its own first-level cache based on the computing task generated by the host computer; the streaming multiprocessor corresponds to an independent first-level cache; the computing task corresponds to instruction data and operand data; if the read request does not hit the data in the first-level cache, the read request is forwarded to a preset cross matrix through the first-level cache, so that the read request is forwarded to the second-level cache shared by several streaming multiprocessors through the preset cross matrix; the preset cross matrix integrates an arbitrator and a router, which is used to forward the read request to the second-level cache according to the arbitration mechanism and the address space corresponding to the read request; then, based on the read request, the instruction data or operand data corresponding to the computing task is read from the second-level cache through the first-level cache; then, the computing task is executed according to the read instruction data or the operand data, and the corresponding calculation result is cached in the second-level cache, so that the host computer can process the calculation result corresponding to the computing task. In this way, the present application constructs a first-level cache in each stream multiprocessor, and at the same time allocates appropriate cache space to each stream multiprocessor in the second-level cache according to the number and address space of the stream multiprocessors; through the hierarchical cache structure of independent first-level cache and shared second-level cache, the data corresponding to the computing task can be read efficiently, and the cache resources can be centrally managed through the shared second-level cache, allowing all stream multiprocessors to access the same cache pool, reducing data replication, reducing data transmission delay, and improving cache hit rate; and such a hierarchical cache structure is highly flexible and adaptable, which can meet the needs of diverse applications.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

FIG1 is a flow chart of a task execution method disclosed in the present application;

FIG2 is a flowchart of a specific task execution method disclosed in the present application;

FIG3 is a schematic diagram of a cross matrix disclosed in the present application;

FIG4 is a schematic diagram of a crosspoint-based M×S crossbar switch architecture disclosed in the present application;

FIG5 is a schematic diagram of a connection relationship between a streaming multiprocessor and a secondary cache disclosed in the present application;

FIG6 is a schematic diagram of the structure of a task execution device disclosed in the present application;

FIG. 7 is a structural diagram of an electronic device disclosed in this application.

DETAILED DESCRIPTION

The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

As shown in FIG1 , an embodiment of the present invention discloses a task execution method, which is applied to a streaming multiprocessor, including:

Step S11, generating a read request for its own first-level cache based on the computing task generated by the host computer; the streaming multiprocessor corresponds to an independent first-level cache; the computing task corresponds to instruction data and operand data.

It can be understood that after the upper computer generates a parallel computing task, it will cache the instruction data and operand data required for the computing task into the first-level cache or second-level cache of the corresponding streaming multiprocessor. The relevant streaming multiprocessor first generates a read request for its own first-level cache based on the computing task to be executed, in order to try to read the instruction data and operand data required for the computing task from the first-level cache.

In a specific embodiment, the generation of a read request for its own first-level cache based on the computing task generated by the host computer may include: activating its own scheduling unit based on the computing task generated by the host computer to generate a number of thread bundles; passing the initial instruction address corresponding to the computing task and the name of the thread bundle to its own instruction fetch unit, so that the instruction fetch unit parses the initial instruction address and the name of the thread bundle, and generates a read request for its own first-level cache according to the parsing result. Specifically, when the streaming multiprocessor needs to execute the computing task of the host computer, it first activates the internal scheduling unit to generate an active thread bundle. If two active thread bundles are generated at the same time, the initial instruction address, the ID (Identity) of the active thread bundle and other parameters are passed to the instruction fetch unit; after parsing the received instruction address and other parameter information, the instruction fetch unit generates a read request signal for reading the instruction data and operand data required for the computing task from the first-level cache.

Step S12: If the read request does not hit data in the first-level cache, the read request is forwarded to a preset cross matrix through the first-level cache, so that the read request is forwarded to the second-level cache shared by several streaming multiprocessors through the preset cross matrix; the preset cross matrix integrates an arbitrator and a router, which are used to forward the read request to the second-level cache according to the arbitration mechanism and the address space corresponding to the read request.

In an embodiment of the present application, if the first-level cache reads the corresponding cache according to the address information corresponding to the read request but does not hit the data, the read request can be forwarded to the preset cross matrix, and the preset cross matrix forwards the read request to the shared second-level cache; the preset cross matrix is integrated with an arbitrator and an address forwarding route, and the arbitrator includes a read arbitrator for the read request and a write arbitrator for the write request; the arbitration mechanism of the preset cross matrix can forward the read request to the corresponding cache area of the second-level cache based on the address space corresponding to the read request. In a specific embodiment, the cross matrix (AXI-Crossbar, a cross matrix based on the AXI (Advanced eXtensible Interface, a bus protocol) bus) can include a master-slave AXI interface coupling module and a Crossbar module, wherein the master-slave interface coupling module is responsible for interconnecting multiple Master (main line, connected to the first-level cache) interfaces or Slave (slave line, connected to the second-level cache) interfaces with the Crossbar (cross matrix) module to form a cross matrix, supporting data caching and data bit width conversion, and the Crossbar module integrates an arbitrator and routing functions internally, and is responsible for data routing and forwarding between master and slave interfaces. If a 3x3 master-slave interconnection network is formed through an AXI-Crossbar cross matrix, Master 0 (the main line connecting the first-level cache corresponding to the streaming multiprocessor with sequence number 0 and the cross matrix) is connected to the Crossbar module through the AXI interface coupling module to realize data forwarding with any interface of Slave0~3.

In a specific embodiment, forwarding the read request to a preset cross matrix through the first-level cache so as to forward the read request to a second-level cache shared by several streaming multiprocessors through the preset cross matrix may include: forwarding the read request to a first interface coupler of a preset cross matrix through the first-level cache via its own corresponding main line interface, so that the preset cross matrix forwards the read request to a second interface coupler using an internally integrated read arbitrator and routing, and passing the read request to the second-level cache through the second interface coupler using a slave line interface corresponding to the second-level cache, so as to complete the process of forwarding the read request to the second-level cache shared by several streaming multiprocessors. Specifically, the first-level caches of different streaming multiprocessors are connected to the preset cross matrix through a separate main line interface, while the preset cross matrix is connected to the second-level cache through a number of slave line interfaces; the first-level cache can forward the read request to the first interface coupler of the preset cross matrix through the main line interface, and the first interface coupler is used to receive the read request forwarded by the first-level cache; then, the preset cross matrix can use the internally integrated read arbitrator and corresponding routing to forward the read request, and the second interface coupler forwards the read request to the corresponding cache area of the second-level cache.

Step S13: Read instruction data or operand data corresponding to the computing task from the second-level cache via the first-level cache based on the read request.

Furthermore, when the read request is forwarded to the L2 cache, the L2 cache will return the instruction data or operand data to the streaming multiprocessor through the L1 cache. It is understandable that the streaming multiprocessor can cache the data read from the cache into its own corresponding preset buffer for task calculation.

In a specific embodiment, it may also include: if the read request hits data in the first-level cache, then the instruction data or operand data corresponding to the computing task is read from the first-level cache through the read request, so as to execute the computing task according to the read instruction data or operand data. Specifically, after the read request issued by the streaming multiprocessor to the first-level cache hits data in the first-level cache, the instruction data or operand data corresponding to the computing task can be directly read from the first-level cache. Further, the reading of the instruction data or operand data corresponding to the computing task from the first-level cache through the read request may include: after reading the corresponding instruction data or operand data locally from the first-level cache according to the read request, the instruction data or operand data read from the first-level cache is cached in a preset buffer corresponding to the streaming multiprocessor; and the instruction data or operand data corresponding to the computing task is read from the preset buffer. The first-level cache may cache the read instruction data or operand data in the preset buffer corresponding to the thread warp of the streaming multiprocessor, and then the streaming multiprocessor may read the corresponding instruction data or operand data from the preset buffer to perform the computing process of the computing task.

Step S14, executing the computing task according to the read instruction data or the operand data, and caching the corresponding computing result into the secondary cache, so that the host computer processes the computing result corresponding to the computing task.

In the embodiment of the present application, when the streaming multiprocessor reads the instruction data and operand data corresponding to the computing task, it executes the computing task and can cache the corresponding operation result to the secondary cache for processing by the host computer.

In a specific embodiment, the execution of the computing task according to the instruction data or operand data read may include: selecting a target computing unit matching the computing task through its own execution unit; the target computing unit includes a logic computing unit, a floating-point computing unit, a memory access unit and a function unit; using the target computing unit to execute the computing task corresponding to the instruction data or the operand data read, and obtaining a corresponding computing result. The execution unit of the streaming multiprocessor will select a target computing unit matching the computing task according to the type of computing task after receiving the operand data and related instruction data, and use the selected target computing unit to complete the computing process of the computing task to obtain a corresponding computing result. Further, caching the corresponding computing result to the secondary cache may include: forwarding the computing result to the third interface coupler of the preset cross matrix through the master line interface of the primary cache, so that the preset cross matrix forwards the computing result to the fourth interface coupler using the internal integrated write arbiter and routing, and caching the computing result to the secondary cache through the fourth interface coupler using the slave line interface corresponding to the secondary cache. Specifically, after the streaming multiprocessor obtains the calculation result corresponding to the calculation task, it forwards the calculation result to the third interface coupler of the preset cross matrix through the main line interface of the first-level cache, and the third interface coupler is used to receive the data write request; then the preset cross matrix forwards the calculation result to the fourth interface coupler through the internally integrated write arbitrator and related routing, and the fourth interface coupler caches the calculation result to the second-level cache through the corresponding slave line interface, so that the host computer can process the calculation result.

It can be seen that the present application constructs a first-level cache in each stream multiprocessor, and at the same time allocates appropriate cache space to each stream multiprocessor in the second-level cache according to the number and address space of the stream multiprocessors; through the hierarchical cache structure of independent first-level cache and shared second-level cache, the data corresponding to the computing task can be read efficiently, and the cache resources can be centrally managed through the shared second-level cache; when different stream multiprocessors interact with each other, data interaction can be achieved only according to the address space corresponding to the stream multiprocessor, which optimizes data access efficiency, reduces conflicts, and improves overall computing performance.

As shown in FIG. 2 , an embodiment of the present application discloses a task execution method, which is applied to a streaming multiprocessor, including:

The stream multiprocessor may include a scheduling unit, L1-Icache (Instruction Cache, instruction cache of the first-level cache), an instruction fetch unit, a decoding unit, an operand collection unit, a register file, L1-Dcache (Data Cache, data cache of the first-level cache), an execution unit, and a write-back unit. The scheduling unit (maintaining the thread bundle PC host table) is responsible for activating the thread bundle and generating relevant parameter information to pass to the instruction fetch unit. The instruction fetch unit parses the relevant parameters passed by the previous level to generate a request signal for L1-Icache. After receiving the request signal from the instruction fetch unit, L1-Icache reads the instruction data according to the corresponding request information. When the read misses, the request information will continue to be passed to L2-Cache (secondary cache) via the AXI-Crossbar cross matrix until it returns to the instruction fetch unit after waiting for the instruction data to be read. The decoding unit receives the instruction data from the instruction fetch unit for parsing, and parses the source operand address, destination address and other information according to the type of instruction, and then passes the operand and other information to the operand collection unit, and reads the register file or L1-Dcache according to the corresponding operand type and address. When the operand is located in the L1-Dcache, the reading process is similar to the instruction data, and a read data request is sent to the L2-Cache when the data does not hit. Further, after the instruction data and operand collection are completed, the operand is passed to the execution unit, which integrates the logic calculation unit, floating point calculation unit, memory access unit and function unit. According to the type of calculation task, the corresponding calculation unit is selected for calculation. After the execution is completed, the calculation result is written back to the corresponding Cache according to the destination type and address, and a complete pipeline operation is completed. Among them, the L1-Cache can include two types, L1-Icache and L1-Dcache, and internally integrates a request arbitration module, a response arbitration module, a response cache buffer, a bypass forwarding module and a cache RAM. When multiple thread bundles receive requests to read data at the same time, the request arbitration module prioritizes the thread bundle IDs for polling. When the relevant data is read from the cache RAM, it is cached in the corresponding buffer according to the thread bundle ID through the response arbitration module for polling. When the data does not hit, the read request is forwarded to the L2-Cache through the bypass forwarding module via the AXI-Crossbar cross matrix to continue reading the data. As shown in Figure 3, the AXI-Crossbar cross matrix schematic diagram includes a read arbitrator for processing read requests AR/R and a write arbitrator for processing write requests AW/W, both of which implement data forwarding based on routing. Further, as shown in Figure 4, an M×S cross switch architecture based on a crosspoint is provided at each crosspoint, where a switch controlled by a selection signal is provided to control the on-off status of the crosspoint to determine the connection between the input port and the output port. The selection signal is generated by the cross switch router. When multiple nodes initiate communication requests to the same node at the same time, the device will decide which port's interface coupling buffer the data entering the router comes from based on an arbitrator using a polling strategy. When the data in the interface buffer is taken out, its corresponding storage resources will also be released. If the buffer is full, the output end will invalidate the buffer available signal and temporarily intercept subsequent communication requests. When there is a transmission task from port M to port S to be processed, it will prepare the data to be sent and send a transmission request signal and transmission information such as the destination port number to the cross switch controller. If the data path is idle and available, the cross switch controller will set the (M, S) switch selection signal to valid and send a ready-to-send feedback signal to port M. After receiving the feedback signal, the data enters the cross switch from port M and leaves the cross switch network from port S along the data path. While setting the (M, S) selection signal valid, the cross switch controller will also send a transmission request signal to port S. When the data is correctly received and cached by port S, port S will send a corresponding reply feedback signal to the cross switch controller. The controller will continue to feed back the reply feedback signal to port M, indicating that the communication task is completed, the storage space can be released, and the corresponding crosspoint switch is closed at the same time.

As shown in FIG5, it is a schematic diagram of the connection relationship between the stream multiprocessor and the secondary cache. It can be seen that the L1-Cache includes two types of L1-Icache and L1-Dcache. A single stream multiprocessor corresponds to one L1-Cache, and the L1-Cache is connected to the AXI-Crossbar cross matrix through the main line. The AXI-Crossbar cross matrix is connected to the secondary cache through the slave line. The secondary cache corresponds to different cache areas according to the address space of the stream multiprocessor; in a specific embodiment, the L1-Cache can integrate a request arbitration module, a response arbitration module, a response cache buffer, a bypass forwarding module and a cache RAM. When multiple thread bundles receive requests to read data at the same time, the request arbitration module divides the priority according to the thread bundle ID for polling processing. When the relevant data is read from the cache RAM, it is cached in the corresponding buffer according to the thread bundle ID through the response arbitration module, and the polling processing is carried out in this way. When the data is not hit, the read request is forwarded to the L2-Cache through the bypass forwarding module via the AXI-Crossbar cross matrix to continue reading the data.

In a specific embodiment, when the host computer generates a parallel computing task, the instructions and operand data required for this computing task are first cached in the L2-Cache, and the required stream multiprocessor and its corresponding L2-Cache cache space are allocated for this task. Correspondingly, the stream multiprocessor SMP0 activates the internal scheduling unit to generate an active thread bundle. If two active thread bundles W0 and W1 are generated at the same time, the initial instruction address, the ID of the active thread bundle and other parameters are passed to the instruction fetch unit. After parsing the received instruction address and other parameter information, the instruction fetch unit generates read request signals Req.W0 and Req.W1 to read the instruction data in the L1-Icache. When the request arbitration module in the L1-ICache detects the read request signal, it reads the corresponding cache in turn according to the address information. If the data hits this time, the instruction data is cached in the response buffer.W0 and buffer.W1 respectively through the response arbitration module and is taken away in turn by the instruction fetch unit. On the contrary, if the data does not hit this time, the bypass forwarding module of L1-ICache will forward the request signals IReq.W0 and IReq.W1 to the AXI-Crossbar cross matrix in turn, and the read request of SMP0 will be connected to the Internet from the Master0 side, and will be forwarded from the Slave0 side to L2-Cache through the arbitrator and routing, and read from the specified SMP0 cache area. After that, the read data will be cached in the corresponding buffer through the response arbitration module of L1-Icache, and finally read by the instruction fetch unit and forward the read instruction data to the decoding unit. After receiving the instruction data from the instruction fetch unit, the decoding unit will parse it, and parse the source operand location, calculation result storage location and required calculation unit type according to the instruction type protocol, and pass it to the operand collection unit, and read the register file or L1-Dcache according to the corresponding operand type and address. When the operand is in the register file, it can be directly read according to the register address. When the operand is in L1-Dcache, a read request signal DReq.W0 and DReq.W1 are generated. The process is the same as the request and response process of the instruction data. In the case of a data hit, the operand is directly returned. In the case of a miss, the read request signal needs to be forwarded to L2-Cache. Finally, after all the required operands are collected, they are passed to the execution unit. Further, after the execution unit receives the operand and the related instruction information, it selects the corresponding computing unit for operation according to the type of computing task, including the logic computing unit, the floating point computing unit, the memory access unit and the function unit. After the execution is completed, the operation result is cached in the corresponding register file or L1-Dcache through the write-back unit according to the destination type and address. At this point, a complete pipeline operation of SMP0 is completed. It can be understood that when the computing task is assigned to multiple SMPs, each SMP will generate multiple active thread bundles to complete the related operations. Repeat the above steps, and finally the operation results of all SMPs will be cached in L2-Cache for processing by the host computer.

The present application constructs a first-level cache in each stream multiprocessor, and at the same time allocates appropriate cache space to each stream multiprocessor in the second-level cache according to the number and address space of the stream multiprocessors; through the hierarchical cache structure of independent first-level cache and shared second-level cache, the data corresponding to the computing task can be read efficiently, and the cache resources can be centrally managed through the shared second-level cache, allowing all stream multiprocessors to access the same cache pool, reducing data replication, reducing data transmission latency, and improving cache hit rate; and such a hierarchical cache structure is highly flexible and adaptable, and can meet diverse application requirements.

It can be seen that the present application constructs a first-level cache in each stream multiprocessor, and at the same time allocates appropriate cache space to each stream multiprocessor in the second-level cache according to the number and address space of the stream multiprocessors; through the hierarchical cache structure of independent first-level cache and shared second-level cache, the data corresponding to the computing task can be efficiently read, and the cache resources can be centrally managed through the shared second-level cache. Compared with the solution of independent second-level cache corresponding to the stream multiprocessor, although the independence and data access speed of each core are guaranteed, there are problems such as resource waste, data redundancy and complex cache consistency maintenance. Especially under highly parallel workloads, data is frequently transferred between cores, and independent cache will cause a large amount of data migration and invalid cache line refresh, which seriously affects system performance and energy efficiency; the hierarchical cache structure of the shared second-level cache of the present application allows all stream multiprocessors to access the same cache pool, reduces data replication, reduces data transmission delay, and improves cache hit rate; and such a hierarchical cache structure has high flexibility and adaptability, which can meet diverse application requirements.

As shown in FIG6 , the embodiment of the present application discloses a task execution device, which is applied to a streaming multiprocessor, including:

A read request generation module 11 is used to generate a read request for its own first-level cache based on a computing task generated by a host computer; the streaming multiprocessor corresponds to an independent first-level cache; the computing task corresponds to instruction data and operand data;

Among them, in scenarios such as artificial intelligence, big data analysis, and high-end graphics rendering, the streaming multiprocessor in the GPU can execute the computing tasks generated by the corresponding host computer, and the computing tasks correspond to instruction data and operand data; a single streaming multiprocessor corresponds to an independent first-level cache. Furthermore, the process of the streaming multiprocessor executing a single computing task is to generate a read request for its own first-level cache based on the computing task through its own read request generation model, so as to read the instruction data and first-level operand data related to the current computing task from its corresponding independent first-level cache through the read request.

A forwarding module 12, configured to forward the read request to a preset cross matrix through the first-level cache when the read request does not hit data in the first-level cache, so as to forward the read request to a second-level cache shared by a plurality of streaming multiprocessors through the preset cross matrix; an arbitrator and a router are integrated in the preset cross matrix, configured to forward the read request to the second-level cache according to an arbitration mechanism and an address space corresponding to the read request;

When a read request generated by a single streaming multiprocessor does not hit data in its corresponding independent first-level cache, the read request can be forwarded to a preset cross matrix through a forwarding module; the preset cross matrix is a cross switch connecting the first-level cache and the second-level cache (shared by several streaming multiprocessors), with an arbitrator and a router integrated inside, which can realize data forwarding between the first-level cache and the second-level cache. The read request can be forwarded to the second-level cache through the preset cross matrix so that the corresponding data can be read from the second-level cache.

A data reading module 13, configured to read instruction data or operand data corresponding to the computing task from the secondary cache via the primary cache based on the read request;

Among them, in the streaming multiprocessor, the data reading module can be used to read the instruction data or operand data corresponding to the current computing task from the second-level cache through the first-level cache; the instruction data and operand data corresponding to the current computing task can be read from the first-level cache and/or the second-level cache through a read request.

The task execution module 14 is used to execute the computing task according to the read instruction data or the operand data, and cache the corresponding computing result to the secondary cache so that the host computer can process the computing result corresponding to the computing task.

Among them, after the streaming multiprocessor reads the instruction data and operand data corresponding to the current computing task, it can execute the computing task through the task execution module, and then cache the corresponding operation result in the secondary cache so that the host computer can process the operation result corresponding to the current computing task. It can be understood that the process of caching the operation result in the secondary cache is also forwarded through the preset cross matrix data.

It can be seen that the present application constructs a first-level cache in each stream multiprocessor, and at the same time allocates appropriate cache space to each stream multiprocessor in the second-level cache according to the number and address space of the stream multiprocessors; through the hierarchical cache structure of independent first-level cache and shared second-level cache, the data corresponding to the computing task can be read efficiently, and the cache resources can be centrally managed through the shared second-level cache, allowing all stream multiprocessors to access the same cache pool, reducing data duplication, reducing data transmission delays, and improving cache hit rates; and such a hierarchical cache structure is highly flexible and adaptable, and can meet the needs of diverse applications.

In a specific embodiment, the read request generating module 11 may include:

A thread warp generation unit, used to activate its own scheduling unit based on the computing tasks generated by the host computer to generate a number of thread warps;

A read request generating unit is used to pass the initial instruction address corresponding to the computing task and the name of the thread bundle to its own instruction fetch unit, so that the instruction fetch unit parses the initial instruction address and the name of the thread bundle, and generates a read request for its own first-level cache according to the parsing result.

In a specific embodiment, the device may further include:

A first-level cache reading module is used to read instruction data or operand data corresponding to the computing task from the first-level cache through the read request when the read request hits data in the first-level cache, so as to execute the computing task according to the read instruction data or the operand data.

In another specific embodiment, the first-level cache reading module may include:

A first data cache unit, configured to read the corresponding instruction data or operand data locally from the first-level cache according to the read request, and cache the instruction data or operand data read from the first-level cache into a preset buffer corresponding to the streaming multiprocessor;

The first data reading unit is used to read instruction data or operand data corresponding to the computing task from the preset buffer.

In a specific embodiment, the forwarding module 12 may include:

A forwarding unit is used to forward the read request to the first interface coupler of a preset cross matrix through the first-level cache via its own corresponding master line interface, so that the preset cross matrix uses the internally integrated read arbiter and routing to forward the read request to the second interface coupler, and transmit the read request to the second-level cache through the second interface coupler using the slave line interface corresponding to the second-level cache, so as to complete the process of forwarding the read request to the second-level cache shared by several streaming multiprocessors.

In a specific embodiment, the task execution module 14 may include:

A computing unit selection unit, used to select a target computing unit matching the computing task through its own execution unit; the target computing unit includes a logic computing unit, a floating-point computing unit, a memory access unit and a function unit;

The task execution unit is used to use the target computing unit to execute the computing task corresponding to the read instruction data or the operand data to obtain a corresponding computing result.

In a specific embodiment, the task execution module 14 may include:

The second data cache unit is used to forward the calculation result to the third interface coupler of the preset cross matrix through the main line interface of the first-level cache, so that the preset cross matrix uses the internally integrated write arbitrator and routing to forward the calculation result to the fourth interface coupler, and cache the calculation result to the second-level cache through the fourth interface coupler using the slave line interface corresponding to the second-level cache.

Furthermore, an embodiment of the present application also discloses an electronic device. FIG. 7 is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content in the diagram cannot be regarded as any limitation on the scope of use of the present application.

FIG7 is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of the present application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input/output interface 25, and a communication bus 26. The memory 22 is used to store a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the task execution method disclosed in any of the aforementioned embodiments. In addition, the electronic device 20 in this embodiment may specifically be an electronic computer.

In this embodiment, the power supply 23 is used to provide working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and the external device, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs and is not specifically limited here.

In addition, the memory 22, as a carrier for storing resources, can be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon can include an operating system 221, a computer program 222, etc., and the storage method can be temporary storage or permanent storage.

The operating system 221 is used to manage and control the hardware devices and computer programs 222 on the electronic device 20, which can be Windows Server, Netware, Unix, Linux, etc. In addition to including computer programs that can be used to complete the task execution method performed by the electronic device 20 disclosed in any of the aforementioned embodiments, the computer program 222 can further include computer programs that can be used to complete other specific tasks.

Furthermore, the present application also discloses a computer-readable storage medium for storing a computer program; wherein the computer program, when executed by a processor, implements the task execution method disclosed above. For the specific steps of the method, reference may be made to the corresponding contents disclosed in the above embodiments, and no further description will be given here.

In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

The technical solution provided by the present application is introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for general technical personnel in this field, according to the idea of the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims (9)

1. A task execution method, characterized in that it is applied to a streaming multiprocessor, comprising: Generate a read request for its own first-level cache based on the computing task generated by the host computer; the streaming multiprocessor corresponds to an independent first-level cache; the computing task corresponds to instruction data and operand data; If the read request does not hit data in the first-level cache, the read request is forwarded to a preset cross matrix through the first-level cache, so that the read request is forwarded to a second-level cache shared by a plurality of streaming multiprocessors through the preset cross matrix; an arbitrator and a router are integrated in the preset cross matrix, and are used to forward the read request to the second-level cache according to an arbitration mechanism and an address space corresponding to the read request; Reading instruction data or operand data corresponding to the computing task from the secondary cache via the primary cache based on the read request; Execute the computing task according to the read instruction data or the operand data, and cache the corresponding computing result in the secondary cache so that the host computer processes the computing result corresponding to the computing task; The forwarding of the read request to a preset cross matrix through the first-level cache, so as to forward the read request to a second-level cache shared by a plurality of streaming multiprocessors through the preset cross matrix, includes: The read request is forwarded by the first-level cache via its own corresponding main line interface to the first interface coupler of a preset cross matrix, so that the preset cross matrix uses the internally integrated read arbiter and routing to forward the read request to the second interface coupler, and the read request is passed to the second-level cache via the second interface coupler using the slave line interface corresponding to the second-level cache, so as to complete the process of forwarding the read request to the second-level cache shared by several streaming multiprocessors. 2. The task execution method according to claim 1, characterized in that the computing task generated by the host computer generates a read request for its own first-level cache, comprising: Activate its own scheduling unit based on the computing tasks generated by the host computer to generate several thread bundles; The initial instruction address corresponding to the computing task and the name of the thread warp are passed to its own instruction fetch unit, so that the instruction fetch unit parses the initial instruction address and the name of the thread warp, and generates a read request for its own first-level cache according to the parsing result. 3. The task execution method according to claim 1, further comprising: If the read request hits data in the first-level cache, instruction data or operand data corresponding to the computing task is read from the first-level cache through the read request, so as to execute the computing task according to the read instruction data or operand data. 4. The task execution method according to claim 3, characterized in that the step of reading the instruction data or operand data corresponding to the computing task from the first-level cache through the read request comprises: After reading the corresponding instruction data or operand data locally from the first-level cache according to the read request, the instruction data or operand data read from the first-level cache is cached in a preset buffer corresponding to the streaming multiprocessor; The instruction data or operand data corresponding to the computing task is read from the preset buffer. 5. The task execution method according to any one of claims 1 to 4, characterized in that executing the computing task according to the read instruction data or the operand data comprises: Selecting a target computing unit that matches the computing task through its own execution unit; the target computing unit includes a logic computing unit, a floating-point computing unit, a memory access unit and a function unit; The target computing unit is used to execute the computing task corresponding to the read instruction data or operand data to obtain a corresponding computing result. 6. The task execution method according to claim 5, characterized in that caching the corresponding operation results into the secondary cache comprises: The calculation result is forwarded to the third interface coupler of the preset cross matrix through the master line interface of the first-level cache, so that the preset cross matrix forwards the calculation result to the fourth interface coupler using the internally integrated write arbiter and routing, and caches the calculation result to the second-level cache through the fourth interface coupler using the slave line interface corresponding to the second-level cache. 7. A task execution device, characterized in that it is applied to a streaming multiprocessor, comprising: A read request generation module, used to generate a read request for its own first-level cache based on a computing task generated by a host computer; the streaming multiprocessor corresponds to an independent first-level cache; the computing task corresponds to instruction data and operand data; A forwarding module, used for forwarding the read request to a preset cross matrix through the first-level cache when the read request does not hit data in the first-level cache, so as to forward the read request to the second-level cache shared by a plurality of streaming multiprocessors through the preset cross matrix; an arbitrator and a router are integrated in the preset cross matrix, and used for forwarding the read request to the second-level cache according to an arbitration mechanism and an address space corresponding to the read request; A data reading module, configured to read instruction data or operand data corresponding to the computing task from the secondary cache via the primary cache based on the read request; A task execution module, used for executing the computing task according to the read instruction data or the operand data, and caching the corresponding computing result into the secondary cache so that the host computer processes the computing result corresponding to the computing task; Wherein, the forwarding module includes: A forwarding unit is used to forward the read request to the first interface coupler of a preset cross matrix through the first-level cache via its own corresponding master line interface, so that the preset cross matrix uses the internally integrated read arbiter and routing to forward the read request to the second interface coupler, and transmit the read request to the second-level cache through the second interface coupler using the slave line interface corresponding to the second-level cache, so as to complete the process of forwarding the read request to the second-level cache shared by several streaming multiprocessors. 8. An electronic device, comprising: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the task execution method according to any one of claims 1 to 6. 9. A computer-readable storage medium, characterized in that it is used to store a computer program, and when the computer program is executed by a processor, it implements the task execution method according to any one of claims 1 to 6.