CN113377538B - A storage computing collaborative scheduling method and system for GPU data reuse - Google Patents

Abstract

The invention discloses a storage computing collaborative scheduling method and a system for GPU data reuse, wherein the method comprises the steps that a kernel program is started to turn over a reverse mark of the kernel program; in a thread block scheduler of the GPU, for thread block scheduling of the kernel program, selecting one thread block dispatch strategy in a forward thread block dispatch strategy and a reverse thread block dispatch strategy in turn according to a reversing mark of the kernel program to select a thread block to be transmitted from a thread block to be transmitted queue; in the GPU driving, for the data page replacement of the kernel program, one of the forward data page replacement strategy and the reverse data page replacement strategy is selected in turn according to the inversion mark of the kernel program to select the GPU side data page from the GPU side data page queue for replacement. The invention realizes the cooperative scheduling of the thread blocks and the data pages, reduces the influence of excessive configuration of the memory on the system performance by reusing shared data, and can effectively improve the system performance.

Description

A storage computing collaborative scheduling method and system for GPU data reuse

technical field

The invention relates to computer computing scheduling technology, in particular to a storage computing collaborative scheduling method and system for GPU data reuse.

Background technique

Due to its high computing throughput and good programmability, GPU has been widely used in high-performance fields including machine learning, object detection, and image denoising. However, due to the limited memory space on the GPU, it has been unable to accommodate the ever-expanding working set of the application (GPU data access per unit time). The introduction of unified virtual memory and on-demand page fetching technology provides good support for memory over-allocation, but due to the extra data page transfer between CPU memory and GPU memory, the loss of system performance is caused. Therefore, how to reduce these redundant data migrations is crucial for performance improvement. After studying a large number of tested assemblies, we found that there are many applications that share data between kernel programs (Kernel). And for most of these programs, each of the kernel programs accesses the same data area in a similar data access sequence. When the GPU's memory cannot hold the working set of the entire kernel program, the old data pages will be swapped out to the CPU's memory and the required data pages will be fetched into the GPU's memory. When a kernel program ends, only the latest accessed data pages will remain in the GPU memory, and subsequent kernel programs will also access those data pages that have been swapped into the CPU memory after startup. We found that although there is a large amount of shared data between kernel programs in these applications, such data sharing characteristics will disappear when memory over-allocation occurs, causing a sharp decline in system performance.

Efficiently using existing data in GPU memory is key to avoiding the long latency overhead caused by page faults, especially in the case of memory oversubscription. Figure 1 shows the performance degradation of an application that shares data between kernels due to memory overcommitment. We found that such applications are insensitive to the degree of memory over-provisioning, and that even a little memory over-provisioning can cause a sharp drop in performance. Figure 2 shows the change of data page access characteristics and page fault rate when the GPU memory can only accommodate 75% of the data access size of the FFT program (multiple kernel programs access the same order in the same order, and the dotted line indicates each kernel program end boundary). We found that each kernel program in FFT has similar data access characteristics and order, and there is a high page fault rate at the boundary of each kernel program start (the area circled in Figure 2), which is because earlier The accessed data pages are replaced by newly accessed data pages, so that subsequent kernel programs accessing these evicted data pages will cause data page invalidation. In order to fundamentally reduce the number of page migrations when memory is over-provisioned, many related techniques have been proposed including prefetching, using computation time to hide transfer time, and batching page failures. However, these techniques do little to improve the performance of such applications. Prefetching will cause system jitter due to the eviction of useful data pages, and the long delay caused by a large number of page failures cannot be hidden by pre-eviction of data pages and batch page failures. Based on the above analysis, we have obtained three findings for applications such as data sharing between kernel programs. First, once memory overcommitment occurs, the performance of the program will drop dramatically. Second, previous optimization methods for program performance when memory overcommitment occurs are not applicable to such programs. Finally, the page fault rate at kernel program boundaries is very high. Therefore, our research goal is to reduce the page fault rate at kernel program boundaries by reusing shared data between kernel programs.

Contents of the invention

The technical problem to be solved by the present invention: Aiming at the above-mentioned problems of the prior art, a storage computing cooperative scheduling method and system oriented to GPU data reuse are provided. The present invention realizes the cooperative scheduling of thread blocks and data pages, and reduces The impact of memory oversubscription on system performance can effectively improve system performance.

In order to solve the problems of the technologies described above, the technical solution adopted in the present invention is:

A storage computing collaborative scheduling method for GPU data reuse, comprising:

1) Under the condition that the current program has excessive GPU memory capacity and there is data sharing between kernel programs, detect whether there is a kernel program started, and if there is a kernel program started, flip the inversion flag of the kernel program;

2) In the GPU driver, for the data page replacement of the kernel program, according to the inversion flag of the kernel program, the two preset data page replacement strategies of the forward data page replacement strategy and the reverse data page replacement strategy are selected in turn One of the data page replacement strategies selects the GPU end data page from the GPU end data page queue to replace, and the forward data page replacement strategy and the reverse data page replacement strategy select the direction of the GPU end data page to be different; In the thread block scheduler, for the thread block scheduling of the kernel program, according to the inversion flag of the kernel program, the two preset thread block dispatch strategies of the forward thread block dispatch strategy and the reverse thread block dispatch strategy are selected in turn. A thread block dispatch strategy is used to select a thread block from the thread block waiting queue for emission, and the forward thread block dispatch strategy and the reverse thread block dispatch strategy select thread blocks in different directions.

Optionally, in the step 1), the inversion sign of the kernel program includes: first detecting whether the inversion sign of the kernel program exists, if the inversion sign of the kernel program does not exist, then for the inversion sign of the kernel program initialization, if If the inversion sign of the kernel program exists, then the inversion sign of the kernel program is turned over.

Optionally, when the inversion flag is initialized, the initialization value of the inversion flag is 0 or 1.

Optionally, said inversion of the inversion flag of the kernel program refers to: if the original value of the inversion flag of the kernel program is 0, then the inversion flag of the kernel program changes from 0 to 1, if the inversion flag of the kernel program If the original value is 1, then the inversion flag of the kernel program is changed from 1 to 0.

Optionally, in step 2), select one of the data page replacement strategies in turn from the two preset data page replacement strategies of the forward data page replacement strategy and the reverse data page replacement strategy according to the inversion flag of the kernel program. When selecting a GPU-side data page for replacement from the GPU-side data page queue, if the reverse flag is 0, select the forward data page replacement strategy; if the reverse flag is 1, select the reverse data page replacement strategy.

Optionally, in step 2), when selecting one of the thread block dispatching strategies in turn from the two preset thread block dispatching strategies of the forward thread block dispatching strategy and the reverse thread block dispatching strategy according to the reversal flag, if the reversal flag If it is 0, the forward thread block dispatch strategy is selected; if the reverse flag is 1, the reverse thread block dispatch strategy is selected.

Optionally, the excess GPU memory capacity of the current program in step 1) refers to: when the GPU driver on the host side monitors the data page request generated by the GPU side, if the length of the data page queue maintained by the GPU driver reaches the GPU memory capacity , it is necessary to evict the data pages in the GPU memory to the CPU memory according to the data page replacement policy, and at the same time set the memory over-allocation flag to indicate that the current program has memory over-allocation.

Optionally, there is data sharing between kernel programs in step 1) and refers to: judging whether the kernel programs share the same pointer according to the information at compile time, as an indicator of data sharing between kernel programs, and for each Assign a data sharing flag to the started kernel program to indicate whether it has data sharing with the previous kernel program; when a kernel program starts, it will judge the memory over-allocation flag bit and the data sharing corresponding to this kernel program respectively Whether the flag bits are all 1, if yes, it is determined that the cooperative scheduling method needs to be used.

In addition, the present invention also provides a GPU data reuse-oriented storage computing cooperative scheduling system, including an interconnected processing unit and a memory, and the processing unit is programmed or configured to execute the aforementioned GPU data reuse-oriented storage computing collaborative scheduling method step.

In addition, the present invention also provides a computer-readable storage medium, in which a computer program programmed or configured to execute the aforementioned GPU data reuse-oriented storage-computing cooperative scheduling method is stored.

Compared with the prior art, the present invention has the following advantages: since the GPU has very high computing throughput and good programmability, it has been widely used in high performance applications including machine learning, target detection and image denoising. field. However, due to the limited memory space on the GPU, it has been unable to accommodate the ever-expanding working set of the application (GPU data access per unit time). The introduction of unified virtual memory and on-demand page fetching technology provides good support for memory over-allocation, but due to the extra data page transfer between CPU memory and GPU memory, the loss of system performance is caused. Therefore, how to reduce these redundant data migrations is crucial for performance improvement. After studying a large number of tested assemblies, we found that there are many applications that share data between kernel programs. And for most of these programs, each of the kernel programs accesses the same data area in a similar data access sequence. When the GPU's memory cannot hold the working set of the entire kernel program, the old data pages will be swapped out to the CPU's memory and the required data pages will be fetched into the GPU's memory. When a kernel program ends, only the latest accessed data pages will be kept in the GPU memory, and subsequent kernel programs will also access those data pages that have been swapped into the CPU memory after starting. We found that although there is a large amount of shared data between kernel programs in these applications, such data sharing characteristics will disappear when memory over-allocation occurs, causing a sharp decline in system performance. Based on the above observations, the present invention proposes a thread block and data page cooperative scheduling method to improve the performance of the system by effectively utilizing the shared data between kernel programs, and by coordinating the switching of the thread block allocation order, that is, changing the thread block allocation order , Data page replacement strategy switching to make full use of the shared data between kernel programs, and evaluate the performance of our method through a large number of GPU test sets. The results show that the method of the present invention improves the performance by 65% compared with the latest research.

Description of drawings

Figure 1 shows the impact of GPU memory oversubscription on system performance.

Figure 2 shows the change of data page access characteristics and page failure rate when the GPU memory can only accommodate 75% of the data access size of the FFT program.

Fig. 3 is a schematic diagram of the basic principle of the method of the embodiment of the present invention.

Fig. 4 is a schematic diagram of performance comparison between the method of the embodiment of the present invention and the existing method.

FIG. 5 shows the relationship between the data page access characteristics and data page failure rate during FFT execution when the GPU memory capacity is set to 75% of the amount of data accessed by FFT in the embodiment of the present invention, when using the method of this embodiment (inverted allocation). Variety.

FIG. 6 shows the change of data page access characteristics and data page failure rate during FFT execution when the GPU memory capacity is set to 75% of the data volume accessed by FFT in an embodiment of the present invention and using the existing cooperative method.

Fig. 7 is the execution process under the benchmark configuration in the embodiment of the present invention.

FIG. 8 is an execution process when inversion allocation is used in the embodiment of the present invention.

Fig. 9 is the execution process when using the collaborative method in the embodiment of the present invention.

FIG. 10 is a schematic diagram of a least recently used data page replacement strategy based on loading time in an embodiment of the present invention.

FIG. 11 is a schematic diagram of a reverse least recently used data page replacement strategy based on loading time in an embodiment of the present invention.

FIG. 12 is an IPC comparison based on different optimization mechanisms when the benchmark configuration is used as a performance reference in the embodiment of the present invention.

Detailed ways

As shown in Figure 3, the storage and computing collaborative scheduling method for GPU data reuse in this embodiment includes:

1) Under the condition that the current program has excessive GPU memory capacity and there is data sharing between kernel programs, detect whether there is a kernel program started, and if there is a kernel program started, flip the inversion flag of the kernel program;

2) In the GPU driver, for the data page replacement of the kernel program, according to the inversion flag of the kernel program, the two preset data page replacement strategies of the forward data page replacement strategy and the reverse data page replacement strategy are selected in turn One of the data page replacement strategies selects the GPU end data page from the GPU end data page queue to replace, and the forward data page replacement strategy and the reverse data page replacement strategy select the direction of the GPU end data page to be different; In the thread block scheduler, for the thread block scheduling of the kernel program, according to the inversion flag of the kernel program, the two preset thread block dispatch strategies of the forward thread block dispatch strategy and the reverse thread block dispatch strategy are selected in turn. A thread block dispatch strategy is used to select a thread block from the thread block waiting queue for emission, and the forward thread block dispatch strategy and the reverse thread block dispatch strategy select thread blocks in different directions.

As shown above, coordinating thread blocks with data in memory is the key to reducing the page fault rate at the kernel program boundary, but how to determine the thread blocks related to these data is the biggest challenge encountered in the design. Based on the observation and analysis of this type of application program, the method of this embodiment finds that in most of these types of programs, each kernel program has similar data access characteristics and sequences (as shown in Figure 2), in each kernel program , the access behavior to the data page is related to the thread block scheduling mechanism, because each thread generally uses its corresponding thread label and thread block label to indicate the data location to be operated. In view of this, the method of this embodiment proposes a thread block allocation mechanism called reverse allocation, as shown by mark b in FIG. 3 , by adjusting the thread block scheduling mechanism to dynamically change the thread block allocation strategy. The default thread block distribution strategy is shown on the left side of mark b in Figure 3, and thread blocks are distributed in the order of sequence numbers from low to high. Whenever a kernel program is started, the inversion flag is flipped, and the scheduler will choose between the two thread block distribution strategies shown in mark b in Figure 3 according to the inversion flag in the command processor. When the inversion flag is 0 When selecting the forward thread block dispatching strategy, otherwise the reverse thread block dispatching strategy is selected, so that the dispatched thread block matches the data retained in the memory to maximize the de-reuse of data.

In this embodiment, the GPU memory capacity excess in the current program in step 1) refers to: when the GPU driver on the host side monitors the data page request generated by the GPU side, if the length of the data page queue maintained by the GPU driver reaches the GPU memory capacity, it is necessary to evict the data pages in the GPU memory to the CPU memory according to the data page replacement policy, and at the same time set the memory over-allocation flag to indicate that the current program has memory over-allocation.

In the present embodiment, in step 1) there is data sharing between the kernel programs and refers to: judging whether the kernel programs share the same pointer according to the information at the time of compiling, as an indicator of data sharing between the kernel programs, and for each A kernel program to be started is assigned a data sharing flag to indicate whether it has data sharing with the previous kernel program; when a kernel program is started, the memory over-allocation flag bit and the data corresponding to the kernel program will be judged respectively Whether the shared flag bits are all 1, if so, it is determined that the cooperative scheduling method needs to be used.

The application scenario of the cooperative scheduling method proposed in this embodiment is when there is memory over-allocation in the application program, and there is data sharing between multiple kernel programs. Therefore, the detection of memory over-allocation and data sharing between kernel programs is the key to triggering this mechanism. When the GPU driver on the host side monitors the data page request generated by the GPU side, if the length of the data page queue maintained by the GPU driver reaches the GPU memory capacity, it needs to replace the data pages in the GPU memory one by one according to the data page replacement policy. out to the CPU memory, and set the memory over-allocation flag at the same time to indicate that the current program has memory over-allocation, and use this flag as one of the indicators for the cooperative scheduling mechanism of this embodiment to start. According to the information at the time of compilation, it is judged whether the kernel programs share the same pointer, which is used as an indicator of data sharing between the kernel programs. And assign a data sharing flag to each kernel program to be started, so as to indicate whether it has data sharing with the previous kernel program. When a kernel program is started, the cooperative scheduling mechanism proposed in this embodiment will judge whether the memory oversubscription flag bit and the data sharing flag bit corresponding to the kernel program are both 1, and if so, trigger the cooperative scheduling mechanism, otherwise follow the default configuration to execute.

In the present embodiment, step 1) inversion of the reversal sign of this kernel program includes: first detect whether the reversal sign of this kernel program exists, if the reversal sign of this kernel program does not exist, then for the initialization reversal sign of this kernel program, If the inversion sign of the kernel program exists, the inversion sign of the kernel program is reversed.

In this embodiment, when the inversion flag is initialized, the initialization value of the inversion flag is 0 or 1.

In the present embodiment, the reversing of the inversion flag of the kernel program refers to: if the original value of the inversion flag of the kernel program is 0, then the inversion flag of the kernel program is changed from 0 to 1, if the inversion flag of the kernel program The original value of is 1, then the inversion flag of the kernel program is changed from 1 to 0.

In this embodiment, in step 2), according to the inversion flag of the kernel program, one of the data page replacement strategies is selected in turn from the two preset data page replacement strategies of the forward data page replacement strategy and the reverse data page replacement strategy When selecting a GPU-side data page for replacement from the GPU-side data page queue, if the reverse flag is 0, select the forward data page replacement strategy; if the reverse flag is 1, select the reverse data page replacement strategy.

In this embodiment, in step 2), when selecting one of the thread block dispatching strategies in turn from the two preset thread block dispatching strategies of the forward thread block dispatching strategy and the reverse thread block dispatching strategy according to the reversal flag, if the reverse If the flag is 0, the forward thread block dispatch strategy is selected; if the reverse flag is 1, the reverse thread block dispatch strategy is selected.

In order to demonstrate the effectiveness of the method of this embodiment, the prior art called Oracle and ETC are respectively introduced in this embodiment to compare with the method of this embodiment (inverted allocation), wherein Oracle can fully utilize the inter-kernel program reserved in the memory shared data. Figure 4 shows the average performance comparison between the method of this embodiment (inverted allocation) and Oracle and ETC using the default configuration as a benchmark, where Baseline is the benchmark performance for comparison. FIG. 5 shows the data page access characteristics and page failure changes of the FFT program when the method of this embodiment (inverted allocation) and the cooperative method are applied. Based on the analysis in Figure 4, it can be known that, firstly, compared with the latest method ETC, the method in this embodiment (inverted allocation) improves the performance by an average of 20.2%, which is consistent with the decrease in the page failure rate at the kernel program boundary shown in Figure 5 Second, compared to Oracle, the performance of reverse allocation is 58.5% worse, indicating that there is still a lot of room for improvement. Based on the analysis of Fig. 5, it can be seen that the execution time of even-numbered kernel programs is shorter than that of odd-numbered kernel programs. Referring to Fig. 5, when the GPU memory capacity is set to 75% of the amount of data accessed by FFT, this embodiment is used method (inverted allocation), the data page access characteristics and data page failure rate changes during FFT execution (multiple kernel programs access the same sequence in the same order, and the dotted line indicates the end boundary of each kernel program).

In order to deeply explore the reasons for the huge performance gap between the method of this embodiment (inverted allocation) and Oracle and the difference in the execution time of different kernel programs, this embodiment analyzes the execution process of a simple test program. This application program contains multiple kernels. program, and each kernel program accesses the same data, the specific execution process is shown in Figure 7-9. To simplify the analysis, we make some assumptions: 1) GPU can only execute one thread block at a time; 2) five kernel programs (A, B, C, D, E) access the same data, and each kernel program is controlled by 5 thread blocks (C1-C5), they only access one data page (respectively access P1-P5); 3) The memory capacity of the GPU is 3 data pages. First, we analyze the program execution process when using the basic configuration as shown in Figure 7. These kernel programs are started sequentially and the related thread blocks are also dispatched to the corresponding execution units in sequence. Initially, the requested data pages are loaded into GPU memory that is not full. However, due to the limited storage space and the least recently used data page replacement strategy based on loading time, the data page first loaded into the memory will be replaced by the newly accessed data page, so the data page will always be generated during the entire execution process Missing exceptions lead to a large performance penalty, which is consistent with the results shown in Figure 1.

The execution process of applying the method of this embodiment (inverted allocation) is shown in FIG. 8 . The difference from FIG. 7 is that the thread block allocation order of each kernel program is constantly switching. We made two observations. First, the page fault rate at the kernel program boundary is very low compared to the execution of the base configuration, which agrees with the results in Fig. 5. Secondly, for the kernel program executed with an odd number of times, the data reserved in the GPU memory is not fully utilized, which also explains the reason for the performance gap between the method of this embodiment (inverted allocation) and Oracle. In a word, although the method of this embodiment (reverse allocation) can improve the performance well, it is impossible to achieve performance improvement similar to that of Oracle. The replacement policy that cooperates with the data page and the method of this embodiment (inverted allocation) are the key to make up this performance gap.

Kernel programs executed an odd number of times do not fully utilize shared data in memory. Figure 10 shows the basic principle of the default data page replacement policy. The GPU driver on the host maintains a linear table used to record the order of data page migration from CPU memory to GPU memory. Although it does not reflect the order in which data pages are accessed, it has lower overhead than the ideal least recently used policy. Because reverse allocation switches the order in which thread blocks are allocated, the basic data page replacement strategy no longer works well with reverse allocation. As shown in Figure 5, data page 3 (P3) instead of data page 5 (P5) is evicted to the CPU memory during the execution of thread block C2 in kernel program B according to the least recently used policy based on load time go. However, for kernel program B, it is more desirable to evict data page 5 (P5), so that subsequent kernel programs can make full use of the shared data.

In order to solve this problem, we propose a data page replacement strategy called reverse replacement, as shown in Figure 10 (the same as the mark a in Figure 3). The cooperative inversion strategy introduces a reverse least recently used data page replacement strategy based on loading time on the basis of thread block inversion replacement, as shown in Figure 11. This new replacement policy switches the allocation and eviction direction of data pages compared to the default one. As shown by mark a in Figure 3, whenever a new kernel program starts, the GPU driver on the host side will select the corresponding data page replacement strategy according to the inversion flag bit in the command processor. When the inversion flag is 0, use The forward data page replacement strategy shown in Figure 7, and the reverse data page replacement strategy shown in Figure 11, can better cooperate with the reverse allocation mechanism to make full use of shared data.

Figure 8 shows the execution process of the simple test program introduced in the previous section when the cooperative inversion mechanism of the inversion allocation mechanism and the inversion replacement mechanism is used at the same time. As expected, data page 5 (P5) is replaced in CPU memory when thread block C2 in kernel program B executes. Therefore, when the kernel program C is executed, all data previously reserved in the GPU memory will be fully used, so that page fault errors will not be generated at the kernel program boundary. Figure 6 shows the data page access characteristics and page failure rate of the FFT program when the cooperative inversion strategy is used. We obtained two findings. First, the performance of the odd-numbered kernel program has been greatly improved, resulting in a ratio inversion Allocate better performance; second, more shared data is reused, resulting in fewer data page failure errors at kernel program boundaries. We conclude that data page failure errors at kernel program boundaries can be effectively reduced by reusing shared data.

In this example, GPGPU-simv4.0.0 is extended for evaluation. The relevant configuration information of the GPU system including cores and storage is shown in Table 1. We carefully simulated the on-demand migration process of data between CPU memory and GPU memory, and set the data page failure processing delay to the most optimistic 20μs. If the memory of the GPU is full, the driver of the GPU will replace the corresponding data page according to the corresponding data page replacement policy. To experiment with varying degrees of memory oversubscription, we set the GPU's memory capacity to a certain percentage (75%-95%) of the space required by each application. We selected 12 applications from CUDASDK, Rodinia, Ispass, and Polibench and other benchmark test suites for experiments. The occupied space of these programs ranged from 1MB to 96MB, with an average occupancy of 18.5MB. Limited simulation speed prevents us from simulating larger footprint applications.

Table 1 Simulator basic configuration information:

To evaluate the effectiveness of our method, we implement the state-of-the-art ETC. Figure 12 shows the performance comparison of multiple applications under reverse allocation, cooperative reverse, ETC and Oracle configurations based on the basic configuration. We got 3 findings, first, ETC exhibits weaker performance optimization ability, improving performance by 15% on average. Compared with ETC, inversion allocation and synergy inversion are improved by 20.2% and 65%, respectively. Second, for almost all applications, cooperative inversion can achieve performance close to that of Oracle, mainly because most of the shared data among kernel programs are effectively utilized. Finally, for both BFS and FWT programs, co-inversion does not improve their performance well, because the different kernel programs of these two programs have irregular data access characteristics, which cannot be well captured and utilized by co-inversion . As a result, applications that share a lot of data between kernels suffer a significant performance penalty when memory is oversubscribed. In the method of this embodiment, a thread block and data page cooperative scheduling method is proposed, aiming at reducing the impact of memory over-allocation on system performance by reusing shared data.

To sum up, although the latest GPUs are equipped with larger and larger memory, they still cannot accommodate the entire working set of large applications. With the support of unified virtual memory and on-demand data page insertion technology, these large programs can still run normally without the active intervention of programmers. However, this advantage has caused a certain performance loss. We found that although there are many current researches on reducing large page migration between CPU and GPU, it is still impossible to obtain good performance improvement for applications with data sharing between kernel programs. In this embodiment, the method of this embodiment proposes a thread block and data page cooperative scheduling method to reduce the impact of memory over-allocation in a transparent manner to programmers. This embodiment is oriented to storage computing for GPU data reuse The basic principle of the cooperative scheduling method is to effectively use the shared data between kernel programs by coordinating the order of thread block allocation and the order of data page replacement. Experiments show that the system performance of the storage computing cooperative scheduling method for GPU data reuse in this embodiment is A 65% improvement over the latest research method.

In addition, this embodiment also provides a GPU data reuse-oriented storage computing collaborative scheduling system, including:

Reversal flag management program unit, used to detect whether there is a kernel program to start, if there is a kernel program to start, the reversal flag of the kernel program is reversed;

The data page replacement program unit is used for the data page replacement of the kernel program in the GPU driver, and according to the inversion flag of the kernel program, there are two preset data pages in the forward data page replacement strategy and the reverse data page replacement strategy In the replacement strategy, one of the data page replacement strategies is selected in turn to select the GPU-side data page from the GPU-side data page queue for replacement. The forward data page replacement strategy and the reverse data page replacement strategy select the GPU-side data page different directions;

The thread block selection program unit is used in the thread block scheduler of the GPU, for the thread block scheduling of the kernel program. One of the thread block dispatch strategies is selected in turn from the thread block dispatch strategy to select the thread block emission from the thread block queue to be launched, and the forward thread block dispatch strategy and the reverse thread block dispatch strategy select the direction of the thread block different.

In addition, this embodiment also provides a GPU data reuse-oriented storage computing collaborative scheduling system, including:

The CPU is used to detect whether a kernel program is started, and if a kernel program is started, the inversion flag of the kernel program is reversed, and in the GPU driver, the data page replacement for the kernel program is performed according to the inversion flag of the kernel program. Select one of the data page replacement strategies in turn from the two preset data page replacement strategies of the data page replacement strategy and the reverse data page replacement strategy to select the GPU-side data page from the GPU-side data page queue for replacement. The forward data page replacement strategy and the reverse data page replacement strategy select different directions for GPU-side data pages;

GPU, used in the thread block scheduler of the GPU, for the thread block scheduling of the kernel program, according to the reverse flag of the kernel program, there are two preset thread blocks in the forward thread block dispatch strategy and the reverse thread block dispatch strategy In the distribution strategy, one of the thread block distribution strategies is selected in turn to select a thread block from the thread block queue to be emitted, and the forward thread block distribution strategy and the reverse thread block distribution strategy select thread blocks in different directions;

The CPU and the GPU are connected to each other.

In addition, this embodiment also provides a GPU data reuse-oriented storage computing cooperative scheduling system, including a processing unit and a memory connected to each other, and the processing unit is programmed or configured to perform the GPU data reuse-oriented storage computing collaborative scheduling method steps.

In addition, this embodiment also provides a computer-readable storage medium, where a computer program programmed or configured to execute the storage-computing cooperative scheduling method oriented to GPU data reuse is stored.

Those skilled in the art should understand that the embodiments of the present application may be provided as methods, systems, or computer program products. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein. The present application is described with reference to flowcharts and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present application. It should be understood that each procedure and/or block in the flowchart and/or block diagram, and a combination of procedures and/or blocks in the flowchart and/or block diagram can be realized by computer program instructions. These computer program instructions may be provided to a general purpose computer, special purpose computer, embedded processor, or processor of other programmable data processing equipment to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing equipment produce a An apparatus for realizing the functions specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram. These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to operate in a specific manner, such that the instructions stored in the computer-readable memory produce an article of manufacture comprising instruction means, the instructions The device realizes the function specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram. These computer program instructions can also be loaded onto a computer or other programmable data processing device, causing a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby The instructions provide steps for implementing the functions specified in the flow chart or blocks of the flowchart and/or the block or blocks of the block diagrams.

The above descriptions are only preferred implementations of the present invention, and the scope of protection of the present invention is not limited to the above examples, and all technical solutions that fall under the idea of the present invention belong to the scope of protection of the present invention. It should be pointed out that for those skilled in the art, some improvements and modifications without departing from the principles of the present invention should also be regarded as the protection scope of the present invention.

Claims (10)

1. The storage computing cooperative scheduling method for GPU data reuse is characterized by comprising the following steps of:

1) Detecting whether a kernel program is started under the conditions that the current program has excessive GPU memory capacity and data sharing exists among the kernel programs, and turning over a reversing mark of the kernel program if the kernel program is started;

2) In GPU driving, selecting one of the data page replacement strategies in turn in two preset data page replacement strategies of a forward data page replacement strategy and a reverse data page replacement strategy according to the inversion mark of the kernel program to select a GPU side data page from a GPU side data page queue for replacement, wherein the directions of the data page selection of the forward data page replacement strategy and the reverse data page replacement strategy are different; in a thread block scheduler of a GPU, for thread block scheduling of the kernel program, one of a forward thread block dispatch strategy and a reverse thread block dispatch strategy is selected in turn according to a reversing mark of the kernel program in two preset thread block dispatch strategies to select thread block emission from a thread block to-be-emitted queue, wherein the directions of the thread blocks selected by the forward thread block dispatch strategy and the reverse thread block dispatch strategy are different; the data page replacement strategy introduces a reverse data page replacement strategy based on the thread block reverse replacement of the GPU, and the reverse data page replacement strategy is a data page replacement strategy which is least recently used and is reversely based on loading time, so that the allocation and the eviction direction of the data page are switched with the forward data page replacement strategy.

2. The GPU data reuse oriented storage computing co-scheduling method of claim 1, wherein flipping the kernel's inversion flag in step 1) comprises: firstly, detecting whether a reverse flag of the kernel program exists or not, initializing the reverse flag for the kernel program if the reverse flag of the kernel program does not exist, and reversing the reverse flag of the kernel program if the reverse flag of the kernel program exists.

3. The GPU-data reuse oriented storage computing co-scheduling method of claim 2, wherein when the inversion flag is initialized, an initialization value of the inversion flag is 0 or 1.

4. The GPU data reuse oriented storage computing co-scheduling method of claim 2, wherein said flipping the inversion flag of the kernel refers to: the reverse flag of the kernel is changed from 0 to 1 if the original value of the reverse flag of the kernel is 0, and from 1 to 0 if the original value of the reverse flag of the kernel is 1.

5. The method for collaborative scheduling of storage and computation for GPU-oriented data reuse according to claim 1, wherein in step 2), one of the two preset data page replacement policies, namely the forward data page replacement policy and the reverse data page replacement policy, is selected in turn according to the inversion flag of the kernel program, when the GPU-oriented data page is selected from the GPU-oriented data page queue for replacement, if the inversion flag is 0, the forward data page replacement policy is selected; if the reverse flag is 1, then the reverse page replacement policy is selected.

6. The method for collaborative scheduling of storage and computation for GPU data reuse according to claim 1, wherein in step 2), when one of the forward thread block dispatch policies and the reverse thread block dispatch policies is selected in turn according to the reverse flag in two preset thread block dispatch policies, if the reverse flag is 0, the forward thread block dispatch policy is selected; if the reverse flag is 1, then the reverse thread block dispatch policy is selected.

7. The collaborative scheduling method for GPU-oriented data reuse according to claim 1, wherein the occurrence of the GPU memory capacity excess in the current program in step 1) means: when the GPU driver at the host computer monitors a data page request generated by the GPU, if the length of a data page queue maintained by the GPU driver reaches the memory capacity of the GPU at the moment, the data page in the GPU memory needs to be evicted into the CPU memory according to a data page replacement strategy, and meanwhile, a memory excess configuration mark is set to indicate that the memory excess configuration of the current program occurs.

8. The collaborative scheduling method for storage computing for GPU-oriented data reuse according to claim 1, wherein the data sharing between kernel programs in step 1) means: judging whether the same pointer is shared between kernel programs according to the information of compiling time, taking the pointer as an index of data sharing between kernel programs, and distributing a data sharing mark for each kernel program to be started so as to indicate whether the kernel programs share data with the previous kernel program; when a kernel program is started, whether the memory excess configuration zone bit and the data sharing zone bit corresponding to the kernel program are 1 or not is respectively judged, and if yes, a cooperative scheduling method is judged to be needed.

9. A GPU data reuse oriented storage computing co-scheduling system comprising a processing unit and a memory connected to each other, wherein the processing unit is programmed or configured to perform the steps of the GPU data reuse oriented storage computing co-scheduling method of any of claims 1 to 8.

10. A computer readable storage medium having stored therein a computer program programmed or configured to perform the GPU-oriented data reuse storage computing co-scheduling method of any of claims 1-8.

Images