CN117742679A - Kernel fusion method and system based on deep neural network - Google Patents

Abstract

The present invention provides a kernel fusion method based on a deep neural network, which includes: compiling the source code into a host-side intermediate code file and a device-side intermediate code file through a compilation framework, inputting the above two files into the fusion framework, and generating the fused The device-side intermediate code file; optimize and compile the fused device-side intermediate code file to obtain the host-side intermediate code file with device-side information; combine the host-side intermediate code file with device-side information with the device-side intermediate code file The code file is input into the fusion framework to generate the fused host-side intermediate code file; the fused host-side intermediate code file is optimized and compiled to obtain the corresponding executable file. The invention also provides a kernel fusion system, a storage medium and an electronic device based on a deep neural network. In this way, the present invention can reduce performance overhead and improve parallel resource utilization, thereby improving the reasoning performance of the deep neural network system.

Description

Kernel fusion method and system based on deep neural network

Technical field

The present invention relates to the field of artificial intelligence neural network technology, and in particular to a kernel fusion method, system, storage medium and electronic device based on a deep neural network.

Background technique

In recent years, DNN (Deep Neural Networks) has been widely used in many fields, including speech recognition, image classification, neural machine translation, automatic driving, etc. In view of the widespread deployment of DNN models and the huge demand for efficient reasoning, both industry and academia have optimized DNN models from multiple aspects such as hardware, software, and algorithms, and have proposed a series of hardware accelerators for DNN optimization. Acceleration libraries, compilers, programming frameworks and algorithms.

Optimization technologies in programming frameworks and compilers include graph-based TACO (Tencent’s AI computing acceleration suite) and APOLLO (Baidu’s open platform for autonomous driving). TACO can efficiently replace operators in the calculation graph with equivalents, thus improving performance. APOLLO improves inference and training performance by dividing the calculation graph into subgraphs and performing kernel fusion through analysis.

Existing operator fusion technologies include pattern-based fusion, rule-based fusion, and program dependency analysis-based fusion. In the fusion scheme based on patterns or rules, users are required to pre-define the patterns or rules that can be fused, so that new operators cannot be processed well. In the fusion solution based on program dependency analysis, the compiler needs to have powerful analysis capabilities, which poses a great challenge to the compiler, and it may not be able to accurately analyze all operators that may be fused. When selecting a fusion solution, a tuning process is also required, which is also very time-consuming. And when there are global dependencies between cores, the fusion framework will not fuse them together. However, research has found that it is still possible to obtain performance gains by fusing kernels with global dependencies. For example, Rammer is a well-known DNN compiler in the industry, also called NNFusion. It horizontally merges operators into different thread blocks during compilation to execute multiple operators in parallel. At the same time, it will also explore co-scheduling between operators and within operators. Apollo is currently the most advanced operator fusion framework for neural network inference optimization. It not only considers the fusion of memory-intensive operators, but also considers the fusion of calculation-intensive operators. It uses manual rules for operator fusion and also optimizes Parallel execution between independent operators.

Existing DNN operator fusion technology either requires users to pre-define the fusion mode or rules, or requires the compiler to have powerful analysis capabilities. However, predefined fusion modes or rules do not cover all operators, and the compiler's analysis capabilities are not perfect and cannot analyze all fusion operators.

In summary, it can be seen that the existing technology obviously has inconveniences and defects in actual use, so it is necessary to improve it.

Contents of the invention

In view of the above defects, the purpose of the present invention is to provide a kernel fusion method, system, storage medium and electronic device based on a deep neural network, which can reduce performance overhead and improve parallel resource utilization, thereby improving the reasoning performance of the deep neural network system. .

In order to solve the above technical problems, the present invention is implemented as follows:

In the first aspect, embodiments of the present invention provide a kernel fusion method based on a deep neural network, including:

Compile the source code into host-side intermediate code files and device-side intermediate code files corresponding to the host side and device side through the compilation framework;

Input the host-side intermediate code file and the device-side intermediate code file as parameters into the fusion framework, and generate a fused device-side intermediate code file;

The fused device-side intermediate code file is optimized and compiled through the compilation framework to obtain a host-side intermediate code file with device-side information;

Input the host-side intermediate code file with device-side information and the device-side intermediate code file as parameters into the fusion framework, and generate a fused host-side intermediate code file;

The integrated host-side intermediate code file is optimized and compiled through the compilation framework to obtain a corresponding executable file.

According to the method of the present invention, the step of optimizing and compiling the merged device-side intermediate code file through the compilation framework to obtain the host-side intermediate code file with device-side information includes:

Optimize the fused device-side intermediate code file through the compilation framework to generate an optimized device-side intermediate code file;

Compile the optimized device-side intermediate code file through the compilation framework to generate a merged device-side binary file;

The source code and the fused device-side binary file are compiled through the compilation framework to generate the host-side intermediate code file with device-side information.

According to the method of the present invention, the host-side intermediate code file with device-side information is a host-side intermediate code file with device-side binary information.

According to the method of the present invention, the step of generating the merged host-side intermediate code file includes:

When generating the fused host-side intermediate code file, extract the dimensional information of the network, thread blocks, and threads each time the kernel is started;

Determine the new dimensional information of the fused kernel network according to different fusion methods;

Set the new dimension information of the merged host-side intermediate code file;

Generate a new name of the merged host-side intermediate code file, and then generate an intermediate code for calling instructions based on the new name.

According to the method of the present invention, the step of optimizing and compiling the fused host-side intermediate code file through a compilation framework to obtain the corresponding executable file includes:

Optimize the merged host-side intermediate code file through the compilation framework to generate an optimized host-side intermediate code file;

The optimized host-side intermediate code file is compiled through the compilation framework to generate the executable file.

According to the method of the present invention, a global synchronization primitive is inserted into the merged device-side intermediate code file to synchronize all threads of the kernel.

According to the method of the present invention, the compilation framework is an LLVM compilation framework; and/or

The executable file is an executable file applied to a unified computing device architecture.

In the second aspect, embodiments of the present invention provide a deep neural network-based kernel fusion system constructed based on the above method, including:

The intermediate code compilation module is used to compile the source code into host-side intermediate code files and device-side intermediate code files corresponding to the host side and device side through the compilation framework;

The first fusion code module is used to input the host-side intermediate code file and the device-side intermediate code file as parameters into the fusion framework, and generate a fused device-side intermediate code file;

The first optimization and compilation module is used to optimize and compile the fused device-side intermediate code file through the compilation framework to obtain a host-side intermediate code file with device-side information;

The second fusion code module inputs the host-side intermediate code file with device-side information and the device-side intermediate code file as parameters into the fusion framework, and generates a fused host-side intermediate code file;

The second optimization and compilation module is used to optimize and compile the fused host-side intermediate code file through the compilation framework to obtain a corresponding executable file.

In a third aspect, embodiments of the present invention provide a storage medium for storing a computer program for executing any one of the deep neural network-based kernel fusion methods described above.

In a fourth aspect, an embodiment of the present invention is an electronic device, including a storage medium, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the The kernel fusion method based on deep neural network according to any one of the above.

In the embodiment of the present invention, the source code is first compiled into a host-side intermediate code file and a device-side intermediate code file through the compilation framework, and is input into the fusion framework as a parameter to generate a fused device-side intermediate code file; the fused device-side intermediate code file is The device-side intermediate code file is optimized and compiled to obtain a host-side intermediate code file with device-side information; the host-side intermediate code file with device-side information and the device-side intermediate code file are input into the fusion framework to generate the fused Host-side intermediate code file; optimize and compile the merged host-side intermediate code file to obtain the corresponding executable file. Thus, the present invention proposes an operator fusion method that fuses all kernels in a deep neural network into one kernel, including fusion code generation at the intermediate code level, which can reduce kernel startup overhead, data transmission overhead and other performance overhead, improve parallel resource utilization, thereby improving the inference performance of deep neural network systems.

Description of drawings

Figure 1 is a schematic flow chart of a kernel fusion method based on a deep neural network provided by Embodiment 1 of the present invention;

Figure 2 is a schematic flowchart of the kernel fusion method based on deep neural networks provided in Embodiment 2 of the present invention;

Figure 3 is a schematic structural diagram of a deep neural network-based kernel fusion system provided in Embodiment 1 of the present invention;

Figure 4 is a schematic structural diagram of a deep neural network-based kernel fusion system provided in Embodiment 2 of the present invention;

FIG. 5 is a schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present invention.

Detailed ways

In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention and are not intended to limit the present invention.

It should be noted that references to "one embodiment", "embodiment", "example embodiment", etc. in this specification mean that the described embodiment may include specific features, structures or characteristics, but not every Embodiments must contain these specific features, structures or characteristics. Furthermore, such statements are not intended to refer to the same embodiment. Further, when particular features, structures or characteristics are described in connection with embodiments, whether or not explicitly described, it has been shown that it is within the knowledge of those skilled in the art to incorporate such features, structures or characteristics into other embodiments. .

In addition, certain words are used in the description and subsequent claims to refer to specific components or parts. It should be understood by those with ordinary knowledge in the art that a manufacturer may use different nouns or terms to refer to the same component or part. This description and the subsequent claims do not use differences in names as a means to distinguish components or parts; rather, differences in functions of components or parts are used as a criterion for distinction. The words "include" and "include" mentioned in this description and the following claims are open-ended terms, and therefore should be interpreted to mean "include but not limited to." Except, the word "connection" here includes any direct and indirect means of electrical connection. Indirect electrical connection means include connection through other devices.

The kernel fusion method based on deep neural networks provided by the embodiments of the present invention will be described in detail below with reference to the accompanying drawings through specific embodiments and application scenarios.

Figure 1 is a schematic flowchart of a kernel fusion method based on a deep neural network provided by Embodiment 1 of the present invention. The method includes the following steps:

Step S101: Compile the source code into host-side intermediate code (Instruction Reference, IR) files and device-side intermediate code files corresponding to the host side and the device side through the compilation framework.

Optionally, the compilation framework is an LLVM (Low Level Virtual Machine) compilation framework.

Step S102: Input the host-side intermediate code file and the device-side intermediate code file as parameters into the fusion framework, and generate a fused device-side intermediate code file.

Optionally, a global synchronization primitive is inserted into the merged device-side intermediate code file to synchronize all threads of this kernel.

Step S103: Optimize and compile the merged device-side intermediate code file through the compilation framework to obtain a host-side intermediate code file with device-side information.

Step S104: Input the host-side intermediate code file with device-side information and the device-side intermediate code file as parameters into the fusion framework, and generate a merged host-side intermediate code file.

Step S105: Optimize and compile the merged host-side intermediate code file through the compilation framework to obtain a corresponding executable file.

Optionally, the executable file is an executable file applied to CUDA (Compute Unified Device Architecture, unified computing device architecture).

In the kernel fusion method based on deep neural network provided by the embodiment of the present invention, the fusion framework is implemented based on the compilation framework and is responsible for code generation and optimization. First, the source code is compiled into host-side intermediate code files and device-side intermediate code files through the compilation framework, and input into the fusion framework as parameters to generate the fused device-side intermediate code files; the fused device-side intermediate code files are optimized and compile to obtain the host-side intermediate code file with device-side information; input the host-side intermediate code file with device-side information and the device-side intermediate code file into the fusion framework to generate the fused host-side intermediate code file; The fused host-side intermediate code file is optimized and compiled to obtain the corresponding executable file. Thus, the present invention proposes an operator fusion method that fuses all kernels in a deep neural network into one kernel, including fusion code generation at the intermediate code level, which can reduce kernel startup overhead, data transmission overhead and other performance overhead, improve parallel resource utilization, thereby improving the inference performance of deep neural network systems.

Figure 2 is a schematic flowchart of the kernel fusion method based on deep neural networks provided in Embodiment 2 of the present invention. The present invention implements the fusion framework based on the LLVM compilation framework and is responsible for code generation and optimization. The fusion framework runs on an x86 machine, and the fused DNN program can run on an NVIDIA GPU. In this embodiment, the LSTM (Long Short-Term Memory) network implementation is used as an example to illustrate the basic idea of the present invention. The method includes the following steps:

Step S201: Compile the source code into host-side intermediate code files and device-side intermediate code files corresponding to the host side and the device side through the compilation framework.

Optionally, the compilation framework in this embodiment is an LLVM compilation framework. Specifically, the LLVM compilation framework is used to compile the C++ and CUDA (Unified Computing Device Architecture) source codes into LLVM IR files corresponding to the host side and device side: lstm-host.ll and lstm-kernel.ll.

Step S202: Input the host-side intermediate code file and the device-side intermediate code file as parameters into the fusion framework, and generate a fused device-side intermediate code file.

This step involves device-side fusion code generation. Optionally, input the host-side LLVM IR file lstm-host.ll and the device-side LLVM IR file lstm-kernel.ll as parameters into the fusion framework to obtain the merged device-side LLVM IR file: fused-lstm-kernel.ll .

Optionally, when generating code for the device side, you may need to insert global synchronization primitives into the fused device-side intermediate code file fused-lstm-kernel.ll to perform synchronization between all threads of a kernel (kernel). Synchronize. It is guaranteed that all threads have generated the results they are responsible for before calculating the next operator. Preferably, the global synchronization primitive can be implemented through the Cooperative Group API of CUDA (Computing Unified Device Architecture).

Step S203: Optimize the integrated device-side intermediate code file through the compilation framework to generate an optimized device-side intermediate code file.

Optionally, perform traditional compilation optimization on the fused device-side LLVM IR file, including constant propagation, etc., to obtain the optimized device-side LLVM IR file: fused-lstm-kernel-opt.ll.

Step S204: Compile the optimized device-side intermediate code file through the compilation framework to generate a merged device-side binary (fatbin) file.

Optionally, use the LLVM compilation framework, ptxas and fatbinary to compile the optimized device-side LLVM IR file into the fused device-side fatbin file: fused-lstm-kernel.fatbin.

Step S205: Compile the source code and the merged device-side binary file through the compilation framework to generate a host-side intermediate code file with device-side information.

Optionally, the host-side intermediate code file with device-side information is a host-side intermediate code file with device-side binary information. Preferably, use LLVM to compile the C++/CUDA source code and the merged device-side fatbin file into a host-side LLVM IR file with device-side fatbin information: lstm-fatbin-host.ll.

Step S206: Input the host-side intermediate code file with device-side information and the device-side intermediate code file as parameters into the fusion framework to generate a merged host-side intermediate code file.

This step involves fusion code generation on the host side. Optionally, input the host-side LLVM IR file (lstm-fatbin-host.ll) with device-side fatbin information and the device-side LLVM IR file (lstm-kernel.ll) to the fusion framework to generate the merged host-side LLVM IR file: fused-lstm-fatbin-host.ll. The merged host-side LLVM IR file contains device-side fatbin information.

Optionally, the step of generating the merged host-side intermediate code file includes:

(1) Extract the dimension information of the network (grid), thread block (block), and thread (thread) each time the kernel (LaunchKernel) is launched;

(2) Determine the new dimensional information of the fused kernel network (grid) according to different fusion methods;

(3) Generate the merged host-side intermediate code file (LaunchKernel IR);

(4) Set the new dimension information of the fused host-side intermediate code file. Given new parameters, the new parameters are the result of splicing the original parameters in order;

(5) Generate a new name for the fused host-side intermediate code file, and then generate the intermediate code for calling the call instruction based on the new name.

Step S207: Optimize the merged host-side intermediate code file through the compilation framework to generate an optimized host-side intermediate code file.

Preferably, perform traditional compilation and optimization on the fused host-side LLVM IR file with device-side fatbin information, including constant propagation, etc., to obtain the optimized host-side LLVM IR file: fused-lstm-fatbin-host-opt.ll .

Step S208: Compile the optimized host-side intermediate code file through the compilation framework to generate an executable file.

Optionally, use the LLVM compilation framework to compile and link the optimized host-side LLVM IR file into an executable file for CUDA (Unified Computing Device Architecture): fused-lstm.

The purpose of the present invention is to solve the problem that the existing technology's kernel fusion solution requires manual definition of modes or rules; or requires precise compilation and analysis, which takes too long in the tuning phase; and the kernel with global dependencies does not require issues that will be integrated. The present invention proposes an operator fusion method that integrates all overhead in the DNN system into one kernel, including fusion code generation at the intermediate code level, reducing startup kernel overhead and data transmission overhead, and improving parallel resource utilization. Thereby improving the reasoning performance of the DNN system.

After evaluating different operator fusion solutions of the LSTM network, it was found that the inference performance of LSTM under the TensorRT system was 6.30ms, and the inference performance under the Rammer system was 1.72ms. This invention integrates the LSTM network into a kernel and performs compilation optimization. The inference performance is 0.80ms, which is significantly better than other fusion solutions.

It should be noted that, for the deep neural network-based kernel fusion method provided by the embodiment of the present invention, the execution subject may be an electronic device, a deep neural network-based kernel fusion system, or a device used in the deep neural network-based kernel fusion system. A control module that executes the kernel fusion method based on deep neural networks. In the embodiment of the present invention, a kernel fusion system based on a deep neural network executing a kernel fusion method based on a deep neural network is used as an example to illustrate the kernel fusion system based on a deep neural network provided by the embodiment of the present invention.

Figure 3 is a schematic structural diagram of a deep neural network-based kernel fusion system provided in Embodiment 1 of the present invention. The system 100 at least includes an intermediate code compilation module 10, a first fusion code module 20, a first optimization compilation module 30, a second Fusion code module 40 and second optimization compilation module 50, wherein:

The intermediate code compilation module 10 is used to compile the source code into host-side intermediate code files and device-side intermediate code files corresponding to the host side and the device side through the compilation framework. Preferably, the compilation framework is an LLVM compilation framework.

The first fusion code module 20 is used to input the host-side intermediate code file and the device-side intermediate code file as parameters into the fusion framework, and generate a fused device-side intermediate code file.

The first optimization and compilation module 30 is used to optimize and compile the merged device-side intermediate code file through a compilation framework to obtain a host-side intermediate code file with device-side information.

The second fusion code module 40 inputs the host-side intermediate code file with device-side information and the device-side intermediate code file as parameters into the fusion framework, and generates the fused host-side intermediate code file.

The second optimization and compilation module 50 is used to optimize and compile the merged host-side intermediate code file through the compilation framework to obtain a corresponding executable file. Preferably, the executable file is an executable file applied to a unified computing device architecture.

Figure 4 is a schematic structural diagram of a deep neural network-based kernel fusion system provided in Embodiment 2 of the present invention. The system 100 at least includes an intermediate code compilation module 10, a first fusion code module 20, a first optimization compilation module 30, a second Fusion code module 40 and second optimization compilation module 50, wherein:

The intermediate code compilation module 10 is used to compile the source code into host-side intermediate code files and device-side intermediate code files corresponding to the host side and the device side through the compilation framework. Preferably, the compilation framework is an LLVM compilation framework.

The first fusion code module 20 is used to input the host-side intermediate code file and the device-side intermediate code file as parameters into the fusion framework, and generate a fused device-side intermediate code file. Preferably, a global synchronization primitive is inserted into the merged device-side intermediate code file to synchronize all threads of this kernel.

The first optimization and compilation module 30 is used to optimize and compile the merged device-side intermediate code file through a compilation framework to obtain a host-side intermediate code file with device-side information. Preferably, the first optimization compilation module 30 further includes:

The first optimization sub-module 31 is used to optimize the fused device-side intermediate code file through the compilation framework and generate an optimized device-side intermediate code file.

The first compilation sub-module 32 is used to compile the optimized device-side intermediate code file through the compilation framework to generate a merged device-side binary file.

The second compilation sub-module 33 is used to compile the source code and the fused device-side binary file through the compilation framework to generate a host-side intermediate code file with device-side information. Preferably, the host-side intermediate code file with device-side information is a host-side intermediate code file with device-side binary information.

The second fusion code module 40 inputs the host-side intermediate code file with device-side information and the device-side intermediate code file as parameters into the fusion framework, and generates the fused host-side intermediate code file. Preferably, the second fusion code module 40 is used to extract the dimensional information of the network, thread blocks, and threads each time the kernel is started when generating the fused host-side intermediate code file. Determine the new dimensional information of the fused kernel network according to different fusion methods; set the new dimensional information of the fused host-side intermediate code file; generate a new name for the fused host-side intermediate code file, and then use the new name to Generate intermediate code for calling instructions.

The second optimization and compilation module 50 is used to optimize and compile the merged host-side intermediate code file through the compilation framework to obtain a corresponding executable file. Preferably, the second optimization compilation module 50 further includes:

The second optimization submodule 51 is used to optimize the merged host-side intermediate code file through a compilation framework to generate an optimized host-side intermediate code file.

The third compilation sub-module 52 is used to compile the optimized host-side intermediate code file through the compilation framework to generate an executable file. Preferably, the executable file is an executable file applied to a unified computing device architecture.

The deep neural network-based kernel fusion system provided by the embodiment of the present invention can implement various processes implemented by the deep neural network-based kernel fusion method in the figure. To avoid duplication, they will not be described again here.

The kernel fusion system based on the deep neural network provided by the embodiment of the present invention first compiles the source code into a host-side intermediate code file and a device-side intermediate code file through a compilation framework, and inputs them as parameters into the fusion framework to generate the fused device-side Intermediate code file; optimize and compile the fused device-side intermediate code file to obtain a host-side intermediate code file with device-side information; input the host-side intermediate code file with device-side information and the device-side intermediate code file Go to the fusion framework and generate the fused host-side intermediate code file; optimize and compile the fused host-side intermediate code file to obtain the corresponding executable file. Thus, the present invention proposes an operator fusion system that fuses all kernels in a deep neural network into one kernel, including fusion code generation at the intermediate code level, which can reduce kernel startup overhead, data transmission overhead and other performance overhead, improve parallel resource utilization, thereby improving the inference performance of deep neural network systems.

The present invention also provides a storage medium for storing the computer program of any deep neural network-based kernel fusion method as shown in Figures 1 to 2. For example, computer program instructions, when executed by a computer, can call or provide methods and/or technical solutions according to the present invention through the operation of the computer, and can achieve the same technical effect. To avoid duplication, they will not be described again here. The program instructions for calling the method of the present invention may be stored in a fixed or removable storage medium, and/or be transmitted through a data stream in a broadcast or other signal-bearing media and/or be stored in a program instruction according to the program instructions. The storage medium of the computer device on which it is running.

According to an embodiment of the present invention, the present invention also provides an electronic device 400 as shown in Figure 5. The electronic device 400 optionally includes a storage medium 200 for storing a computer program and a processor for executing the computer program. 300, wherein when the computer program is executed by the processor 300, any one of the above-mentioned deep neural network-based kernel fusion methods is implemented, triggering the electronic device 400 to execute the methods and/or technical solutions based on the foregoing embodiments, And can achieve the same technical effect. To avoid repetition, they will not be described again here. It should be noted that electronic devices in embodiments of the present invention include mobile electronic devices and non-mobile electronic devices. For example, mobile electronic devices may be mobile phones, tablet computers, notebook computers, PDAs, vehicle-mounted electronic devices, wearable devices, super mobile personal computers, netbooks, or personal digital assistants, etc., and non-mobile electronic devices may be servers, network attachments, etc. Storage (Network Attached Storage, NAS), personal computer (PC), television (television, TV), teller machine or self-service machine, etc. are not specifically limited in the embodiment of the present invention.

It should be noted that the present invention may be implemented in software and/or a combination of software and hardware, for example, using an application specific integrated circuit (ASIC), a general purpose computer or any other similar hardware device. In one embodiment, the software program of the present invention can be executed by a processor to implement the above steps or functions. Likewise, the software program of the present invention (including associated data structures) may be stored in a computer-readable recording medium, such as a RAM memory, a magnetic or optical drive or a floppy disk and similar devices. In addition, some steps or functions of the present invention may be implemented in hardware, for example, as a circuit that cooperates with a processor to perform each step or function.

The present invention can be implemented on a computer as a computer implementation method, or in dedicated hardware, or in a combination of both. Executable code for the method according to the invention or parts thereof may be stored on a computer program product. Examples of computer program products include memory devices, optical storage devices, integrated circuits, servers, online software, and the like. Optionally, the computer program product includes non-transitory program code means stored on a computer-readable medium for performing the method according to the invention when the program product is executed on a computer.

In an alternative embodiment, the computer program comprises computer program code means adapted to perform all steps of the method according to the invention when the computer program is run on a computer. Optionally, the computer program is embodied on a computer-readable medium.

It should be noted that, in this document, the terms "comprising", "comprises" or any other variation thereof are intended to cover a non-exclusive inclusion, such that a process, method, article or device that includes a series of elements not only includes those elements, It also includes other elements not expressly listed or inherent in the process, method, article or apparatus. Without further limitation, an element defined by the statement "comprises a..." does not exclude the presence of additional identical elements in a process, method, article, or device that includes that element. In addition, it should be pointed out that the scope of the methods and apparatuses in the embodiments of the present invention is not limited to performing functions in the order shown or discussed, but may also include performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved. Functions may be performed, for example, the methods described may be performed in an order different from that described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

Of course, the present invention can also have various other embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art can make various corresponding changes and deformations according to the present invention. However, these corresponding Changes and deformations should fall within the protection scope of the appended claims of the present invention.

Claims (10)

1. The kernel fusion method based on the deep neural network is characterized by comprising the following steps of:

compiling the source codes into host end intermediate code files and equipment end intermediate code files corresponding to the host end and the equipment end respectively through a compiling frame;

inputting the host side intermediate code file and the equipment side intermediate code file as parameters into a fusion framework to generate a fused equipment side intermediate code file;

optimizing and compiling the fused equipment-end intermediate code file through the compiling framework to obtain a host-end intermediate code file with equipment-end information;

inputting the host end intermediate code file with the equipment end information and the equipment end intermediate code file as parameters into the fusion frame to generate a fused host end intermediate code file;

and optimizing and compiling the fused host side intermediate code file through the compiling framework to obtain a corresponding executable file.

2. The method according to claim 1, wherein the step of optimizing and compiling the fused device-side intermediate code file by the compiling framework to obtain a host-side intermediate code file with device-side information includes:

optimizing the fused equipment end intermediate code file through the compiling framework to generate an optimized equipment end intermediate code file;

compiling the optimized equipment-side intermediate code file through the compiling framework to generate a fused equipment-side binary file;

and compiling the source code and the fused equipment-side binary file through the compiling framework to generate the host-side intermediate code file with the equipment-side information.

3. The method of claim 2, wherein the host side intermediate code file with device side information is a host side intermediate code file with device side binary information.

4. The method of claim 1, wherein the step of generating the fused host-side intermediate code file comprises:

when the fused host side intermediate code file is generated, extracting dimension information of a network, a thread block and a thread when the kernel is started each time;

determining new dimension information of the network of the fused kernel according to different fusion modes;

setting the new dimension information of the fused host side intermediate code file;

and generating a new name of the fused host side intermediate code file, and then generating an intermediate code of the calling instruction according to the new name.

5. The method of claim 1, wherein the step of optimizing and compiling the fused host-side intermediate code file by a compiling framework to obtain a corresponding executable file comprises:

optimizing the fused host side intermediate code file through the compiling framework to generate an optimized host side intermediate code file;

and compiling the optimized host side intermediate code file through the compiling framework to generate the executable file.

6. The method of claim 1, wherein global synchronization primitives are inserted into the fused device side intermediate code file to synchronize all threads of the kernel.

7. The method of claim 1, wherein the compilation framework is an LLVM compilation framework; and/or

The executable file is an executable file applied to a unified computing device architecture.

8. A deep neural network-based kernel fusion system constructed based on the method of any one of claims 1-7, comprising:

the intermediate code compiling module is used for compiling the source codes into a host end intermediate code file and a device end intermediate code file corresponding to the host end and the device end respectively through a compiling framework;

the first fusion code module is used for inputting the host side intermediate code file and the equipment side intermediate code file as parameters into a fusion frame to generate a fused equipment side intermediate code file;

the first optimizing and compiling module is used for optimizing and compiling the fused equipment-end intermediate code file through the compiling framework to obtain a host-end intermediate code file with equipment-end information;

the second fusion code module is used for inputting the host end intermediate code file with the equipment end information and the equipment end intermediate code file into the fusion frame as parameters to generate a fused host end intermediate code file;

and the second optimizing and compiling module is used for optimizing and compiling the fused host side intermediate code file through the compiling framework to obtain a corresponding executable file.

9. A storage medium storing a computer program for performing the deep neural network-based kernel fusion method of any one of claims 1 to 7.

10. An electronic device comprising a storage medium, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the deep neural network based kernel fusion method of any one of claims 1-7 when the computer program is executed.