CN115543554A - Method and device for scheduling calculation jobs and computer readable storage medium - Google Patents

Abstract

The present application discloses a computing job scheduling method, device, and computer-readable storage medium. By obtaining the queuing reason of the queuing job, when the queuing reason is that the remaining resource amount of the computing resource pool corresponding to the queuing job is less than the required resource amount of the queuing job When , obtain the remaining resources of other computing resource pools in the cluster. When the remaining resources of other computing resource pools are greater than or equal to the required resources, and other computing resource pools match the queued jobs, assign the queued jobs to other computing resource pools Computing; it can not only improve the effective utilization of computing resources in the cluster, but also reduce the queuing time of computing jobs, thereby improving the computing efficiency and utilization of the cluster.

Description

A scheduling method, device, and computer-readable storage medium for computing jobs

technical field

The present application relates to the technical field of computing scheduling, and in particular to a computing job scheduling method, device, and computer-readable storage medium.

Background technique

High Performance Computing (HPC for short) refers to computer technology aimed at improving scientific computing capabilities. HPC simulation computing is a kind of parallel computing, which divides an application program into multiple parts that can be executed in parallel and assigns them to multiple processors for execution.

HPC simulation computing needs to rely on scheduling software to manage the computing scheduling of multiple applications (such as simulation software, etc.), but the usual scheduling software can only perform computing scheduling for applications in a cluster of a single environment (such as a single computing pool), which is difficult It meets the needs of computing resource scheduling problems in complex environments.

Contents of the invention

In order to solve the above-mentioned technical problems, the present application is proposed. Embodiments of the present application provide a computing job scheduling method, device, and computer-readable storage medium, which solve the above-mentioned technical problems.

According to one aspect of the present application, a scheduling method for computing jobs is provided, including: obtaining the queuing reason of the queued job; wherein, the queuing job represents a computing job that is queuing up for processing; when the queuing reason is the queuing When the remaining resource amount of the computing resource pool corresponding to the job is less than the required resource amount of the queued job, obtain the remaining resource amount of other computing resource pools in the cluster; wherein, the cluster includes the computing resource pool corresponding to the queued job and the the other computing resource pools; and when the remaining resource amount of the other computing resource pools is greater than or equal to the required resource amount, and the other computing resource pools match the queued jobs, assigning the queued jobs to the other computing resource pools for computing.

In an embodiment, when the queuing reason is that the remaining resource amount of the computing resource pool corresponding to the queued job is less than the required resource amount of the queued job, obtaining the remaining resource amount of other computing resource pools in the cluster includes :

When the reason for queuing is that the remaining resource amount of the computing resource pool corresponding to the queued job minus the amount of reserved resources is less than the required resource amount of the queued job, obtain the remaining resource amount of other computing resource pools in the cluster; wherein, The reserved resource amount represents the reserved resource amount in a single computing resource pool, and the reserved resource amount is positively correlated with the remaining resource amount of the corresponding computing resource pool.

In an embodiment, when the remaining resource amount of the other computing resource pool is greater than or equal to the required resource amount, and the other computing resource pool matches the queued job, the queued job Allocation to the other computing resource pools for computing includes:

When the remaining resource amount of the other computing resource pool minus the reserved resource amount is greater than or equal to the required resource amount, and the other computing resource pool matches the queued job, assign the queued job to The other computing resource pools perform calculations; wherein, the reserved resource amount represents the reserved resource amount in a single computing resource pool, and the reserved resource amount is positively correlated with the remaining resource amount of the corresponding computing resource pool.

In an embodiment, the scheduling method of the computing job also includes:

When the remaining resource amounts of the other computing resource pools are all smaller than the required resource amounts, stop scheduling the queued jobs.

In an embodiment, after the scheduling of the queued job is stopped, the method for scheduling the computing job further includes:

Scheduling computing jobs subsequent to the queued jobs.

In an embodiment, the scheduling method of the computing job also includes:

When the sum of the required resources of the computing jobs corresponding to a single user is greater than the upper limit of single user resources, stop scheduling the computing jobs of the single user; The amount of remaining resources in the resource pool is positively related.

In an embodiment, before the acquisition of the queuing cause of the queued job, the scheduling method of the computing job further includes:

According to the requirements of all computing jobs and the computing characteristics of each computing resource pool in the cluster, the computing jobs are allocated to each of the computing resource pools for matching.

In one embodiment, the queuing reasons for obtaining the queued jobs include:

When the queuing job exists in the cluster, the queuing reason of the queuing job is acquired.

According to another aspect of the present application, a computing job scheduling device is provided, including: a first acquisition module, configured to acquire the queuing reason of the queued job; wherein, the queued job represents a computing job that is queued for processing; 2. An acquisition module, configured to acquire the remaining resources of other computing resource pools in the cluster when the queuing cause is that the remaining resources of the computing resource pool corresponding to the queuing job are less than the required resource amount of the queuing job; wherein, The cluster includes the computing resource pool corresponding to the queued job and the other computing resource pools; and a scheduling execution module, configured to, when the remaining resource amount of the other computing resource pool is greater than or equal to the required resource amount, And when the other computing resource pool matches the queued job, assign the queued job to the other computing resource pool for computing.

According to another aspect of the present application, a computer-readable storage medium is provided, the storage medium stores a computer program, and the computer program is used to execute any of the methods for scheduling computing jobs described above.

The application provides a computing job scheduling method, device, and computer-readable storage medium. When there are queuing jobs, by obtaining the queuing reasons of the queuing jobs, when the queuing reason is that the remaining resources of the computing resource pool corresponding to the queuing jobs are less than When queuing the required resources of the job, obtain the remaining resources of other computing resource pools in the cluster. When the remaining resources of other computing resource pools are greater than or equal to the required resources and the other computing resource pools match the queued jobs, the queued jobs will be Allocate to other computing resource pools for calculation; that is, to judge the queuing reason of the queued job, if the remaining resource amount of the computing resource pool corresponding to the queued job is insufficient, obtain the remaining resource amount of other computing resource pools in the cluster, if other If the remaining resources of the computing resource pool meet the resource requirements of the queued job and the other computing resource pool matches the queued job, then the queued job is allocated to the other computing resource pool for computing, which can not only improve the effective utilization of computing resources in the cluster Utilization, and can reduce the queuing time of computing jobs, thereby improving the computing efficiency and utilization of the cluster.

Description of drawings

The above and other objects, features and advantages of the present application will become more apparent through a more detailed description of the embodiments of the present application in conjunction with the accompanying drawings. The accompanying drawings are used to provide a further understanding of the embodiments of the present application, and constitute a part of the specification, and are used together with the embodiments of the present application to explain the present application, and do not constitute limitations to the present application. In the drawings, the same reference numerals generally represent the same components or steps.

Fig. 1 is a schematic flowchart of a scheduling method for computing jobs provided by an exemplary embodiment of the present application.

Fig. 2 is a schematic flowchart of a scheduling method for computing jobs provided by another exemplary embodiment of the present application.

Fig. 3 is a schematic flowchart of a scheduling method for computing jobs provided by another exemplary embodiment of the present application.

Fig. 4 is a schematic flowchart of a scheduling method for computing jobs provided by another exemplary embodiment of the present application.

Fig. 5 is a schematic flowchart of a scheduling method for computing jobs provided by another exemplary embodiment of the present application.

Fig. 6 is a schematic flowchart of a method for scheduling computing jobs provided by another exemplary embodiment of the present application.

Fig. 7 is a schematic structural diagram of a computing job scheduling device provided by an exemplary embodiment of the present application.

Fig. 8 is a schematic structural diagram of an apparatus for scheduling computing jobs provided by another exemplary embodiment of the present application.

Fig. 9 is a structural diagram of an electronic device provided by an exemplary embodiment of the present application.

detailed description

Hereinafter, exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Apparently, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments of the present application. It should be understood that the present application is not limited by the exemplary embodiments described here.

The application of HPC simulation computing needs to rely on HPC scheduling software to realize computing job scheduling, but the computing job scheduling strategy of common HPC scheduling software can only be scheduled for a single computing cluster, and cannot satisfy multiple computing clusters and local computing clusters. An environment combined with cloud computing clusters.

In order to solve the mutual scheduling between multiple computing clusters (including local computing clusters and/or cloud computing clusters), this application proposes a computing job scheduling method, device, and computer-readable storage medium, which manage all clusters in the same scheduling , by monitoring the queued jobs in each cluster (or computing resource pool), when a queued job appears in a cluster, detect the remaining computing resources of other clusters, if the remaining computing resources of other clusters meet the needs of the queued job , dispatch the queued job to other clusters for computing processing, so as to improve the computing efficiency of the HPC simulation computing program and the resource utilization of all computing clusters.

The specific solutions and implementations of a computing job scheduling method, device, and computer-readable storage medium provided in the embodiments of the present application are described in detail below with reference to the accompanying drawings.

Fig. 1 is a schematic flowchart of a scheduling method for computing jobs provided by an exemplary embodiment of the present application. As shown in Figure 1, the scheduling method of the computing job includes the following steps:

Step 110: Obtain the queuing reason of the queuing job.

Wherein, queued jobs represent computing jobs that are queued for processing. In an embodiment, the specific implementation manner of step 110 may be: when there are queued jobs waiting in queue in the cluster, acquiring the queuing reason of the queued jobs.

Specifically, this application is applied to an HPC cluster scenario of multiple computing resource pools, for example, including three computing resource pools (the first local computing resource pool, the second local computing resource pool, and the cloud computing resource pool), wherein the first The local computing resource pool includes 36 servers (CPU), the second local computing resource pool includes 48 servers (CPU), and the cloud computing resource pool includes 64 servers (CPU), and the software required for this application includes STAR-CCM+ , Fluent, Abaqus, LS_Dyna, MechanicalAPDL, Optistruct, etc. It should be understood that the number of computing resource pools and the number of corresponding servers in this application are only exemplary, and are not limited to the specific number of computing resource pools and the specific number of corresponding servers. When a user submits a software calculation, it may be because the resources corresponding to the software or the customer's computing resource pool are all used up or the remaining resources are not enough to calculate the application software, then it can be determined that the application software is a queued job. When there is a queuing job in the cluster, activate the scheduler, that is, judge the queuing reason of the queuing job, so as to schedule resources according to the queuing reason. Specifically, the scheduler can periodically (for example, every minute) determine whether there are queued jobs in the cluster, so as to avoid long-time queues without being discovered.

Step 120: When the reason for queuing is that the remaining resource amount of the computing resource pool corresponding to the queued job is less than the required resource amount of the queued job, obtain the remaining resource amount of other computing resource pools in the cluster.

Wherein, the cluster includes the computing resource pool corresponding to the queuing job and other computing resource pools. The other computing resource pools may be one or more. For example, in the above example, if the first local computing resource pool is the computing resource pool corresponding to the queuing job , the second local computing resource pool and the cloud computing resource pool are other computing resource pools. When it is determined that the queuing reason of the queued job is that the remaining resource amount of the corresponding computing resource pool is less than the required resource amount, the scheduling software obtains the remaining resource amount of other computing resource pools in the cluster to determine whether scheduling can be performed. For example, when there is a queued job at the first local computing resource pool, and the reason for queuing is that the remaining resources of the first local computing resource pool are not enough to meet the required resources of the queued job, obtain the second local computing resource pool and cloud computing resources The amount of remaining resources in the pool. It should be understood that the present application may obtain the remaining resources of other computing resource pools in a preset order, for example, first obtain the remaining resources of the second local computing resource pool, if the remaining resources of the second local computing resource pool do not meet the requirements Then obtain the remaining resource amount of the computing resource pool on the cloud, so as to save the calculation amount of calculating the remaining resource amount.

Step 130: When the remaining resources of other computing resource pools are greater than or equal to the required resource amount, and other computing resource pools match the queued jobs, assign the queued jobs to other computing resource pools for calculation.

Due to the different requirements of different simulation computing software, for example, STAR-CCM software is suitable for multi-core servers but has no requirement for the main frequency, so the computing resource pool on the cloud is selected for calculation. Optistruct software is suitable for high main frequency but does not require the number of core servers. Then, the first local computing resource pool is selected. Therefore, before acquiring the remaining resources of other computing resource pools, it is preferable to select the computing resource pool that meets the requirements of the queued job as the target. After obtaining the remaining resources of other computing resource pools, if the remaining resources of other computing resource pools are greater than or equal to the required resources of the queued job (that is, the remaining resources of the other computing resource pools meet the needs of the queued job) , and the other computing resource pool matches the queued job, at this time, the queued job can be assigned to the other computing resource pool for computing, so as to reduce the number of simulation computing software waiting in the queue, thereby improving the effective utilization rate and computing efficiency of the cluster .

A computing job scheduling method provided by this application, when there is a queued job, by obtaining the queuing reason of the queued job, when the queuing reason is that the remaining resource amount of the computing resource pool corresponding to the queued job is less than the required resource amount of the queued job, Obtain the remaining resources of other computing resource pools in the cluster. When the remaining resources of other computing resource pools are greater than or equal to the required resources, and other computing resource pools match the queued jobs, assign the queued jobs to other computing resource pools for computing ; That is, to judge the queuing reason of the queuing job, if the remaining resource amount of the computing resource pool corresponding to the queuing job is insufficient, obtain the remaining resource amount of other computing resource pools in the cluster, if the remaining resource amount of other computing resource pools satisfies If the resource requirements of the queued job and the other computing resource pool match the queued job, then the queued job is allocated to the other computing resource pool for computing, which can not only improve the effective utilization of computing resources in the cluster, but also reduce the workload of computing jobs. Queue waiting time, which can improve the computing efficiency and utilization of the cluster.

Fig. 2 is a schematic flowchart of a scheduling method for computing jobs provided by another exemplary embodiment of the present application. As shown in Figure 2, the above step 120 may include:

Step 121: When the queuing reason is that the remaining resource amount of the computing resource pool corresponding to the queued job minus the reserved resource amount is less than the required resource amount of the queued job, obtain the remaining resource amount of other computing resource pools in the cluster.

Wherein, the reserved resource amount represents the reserved resource amount in a single computing resource pool, and the reserved resource amount is positively correlated with the remaining resource amount of the corresponding computing resource pool. In order to ensure the calculation speed of the computing resource pool, a reserved resource amount can be set for each computing resource pool. On the one hand, it can avoid the oversaturated operation of the computing resource pool, and on the other hand, it can also provide new users or more urgent computing jobs. Reserve a certain amount of computing resources. Specifically, the amount of reserved resources is positively correlated with the amount of remaining resources, that is, the more remaining resources in a single computing resource pool, the more resources are reserved. For example, if the number of servers in a single computing resource pool is 50, when the resource utilization is 0 (i.e. completely idle), at this time, the required resources of a single simulation computing software can be allowed to be up to 30 servers (i.e., the reserved resources are 20 servers); when the resource utilization rate is 80% (i.e., the remaining resources are 10 servers), at this time, the required resources of a single simulation computing software can be allowed to be up to 5 servers (that is, the reserved resources are 5 servers). Preferably, the application may preset corresponding retention ratios according to different resource utilization rates, for example, when the resource utilization rate is 0, the retention ratio is 40%, and when the resource utilization rate is 80%, the retention ratio is 50%.

When the amount of remaining resources in the computing resource pool minus the amount of reserved resources is less than the required amount of resources for queued jobs, that is, when the amount of resources that can be used by a single simulation computing software in the computing resource pool is less than the required amount of resources, other computing resources in the cluster will be obtained The amount of remaining resources in the pool is used to schedule the single simulation computing software.

Fig. 3 is a schematic flowchart of a scheduling method for computing jobs provided by another exemplary embodiment of the present application. As shown in Figure 3, the above step 130 may include:

Step 131: When the remaining resource amount of other computing resource pools minus the reserved resource amount is greater than or equal to the required resource amount, and other computing resource pools match the queued jobs, assign the queued jobs to other computing resource pools for calculation.

Wherein, the reserved resource amount represents the reserved resource amount in a single computing resource pool, and the reserved resource amount is positively correlated with the remaining resource amount of the corresponding computing resource pool. In order to ensure the computing speed of the computing resource pool, each computing resource pool sets a reserved resource amount to avoid the oversaturated operation of the computing resource pool, and can also reserve certain computing resources for new users or more urgent computing jobs . Therefore, when obtaining the available resources of other computing resource pools, it is also necessary to reserve a part of the resources, that is, only when the remaining resources of other computing resource pools minus the corresponding reserved related) after the resource amount can still meet the demand resource amount of the queued job, and the computing resource pool matches the queued job, the queued job is assigned to the other computing resource pool for computing, so as to ensure that the other computing resources Normal computing operation of the pool.

Fig. 4 is a schematic flowchart of a scheduling method for computing jobs provided by another exemplary embodiment of the present application. As shown in Figure 4, the scheduling method of the above calculation job may also include:

Step 140: When the remaining resources of other computing resource pools are less than the required resources, stop scheduling the queued jobs.

If the remaining resources of other computing resource pools are less than the required resources of the queuing job, that is, the remaining resources of all computing resource pools in the cluster cannot meet the requirements of the queuing job, the only way to exit the scheduling level is to stop processing Queued jobs are scheduled, maintaining the existing queued state. And in the next cycle, it is judged again whether there are queued jobs and whether there are other computing resource pools that meet the requirements of the queued jobs. If the judgment results are all yes, the queued jobs are scheduled.

In an embodiment, as shown in FIG. 4, after step 140, the above-mentioned computing job scheduling method may further include:

Step 150: Scheduling the computing jobs following the queued jobs.

Due to the different resource requirements of different simulation computing software, if there are multiple queued jobs in a certain computing resource pool, and the resource requirements of the queued jobs at the front of the queue exceed the remaining resources of all computing resource pools in the cluster, if When the amount of required resources for the subsequent queued jobs is small, there may be a computing resource pool that can be used for calculation, and the subsequent queued jobs can be scheduled (the specific scheduling method is as described in the above-mentioned steps 110-130), so that as much as possible Improve the resource utilization of the entire cluster and the computing efficiency of the simulation computing software.

Fig. 5 is a schematic flowchart of a scheduling method for computing jobs provided by another exemplary embodiment of the present application. As shown in Figure 5, the scheduling method of the above calculation job may also include:

Step 160: When the sum of required resource amounts of computing jobs corresponding to a single user is greater than the upper limit of resource amounts of a single user, stop scheduling the computing jobs of a single user.

Due to the limited computing resources in the computing resource pool, if a single user submits too many simulation computing software, the single user may occupy too many resources and cause other users to fail to use it normally. Therefore, this application can limit the total resources of a single user (setting the upper limit of single-user resources). The computing jobs submitted by users can only be queued up, and the scheduling software stops scheduling the computing jobs of this single user.

Wherein, the upper limit of the single-user resource amount is positively correlated with the remaining resource amount of the corresponding computing resource pool. Specifically, the upper limit of single-user resources is positively correlated with the remaining resources, that is, the more remaining resources in a single computing resource pool, the higher the upper limit of single-user resources. For example, if the number of servers in a single computing resource pool is 50, when the resources When the utilization rate is 0 (that is, completely idle), the total computing resources of the simulation computing software submitted by a single user can be up to 30 servers; when the resource utilization rate is 80%, the simulation calculation software submitted by a single user can be allowed at this time The total computing resources of the computing software are up to 5 servers (that is, 5 servers are reserved, and the 5 reserved servers can be labeled, and common software is prohibited from running, so as to avoid the oversaturation problem caused by the 5 reserved servers being used as much as possible ). Preferably, this application can pre-set the ratio of the upper limit of the corresponding single-user resource amount to the remaining resource amount according to different resource utilization rates. For example, when the above-mentioned resource utilization rate is 0, the ratio is 40%, while the resource utilization rate is 80%. When the ratio is 50%.

Fig. 6 is a schematic flowchart of a method for scheduling computing jobs provided by another exemplary embodiment of the present application. As shown in FIG. 6, before step 110, the above-mentioned scheduling method for computing jobs may further include:

Step 170: According to the requirements of all computing jobs and the computing characteristics of each computing resource pool in the cluster, assign the computing jobs to each computing resource pool for matching.

Due to the different requirements of different simulation computing software, for example, STAR-CCM software is suitable for multi-core servers but does not require the main frequency, so choose a computing resource pool with a large number of servers for calculation. Optistruct software is suitable for high main frequency but has no requirements for the number of core servers. If there is no requirement, choose a computing resource pool with a higher frequency. In order to achieve higher computing efficiency and computing effects as much as possible, the application can pre-allocate each simulation computing software to a matching computing resource pool, that is, according to the requirements of the simulation computing software and the computing characteristics of the computing resource pool, the simulation Computing software and computing resource pools are paired to ensure that each simulation computing software can perform computing processing in a better computing resource pool without scheduling, so as to ensure the simulation effect. Moreover, in order to further improve the utilization rate and balance of computing resources in the cluster, this application can evenly distribute each simulation computing software to each computing resource pool under the premise of satisfying the matching principle according to the required resource amount of each simulation computing software , to minimize the risk of computing resource pool saturation, thereby reducing scheduling. In addition, after a period of time, the application can also comprehensively match each simulation computing software and computing resource pool according to the usage frequency of each simulation computing software during this period of time, so as to further reduce the risk of computing resource pool saturation, thereby reducing scheduling.

Fig. 7 is a schematic structural diagram of a computing job scheduling device provided by an exemplary embodiment of the present application. As shown in FIG. 7 , the scheduling device 70 for computing jobs includes: a first acquiring module 71, configured to acquire the queuing reasons of the queued jobs; wherein, the queued jobs represent computing jobs that are queuing up for processing; a second acquiring module 72 is used for When the reason for queuing is that the remaining resource amount of the computing resource pool corresponding to the queued job is less than the required resource amount of the queued job, obtain the remaining resource amount of other computing resource pools in the cluster; where the cluster includes the computing resource pool corresponding to the queued job and other computing resource pools Resource pool; and scheduling execution module 73, for when the remaining resource amount of other computing resource pools is greater than or equal to the required resource amount, and when other computing resource pools match the queued jobs, assign the queued jobs to other computing resource pools for calculation .

A computing job scheduling device provided by the present application, when there is a queued job, obtains the queuing reason of the queued job through the first acquisition module 71, when the queuing reason is that the remaining resource amount of the computing resource pool corresponding to the queued job is less than that of the queued job When the amount of resources is required, the second acquisition module 72 acquires the remaining resource amounts of other computing resource pools in the cluster, and when the remaining resource amounts of other computing resource pools are greater than or equal to the required resource amount, and other computing resource pools match the queued jobs, scheduling Execution module 73 assigns the queued job to other computing resource pools for calculation; that is, judges the reason for queuing the queued job, and if the remaining resource amount of the computing resource pool corresponding to the queued job is insufficient, then obtain the other computing resource pools in the cluster. The amount of remaining resources, if the remaining resources of other computing resource pools meet the resource requirements of the queued job and the other computing resource pool matches the queued job, then the queued job is assigned to the other computing resource pool for calculation, which can not only improve The effective use of computing resources in the cluster can reduce the queuing time of computing jobs, thereby improving the computing efficiency and utilization of the cluster.

In an embodiment, the first acquiring module 71 may be further configured to: acquire the queuing reason of the queuing job when there is a queuing job waiting in line in the cluster.

In an embodiment, the second acquisition module 72 may be further configured to: when the reason for queuing is that the remaining resource amount of the computing resource pool corresponding to the queued job minus the amount of reserved resources is less than the required resource amount of the queued job, obtain other resources in the cluster. Calculate the remaining resources of the resource pool. Wherein, the reserved resource amount represents the reserved resource amount in a single computing resource pool, and the reserved resource amount is positively correlated with the remaining resource amount of the corresponding computing resource pool.

In an embodiment, the scheduling execution module 73 may be further configured to: when the remaining resource amount of other computing resource pools minus the reserved resource amount is greater than or equal to the required resource amount, and other computing resource pools match the queued jobs, queue Jobs are assigned to other computing resource pools for computing. Wherein, the reserved resource amount represents the reserved resource amount in a single computing resource pool, and the reserved resource amount is positively correlated with the remaining resource amount of the corresponding computing resource pool.

Fig. 8 is a schematic structural diagram of an apparatus for scheduling computing jobs provided by another exemplary embodiment of the present application. As shown in FIG. 8 , the computing job scheduling device 70 may include: a scheduling termination module 74 configured to stop scheduling queued jobs when the remaining resource amounts of other computing resource pools are less than the required resource amounts. Correspondingly, the above computing job scheduling device 70 may be further configured to: schedule computing jobs after the queued jobs.

In an embodiment, the scheduling termination module 74 may be further configured to stop scheduling the computing jobs of a single user when the sum of required resource amounts of computing jobs corresponding to a single user is greater than the upper limit of resource amounts of a single user.

In one embodiment, as shown in FIG. 8 , the scheduling device 70 for the computing jobs mentioned above may include: a pre-allocation module 75, configured to assign computing jobs to Assigned to each computing resource pool match.

Next, an electronic device according to an embodiment of the present application will be described with reference to FIG. 9 . The electronic device may be either or both of the first device and the second device, or a stand-alone device independent of them, and the stand-alone device may communicate with the first device and the second device to receive collected data from them. input signal.

FIG. 9 illustrates a block diagram of an electronic device according to an embodiment of the present application.

As shown in FIG. 9 , an electronic device 10 includes one or more processors 11 and a memory 12 .

Processor 11 may be a central processing unit (CPU) or other form of processing unit having data processing capabilities and/or instruction execution capabilities, and may control other components in electronic device 10 to perform desired functions.

Memory 12 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and/or cache memory (cache). The non-volatile memory may include, for example, a read-only memory (ROM), a hard disk, a flash memory, and the like. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 11 may execute the program instructions to implement the above-mentioned method for scheduling computing jobs in various embodiments of the present application and/or or other desired functionality. Various contents such as input signal, signal component, noise component, etc. may also be stored in the computer-readable storage medium.

In one example, the electronic device 10 may further include: an input device 13 and an output device 14, and these components are interconnected through a bus system and/or other forms of connection mechanisms (not shown).

When the electronic device is a stand-alone device, the input device 13 may be a communication network connector for receiving collected input signals from the first device and the second device.

In addition, the input device 13 may also include, for example, a keyboard, a mouse, and the like.

The output device 14 can output various information to the outside, including determined distance information, direction information, and the like. The output device 14 may include, for example, a display, a speaker, a printer, a communication network and its connected remote output devices, and the like.

Of course, for the sake of simplicity, only some of the components related to the present application in the electronic device 10 are shown in FIG. 9 , and components such as bus, input/output interface, etc. are omitted. In addition, according to specific application conditions, the electronic device 10 may also include any other suitable components.

The computer program product can be written in any combination of one or more programming languages to execute the program codes for performing the operations of the embodiments of the present application, and the programming languages include object-oriented programming languages, such as Java, C++, etc. , also includes conventional procedural programming languages, such as the "C" language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server to execute.

The computer readable storage medium may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may include, but not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or devices, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: electrical connection with one or more conductors, portable disk, hard disk, random access memory (RAM), read only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

The foregoing description has been presented for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of the application to the forms disclosed herein. Although a number of example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims (10)

1. A scheduling method for computing jobs, comprising: Obtain the queuing reason of the queued job; wherein, the queued job represents a computing job that is queued for processing; When the queuing reason is that the remaining resource amount of the computing resource pool corresponding to the queued job is less than the required resource amount of the queued job, obtain the remaining resource amount of other computing resource pools in the cluster; wherein, the cluster includes the The computing resource pool corresponding to the queued job and the other computing resource pools; and When the remaining resource amount of the other computing resource pool is greater than or equal to the required resource amount, and the other computing resource pool matches the queued job, assign the queued job to the other computing resource pool Calculation. 2. The method for scheduling computing jobs according to claim 1, wherein when the reason for queuing is that the amount of remaining resources in the computing resource pool corresponding to the queued job is less than the required resource amount of the queued job , to obtain the remaining resources of other computing resource pools in the cluster include: When the reason for queuing is that the remaining resource amount of the computing resource pool corresponding to the queued job minus the amount of reserved resources is less than the required resource amount of the queued job, obtain the remaining resource amount of other computing resource pools in the cluster; wherein, The reserved resource amount represents the reserved resource amount in a single computing resource pool, and the reserved resource amount is positively correlated with the remaining resource amount of the corresponding computing resource pool. 3. The computing job scheduling method according to claim 1, wherein when the remaining resource amount of the other computing resource pool is greater than or equal to the required resource amount and the other computing resource pool When matching with the queued job, assigning the queued job to the other computing resource pool for computing includes: When the remaining resource amount of the other computing resource pool minus the reserved resource amount is greater than or equal to the required resource amount, and the other computing resource pool matches the queued job, assign the queued job to The other computing resource pools perform calculations; wherein, the reserved resource amount represents the reserved resource amount in a single computing resource pool, and the reserved resource amount is positively correlated with the remaining resource amount of the corresponding computing resource pool. 4. The scheduling method for computing jobs according to any one of claims 1 to 3, wherein the scheduling method for computing jobs further comprises: When the remaining resource amounts of the other computing resource pools are all smaller than the required resource amounts, stop scheduling the queued jobs. 5. The method for scheduling computing jobs according to claim 4, wherein after the scheduling of the queued jobs is stopped, the method for scheduling computing jobs further comprises: Scheduling computing jobs subsequent to the queued jobs. 6. The scheduling method for computing jobs according to any one of claims 1 to 3, wherein the scheduling method for computing jobs further comprises: When the sum of the required resources of the computing jobs corresponding to a single user is greater than the upper limit of single user resources, stop scheduling the computing jobs of the single user; The amount of remaining resources in the resource pool is positively related. 7. The method for scheduling a computing job according to any one of claims 1 to 3, wherein, before acquiring the queuing cause of the queued job, the method for scheduling the computing job further comprises: According to the requirements of all computing jobs and the computing characteristics of each computing resource pool in the cluster, the computing jobs are allocated to each of the computing resource pools for matching. 8. The scheduling method for computing jobs according to any one of claims 1 to 3, wherein said acquisition of queuing reasons for queuing jobs includes: When the queuing job exists in the cluster, the queuing reason of the queuing job is acquired. 9. A scheduling device for computing jobs, comprising: The first obtaining module is used to obtain the queuing reason of the queuing job; wherein, the queuing job represents a computing job that is queuing for processing; The second obtaining module is used to obtain the remaining resources of other computing resource pools in the cluster when the queuing cause is that the remaining resources of the computing resource pool corresponding to the queued job are less than the required resource amount of the queued job; wherein , the cluster includes the computing resource pool corresponding to the queued job and the other computing resource pools; and A scheduling execution module, configured to allocate the queued job when the remaining resource amount of the other computing resource pool is greater than or equal to the required resource amount, and the other computing resource pool matches the queued job to the other computing resource pools for computing. 10. A computer-readable storage medium, the storage medium stores a computer program, and the computer program is used to execute the computing job scheduling method according to any one of claims 1-8.

Images