CN114265676B - Cluster resource scheduling method, device, equipment and medium - Google Patents

Abstract

The invention provides a cluster resource scheduling method, a device, equipment and a medium, wherein the method comprises the following steps: acquiring resource scheduling tasks of each tenant in a preset time period; determining the scheduling priority corresponding to each resource scheduling task according to each resource scheduling task and the matched preset resource scheduling threshold range; determining cluster scheduling resources corresponding to each resource scheduling task in each scheduling priority according to each scheduling priority and a preset scheduling priority sequence; and scheduling the cluster scheduling resources corresponding to each resource scheduling task. The cluster resource scheduling method considers the scheduling priority of the resource scheduling task of each tenant in the multi-tenant scene, so that the resource scheduling can be performed by taking the scheduling priority as a reference, and the utilization efficiency of the whole cluster resource is improved.

Description

Cluster resource scheduling method, device, equipment and medium

Technical Field

The present invention relates to the field of data processing technology, and in particular to a cluster resource scheduling method, device, equipment and medium.

Background Art

The concept of tenants is not limited to cluster users, but can include a set of workloads consisting of computing, memory, storage and other resources. In a multi-tenant cluster, it is necessary to provide as much security isolation as possible for different tenants within a cluster to avoid resource contention between tenants to the greatest extent possible, while ensuring that shared cluster resources are fairly distributed among tenants.

However, the current cluster resource scheduling method, such as the resource scheduling method of the Kubernetes cluster, allocates resource scheduling tasks for each tenant in turn, only considering the optimal resource scheduling method under the current resource scheduling task, and does not consider the scenario of multiple resource scheduling tasks formed by multiple tenants. As a result, in the scenario of multiple resource scheduling tasks for multiple tenants, the overall cluster resource utilization efficiency is low.

Summary of the invention

The present invention provides a cluster resource scheduling method, device, equipment and medium, which are used to solve the problem that the overall cluster resource utilization efficiency is low in the current cluster resource scheduling method in the scenario of multi-tenant multi-resource scheduling tasks.

A first aspect of the present invention provides a cluster resource scheduling method, comprising:

Obtain resource scheduling tasks for each tenant within a preset time period;

Determine the scheduling priority corresponding to each resource scheduling task according to each resource scheduling task and the matching preset resource scheduling threshold range;

Determine the cluster scheduling resources corresponding to each resource scheduling task in each scheduling priority according to each scheduling priority and the preset scheduling priority sequence;

Schedule the cluster scheduling resources corresponding to each resource scheduling task.

Further, in the method as described above, the resource scheduling task includes resources to be scheduled requested by the tenant;

The determining the scheduling priority corresponding to each resource scheduling task according to each resource scheduling task and the matching preset resource scheduling threshold range includes:

Comparing each of the to-be-scheduled resources with a matching preset resource scheduling threshold range to generate a comparison result;

The scheduling priority corresponding to each resource scheduling task is determined according to each comparison result.

Furthermore, in the above method, determining the scheduling priority corresponding to each resource scheduling task according to each comparison result includes:

If the comparison result is that each of the resources to be scheduled is less than the minimum value of the matched preset resource scheduling threshold range, then determining that the scheduling priority of the resource scheduling task is high;

If the comparison result is that each of the resources to be scheduled is within the matching preset resource scheduling threshold range, then determining that the scheduling priority of the resource scheduling task is medium;

If the comparison result is that each of the resources to be scheduled is equal to the maximum value of the matched preset resource scheduling threshold range, then determining that the scheduling priority of the resource scheduling task is low;

If the comparison result is that each of the resources to be scheduled is greater than the maximum value of the matched preset resource scheduling threshold range, the resource scheduling task is determined as the resource scheduling task for the next preset time period.

Furthermore, in the above method, determining the cluster scheduling resources corresponding to each resource scheduling task in each scheduling priority according to each scheduling priority and a preset scheduling priority order includes:

In order from high scheduling priority to low scheduling priority, each resource scheduling task of each scheduling priority is processed as follows:

Determine whether there is a cluster node in the cluster that matches each resource scheduling task within the current scheduling priority; the matching cluster node includes multiple servers; the idle resources of the multiple servers are greater than the to-be-scheduled resources of the matching resource scheduling task;

If it is determined to exist, then determine the server with the lowest idle ratio among the cluster nodes that match each resource scheduling task;

The idle resources of the server with the lowest idle ratio are determined as cluster scheduling resources corresponding to each resource scheduling task.

Furthermore, in the above method, the step of determining the server with the lowest idle ratio among the cluster nodes matching each resource scheduling task includes:

Determining idle resources of the plurality of servers that match each resource scheduling task;

Divide the difference between the idle resources of each server and the resources to be scheduled of the corresponding resource scheduling task by the total resource limit of each server to generate the idle ratio corresponding to each server;

A server with the lowest idle ratio is determined according to the idle ratios corresponding to the servers.

Furthermore, in the above method, before determining the server with the lowest idle ratio among the cluster nodes matching each resource scheduling task, the method further includes:

Determine whether there are multiple cluster nodes matching each resource scheduling task within the current scheduling priority;

If it is determined that there are multiple matching cluster nodes, then for each matching cluster node, determine the server with the lowest idle ratio among the cluster nodes matching each resource scheduling task;

Determine the minimum value output by the preset cluster idle ratio algorithm according to the server with the lowest idle ratio in each cluster node and the preset cluster idle ratio algorithm;

The server with the lowest idle ratio matched by each resource scheduling task when the preset cluster idle ratio algorithm outputs the minimum value is determined as the target server;

The step of determining the idle resources of the server with the lowest idle ratio as the cluster scheduling resources corresponding to each resource scheduling task includes:

The idle resources of each target server are determined as the cluster scheduling resources corresponding to each resource scheduling task.

Furthermore, in the above method, if the current scheduling priority is high, and if there is no cluster node in the cluster that matches each resource scheduling task within the current scheduling priority, the method further includes:

Get the scheduled cluster scheduling resources corresponding to the medium or low scheduling priority in the cluster;

According to the preset scheduling strategy, the scheduled cluster scheduling resources corresponding to the medium or low scheduling priority are allocated to each resource scheduling task of the current scheduling priority; wherein the preset scheduling strategy is to make each of the scheduled cluster scheduling resources after resource scheduling greater than or equal to the minimum value of the corresponding preset resource scheduling threshold range.

A second aspect of the present invention provides a cluster resource scheduling device, comprising:

The acquisition module is used to obtain the resource scheduling tasks of each tenant within a preset time period;

A priority determination module, used to determine the scheduling priority corresponding to each resource scheduling task according to each resource scheduling task and a matching preset resource scheduling threshold range;

A resource determination module is used to determine the cluster scheduling resources corresponding to each resource scheduling task in each scheduling priority according to each scheduling priority and a preset scheduling priority sequence;

The resource scheduling module is used to schedule the cluster scheduling resources corresponding to each resource scheduling task.

Further, in the above-mentioned device, the resource scheduling task includes resources to be scheduled requested by the tenant;

The priority determination module is specifically used for:

Compare each of the to-be-scheduled resources with a matching preset resource scheduling threshold range to generate a comparison result; and determine a scheduling priority corresponding to each resource scheduling task according to each comparison result.

Furthermore, in the above-mentioned device, when determining the scheduling priority corresponding to each resource scheduling task according to each comparison result, the priority determination module is specifically used to:

If the comparison result is that each of the resources to be scheduled is less than the minimum value of the matching preset resource scheduling threshold range, then the scheduling priority of the resource scheduling task is determined to be high; if the comparison result is that each of the resources to be scheduled is within the matching preset resource scheduling threshold range, then the scheduling priority of the resource scheduling task is determined to be medium; if the comparison result is that each of the resources to be scheduled is equal to the maximum value of the matching preset resource scheduling threshold range, then the scheduling priority of the resource scheduling task is determined to be low; if the comparison result is that each of the resources to be scheduled is greater than the maximum value of the matching preset resource scheduling threshold range, then the resource scheduling task is determined as the resource scheduling task for the next preset time period.

Further, in the above device, the resource determination module is specifically used for:

In order from high scheduling priority to low scheduling priority, the following processing is performed for each resource scheduling task of each scheduling priority: determine whether there is a cluster node in the cluster that matches each resource scheduling task within the current scheduling priority; the matching cluster node includes multiple servers; the idle resources of the multiple servers are greater than the resources to be scheduled of the matching resource scheduling task; if it is determined that it exists, determine the server with the lowest idle ratio among the cluster nodes that match each resource scheduling task; determine the idle resources of the server with the lowest idle ratio as the cluster scheduling resources corresponding to each resource scheduling task.

Furthermore, in the above-mentioned device, when determining the server with the lowest idle ratio among the cluster nodes matching each resource scheduling task, the resource determination module is specifically used to:

Determine the idle resources of the multiple servers that match each resource scheduling task; divide the difference between the idle resources of each server and the resources to be scheduled of the corresponding resource scheduling task by the total resource limit of each server to generate the idle ratio corresponding to each server; determine the server with the lowest idle ratio based on the idle ratios corresponding to each server.

Further, in the above device, the resource determination module is also used for:

Determine whether there are multiple cluster nodes that match each resource scheduling task within the current scheduling priority; if it is determined that there are multiple matching cluster nodes, then for each matching cluster node, determine the server with the lowest idle ratio among the cluster nodes that match each resource scheduling task; determine the minimum value output by the preset cluster idle ratio algorithm based on the server with the lowest idle ratio among each cluster node and the preset cluster idle ratio algorithm; determine the server with the lowest idle ratio that matches each resource scheduling task corresponding to the minimum value output by the preset cluster idle ratio algorithm as the target server;

When the resource determination module determines the idle resources of the server with the lowest idle ratio as the cluster scheduling resources corresponding to each resource scheduling task, it is specifically used to:

The idle resources of each target server are determined as the cluster scheduling resources corresponding to each resource scheduling task.

Further, in the above-mentioned device, if the current scheduling priority is high, and if there is no cluster node in the cluster that matches each resource scheduling task within the current scheduling priority, the resource determination module is further used to:

Obtain the scheduled cluster scheduling resources corresponding to the medium or low scheduling priority in the cluster; allocate the scheduled cluster scheduling resources corresponding to the medium or low scheduling priority to each resource scheduling task of the current scheduling priority according to the preset scheduling strategy; wherein the preset scheduling strategy is to make each of the scheduled cluster scheduling resources after resource scheduling greater than or equal to the minimum value of the corresponding preset resource scheduling threshold range.

A third aspect of the present invention provides an electronic device, comprising: a memory, a processor;

Memory; Memory for storing instructions executable by the processor;

The processor is configured to execute the cluster resource scheduling method described in any one of the first aspects.

A fourth aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the cluster resource scheduling method described in any one of the first aspects.

A fifth aspect of the present invention provides a computer program product, including a computer program, which, when executed by a processor, implements the cluster resource scheduling method described in any one of the first aspects.

The present invention provides a cluster resource scheduling method, device, equipment and medium, the method comprising: obtaining resource scheduling tasks of each tenant within a preset time period; determining the scheduling priority corresponding to each resource scheduling task according to each resource scheduling task and the matching preset resource scheduling threshold range; determining the cluster scheduling resources corresponding to each resource scheduling task in each scheduling priority according to each scheduling priority and the preset scheduling priority order; scheduling the cluster scheduling resources corresponding to each resource scheduling task. The cluster resource scheduling method of the present invention obtains the resource scheduling tasks of each tenant within a preset time period, and determines the scheduling priority corresponding to each resource scheduling task according to each resource scheduling task and the matching preset resource scheduling threshold range, thereby determining the cluster scheduling resources corresponding to each resource scheduling task in each scheduling priority according to the preset scheduling priority order for each scheduling priority, so as to realize the scheduling of the cluster scheduling resources corresponding to each resource scheduling task. The cluster resource scheduling method of the present invention takes into account the scheduling priority of the resource scheduling task of each tenant in a multi-tenant scenario, so that resource scheduling can be performed based on the scheduling priority, thereby improving the utilization efficiency of the overall cluster resources.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

FIG1 is a scenario diagram of a cluster resource scheduling method that can implement an embodiment of the present invention;

FIG2 is a schematic diagram of a flow chart of a cluster resource scheduling method provided by a first embodiment of the present invention;

FIG3 is a schematic diagram of a flow chart of a cluster resource scheduling method provided by a second embodiment of the present invention;

FIG4a is a schematic diagram of a preset resource scheduling threshold range of tenant A in a cluster resource scheduling method provided in a second embodiment of the present invention;

FIG4b is a schematic diagram of a preset resource scheduling threshold range for tenant B in the cluster resource scheduling method provided in the second embodiment of the present invention;

FIG4c is a schematic diagram of a preset resource scheduling threshold range of tenant C in the cluster resource scheduling method provided in the second embodiment of the present invention;

FIG5 is a schematic diagram of scheduling priorities of a cluster resource scheduling method provided by a second embodiment of the present invention;

FIG6 is a schematic diagram of the structure of a cluster resource scheduling device provided by a third embodiment of the present invention;

FIG. 7 is a schematic diagram of the structure of an electronic device provided by a fourth embodiment of the present invention.

The above drawings have shown clear embodiments of the present invention, which will be described in more detail below. These drawings and text descriptions are not intended to limit the scope of the present invention in any way, but to illustrate the concept of the present invention for those skilled in the art by referring to specific embodiments.

DETAILED DESCRIPTION

Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are merely examples of devices and methods consistent with some aspects of the present invention as detailed in the appended claims.

The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present invention will be described below in conjunction with the accompanying drawings.

In order to clearly understand the technical solution of the present application, the solution of the prior art is first introduced in detail. With the gradual development of cloud data, renting cluster resources has become a common choice for companies or users. The concept of tenants is not limited to cluster users, it can include a set of workloads consisting of a group of computing, memory, storage and other resources. Since the time when tenants use cluster resources is independent of each other, scenarios where multiple tenants demand cluster resources at the same time often occur. At present, the resource scheduling of the cluster mainly provides local optimal choices for the resource scheduling tasks of a single tenant. Even if there are resource scheduling tasks for multiple tenants at the same time, the local optimal choice for each resource scheduling task will be provided one by one. However, this method may cause resource snatching by tenants due to the weak isolation of tenant resources. At the same time, the utilization efficiency of the overall cluster resources is low.

Therefore, in view of the problem that the overall cluster resource utilization efficiency is low in the current cluster resource scheduling method in the scenario of multi-tenant multi-resource scheduling tasks, the inventors have found in the research that in order to solve this problem, the uniqueness of each tenant's resource scheduling task can be comprehensively considered. For example, some resource scheduling tasks require more resources, while some resource scheduling tasks require fewer resources, etc., and the scheduling priorities of each resource scheduling task are divided according to the uniqueness of each tenant's resource scheduling task, so as to perform the overall optimal resource scheduling for each resource scheduling task according to the scheduling priority.

Specifically, firstly, the resource scheduling tasks of each tenant within a preset time period are obtained, and the scheduling priority corresponding to each resource scheduling task is determined according to each resource scheduling task and the matching preset resource scheduling threshold range, so as to determine the cluster scheduling resources corresponding to each resource scheduling task in each scheduling priority according to the preset scheduling priority order, so as to realize the scheduling of the cluster scheduling resources corresponding to each resource scheduling task. The cluster resource scheduling method of the present invention takes into account the scheduling priority of the resource scheduling task of each tenant in a multi-tenant scenario, so that resource scheduling can be performed based on the scheduling priority, thereby improving the overall resource utilization efficiency of the cluster.

Based on the above creative findings, the inventor proposed the technical solution of the present application.

The following is an introduction to the application scenario of the cluster resource scheduling method provided in an embodiment of the present invention. As shown in Figure 1, 1 is an electronic device, 2 is a tenant, and 3 is a cluster. The network architecture of the application scenario corresponding to the cluster resource scheduling method provided in an embodiment of the present invention includes: electronic device 1, tenant 2, and cluster 3. Tenant 2 can establish a network connection with electronic device 1 using the electronic devices used by each user, wherein tenant 2 includes multiple tenants such as tenant A, tenant B, and tenant N, and cluster 3 includes multiple cluster nodes such as cluster node A, cluster node B, and cluster node N.

When multiple tenants in tenant 2, such as tenant A, tenant B, and tenant N, send corresponding resource scheduling tasks A, B, and N to electronic device 1, electronic device 1 determines the scheduling priority corresponding to each resource scheduling task according to resource scheduling tasks A, B, and N and the preset resource scheduling threshold range matching resource scheduling tasks A, B, and N. Assuming that resource scheduling task A is less than the minimum value of the preset resource scheduling threshold range, the scheduling priority of resource scheduling task A can be determined as high, assuming that resource scheduling task B is within the preset resource scheduling threshold range, the scheduling priority of resource scheduling task B can be determined as medium, and assuming that resource scheduling task N is equal to the maximum value of the preset resource scheduling threshold range, the scheduling priority of resource scheduling task N can be determined as low. Then, electronic device 1 determines the cluster scheduling resources corresponding to each resource scheduling task in each scheduling priority according to the scheduling priorities of resource scheduling tasks A, B, and N and the preset scheduling priority order. Finally, electronic device 1 connects to cluster 3 to schedule the cluster scheduling resources corresponding to each resource scheduling task.

The embodiments of the present invention are described below in conjunction with the accompanying drawings.

FIG2 is a flow chart of a cluster resource scheduling method provided by the first embodiment of the present invention. As shown in FIG2, in this embodiment, the execution subject of the embodiment of the present invention is a cluster resource scheduling device, which can be integrated in an electronic device. The cluster resource scheduling method provided by this embodiment includes the following steps:

Step S101, obtaining resource scheduling tasks for each tenant within a preset time period.

In this embodiment, the preset time period can be set according to demand, such as ten minutes, one minute, etc., so that the multiple resource scheduling tasks to be processed are divided in the form of time periods.

The resource scheduling tasks of each tenant generally include data such as the tenant's resources to be scheduled and the tenant ID. The tenant's resources to be scheduled refer to the cluster resources that the tenant requests to schedule.

Cluster resources include memory, processor resources, disk resources, etc. of each server in the cluster node. When performing resource scheduling, the resources to be scheduled of the resource scheduling task can be rounded up.

Step S102: determining the scheduling priority corresponding to each resource scheduling task according to each resource scheduling task and a matching preset resource scheduling threshold range.

In this embodiment, a resource scheduling threshold range is pre-set for each tenant, the minimum value of the resource scheduling threshold range corresponds to the minimum amount of resources that can meet the tenant's normal operation, and the maximum value of the resource scheduling threshold range corresponds to the maximum amount of resources when the tenant operates in excess. Generally, the cluster resources required by the tenant are within the preset resource scheduling threshold range.

If the cluster resources that a resource scheduling task needs to schedule are less than the minimum value of the resource scheduling threshold range, the tenant cannot operate normally. Therefore, the scheduling priority of the resource scheduling task can be set to the highest level. Similarly, if the cluster resources that a resource scheduling task needs to schedule are within the resource scheduling threshold range, it means that the scheduled cluster resources can meet the tenant's normal operation and there is still room to be scheduled. The scheduling priority of the resource scheduling task can be set to the next level. If the cluster resources that a resource scheduling task needs to schedule are equal to the maximum value of the resource scheduling threshold range, it means that the scheduled cluster resources can meet the tenant's normal operation and there is still a large space to be scheduled. The scheduling priority of the resource scheduling task can be set to the next level. If the cluster resources that a resource scheduling task needs to schedule are greater than the maximum value of the resource scheduling threshold range, it means that the tenant is currently in an abnormal operation state. You can wait for the tenant to complete the execution of certain programs before scheduling cluster resources. Therefore, the scheduling priority of the resource scheduling task can be set to the lowest level.

Step S103: determining the cluster scheduling resources corresponding to each resource scheduling task in each scheduling priority according to each scheduling priority and a preset scheduling priority sequence.

In this embodiment, the preset scheduling priority order is preferably in the order from the highest gear to the lowest gear, and can also be set according to needs, which is not limited in this embodiment.

For each resource scheduling task in each scheduling priority, the cluster scheduling resource corresponding to each resource scheduling task may be determined in a random or preset order.

Step S104: Schedule the cluster scheduling resources corresponding to each resource scheduling task.

In this embodiment, after the cluster scheduling resources corresponding to each resource scheduling task are determined, the cluster scheduling resources corresponding to each resource scheduling task can be scheduled.

A cluster resource scheduling method provided by an embodiment of the present invention includes: obtaining resource scheduling tasks of each tenant within a preset time period. Determining the scheduling priority corresponding to each resource scheduling task according to each resource scheduling task and a matching preset resource scheduling threshold range. Determining the cluster scheduling resources corresponding to each resource scheduling task in each scheduling priority according to each scheduling priority and a preset scheduling priority sequence. Scheduling the cluster scheduling resources corresponding to each resource scheduling task.

The cluster resource scheduling method of the embodiment of the present invention obtains the resource scheduling tasks of each tenant within a preset time period, and determines the scheduling priority corresponding to each resource scheduling task according to each resource scheduling task and the matching preset resource scheduling threshold range, thereby determining the cluster scheduling resources corresponding to each resource scheduling task in each scheduling priority in accordance with the preset scheduling priority order for each scheduling priority, so as to schedule the cluster scheduling resources corresponding to each resource scheduling task. The cluster resource scheduling method of the embodiment of the present invention takes into account the scheduling priority of the resource scheduling task of each tenant in a multi-tenant scenario, so that resource scheduling can be performed based on the scheduling priority, thereby improving the overall resource utilization efficiency of the cluster.

FIG3 is a flow chart of a cluster resource scheduling method provided by a second embodiment of the present invention. As shown in FIG3, the cluster resource scheduling method provided by this embodiment is based on the cluster resource scheduling method provided by the previous embodiment of the present invention, and each step is further refined. The cluster resource scheduling method provided by this embodiment includes the following steps.

Step S201, obtaining resource scheduling tasks for each tenant within a preset time period.

In this embodiment, the implementation of step 201 is similar to the implementation of step 101 in the previous embodiment of the present invention, and will not be described in detail here.

Step S202: compare each resource to be scheduled with a matching preset resource scheduling threshold range to generate a comparison result, wherein the resource scheduling task includes the resource to be scheduled requested by the tenant.

In this embodiment, the comparison result may be less than the minimum value of the preset resource scheduling threshold range, within the preset resource scheduling threshold range, equal to the maximum value of the preset resource scheduling threshold range, or greater than the maximum value of the preset resource scheduling threshold range.

Step S203: determining the scheduling priority corresponding to each resource scheduling task according to each comparison result.

The scheduling priority corresponding to each resource scheduling task can be determined according to the comparison results:

If the comparison result is that each resource to be scheduled is smaller than the minimum value of the matched preset resource scheduling threshold range, it is determined that the scheduling priority of the resource scheduling task is high.

If the comparison result is that each resource to be scheduled is within the matching preset resource scheduling threshold range, then the scheduling priority of the resource scheduling task is determined to be medium.

If the comparison result is that each resource to be scheduled is equal to the maximum value of the matching preset resource scheduling threshold range, it is determined that the scheduling priority of the resource scheduling task is low.

If the comparison result is that each resource to be scheduled is greater than the maximum value of the matched preset resource scheduling threshold range, the resource scheduling task is determined as the resource scheduling task for the next preset time period.

A resource scheduling threshold range is pre-set for each tenant. The minimum value of the resource scheduling threshold range corresponds to the minimum amount of resources that can meet the tenant's normal operation, and the maximum value of the resource scheduling threshold range corresponds to the maximum amount of resources when the tenant operates in excess. Generally, the cluster resources required by the tenant are within the preset resource scheduling threshold range.

If the cluster resources that a resource scheduling task needs to schedule are less than the minimum value of the resource scheduling threshold range, it cannot meet the normal operation of the tenant, so the scheduling priority of the resource scheduling task can be set to high. Similarly, if the cluster resources that a resource scheduling task needs to schedule are within the resource scheduling threshold range, it means that the scheduled cluster resources can meet the normal operation of the tenant and there is still room for scheduling, so the scheduling priority of the resource scheduling task can be set to medium.

If the cluster resources that a resource scheduling task needs to schedule are equal to the maximum value of the resource scheduling threshold range, it means that the scheduled cluster resources can meet the normal operation of the tenant and there is still a large scheduling space. The scheduling priority of the resource scheduling task can be set to low.

If the cluster resources that need to be scheduled by a resource scheduling task are greater than the maximum value of the resource scheduling threshold range, it means that the tenant is currently in an abnormal operation state. You can wait until the tenant completes the execution of certain programs before performing cluster resource scheduling. Therefore, you can determine the resource scheduling task as the resource scheduling task for the next preset time period, so that it can be processed within the next preset time period, thereby improving the overall scheduling efficiency of all tenants.

Step S204: determining the cluster scheduling resources corresponding to each resource scheduling task in each scheduling priority according to each scheduling priority and a preset scheduling priority sequence.

In this embodiment, the implementation of step 204 is similar to the implementation of step 103 in the previous embodiment of the present invention, and will not be described in detail here.

Optionally, in this embodiment, determining the cluster scheduling resources corresponding to each resource scheduling task in each scheduling priority according to each scheduling priority and a preset scheduling priority order includes:

In order from high scheduling priority to low scheduling priority, each resource scheduling task of each scheduling priority is processed as follows:

It is determined whether there is a cluster node in the cluster that matches each resource scheduling task within the current scheduling priority. The matching cluster nodes include multiple servers. The idle resources of the multiple servers are greater than the to-be-scheduled resources of the matching resource scheduling task.

If it is determined that it exists, then the server with the lowest idle ratio among the cluster nodes that match each resource scheduling task is determined.

The idle resources of the server with the lowest idle ratio are determined as the cluster scheduling resources corresponding to each resource scheduling task.

In this embodiment, resource scheduling tasks with high scheduling priorities are processed first in descending order of scheduling priority. At the same time, for each resource scheduling task in each scheduling priority, it is necessary to determine the server with the lowest idle ratio in the matching cluster node. The lowest idle ratio means that after the server provides resources to tenants, the proportion of the server's idle resources to the server's total resources is the smallest among all servers in the same cluster node. The purpose of this selection is to enable the scheduled server to be in a fully loaded operating state as much as possible after resource scheduling, thereby improving the resource utilization of the server.

Optionally, in this embodiment, determining the server with the lowest idle ratio among the cluster nodes matching each resource scheduling task includes:

Idle resources of multiple servers matching each resource scheduling task are determined.

The difference between the idle resources of each server and the resources to be scheduled of the corresponding resource scheduling task is divided by the total resource limit of each server to generate the idle ratio corresponding to each server.

A server with the lowest idle ratio is determined according to the idle ratios corresponding to the servers.

In this embodiment, the lowest idle ratio means that the scheduled server can be in the most fully loaded running state possible. Therefore, it is necessary to consider the difference between the idle resources of each server and the resources to be scheduled of the corresponding resource scheduling task divided by the total resource limit of each server.

Server resources can include disk, memory, and processor resources, so all resources are taken into consideration comprehensively.

Optionally, in this embodiment, before determining the server with the lowest idle ratio among the cluster nodes matching each resource scheduling task, the method further includes:

Determine whether there are multiple cluster nodes matching each resource scheduling task within the current scheduling priority.

If it is determined that there are multiple matching cluster nodes, then for each matching cluster node, a server with the lowest idle ratio among the cluster nodes matching each resource scheduling task is determined.

The minimum value output by the preset cluster idle ratio algorithm is determined according to the server with the lowest idle ratio in each cluster node and the preset cluster idle ratio algorithm.

The server with the lowest idle ratio matched by each resource scheduling task corresponding to the minimum value output by the preset cluster idle ratio algorithm is determined as the target server.

The idle resources of the server with the lowest idle ratio are determined as the cluster scheduling resources corresponding to each resource scheduling task, including:

The idle resources of each target server are determined as the cluster scheduling resources corresponding to each resource scheduling task.

In this embodiment, since there are multiple resource scheduling tasks in each scheduling priority, when there are multiple cluster nodes matching the resource scheduling tasks, there will also be multiple selectable solutions for the number of resource scheduling tasks. In order to trigger the whole and make the overall resource scheduling effect better, it is necessary to further screen each selectable solution.

Assuming that there are 10 resource scheduling tasks and each resource scheduling task matches 3 cluster nodes, there may be 3 to the power of 10 resource scheduling solutions. The best one needs to be selected from these solutions as the final resource scheduling solution. In general, the server with the lowest idle ratio among the cluster nodes matched by each resource scheduling task will be selected as the final solution.

Optionally, in this embodiment, if the current scheduling priority is high, and if there is no cluster node in the cluster that matches each resource scheduling task within the current scheduling priority, the method of this embodiment further includes:

Get the scheduled cluster resources corresponding to the medium or low scheduling priority in the cluster.

According to the preset scheduling strategy, the scheduled cluster scheduling resources corresponding to the medium or low scheduling priority are allocated to each resource scheduling task of the current scheduling priority. The preset scheduling strategy is to make each scheduled cluster scheduling resource after resource scheduling greater than or equal to the minimum value of the corresponding preset resource scheduling threshold range.

In this embodiment, in order to ensure resource scheduling requirements with high scheduling priority, when the idle resources of the servers of each cluster node cannot meet the resource scheduling requirements with high scheduling priority, resources can be obtained from the historical resource scheduling tasks of the scheduled cluster scheduling resources.

For example, in the previous time period, a resource scheduling task with a medium or low scheduling priority of cluster resources was scheduled, and is currently occupying server resources in the cluster. At this time, the server resources occupied by it can be obtained, and while ensuring that the resource scheduling task and the corresponding tenant can meet normal operation, the scheduled cluster scheduling resources can be allocated to each resource scheduling task with a high scheduling priority, thereby ensuring the current high scheduling priority resource scheduling needs.

Specifically, the resources to be scheduled to the current high scheduling priority can be determined based on the number of resources scheduled by the scheduled cluster. If the resources scheduled by the scheduled cluster can meet the normal operation of the corresponding tenants after re-scheduling, more resources can be scheduled to the current high scheduling priority resource scheduling tasks as much as possible.

Step S205: Schedule the cluster scheduling resources corresponding to each resource scheduling task.

In this embodiment, the implementation of step 205 is similar to the implementation of step 104 in the previous embodiment of the present invention, and will not be described in detail here.

In order to better understand the cluster resource scheduling method of this embodiment, a Kubernetes cluster is taken as an example for detailed description.

Kubernetes provides a default scheduler, which is generally used by all systems to manage service clusters. The scheduler schedules cluster resources for each tenant's resource scheduling task one by one, thus achieving local optimality, but ignoring the overall situation.

In this embodiment, the cluster resource scheduling method of this embodiment can be applied while retaining the original scheduling method of Kubernetes. For each tenant, a corresponding resource scheduling threshold range is pre-set, and the two ends of the resource scheduling threshold range are the minimum resource value that needs to be guaranteed and the maximum resource value that can be used by the tenant. When the minimum resource value of a tenant is not met, it will have priority over other tenants of the same level to obtain resources. The minimum resource value is the lower limit of the tenant's reserved resources, and the resources applied by the tenant cannot exceed the maximum resource value.

As shown in Figures 4a, 4b, and 4c, tenants can be divided into multiple situations. For example, the overall resource demand of tenant B is relatively small, while the overall resource demand of tenant A is relatively large. The minimum resource value required by tenant A is greater than the minimum resource value of tenant B, and the maximum resource value of tenant A is also greater than the maximum resource value of tenant B. The minimum resource value of tenant C is equal to the maximum resource value, which means that the resource demand of this tenant is very rigid and it is not allowed to share resources with other tenants.

As shown in FIG5 , resource scheduling tasks can be divided into multiple queues according to the scheduling priority of the resource scheduling tasks, and resource scheduling tasks of different scheduling priorities are assigned to different queues. Assuming that the scheduling priorities are divided into 1-3, multiple queues can be divided according to the scheduling priorities, and scheduling priority 1 includes resource scheduling tasks 1-1, resource scheduling tasks 1-2, resource scheduling tasks 1-3, etc. Similarly, scheduling priority 2 includes resource scheduling tasks 2-1, resource scheduling tasks 2-2, resource scheduling tasks 2-3, resource scheduling tasks 2-4, etc., and scheduling priority 3 includes resource scheduling tasks 3-1, resource scheduling tasks 3-2, and resource scheduling tasks 3-3.

Scheduling priority 1 is the highest priority and supports preemption of other resource scheduling tasks. Resource scheduling tasks with priority 2 support allocation failure, but do not support preemption. Resource scheduling tasks with priority 3 support allocation failure and also support preemption.

When performing resource scheduling for tenants, it can be determined whether to use the default scheduling method of Kubernetes or the cluster resource scheduling method of this embodiment to be compatible with both scheduling methods.

After determining the scheduling priorities corresponding to the resource scheduling tasks corresponding to each tenant, the resource scheduling tasks with a scheduling priority of 1 are processed first, and then the resource scheduling tasks with priorities 2 and 3 are processed one by one.

For each resource scheduling task with a priority of 1, each resource scheduling task can be assigned one by one according to a preset order or a random order (and multiple random orders). For example, it is determined whether there are multiple servers in the cluster node whose idle resources are greater than the resources to be scheduled for the matching resource scheduling task. It is determined whether the server is operating normally, and whether the resource scheduling task has requirements for the server model, etc.

Then, the server with the smallest idle ratio is selected from the multiple matching servers.

In order to improve the accuracy and efficiency of resource scheduling, the process of selecting the server with the smallest idle ratio for each resource scheduling task of priority 1 can be repeated multiple times, with different orders and different matching cluster nodes used each time.

If the resources cannot be satisfied, determine whether to preempt the resources with a historically low scheduling priority that are currently occupying the server resources. When preempting, care needs to be taken to reserve the minimum resources for the preempted tenant.

Then, priority 2 and priority 3 are scheduled in the same way, but no preemption-related calculations are performed during the execution process, thereby completing the scheduling of all resource scheduling tasks.

FIG6 is a schematic diagram of the structure of a cluster resource scheduling device provided in a third embodiment of the present invention. As shown in FIG6 , in this embodiment, the cluster resource scheduling device 300 includes:

The acquisition module 301 is used to acquire the resource scheduling tasks of each tenant within a preset time period.

The priority determination module 302 is used to determine the scheduling priority corresponding to each resource scheduling task according to each resource scheduling task and the matching preset resource scheduling threshold range.

The resource determination module 303 is used to determine the cluster scheduling resources corresponding to each resource scheduling task in each scheduling priority according to each scheduling priority and a preset scheduling priority sequence.

The resource scheduling module 304 is used to schedule the cluster scheduling resources corresponding to each resource scheduling task.

The cluster resource scheduling device provided in this embodiment can execute the technical solution of the method embodiment shown in FIG. 2 . Its implementation principle and technical effects are similar to those of the method embodiment shown in FIG. 2 , and will not be described in detail here.

Meanwhile, the cluster resource scheduling device provided by the present invention further refines the cluster resource scheduling device 300 based on the cluster resource scheduling device provided by the previous embodiment.

Optionally, in this embodiment, the resource scheduling task includes resources to be scheduled requested by the tenant.

The priority determination module 302 is specifically used for:

Each resource to be scheduled is compared with a matching preset resource scheduling threshold range to generate a comparison result, and a scheduling priority corresponding to each resource scheduling task is determined according to each comparison result.

Optionally, in this embodiment, when determining the scheduling priority corresponding to each resource scheduling task according to each comparison result, the priority determination module 302 is specifically used to:

If the comparison result is that each resource to be scheduled is less than the minimum value of the matching preset resource scheduling threshold range, the scheduling priority of the resource scheduling task is determined to be high. If the comparison result is that each resource to be scheduled is within the matching preset resource scheduling threshold range, the scheduling priority of the resource scheduling task is determined to be medium. If the comparison result is that each resource to be scheduled is equal to the maximum value of the matching preset resource scheduling threshold range, the scheduling priority of the resource scheduling task is determined to be low. If the comparison result is that each resource to be scheduled is greater than the maximum value of the matching preset resource scheduling threshold range, the resource scheduling task is determined as the resource scheduling task for the next preset time period.

Optionally, in this embodiment, the resource determination module 303 is specifically used to:

In order from high scheduling priority to low scheduling priority, the following processing is performed for each resource scheduling task of each scheduling priority: Determine whether there is a cluster node in the cluster that matches each resource scheduling task within the current scheduling priority. The matching cluster nodes include multiple servers. The idle resources of multiple servers are greater than the resources to be scheduled of the matching resource scheduling task. If it is determined that there is, determine the server with the lowest idle ratio among the cluster nodes matched by each resource scheduling task. Determine the idle resources of the server with the lowest idle ratio as the cluster scheduling resources corresponding to each resource scheduling task.

Optionally, in this embodiment, when determining the server with the lowest idle ratio among the cluster nodes matching each resource scheduling task, the resource determination module 303 is specifically configured to:

Determine the idle resources of multiple servers that match each resource scheduling task. Divide the difference between the idle resources of each server and the resources to be scheduled of the corresponding resource scheduling task by the total resource limit of each server to generate the idle ratio corresponding to each server. Determine the server with the lowest idle ratio based on the idle ratios corresponding to each server.

Optionally, in this embodiment, the resource determination module 303 is further used to:

Determine whether there are multiple cluster nodes that match each resource scheduling task within the current scheduling priority. If it is determined that there are multiple matching cluster nodes, then for each matching cluster node, determine the server with the lowest idle ratio among the cluster nodes that match each resource scheduling task. Determine the minimum value output by the preset cluster idle ratio algorithm based on the server with the lowest idle ratio among each cluster node and the preset cluster idle ratio algorithm. Determine the server with the lowest idle ratio that matches each resource scheduling task when the preset cluster idle ratio algorithm outputs the minimum value as the target server.

When determining the idle resources of the server with the lowest idle ratio as the cluster scheduling resources corresponding to each resource scheduling task, the resource determination module 303 is specifically used to:

The idle resources of each target server are determined as the cluster scheduling resources corresponding to each resource scheduling task.

Optionally, in this embodiment, if the current scheduling priority is high, and if there is no cluster node in the cluster that matches each resource scheduling task within the current scheduling priority, the resource determination module 303 is further used to:

Obtain the scheduled cluster scheduling resources corresponding to the medium or low scheduling priority in the cluster. Allocate the scheduled cluster scheduling resources corresponding to the medium or low scheduling priority to each resource scheduling task of the current scheduling priority according to the preset scheduling strategy. Among them, the preset scheduling strategy is to make each scheduled cluster scheduling resource after resource scheduling greater than or equal to the minimum value of the corresponding preset resource scheduling threshold range.

The cluster resource scheduling device provided in this embodiment can execute the technical solutions of the method embodiments shown in Figures 2 to 5. Its implementation principle and technical effects are similar to those of the method embodiments shown in Figures 2 to 5, and will not be described in detail here.

According to an embodiment of the present invention, the present invention also provides an electronic device, a computer-readable storage medium and a computer program product.

As shown in Figure 7, Figure 7 is a schematic diagram of the structure of an electronic device provided by a fourth embodiment of the present invention. The electronic device is intended to be a digital computer in various forms, such as a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and/or required herein.

As shown in Fig. 7, the electronic device includes: a processor 401 and a memory 402. The various components are connected to each other using different buses and can be installed on a common mainboard or in other ways as required. The processor can process instructions executed in the electronic device.

The memory 402 is a non-transient computer-readable storage medium provided by the present invention. The memory stores instructions executable by at least one processor to enable at least one processor to execute the cluster resource scheduling method provided by the present invention. The non-transient computer-readable storage medium of the present invention stores computer instructions, which are used to enable a computer to execute the cluster resource scheduling method provided by the present invention.

The memory 402, as a non-transient computer-readable storage medium, can be used to store non-transient software programs, non-transient computer executable programs and modules, such as program instructions/modules corresponding to the cluster resource scheduling method in the embodiment of the present invention (for example, the acquisition module 301, the priority determination module 302, the resource determination module 303 and the resource scheduling module 304 shown in FIG. 6). The processor 401 executes various functional applications and data processing of the server by running the non-transient software programs, instructions and modules stored in the memory 402, that is, the cluster resource scheduling method in the above method embodiment is implemented.

At the same time, this embodiment also provides a computer product. When instructions in the computer product are executed by a processor of an electronic device, the electronic device can execute the cluster resource scheduling methods of the above-mentioned embodiments 1 to 2.

Those skilled in the art will readily appreciate other implementations of the embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses, or adaptations of the embodiments of the present invention that follow the general principles of the embodiments of the present invention and include common knowledge or customary techniques in the art that are not disclosed in the embodiments of the present invention. The specification and examples are to be considered merely as exemplary, and the true scope and spirit of the embodiments of the present invention are indicated by the following claims.

It should be understood that the embodiments of the present invention are not limited to the precise structures described above and shown in the drawings, and various modifications and changes may be made without departing from the scope thereof. The scope of the embodiments of the present invention is limited only by the appended claims.

Claims (8)

1. The cluster resource scheduling method is characterized by comprising the following steps of:

Acquiring resource scheduling tasks of each tenant in a preset time period;

determining the scheduling priority corresponding to each resource scheduling task according to each resource scheduling task and the matched preset resource scheduling threshold range;

Determining cluster scheduling resources corresponding to each resource scheduling task in each scheduling priority according to each scheduling priority and a preset scheduling priority sequence;

scheduling cluster scheduling resources corresponding to each resource scheduling task;

The determining cluster scheduling resources corresponding to each resource scheduling task in each scheduling priority according to each scheduling priority and a preset scheduling priority sequence comprises the following steps:

The following processing is performed for each resource scheduling task of each scheduling priority according to the order from the higher scheduling priority to the lower scheduling priority:

Judging whether cluster nodes matched with each resource scheduling task in the current scheduling priority exist in the clusters or not; the matched cluster node comprises a plurality of servers; the idle resources of the servers are larger than the resources to be scheduled of the matched resource scheduling task;

if yes, determining a server with the lowest idle rate in cluster nodes matched with each resource scheduling task;

determining the idle resources of the server with the lowest idle rate as cluster scheduling resources corresponding to each resource scheduling task;

Before determining the server with the lowest idle rate in the cluster node matched with each resource scheduling task, the method further comprises the following steps:

Judging whether a plurality of cluster nodes matched with each resource scheduling task in the current scheduling priority are provided;

If the matched cluster nodes are determined to be a plurality of, determining a server with the lowest idle rate in the cluster nodes matched with each resource scheduling task aiming at each matched cluster node;

determining a preset cluster idle rate algorithm output minimum value according to a server with the lowest idle rate in each cluster node and the preset cluster idle rate algorithm;

Determining a server with the lowest idle rate, which is matched with each corresponding resource scheduling task when a preset cluster idle rate algorithm outputs the minimum value, as a target server;

the determining the idle resources of the server with the lowest idle rate as cluster scheduling resources corresponding to each resource scheduling task comprises the following steps:

and determining the idle resources of each target server as cluster scheduling resources corresponding to each resource scheduling task.

2. The method of claim 1, wherein the resource scheduling task comprises a tenant requested resource to be scheduled;

determining the scheduling priority corresponding to each resource scheduling task according to each resource scheduling task and the matched preset resource scheduling threshold range, including:

Comparing each resource to be scheduled with a matched preset resource scheduling threshold range to generate a comparison result;

and determining the scheduling priority corresponding to each resource scheduling task according to each comparison result.

3. The method according to claim 2, wherein determining the scheduling priority corresponding to each resource scheduling task according to each comparison result includes:

If the comparison result is that each resource to be scheduled is smaller than the minimum value of the matched preset resource scheduling threshold range, determining that the scheduling priority of the resource scheduling task is high;

If the comparison result shows that each resource to be scheduled is in the matched preset resource scheduling threshold range, determining the scheduling priority of the resource scheduling task as the middle;

If the comparison result is that each resource to be scheduled is equal to the maximum value of the matched preset resource scheduling threshold range, determining that the scheduling priority of the resource scheduling task is low;

And if the comparison result is that each resource to be scheduled is greater than the maximum value of the matched preset resource scheduling threshold range, determining the resource scheduling task as the resource scheduling task in the next preset time period.

4. A method according to claim 3, wherein determining the server with the lowest idle rate in the cluster node to which each resource scheduling task matches comprises:

determining idle resources of the plurality of servers matched with each resource scheduling task;

dividing the difference between the idle resources of each server and the resources to be scheduled of the corresponding resource scheduling task by the total limit of the resources of each server to generate the idle ratio corresponding to each server;

And determining the server with the lowest idle rate according to the idle rate corresponding to each server.

5. The method of any of claims 1-4, wherein if the current scheduling priority is high and if there are no cluster nodes in the cluster that match resource scheduling tasks within the current scheduling priority, the method further comprises:

the method comprises the steps of obtaining scheduled cluster scheduling resources with medium or low scheduling priorities in clusters;

According to a preset scheduling strategy, scheduling resources of a scheduled cluster with a scheduling priority being middle or low are allocated to each resource scheduling task with the current scheduling priority; the preset scheduling policy is that each scheduled cluster scheduling resource after resource scheduling is larger than or equal to a minimum value corresponding to a preset resource scheduling threshold range.

6. A cluster resource scheduling apparatus, comprising:

the acquisition module is used for acquiring resource scheduling tasks of each tenant in a preset time period;

The priority determining module is used for determining the scheduling priority corresponding to each resource scheduling task according to each resource scheduling task and the matched preset resource scheduling threshold range;

The resource determining module is used for determining cluster scheduling resources corresponding to each resource scheduling task in each scheduling priority according to each scheduling priority and a preset scheduling priority sequence;

The resource scheduling module is used for scheduling cluster scheduling resources corresponding to each resource scheduling task;

the resource determining module is specifically configured to:

The following processing is performed for each resource scheduling task of each scheduling priority according to the order from the higher scheduling priority to the lower scheduling priority:

Judging whether cluster nodes matched with each resource scheduling task in the current scheduling priority exist in the clusters or not; the matched cluster node comprises a plurality of servers; the idle resources of the servers are larger than the resources to be scheduled of the matched resource scheduling task;

if yes, determining a server with the lowest idle rate in cluster nodes matched with each resource scheduling task;

determining the idle resources of the server with the lowest idle rate as cluster scheduling resources corresponding to each resource scheduling task;

before determining the server with the lowest idle rate in the cluster nodes matched with each resource scheduling task, the resource determining module is further configured to:

Judging whether a plurality of cluster nodes matched with each resource scheduling task in the current scheduling priority are provided;

If the matched cluster nodes are determined to be a plurality of, determining a server with the lowest idle rate in the cluster nodes matched with each resource scheduling task aiming at each matched cluster node;

determining a preset cluster idle rate algorithm output minimum value according to a server with the lowest idle rate in each cluster node and the preset cluster idle rate algorithm;

Determining a server with the lowest idle rate, which is matched with each corresponding resource scheduling task when a preset cluster idle rate algorithm outputs the minimum value, as a target server;

The resource determining module is specifically configured to, when determining the idle resource of the server with the lowest idle rate as a cluster scheduling resource corresponding to each resource scheduling task:

and determining the idle resources of each target server as cluster scheduling resources corresponding to each resource scheduling task.

7. An electronic device, comprising: a processor, and a memory communicatively coupled to the processor;

The memory stores computer-executable instructions;

The processor executes computer-executable instructions stored by the memory to implement the cluster resource scheduling method of any one of claims 1 to 5.

8. A computer readable storage medium, wherein computer executable instructions are stored in the computer readable storage medium, which when executed by a processor is configured to implement the cluster resource scheduling method of any one of claims 1 to 5.