CN116149843A - Resource allocation method, network, storage medium and processor based on dynamic programming - Google Patents

Abstract

This application applies to the technical field of point cloud semantic segmentation, and provides a resource allocation method based on dynamic programming, a network, a storage medium, and a processor. The a priori data set is recorded by the experimental test, and the amount of resources provided in the computer is reasonably adjusted through the a priori data set, and the allocation of resources is dynamically adjusted as the load changes, and at the same time, it can improve resource utilization without causing The purpose of the problem of resource scarcity. It effectively avoids the disadvantage that the existing resource allocation method cannot balance stability and efficiency.

Description

Resource allocation method, network, storage medium and processor based on dynamic programming

technical field

The present application belongs to the field of information technology, and in particular relates to a resource allocation method based on dynamic programming, a network, a storage medium and a processor.

Background technique

Resource scheduling algorithm is a long-term research direction in the field of cloud computing. For different scenarios, many researchers have conducted in-depth exploration on the improvement of resource efficiency. The main problem to be solved by the resource scheduling algorithm is in a cluster composed of a large number of physical machines, which strategy is used to use the resources of these physical machines to ensure the highest efficiency of resource use. These strategies include but are not limited to how to select the appropriate machine for application deployment, how to dynamically adjust the resource allocation quota of the application when it is running, and how to adjust the application deployment location to ensure stability when the resources on a single physical machine are insufficient. These series of strategies ensure that the application can run stably (can use the minimum resource quota that guarantees its operation), and at the same time minimize the idle resources in the entire cluster. There are many types of resources, including but not limited to CPU, memory, network card, and cache.

Existing technologies require the application to provide two parameters when applying for an application, namely request and limit. request represents the minimum resources required for the application to run, and limit represents the maximum resources that the application needs to use. Existing technologies use these two parameters to allocate resources. The resource allocation algorithm based on the request parameter will judge whether the remaining amount of resources on the machine can meet the value of the requested request when the application applies for resources, and if it is satisfied, the application will be launched on the machine; resources based on the limit parameter When applying for resources, the allocation algorithm will judge whether the remaining amount of resources on the machine can still meet the value of the applied limit, and when applying for resources, it will judge whether the remaining amount of resources on the machine can still meet the value of the applied request.

Resource allocation algorithm based on request parameters When too many applications are deployed, the sum of the requests of all applications will be close to the sum of the resources of the whole machine, and the sum of the limits of all applications will be greater than the sum of resources of the whole machine. When some applications suddenly need to use the resources of the limit quota, it will cause insufficient resources on a single machine and cause machine failure.

Although the resource allocation algorithm based on the limit parameter will never cause machine failures due to insufficient resources, in most cases, the resources requested by the application (limit-request) are in a state of no use or a small amount of use , resulting in the problem of low utilization of machine resources. There are deficiencies in the prior art.

Contents of the invention

The purpose of this application is to provide a resource allocation method, network, storage medium and processor based on dynamic programming, to accumulate prior knowledge by conducting prior tests on applications to be deployed, and then determine their appropriate resource quotas under certain load conditions , in the case of limited single-machine resources as a whole, the resource utilization rate is improved compared with the previous method, and the application can provide stable service quality, and solve the technical problem of establishing a safe coexistence relationship between authorized users and cognitive users.

On the one hand, the present application provides a resource allocation method based on dynamic programming, including the following steps:

s1. The application to be deployed applies to the application container orchestration system for deployment;

s2. The application container orchestration system obtains the optimized quota resource value from the Hestia module through the scheduling module from the perspective of improving resource efficiency, and replaces the resource value set by the developer himself;

s3. The scheduling module selects a most suitable machine from the cluster according to the optimized quota resource value;

s4. The application container orchestration system notifies the selected machine to download a copy of the application to be deployed and allocate corresponding resources.

On the other hand, the present application also provides a dynamic programming-based resource allocation network using the above-mentioned resource allocation method, including a sequentially connected application container orchestration system, a scheduling module, a Hestia module, and a computer cluster;

The application container orchestration system accepts a deployment application of an application to be deployed;

The Hestia module obtains the value of quota resources optimized from the perspective of improving resource efficiency, instead of the resource value set by the developer itself;

The scheduling module selects a most suitable machine from the computer cluster according to the optimized quota resource value;

The application container orchestration system notifies the selected machine to download a copy of the application to be deployed and allocate corresponding resources.

On the other hand, the present application also provides a storage medium, wherein the storage medium stores program files capable of implementing the above resource allocation method based on dynamic programming.

On the other hand, the present application also provides a processor, the processor is configured to run a program, wherein the above-mentioned method for resource allocation based on dynamic programming is executed when the program is running.

This application proposes a resource allocation method based on dynamic programming, a network, a storage medium and a processor. When allocating resources, record a priori data set through some prior tests, adjust the quota of resources reasonably through the prior data set, and dynamically adjust the allocation of resources as the load changes, and at the same time achieve the goal of improving resource utilization. without causing a shortage of resources. It effectively avoids the disadvantage that the existing resource allocation method cannot balance stability and efficiency.

Description of drawings

FIG. 1 is a schematic diagram of the structure of the resource allocation method based on dynamic programming in the present application;

Fig. 2 is a schematic block diagram of the specific flow of prototype resources in this application;

FIG. 3 is a schematic diagram of main steps of the resource allocation method based on dynamic programming in this application.

Detailed ways

In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings 1-3 and embodiments. It should be understood that the specific embodiments described here are only used to explain the present application, and are not intended to limit the present application.

The specific implementation of the present application is described in detail below in conjunction with specific embodiments:

Embodiment one:

Figures 1-3 show the implementation process of the resource allocation method based on dynamic programming provided by the first embodiment of the present application. For the convenience of explanation, only the parts related to the embodiment of the present application are shown, and the details are as follows:

As shown in Figure 3, the present application provides a resource allocation method based on dynamic programming, including the following steps:

s1. The application to be deployed applies to the application container orchestration system for deployment;

s2. The application container orchestration system obtains the optimized quota resource value from the Hestia module through the scheduling module from the perspective of improving resource efficiency, and replaces the resource value set by the developer himself;

s3. The scheduling module selects a most suitable machine from the cluster according to the optimized quota resource value;

s4. The application container orchestration system notifies the selected machine to download a copy of the application to be deployed and allocate corresponding resources.

Specifically, in Figure 1, application is the application that needs to be deployed. Kubernetes (K8s) is an application container orchestration system on a large-scale computing (server) cluster. The main tasks are automated deployment, planning, operation and maintenance and To update the application, our application is deployed to a designated machine by applying to K8s for deployment and executing a set of automated processes. There are many modules for K8s connection. We mainly design the scheduling module here, that is, the Scheduler module in the figure. This module will select the most suitable one from the cluster according to the characteristics of the application to be deployed, the amount of resources requested, and other information. The machine is deployed. When the machine is selected, K8s will notify the machine to download a copy of the application, and then the machine allocates the resources it pre-applied to the application and starts the application. So far, the application is deployed successfully. up.

The main operating point of our algorithm is that when a single machine allocates resources, the original process is that the machine will allocate the resource quota notified by K8s to the application. This quota is set by the application developer himself. We use our algorithm to allocate This quota is further optimized, and the value of this quota is reset from the perspective of improving resource efficiency, so as to achieve the goal. That is, the Hestia module in Figure 1. This module will update the quota resource value in the scheduling module to replace the value set in the application by the developer of the application to be deployed (such as request and limit).

As shown in accompanying drawing 2, described step s2 comprises the following steps:

s21. The Hestia module judges the application to be deployed, and if it is a new application, conducts an advance test to test the service quality corresponding to different resource quotas of the application to be deployed under different loads; establish a priori data set and store it in the database ;

The main task of this step is to change the resource quota of a new type of application under different loads to obtain its service quality. Specifically, we will test each new application for deployment before deployment, and the load will be ranked from low to high, and the resources will also be measured from the first to the highest, and the service quality will be measured. We use the average access load To quantify service quality by delay, the lower the delay, the higher the service quality.

If it is not a new application, read the prior data set corresponding to the application to be deployed in the database;

s22. Execute a set of dynamic programming procedures based on the prior data set to determine the specific value of the allocated resource quota.

Further, the prior data set is expressed as:

f _S (R ₁ , R ₂ ,...,L) = QoS _S = T _requests ;

T _requests represents the prior data set, S represents the application, R represents the resource, and L represents the load.

This prior data set T _requests can obtain the quantitative data of the service quality that this application can bring to it under different load conditions and different resource configurations.

Further, the resources include cup resources and memory resources.

Further, the following dynamic programming transfer equation is adopted in the dynamic programming process in the step s22:

result(S _n ,CPU,mem,L)=min[result(S _n-1 ,CPU _total -CPU,mem _total -mem,L _total -

L)*(n-1)/n+f _S (CPU,mem,L)/n*f _S (CPU _total ,mem _total ,L)];

result(S ₁ ,CPU,mem,L)=f _S (CPU,mem,L)/f _S (CPU _total ,mem _total ,L);

Among them, n represents the number, S ₂ represents the second service, total represents the upper limit of machine resources, and CPU _total represents all CPU resources on the machine;

The meaning of the dynamic programming transfer equation is that the result of the nth service needs to be obtained through the result of the n-1th service and the f _s of the nth service. Result() represents the calculation result of the formula, and f _s represents the QoS value of this s service when the resource quota is R ₁ , R ₂ , ..., L, that is, the time taken to complete all requests.

The meaning of Result is the relative value of service quality; CPU means processor resource; mem means memory resource.

Specifically, here we define resources as CPU and memory (mem). Through these two formulas, we can obtain the optimal state of each application resource when the total amount of resources is limited. This state can guarantee all The service provides relatively optimal service quality while maximizing the resource utilization of the machine.

Further, the following steps are also included:

s5. In the case that the step s1 is not triggered, repeat the step s2 to the step s4 every 1 minute.

The quota of resources will be adjusted in real time according to the application load every minute, which can effectively improve the utilization efficiency of resources

Further, when the application to be deployed is deployed, the Hestia module will sequentially calculate the value of the resource quota of the deployed application and the application to be deployed in the machine downloading the application to be deployed through the dynamic programming transfer equation, and the application The container orchestration system will reallocate resources.

In order to ensure the quality of service, it will be called once every 1 minute, mainly to prevent the risk of resource shortage caused by a sudden increase in the load of some applications, or when a new application is deployed, it will perform a priori test and quota calculation at the same time, a priori test The results will be stored in the database, so that they can be used directly when the same application is deployed in the future.

Embodiment two:

On the other hand, the present application also provides a resource allocation network based on dynamic programming using the above-mentioned semantic segmentation method, including a sequentially connected application container orchestration system, a scheduling module, a Hestia module, and a computer cluster;

The application container orchestration system accepts a deployment application of an application to be deployed;

The Hestia module obtains the value of quota resources optimized from the perspective of improving resource efficiency, instead of the resource value set by the developer itself;

The scheduling module selects a most suitable machine from the computer cluster according to the optimized quota resource value;

The application container orchestration system notifies the selected machine to download a copy of the application to be deployed and allocate corresponding resources.

Specifically, based on a dynamic programming resource allocation algorithm, this algorithm can adjust the resource allocation of the entire stand-alone machine in real time according to the load, so as to achieve the purpose of improving resource utilization. Compared with the resource allocation algorithm based on request, the possibility of resource shortage is reduced, and compared with the resource allocation algorithm based on limit, the resource allocation efficiency of the whole machine is improved.

Embodiment three:

On the other hand, the present application also provides a storage medium, wherein the storage medium stores program files capable of implementing the above resource allocation method based on dynamic programming.

Those of ordinary skill in the art can understand that all or part of the steps in the method of the above-mentioned embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, and the storage Media such as ROM/RAM, magnetic disk, optical disk, etc.

Embodiment four:

On the other hand, the present application also provides a processor, the processor is configured to run a program, wherein the above-mentioned method for resource allocation based on dynamic programming is executed when the program is running.

In the embodiment of the present application, the resource allocation method based on dynamic programming can be implemented by corresponding hardware or software units, and each unit can be an independent software and hardware unit, or can be integrated into a software and hardware unit, which is not limited here this application. For the specific implementation manner of each unit, reference may be made to the description of Embodiment 1, and details are not repeated here.

The above descriptions are only preferred embodiments of the application, and are not intended to limit the application. Any modifications, equivalent replacements and improvements made within the spirit and principles of the application should be included in the protection of the application. within range.

Claims (10)

1. The resource allocation method based on dynamic programming is characterized by comprising the following steps:

s1. the application to be deployed applies for deployment to the application container orchestration system;

s2, the application container arrangement system acquires the value of the quota resource optimized from the viewpoint of improving the resource efficiency from the Herica module through the scheduling module, and replaces the resource value set by a developer in the application to be deployed;

s3. the scheduling module selects a most suitable machine from the cluster according to the optimized limit resource value;

s4. the application container orchestration system informs the selected machines to download copies of the application to be deployed and to allocate corresponding resources.

2. The resource allocation method according to claim 1, wherein said step s2 comprises the steps of:

s21, judging the application to be deployed by the Herica module, if the application is a new application, testing in advance, and testing service quality corresponding to different resource units of the application to be deployed under different degrees of load; establishing a priori data set and storing the priori data set into a database;

if the application is not a new application, reading a priori data set corresponding to the application to be deployed in a database;

s22. executing a set of dynamic programming procedures based on the prior data set to determine the optimized value of the quota resource.

3. The resource allocation method of claim 2, wherein the a priori data set is represented as:

f _S (R ₁ ,R ₂ ,...,L)＝QoS _S ＝T _requests ；

T _requests representing the a priori dataset, S representing the application, R representing the resource, and L representing the load.

4. The resource allocation method of claim 3, wherein the resources comprise a cup resource and a memory resource.

5. The method for allocating resources according to claim 4, wherein the following dynamic programming transfer equation is adopted in the dynamic programming flow in step s 22:

result(S _n ,CPU,mem,L)＝min[result(S _n-1 ,CPU _total -CPU,mem _total -mem,L _total -

L)*(n-1)/n+f _S (CPU,mem,L)/n*f _S (CPU _total ,mem _total ,L)]；

result(S ₁ ,CPU,mem,L)＝f _S (CPU,mem,L)/f _S (CPU _total ,mem _total ,L)；

wherein, result means the relative value of the service quality; the CPU is represented as a processor resource; mem represents a memory resource; n represents what number, S ₂ I.e. representing service 2, total represents the upper limit of machine resources, CPU _total Representing all of the CPU resources on the machine.

6. The resource allocation method according to claim 5, further comprising the step of:

s5. steps s2 to s4 are repeated every 1 minute without triggering said step s1.

7. The method of claim 6, wherein when an application to be deployed is deployed, the nestia module sequentially calculates a value of a resource limit of an application already deployed and an application to be deployed in a machine that downloads the application to be deployed through a dynamic programming transfer equation, and the application container arrangement system reallocates resources.

8. A dynamic programming-based resource distribution network employing the resource distribution method according to any of claims 1 to 7, comprising an application container orchestration system, a scheduling module, a nestia module and a computer cluster connected in sequence;

the application container arrangement system receives a deployment application of an application to be deployed;

the Hestin module acquires the value of the quota resource after optimization from the viewpoint of improving the resource efficiency, and replaces the resource value set by a developer;

the scheduling module selects a most suitable machine from the computer cluster according to the optimized limit resource value;

the application container orchestration system notifies the selected machines to download copies of the application to be deployed and to allocate corresponding resources.

9. A storage medium storing a program file capable of implementing the dynamic programming-based resource allocation method according to any one of claims 1 to 7.

10. A processor for running a program, wherein the program when run performs the dynamic programming based resource allocation method of any one of claims 1 to 7.

Images