CN119166365B - Dynamic allocation method and device for multi-core CPU (Central processing Unit) resources based on SPDK architecture - Google Patents
Dynamic allocation method and device for multi-core CPU (Central processing Unit) resources based on SPDK architecture Download PDFInfo
- Publication number
- CN119166365B CN119166365B CN202411650909.1A CN202411650909A CN119166365B CN 119166365 B CN119166365 B CN 119166365B CN 202411650909 A CN202411650909 A CN 202411650909A CN 119166365 B CN119166365 B CN 119166365B
- Authority
- CN
- China
- Prior art keywords
- cpu
- processing
- target
- processing area
- subsystem
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
- G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Multi Processors (AREA)
Abstract
The embodiment of the specification discloses a dynamic allocation method and device for multi-core CPU resources based on SPDK architecture. The method comprises the steps of dividing CPU resources into a plurality of processing areas, searching a first CPU in a target processing area corresponding to a target polling thread when detecting CPU use application of any target polling thread, determining a second CPU with the minimum current reference count when the first CPU does not exist, and binding the second CPU with the target polling thread. According to the embodiment of the specification, all subsystems of SPDK architecture can be isolated, resource competition among all subsystems is avoided, unused CPUs or CPUs with the smallest current reference count are selected to bind when a target polling thread applies for using the CPUs, dynamic management capacity of CPU resources is improved, effective utilization of the CPU resources can be guaranteed under the scene that the polling thread frequently creates and deletes the CPU resources, and resource utilization rate is improved.
Description
Technical Field
One or more embodiments of the present disclosure relate to information processing technologies, and in particular, to a method and apparatus for dynamically allocating multi-core CPU resources based on SPDK architecture.
Background
Storage performance development kits (Storage Performance Development Kit, SPDK) are widely used to improve the performance of storage and web applications by running in user space to bypass kernels, reducing overhead on the data path. SPDK consists of multiple subsystems, each of which handles different types of transactions by means of polling threads. However, there are still difficulties in efficiently managing and dynamically allocating CPU resources to optimize performance of SPDK architectures. In a traditional operating system, allocation and management of a CPU are generally controlled by a kernel scheduler, the kernel scheduler cannot schedule resources inside a process, only can allocate resources to SPDK processes in their entirety, the requirements of SPDK architecture for fine management of CPU resources cannot be fully met, and SPDK architecture only provides CPU resources used by using CPU masks to set up when running on a multi-core processor, lacks effective CPU isolation and load balancing strategies, and cannot perform more accurate resource management control to reduce delay and improve throughput.
Disclosure of Invention
In order to solve the above problems, one or more embodiments of the present disclosure describe a method and apparatus for dynamically allocating multi-core CPU resources based on SPDK architecture.
According to a first aspect, there is provided a method for dynamically allocating multi-core CPU resources based on SPDK architecture, the method comprising:
Dividing CPU resources into a plurality of processing areas based on subsystem types of SPDK architecture, wherein each processing area is used for processing polling threads of a subsystem of a specified type, and the CPU resources configured by each processing area are determined based on subsystem pressure;
When detecting a CPU use application of any target polling thread, searching a first CPU which is not used currently in a target processing area corresponding to the target polling thread;
Binding any one of the first CPUs with the target polling thread when the first CPU exists;
And when the first CPU does not exist, determining a second CPU with the minimum current reference count in the target processing area, and binding the second CPU with the target polling thread, wherein the current reference count is the number of the CPU which is simultaneously reference-bound.
Preferably, the subsystem class based on SPDK architecture divides CPU resources into a plurality of processing areas, including:
Querying requirement information corresponding to each subsystem type of the SPDK framework, wherein the requirement information comprises processing speed requirements and resource quantity requirements;
and dividing a processing area for each subsystem type, and distributing CPU resources to each processing area based on the requirement information.
Preferably, the allocating CPU resources to each of the processing areas based on the requirement information includes:
Determining a CPU allocation number ratio of each processing area based on the resource number requirement, and determining the CPU allocation number of each processing area according to the total core number of the CPU;
Performing first sequencing on the CPUs from high to low according to the processing speed, and performing second sequencing on the processing areas from high to low according to the processing speed requirement;
And sequentially selecting the processing areas according to the second sequence, selecting the CPU with the CPU distribution number according to the first sequence, and distributing the CPU to the processing areas.
Preferably, after the subsystem class based on SPDK architecture divides the CPU resources into a plurality of processing areas, the method further includes:
initializing CPU resources in the processing areas, clearing the current reference count of each CPU, and marking each CPU as a current unused state.
Preferably, the searching the first CPU that is not currently used in the target processing area corresponding to the target polling thread includes:
Determining the subsystem type to which the target polling thread belongs, determining a target processing area according to the subsystem type, and inquiring a first CPU which is not used currently in the target processing area.
Preferably, the method further comprises:
For any CPU, when the CPU is newly bound with a polling thread, the current reference count of the CPU is increased by one.
Preferably, the method further comprises:
For any CPU, when the CPU detects a CPU release instruction of any polling thread, the CPU is unbinding with the polling thread, and the current reference count of the CPU is reduced by one.
According to a second aspect, there is provided a dynamic allocation apparatus for multi-core CPU resources based on SPDK architecture, the apparatus comprising:
The dividing module is used for dividing CPU resources into a plurality of processing areas based on subsystem types of SPDK architecture, each processing area is used for processing polling threads of a subsystem of a specified type, and the CPU resources configured by each processing area are determined based on subsystem pressure;
the searching module is used for searching a first CPU which is not used currently in a target processing area corresponding to the target polling thread when detecting a CPU use application of any target polling thread;
The first judging module is used for binding any one of the first CPUs with the target polling thread when the first CPU exists;
And the second judging module is used for determining a second CPU with the least current reference count in the target processing area when the first CPU does not exist, binding the second CPU with the target polling thread, and the current reference count is the number of the CPU which are simultaneously reference-bound.
According to a third aspect, there is provided an electronic device comprising a processor and a memory;
the processor is connected with the memory;
The memory is used for storing executable program codes;
The processor runs a program corresponding to executable program code stored in the memory by reading the executable program code for performing the steps of the method as provided in the first aspect or any one of the possible implementations of the first aspect.
According to a fourth aspect, there is provided a computer readable storage medium having stored thereon a computer program having instructions stored therein which, when run on a computer or processor, cause the computer or processor to perform a method as provided by any one of the possible implementations of the first aspect or the first aspect.
According to the method and the device provided by the embodiment of the specification, the CPU resources among the subsystems are isolated by dividing the processing areas, the system stability is improved, the number of the CPUs in each processing area is configured according to the pressure of the subsystems by a load balancing strategy, the allocation of the CPU resources is accurately controlled, the resource competition among the subsystems is avoided, the performance of the CPUs is fully exerted, the system delay is reduced, and the throughput of the system is improved. In addition, when the target polling thread applies for using the CPU, the unused CPU or the CPU with the smallest current reference count is selected for binding, so that the dynamic management capability of CPU resources is improved, the effective utilization of the CPU resources can be ensured under the scene that the polling thread frequently creates and deletes, and the resource utilization rate is improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required to be used in the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present application, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of a dynamic allocation method of multi-core CPU resources based on SPDK architecture in one embodiment of the present disclosure.
FIG. 2 is a schematic structural diagram of a dynamic allocation device for multi-core CPU resources based on SPDK architecture according to one embodiment of the present disclosure.
Fig. 3 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application.
In the following description, the terms "first," "second," and "first," are used for descriptive purposes only and are not to be construed as indicating or implying relative importance. The following description provides various embodiments of the application that may be substituted or combined between different embodiments, and thus the application is also to be considered as embracing all possible combinations of the same and/or different embodiments described. Thus, if one embodiment includes feature A, B, C and another embodiment includes feature B, D, then the application should also be seen as embracing one or more of all other possible combinations of one or more of A, B, C, D, although such an embodiment may not be explicitly recited in the following.
The following description provides examples and does not limit the scope, applicability, or examples set forth in the claims. Changes may be made in the function and arrangement of elements described without departing from the scope of the application. Various examples may omit, replace, or add various procedures or components as appropriate. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Furthermore, features described with respect to some examples may be combined into other examples.
Referring to fig. 1, fig. 1 is a flow chart of a dynamic allocation method for multi-core CPU resources based on SPDK architecture according to an embodiment of the present application. In an embodiment of the present application, the method includes:
s101, dividing CPU resources into a plurality of processing areas based on subsystem types of SPDK architecture.
Each processing area is used for processing polling threads of a subsystem of a specified type, and CPU resources configured by each processing area are determined based on subsystem pressure.
The execution subject of the present application may be a cloud server.
In the embodiment of the present disclosure, SPDK architecture has different kinds of subsystems, and the different kinds of subsystems have different functions, for example, some subsystems are responsible for processing tasks such as coordination among the subsystems of SPDK architecture and scheduling of each polling thread, some subsystems are responsible for processing transactions of the block device subsystem in SPDK architecture, some subsystems are responsible for providing underlying dependencies for other subsystems, and so on. The application sets a plurality of processing areas according to the types of the subsystems, so that each processing area is specially used for processing the polling thread generated in a subsystem of a specific type, and the specific amount of CPU resources divided into each processing area is determined according to the pressure of the subsystem responsible for the processing area. The CPU resources configured in each processing area mainly comprise the number of CPUs and the specific core number of the used CPUs, and the number of the CPUs can influence the concurrent data processing performance of the processing area. In a multi-core processor, the processing efficiency of CPU cores at different positions is different, and taking a processor with a NUMA architecture of 64 cores as an example, the processing efficiency of CPU cores closer to 0 cores is generally higher. Meanwhile, the subsystem pressure is mainly caused by that the processing of other data is delayed due to the fact that the data to be processed is too much or the data being processed is too large, and the data processing speed is slow, so that the cloud server needs to determine the amount of CPU resources configured to each processing area according to the subsystem pressure. The specific configuration mode can be a quantitative mode, namely, the specific subsystem pressure is quantitatively determined, namely, the required concurrency and the required processing rate are set for the subsystem in advance according to the processing requirement, a mapping table is preset, the CPU number corresponding to the concurrency and the CPU number corresponding to the processing rate and the core number corresponding to the processing rate are represented, and finally, the CPU is distributed according to the mapping table. The cloud server can send an allocation request to the staff, the staff returns allocation information after the allocation is finished according to experience, and the cloud server can directly allocate according to the allocation information, and the cloud server can be in other implementation modes.
In one embodiment, the partitioning of CPU resources into a plurality of processing regions based on subsystem class of SPDK architecture includes:
Querying requirement information corresponding to each subsystem type of the SPDK framework, wherein the requirement information comprises processing speed requirements and resource quantity requirements;
and dividing a processing area for each subsystem type, and distributing CPU resources to each processing area based on the requirement information.
In this embodiment of the present disclosure, a worker may set, in advance, demand information for each subsystem in the SPDK framework according to the subsystem type, and store the demand information in the cloud server, where the demand information includes a quantitatively set processing rate demand and a quantitatively set resource quantity demand. After the processing areas are divided for each subsystem type, the cloud server distributes CPU resources according to the demand information and by combining the arbitrary configuration mode, so that the CPU resources with proper quantity and performance are distributed in each processing area.
In one embodiment, the allocating CPU resources to each of the processing regions based on the requirement information includes:
Determining a CPU allocation number ratio of each processing area based on the resource number requirement, and determining the CPU allocation number of each processing area according to the total core number of the CPU;
Performing first sequencing on the CPUs from high to low according to the processing speed, and performing second sequencing on the processing areas from high to low according to the processing speed requirement;
And sequentially selecting the processing areas according to the second sequence, selecting the CPU with the CPU distribution number according to the first sequence, and distributing the CPU to the processing areas.
In the embodiment of the present disclosure, the cloud server may determine the resource number requirements of each processing area, and determine the CPU allocation number ratio of each processing area according to the ratio between the resource number requirements. And then determining the CPU distribution number actually distributed in each processing area according to the calculation of the CPU distribution number ratio and the CPU total core number. In addition, the cloud server also performs first sorting on each CPU according to the processing speed, so that the CPU with higher processing speed is sorted forward. Similarly, the processing regions are also second ordered according to the processing speed requirements. After the first sorting and the second sorting are completed, the cloud server still selects a processing area according to the second sorting, sequentially selects a corresponding number of CPUs in the first sorting according to the CPU allocation number corresponding to the processing area, and allocates the corresponding number of CPUs to the processing area, so that the CPU resource allocated by each processing area is finally ensured to meet the actual operation requirement of the subsystem of the type.
As an example, taking a 64-core processor of a NUMA architecture as an example, after dividing a plurality of processing regions according to subsystem types, each processing region may be named differently according to the subsystem type name. For example, MAIN region may be responsible for processing SPDK tasks such as cooperation among subsystems of the architecture, scheduling of each poll, and the like, and has a high requirement on processing speed, so that the 0-7 core with the highest speed is allocated to MAIN region. The BDEV region may be responsible for processing transactions of the block device subsystem in SPDK architecture, which is the underlying dependency of all other block device related subsystems, requiring higher processing speeds, thus assigning faster 7-15 core CPUs to the BDEV region. LVSTORE region, ISCSI region, etc. require handling a large number of IO-related transactions, require high concurrency, but require relatively low single CPU performance. The amount of CPU resources allocated to these regions is therefore high, but the individual CPU performance is relatively weak, etc.
In one embodiment, after the partitioning of the CPU resources into the plurality of processing regions based on the subsystem class of SPDK architecture, the method further includes:
initializing CPU resources in the processing areas, clearing the current reference count of each CPU, and marking each CPU as a current unused state.
In the embodiment of the present disclosure, after the division of the CPU resources in each processing area is completed, the CPU resources first need to be initialized before the formal work. Specifically, the current reference count of the CPUs may be cleared entirely and each CPU may be marked as currently unused. In addition, the related information such as the number of the CPUs in the processing area, the CPU affinity and the like can be counted again for storage, so that the inquiry and the scheduling can be conveniently carried out when the problem occurs later.
S102, when detecting the CPU use application of any target polling thread, searching a first CPU which is not used currently in a target processing area corresponding to the target polling thread.
In the embodiment of the present specification, the target polling thread is only distinguished from the rest of the polling threads, so that the target polling thread is not distinguished from the rest of the polling threads for convenience of explanation. When a target polling thread is generated and a CPU needs to be bound for specific task processing, the cloud server detects a CPU use application sent by the target polling thread. At this time, the cloud server determines a target processing area corresponding to the target polling thread according to the subsystem type, and searches for a first CPU that is not currently used in all CPUs in the target processing area.
In one implementation manner, the searching the first CPU that is not currently used in the target processing area corresponding to the target polling thread includes:
Determining the subsystem type to which the target polling thread belongs, determining a target processing area according to the subsystem type, and inquiring a first CPU which is not used currently in the target processing area.
In this embodiment of the present disclosure, the cloud server determines the subsystem to which the cloud server belongs according to the location generated by the target polling thread, further determines the type of the subsystem to which the cloud server belongs, then determines the corresponding target processing area according to the type of the subsystem, and finally queries the first CPU in the target processing area.
S103, binding any one of the first CPUs with the target polling thread when the first CPU exists.
In the embodiment of the present disclosure, if a first CPU does currently exist in the target processing area, the cloud server may select any one of the first CPUs, and bind the first CPU with the target polling thread, so that the first CPU can support data processing of the target polling thread.
S104, when the first CPU does not exist, determining a second CPU with the minimum current reference count in the target processing area, and binding the second CPU with the target polling thread.
Wherein the current reference count is the number of simultaneous reference bindings for the CPU.
In the embodiment of the present specification, if the first CPU does not exist in the target processing area, it is indicated that all the CPUs are being used, and only a certain CPU being used can be bound to the target polling thread, so that the CPU processes multiple threads simultaneously. In order to make the pressure of each CPU similar as much as possible, the performance of each CPU is fully exerted, and each CPU is provided with a current reference count to represent how many threads the CPU is currently processing simultaneously. The cloud server determines the second CPU with the least current reference count, and binds the second CPU with the target polling thread, so as to avoid further increasing the load of some CPUs with more processed numbers.
In one embodiment, the method further comprises:
For any CPU, when the CPU is newly bound with a polling thread, the current reference count of the CPU is increased by one.
In the embodiment of the present disclosure, for any CPU, as long as the CPU is newly bound to a polling thread, the current reference count corresponding to the CPU is increased by one at the same time, so as to ensure that the current reference count can represent the actual load condition of each CPU in real time.
In one embodiment, the method further comprises:
For any CPU, when the CPU detects a CPU release instruction of any polling thread, the CPU is unbinding with the polling thread, and the current reference count of the CPU is reduced by one.
In the embodiment of the present specification, under SPDK architecture, a polling thread is frequently created and deleted, and if a CPU release instruction of any polling thread is detected, it indicates that a certain polling thread has been processed and is about to be deleted. At this time, the cloud server unbinds the CPU controlling the binding of the polling thread from the polling thread, and simultaneously decrements the current reference count corresponding to the CPU by one.
The following describes in detail the dynamic allocation device for multi-core CPU resources based on SPDK architecture according to the embodiment of the present application with reference to fig. 2. It should be noted that, the dynamic allocation device of multi-core CPU resources based on SPDK architecture shown in fig. 2 is used to execute the method of the embodiment shown in fig. 1 of the present application, for convenience of explanation, only the relevant parts of the embodiment of the present application are shown, and specific technical details are not disclosed, please refer to the embodiment shown in fig. 1 of the present application.
Referring to fig. 2, fig. 2 is a schematic structural diagram of a dynamic allocation device for multi-core CPU resources based on SPDK architecture according to an embodiment of the present application. As shown in fig. 2, the apparatus includes:
The dividing module 201 is configured to divide the CPU resources into a plurality of processing areas based on subsystem types of SPDK architecture, where each processing area is configured to process a polling thread of a subsystem of a specified type, and the CPU resources configured by each processing area are determined based on subsystem pressure;
The searching module 202 is configured to, when detecting a CPU usage application of any target polling thread, search a first CPU that is not currently used in a target processing area corresponding to the target polling thread;
A first judging module 203, configured to bind any one of the first CPUs with the target polling thread when the first CPU exists;
And a second judging module 204, configured to determine, when the first CPU is not present, a second CPU with a minimum current reference count in the target processing area, bind the second CPU to the target polling thread, where the current reference count is the number of simultaneous reference bindings of CPUs.
In one embodiment, the partitioning module 201 is specifically configured to:
Querying requirement information corresponding to each subsystem type of the SPDK framework, wherein the requirement information comprises processing speed requirements and resource quantity requirements;
and dividing a processing area for each subsystem type, and distributing CPU resources to each processing area based on the requirement information.
In one embodiment, the partitioning module 201 is specifically further configured to:
Determining a CPU allocation number ratio of each processing area based on the resource number requirement, and determining the CPU allocation number of each processing area according to the total core number of the CPU;
Performing first sequencing on the CPUs from high to low according to the processing speed, and performing second sequencing on the processing areas from high to low according to the processing speed requirement;
And sequentially selecting the processing areas according to the second sequence, selecting the CPU with the CPU distribution number according to the first sequence, and distributing the CPU to the processing areas.
In one embodiment, the partitioning module 201 is specifically further configured to:
initializing CPU resources in the processing areas, clearing the current reference count of each CPU, and marking each CPU as a current unused state.
In one embodiment, the lookup module 202 is specifically configured to:
Determining the subsystem type to which the target polling thread belongs, determining a target processing area according to the subsystem type, and inquiring a first CPU which is not used currently in the target processing area.
In one embodiment, the apparatus further comprises:
The first counting module is used for adding one to the current reference count of any CPU when the CPU is newly bound with the polling thread.
In one embodiment, the apparatus further comprises:
The second counting module is used for unbinding the CPU and the polling thread when the CPU detects a CPU release instruction of any polling thread aiming at any CPU, and reducing the current reference count of the CPU by one.
It will be clear to those skilled in the art that the technical solutions of the embodiments of the present application may be implemented by means of software and/or hardware. "unit" and "module" in this specification refer to software and/or hardware capable of performing a particular function, either alone or in combination with other components, such as Field-Programmable gate arrays (Field-Programmable GATE ARRAY, FPGA), integrated circuits (INTEGRATED CIRCUIT, ICs), and the like.
The processing units and/or modules of the embodiments of the present application may be implemented by an analog circuit that implements the functions described in the embodiments of the present application, or may be implemented by software that executes the functions described in the embodiments of the present application.
Referring to fig. 3, a schematic structural diagram of an electronic device according to an embodiment of the present application is shown, where the electronic device may be used to implement the method in the embodiment shown in fig. 1. As shown in fig. 3, the electronic device 300 may include at least one processor 301, at least one network interface 304, a user interface 303, a memory 305, and at least one communication bus 302.
Wherein the communication bus 302 is used to enable connected communication between these components.
The user interface 303 may include a Display screen (Display), a Camera (Camera), and the optional user interface 303 may further include a standard wired interface, and a wireless interface.
The network interface 304 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface), among others.
Wherein the processor 301 may include one or more processing cores. The processor 301 utilizes various interfaces and lines to connect various portions of the overall electronic device 300, perform various functions of the electronic device 300, and process data by executing or executing instructions, programs, code sets, or instruction sets stored in the memory 305, and invoking data stored in the memory 305. Alternatively, the processor 301 may be implemented in at least one hardware form of digital signal Processing (DIGITAL SIGNAL Processing, DSP), field-Programmable gate array (Field-Programmable GATE ARRAY, FPGA), programmable logic array (Programmable Logic Array, PLA). The processor 301 may integrate one or a combination of several of a central processing unit (Central Processing Unit, CPU), an image central processing unit (Graphics Processing Unit, GPU), and a modem, etc. The CPU mainly processes an operating system, a user interface, an application program and the like, the GPU is used for rendering and drawing contents required to be displayed by the display screen, and the modem is used for processing wireless communication. It will be appreciated that the modem may not be integrated into the processor 301 and may be implemented by a single chip.
The memory 305 may include a random access memory (Random Access Memory, RAM) or a Read-only memory (Read-only memory). Optionally, the memory 305 includes a non-transitory computer readable medium (non-transitory computer-readable storage medium). Memory 305 may be used to store instructions, programs, code, sets of codes, or sets of instructions. The memory 305 may include a stored program area that may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing the above-described respective method embodiments, etc., and a stored data area that may store data, etc., referred to in the above-described respective method embodiments. Memory 305 may also optionally be at least one storage device located remotely from the aforementioned processor 301. As shown in fig. 3, an operating system, a network communication module, a user interface module, and program instructions may be included in the memory 305, which is a type of computer storage medium.
In the electronic device 300 shown in fig. 3, the user interface 303 is mainly used for providing an input interface for a user to obtain data input by the user, and the processor 301 may be used for calling a multi-core CPU resource dynamic allocation application program based on SPDK architecture stored in the memory 305, and specifically performing the following operations:
Dividing CPU resources into a plurality of processing areas based on subsystem types of SPDK architecture, wherein each processing area is used for processing polling threads of a subsystem of a specified type, and the CPU resources configured by each processing area are determined based on subsystem pressure;
When detecting a CPU use application of any target polling thread, searching a first CPU which is not used currently in a target processing area corresponding to the target polling thread;
Binding any one of the first CPUs with the target polling thread when the first CPU exists;
And when the first CPU does not exist, determining a second CPU with the minimum current reference count in the target processing area, and binding the second CPU with the target polling thread, wherein the current reference count is the number of the CPU which is simultaneously reference-bound.
The present application also provides a computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of the above method. The computer-readable storage medium may include, among other things, any type of disk including floppy disks, optical disks, DVDs, CD-ROMs, micro-drives, and magneto-optical disks, ROM, RAM, EPROM, EEPROM, DRAM, VRAM, flash memory devices, magnetic or optical cards, nanosystems (including molecular memory ICs), or any type of media or device suitable for storing instructions and/or data.
It should be noted that, for simplicity of description, the foregoing method embodiments are all described as a series of acts, but it should be understood by those skilled in the art that the present application is not limited by the order of acts described, as some steps may be performed in other orders or concurrently in accordance with the present application. Further, those skilled in the art will also appreciate that the embodiments described in the specification are all preferred embodiments, and that the acts and modules referred to are not necessarily required for the present application.
In the foregoing embodiments, the descriptions of the embodiments are emphasized, and for parts of one embodiment that are not described in detail, reference may be made to related descriptions of other embodiments.
In the several embodiments provided by the present application, it should be understood that the disclosed apparatus may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, such as the division of the units, merely a logical function division, and there may be additional manners of dividing the actual implementation, such as multiple units or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be through some service interface, device or unit indirect coupling or communication connection, electrical or otherwise.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.
The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable memory. Based on this understanding, the technical solution of the present application may be embodied essentially or partly in the form of a software product, or all or part of the technical solution, which is stored in a memory, and includes several instructions for causing a computer device (which may be a personal computer, a server, a network device, or the like) to perform all or part of the steps of the method according to the embodiments of the present application. The Memory includes a U disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a removable hard disk, a magnetic disk, or an optical disk, etc. which can store the program codes.
Those of ordinary skill in the art will appreciate that all or a portion of the steps of the various methods of the above embodiments may be implemented by hardware associated with a program of instructions, which may be stored in a computer readable Memory, which may include a flash disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk, or an optical disk, etc.
The foregoing is merely exemplary embodiments of the present disclosure and is not intended to limit the scope of the present disclosure. That is, equivalent changes and modifications are contemplated by the teachings of this disclosure, which fall within the scope of the present disclosure. Embodiments of the present disclosure will be readily apparent to those skilled in the art from consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a scope and spirit of the disclosure being indicated by the claims.
Claims (9)
1. The method for dynamically allocating the multi-core CPU resources based on SPDK architecture is characterized by comprising the following steps:
Dividing CPU resources into a plurality of processing areas based on subsystem types of SPDK architecture, wherein each processing area is used for processing polling threads of a subsystem of a specified type, and the CPU resources configured by each processing area are determined based on subsystem pressure;
When detecting a CPU use application of any target polling thread, searching a first CPU which is not used currently in a target processing area corresponding to the target polling thread;
Binding any one of the first CPUs with the target polling thread when the first CPU exists;
When the first CPU does not exist, determining a second CPU with the minimum current reference count in the target processing area, and binding the second CPU with the target polling thread, wherein the current reference count is the number of the CPU which is simultaneously reference-bound;
Wherein the subsystem class based on SPDK architecture divides the CPU resources into a plurality of processing areas, including:
Querying requirement information corresponding to each subsystem type of the SPDK framework, wherein the requirement information comprises processing speed requirements and resource quantity requirements;
and dividing a processing area for each subsystem type, and distributing CPU resources to each processing area based on the requirement information.
2. The method of claim 1, wherein said allocating CPU resources to each of said processing regions based on said demand information comprises:
Determining a CPU allocation number ratio of each processing area based on the resource number requirement, and determining the CPU allocation number of each processing area according to the total core number of the CPU;
Performing first sequencing on the CPUs from high to low according to the processing speed, and performing second sequencing on the processing areas from high to low according to the processing speed requirement;
And sequentially selecting the processing areas according to the second sequence, selecting the CPU with the CPU distribution number according to the first sequence, and distributing the CPU to the processing areas.
3. The method of claim 1, wherein after partitioning the CPU resources into the plurality of processing regions based on the subsystem class of SPDK architecture, further comprising:
initializing CPU resources in the processing areas, clearing the current reference count of each CPU, and marking each CPU as a current unused state.
4. The method of claim 1, wherein the searching for the first CPU that is not currently used in the target processing region corresponding to the target polling thread comprises:
Determining the subsystem type to which the target polling thread belongs, determining a target processing area according to the subsystem type, and inquiring a first CPU which is not used currently in the target processing area.
5. The method according to claim 1, wherein the method further comprises:
For any CPU, when the CPU is newly bound with a polling thread, the current reference count of the CPU is increased by one.
6. The method according to claim 1, wherein the method further comprises:
For any CPU, when the CPU detects a CPU release instruction of any polling thread, the CPU is unbinding with the polling thread, and the current reference count of the CPU is reduced by one.
7. A multi-core CPU resource dynamic allocation apparatus based on SPDK architecture, the apparatus comprising:
The dividing module is used for dividing CPU resources into a plurality of processing areas based on subsystem types of SPDK architecture, each processing area is used for processing polling threads of a subsystem of a specified type, and the CPU resources configured by each processing area are determined based on subsystem pressure;
the searching module is used for searching a first CPU which is not used currently in a target processing area corresponding to the target polling thread when detecting a CPU use application of any target polling thread;
The first judging module is used for binding any one of the first CPUs with the target polling thread when the first CPU exists;
the second judging module is used for determining a second CPU with the least current reference count in the target processing area when the first CPU does not exist, binding the second CPU with the target polling thread, wherein the current reference count is the number of the CPU which are simultaneously referenced and bound;
the dividing module is specifically configured to:
Querying requirement information corresponding to each subsystem type of the SPDK framework, wherein the requirement information comprises processing speed requirements and resource quantity requirements;
and dividing a processing area for each subsystem type, and distributing CPU resources to each processing area based on the requirement information.
8. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the method according to any of claims 1-6 when the computer program is executed.
9. A computer readable storage medium having stored thereon a computer program having instructions stored therein, which when run on a computer or processor, cause the computer or processor to perform the steps of the method according to any of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202411650909.1A CN119166365B (en) | 2024-11-19 | 2024-11-19 | Dynamic allocation method and device for multi-core CPU (Central processing Unit) resources based on SPDK architecture |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202411650909.1A CN119166365B (en) | 2024-11-19 | 2024-11-19 | Dynamic allocation method and device for multi-core CPU (Central processing Unit) resources based on SPDK architecture |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN119166365A CN119166365A (en) | 2024-12-20 |
| CN119166365B true CN119166365B (en) | 2025-03-14 |
Family
ID=93891568
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202411650909.1A Active CN119166365B (en) | 2024-11-19 | 2024-11-19 | Dynamic allocation method and device for multi-core CPU (Central processing Unit) resources based on SPDK architecture |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN119166365B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113821174A (en) * | 2021-09-26 | 2021-12-21 | 迈普通信技术股份有限公司 | Storage processing method, device, network card equipment and storage medium |
| CN114138471A (en) * | 2021-11-18 | 2022-03-04 | 苏州浪潮智能科技有限公司 | Method for binding software service to CPU core and related device |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110333899B (en) * | 2019-06-27 | 2022-11-01 | 腾讯科技(深圳)有限公司 | Data processing method, device and storage medium |
| KR102802321B1 (en) * | 2019-12-11 | 2025-05-07 | 한국전자통신연구원 | Method and apparatus for processing to support scalability in many core environment |
| CN117742876A (en) * | 2022-09-14 | 2024-03-22 | 腾讯科技(深圳)有限公司 | Binding method, device and equipment of processor core and computer storage medium |
| US20230401109A1 (en) * | 2023-06-27 | 2023-12-14 | Intel Corporation | Load balancer |
| CN118193210A (en) * | 2024-03-27 | 2024-06-14 | 中国联合网络通信集团有限公司 | Coroutine execution method and device based on storage performance development kit SPDK |
-
2024
- 2024-11-19 CN CN202411650909.1A patent/CN119166365B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113821174A (en) * | 2021-09-26 | 2021-12-21 | 迈普通信技术股份有限公司 | Storage processing method, device, network card equipment and storage medium |
| CN114138471A (en) * | 2021-11-18 | 2022-03-04 | 苏州浪潮智能科技有限公司 | Method for binding software service to CPU core and related device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN119166365A (en) | 2024-12-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9965324B2 (en) | Process grouping for improved cache and memory affinity | |
| US8819675B2 (en) | Virtual machine monitor and multiprocessor system | |
| US9875139B2 (en) | Graphics processing unit controller, host system, and methods | |
| US11876731B2 (en) | System and methods for sharing memory subsystem resources among datacenter applications | |
| CN111124687B (en) | A CPU resource reservation method, device and related equipment | |
| US12411716B2 (en) | Serverless function colocation with storage pools | |
| JP6262360B2 (en) | Computer system | |
| US10146446B1 (en) | Data storage system with port congestion hinting for host multipathing | |
| CN103548324A (en) | Virtual machine distribution method and virtual machine distribution device | |
| US9189279B2 (en) | Assignment method and multi-core processor system | |
| US10942772B2 (en) | Dispatching jobs for execution in parallel by multiple processors | |
| CN106325996A (en) | GPU resource distribution method and system | |
| CN116302327A (en) | Resource scheduling method and related equipment | |
| US20150212859A1 (en) | Graphics processing unit controller, host system, and methods | |
| CN111352735A (en) | Data acceleration method, device, storage medium and equipment | |
| CN119166365B (en) | Dynamic allocation method and device for multi-core CPU (Central processing Unit) resources based on SPDK architecture | |
| CN112667356B (en) | NVMe storage virtualization method and system with predictable time delay | |
| CN117707706A (en) | Virtual machine creation method, device, computing equipment and storage medium | |
| CN112783651A (en) | Load balancing scheduling method, medium and device for vGPU of cloud platform | |
| US20140047454A1 (en) | Load balancing in an sap system | |
| US8245229B2 (en) | Temporal batching of I/O jobs | |
| JPH0612395A (en) | Task allocation method in multiprocessor system | |
| TW202201225A (en) | Cloud service resource processing method, device, computing equipment, and storage medium | |
| EP2698713A2 (en) | Load balancing in data processing system | |
| CN117519604A (en) | Virtual machine disk performance adjustment method, device, equipment and storage based on SPDK |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |