CN120449150A

CN120449150A - A transparent virtual machine monitoring method for cloud security

Info

Publication number: CN120449150A
Application number: CN202510468663.4A
Authority: CN
Inventors: 袁平海; 孙宇航; 陈付龙
Original assignee: Anhui Normal University
Current assignee: Anhui Normal University
Priority date: 2025-04-15
Filing date: 2025-04-15
Publication date: 2025-08-08

Abstract

The invention discloses a cloud security-oriented transparent virtual machine monitoring method which comprises the steps of S1, activating and deactivating LBR and PMU, S2, integrity verification and user space interaction, S3, hardware configuration and interrupt processing, S4, kernel cache and user space dual mapping, S5, user space interaction and CFI verification. By skillfully utilizing the hardware characteristics of a modern processor, the real-time monitoring and the integrity verification of the control flow of the virtual machine kernel are realized, the control flow hijacking attack is effectively resisted on the premise of not influencing the normal operation of the virtual machine, and the security of the virtual machine kernel in the cloud computing environment is improved.

Description

Cloud security-oriented transparent virtual machine monitoring method

Technical Field

The invention relates to the field of security protection of a virtual machine kernel in a cloud computing environment, in particular to a method for realizing integrity protection of a virtual machine kernel control flow based on a processor hardware characteristic combination. By utilizing the hardware characteristics of a modern processor, the method provides efficient, transparent and non-invasive control flow integrity protection for the virtual machine kernel, and aims to solve security threats such as control flow hijacking and the like faced by the virtual machine kernel in a cloud computing scene.

Background

The popularization of cloud computing enables a large number of computing tasks to be migrated to the cloud, wherein a platform serving mode is widely adopted. In this mode, the infrastructure provider provides a Virtual Machine (VM) to the customer, who can either process private data on the VM or share information through a server program in the VM. However, the attack surface of the cloud computing environment is wider than that of the local environment, various private data are uploaded and processed in the cloud, and cloud users expect that remote VMs thereof can be comprehensively protected, and even better than the security level of the local system.

The importance and the threat of kernel security are that the kernel is used as the core of a computer system, and the security is the basic stone of the security of the whole system. Kernel vulnerabilities can cause serious problems such as memory corruption, information leakage or denial of service (Denial of Service, doS), which pose a significant threat to system security. Unfortunately, kernel vulnerabilities are constantly exposed, and only in 2022, the number of vulnerabilities assigned to CVE IDs exceeds 300. Meanwhile, researchers continue to discover new kernel vulnerabilities through fuzzy testing and other techniques. Corruption of function pointers or return addresses in memory can cause control flow hijacking whereby malicious attackers can redirect control flow to kernel space to execute malicious code, even building a kernel-mode Rootkit. Therefore, protecting the virtual machine kernel from control flow hijacking attacks is a critical issue to be addressed.

The limitation of the existing Control flow integrity protection scheme is that various mechanisms such as shadow stack, layout randomization, control flow integrity (Control-FlowIntegrity, CFI) and the like have been proposed for coping with kernel Control flow hijacking attacks. CFI defends against hijacking attacks by requiring execution of Control Flow instructions to conform to a Control-Flow Graph (CFG) generated at program compile time. However, most existing CFI schemes are designed for local environments where they are difficult to apply directly due to "ownership separation". Specifically, the cloud platform provider controls the physical machine and has the highest hypervisor authority, while the cloud user has full control of its VM guest operating system. This results in the cloud user being unable to supervise the VM with the VM hypervisor or hardware functions, nor is the cloud provider able to tamper with the client kernel to insert security check code. In addition, some CFI schemes based on software instrumentation require modification of kernel code, may destroy the integrity and stability of the kernel, and are subject to significant compatibility issues in the face of numerous different types and versions of operating systems and kernel modules, severely impacting the utility of the scheme.

The potential and challenges of hardware-assisted solutions are that modern Intel x86 CPUs offer some hardware features for program debugging and system analysis, such as branch trace storage (Branch Trace Store, BTS), last branch record (Last Branch Record, LBR), and performance monitoring units (Performance Monitoring Unit, PMU), etc., providing a new approach to solving kernel security problems in cloud computing environments. However, these hardware features each have limitations. The BTS does not support instruction type filtering, can record all branch instructions, and has higher performance cost, while the LBR supports instruction type filtering and has low recording cost, the buffer area is full and can not inform the CPU, and the PMU can monitor various hardware events and can not independently complete effective recording of control flow. Therefore, there is a need for the present invention to effectively combine these hardware features through an innovative approach to achieve effective protection of virtual machine kernel control flow integrity in a cloud computing environment.

Disclosure of Invention

The invention aims to overcome the defects of the prior art, provides a method for protecting the integrity of the control flow of the virtual machine kernel based on the hardware characteristic combination of a processor, and solves the problem of protecting the integrity of the control flow of the virtual machine kernel in a cloud computing environment. By skillfully utilizing the hardware characteristics of a modern processor, the real-time monitoring and the integrity verification of the control flow of the virtual machine kernel are realized, the control flow hijacking attack is effectively resisted on the premise of not influencing the normal operation of the virtual machine, and the security of the virtual machine kernel in the cloud computing environment is improved.

In order to achieve the purpose, the technical scheme adopted by the invention is that the cloud security-oriented transparent virtual machine monitoring method comprises the following steps:

S1, activating and deactivating LBR and PMU, namely adding a probe in the KVM module, and activating or deactivating the LBR and the PMU when the protected virtual machine VM dispatches to enter or exit;

S2, integrity verification and user space interaction, wherein a host user space code (code residing in a host) performs integrity verification, and a kernel driving module KvmMon derives a device file to the user space as an operation handle for interaction between the user space code and a kernel;

S3, hardware configuration and interrupt processing, namely configuring control, data and status registers of LBR and PMU by setting a VMCS hardware structure;

S4, dual mapping of the kernel buffer and the user space, wherein KvmMon uses the kernel memory buffer to buffer all ICFTRANSFER execution records, and maps the buffer to the user space;

S5, user space interaction and CFI verification, wherein the prototype system exports a device file to a host user space, and user space codes interact with the KvmMon module through customized mmap (), close () and ioctl () system calls to realize double mapping and release of a kernel buffer and communication with KvmMon drive.

The prototype system is realized on the Linux kernel by expanding a KVM management program, the functions of the prototype system comprise driving LBR and PMU characteristics to be realized in a KvmMon module and an integrity checker of a host user space, the KvmMon module is responsible for managing the configuration and data collection of hardware characteristics, and the user space program is responsible for executing specific integrity verification logic.

When the LBR is configured, instruction type filtering is realized by configuring an MSR_LBR_SELECT register so that the MSR_LBR_SELECT register only records indirect control flow transfer in a client kernel space, a PMU related PMC counter is increased when the LBR buffer is inserted into the record each time, PMC overflow is ensured when the LBR buffer is full by initializing PMC to be the complement of the LBR stack size, and at each VMEnter KvmMon initializes and clears the LBR and PMU related MSR registers by setting a VMCS structure so as to ensure the correct operation of hardware characteristics.

The original KVM module is minimally modified, only a plurality of probes are added, wherein the probes refer to function pointers, the function pointers kvm_ lbrpmu _ setter () and kvm_ lbrpmu _dump ()' are initialized when KvmMon modules are loaded, the kvm_ lbrpmu _ setter () function is called to set VMCS hardware structure before VM entry when vCPU is in monitoring state, LBR and PMU functions are enabled, and the kvm_ lbrpmu _dump () function is called to dump indirect control Flow Transfer records (Indirect Control-Flow Transfer, ICFTRANSFER) in the LBR buffer to the buffer when VM exits.

In the initialization stage, the KvmMon module creates a buffer pool by using a memory, each buffer consists of several continuous memory pages, the first page contains a header for managing the metadata of the current buffer, the buffers are used in a cyclic manner, when the buffers overflow, the earliest record will be covered, the buffer and the header thereof can be mapped to a user space through the API exported by the KvmMon module, and a user space program can read ICFTRANSFER records in the buffers and acquire the buffer status through analyzing the buffer header.

The prototype system exports the device file to the host user space, and the user space program realizes the dual mapping from the kernel buffer to the user space by customizing mmap () system call, and simultaneously realizes the communication between the user space code and KvmMon modules by using ioctl () system call.

And (3) carrying out integrity verification by using a user space program, wherein the program adopts a multithreading mode, and a working thread is created for each monitored vCPU to carry out verification.

In step S3, the LBR records execution in client kernel space ICFTRANSFERS, the PMU monitors the record insertion event of the LBR buffer, when the LBR buffer is full, the PMU causes a hardware interrupt, resulting in VM exit, the VMExit handler checks the cause of the exit, and dumps the LBR record to the buffer.

In step S5, the policies of CFIGuard and CFIMon are adopted to extract the control flow graph CFG, and a tool chain is developed to organize the CFG into a desired structure, and for each ICFTRANSFER record, if the source address can be located to a valid ICFInstructions entry in the CFG, and the target address is a member of the corresponding permission set, the record is valid.

The method has the advantage of solving the problem of integrity protection of the kernel control flow of the virtual machine in the cloud computing environment. By skillfully utilizing the hardware characteristics of a modern processor, the real-time monitoring and the integrity verification of the control flow of the kernel of the virtual machine are realized, the control flow hijacking attack is effectively resisted on the premise of not influencing the normal operation of the virtual machine, and the safety of the virtual machine in the cloud computing environment is improved.

Drawings

The contents of the drawings and the marks in the drawings of the present specification are briefly described as follows:

fig. 1 is a system block diagram provided in an embodiment of the present invention.

Fig. 2 is a system workflow diagram provided in an embodiment of the present invention.

FIG. 3 is a performance evaluation graph provided by an embodiment of the present invention, running an sftp server.

Fig. 4 is a performance evaluation diagram provided by an embodiment of the present invention, and an http server is operated.

FIG. 5 is a performance evaluation graph provided by an embodiment of the present invention, running a mysql server.

FIG. 6 is a performance evaluation graph provided by an embodiment of the present invention, running an octane benchmark test set in Windows and Linux virtual machines.

In the scheme, the technical terms Chinese and English are compared as follows:

VM Virtual Machine, virtual Machine

KVM (Kernel-based virtual machine) Kernel-based Virtual Machine

LBR Last Branch Record, last Branch record

PMU Performance Monitoring Unit, performance monitoring Unit

PMC Performance Monitoring Counter Performance monitoring counter

MSR Model SPECIFIC REGISTER, model-specific register

VMCS Virtual Machine Control Structure virtual machine control structure

CFI Control-Flow Integrity, control Flow Integrity

CFG Control Flow Graph, control flow graph

KvmCFI KVM-INTEGRATED CONTROL-Flow Integrity, control Flow Integrity integrated with KVM

KvmMon KVM-INTEGRATED CONTROL-flow Monitor, control flow Monitor integrated with KVM

ICFTRANSFER: indirect Control Flow Transfer, indirect control flow transfer;

VMEnter Virtual MACHINE ENTER, virtual machine entry

VMExit: virtual Machine Exit, virtual machine exit

Detailed Description

The following detailed description of the invention refers to the accompanying drawings, which illustrate preferred embodiments of the invention in further detail.

The invention aims to provide a method for protecting the integrity of a virtual machine kernel control flow based on a processor hardware characteristic combination, which solves the problem of protecting the integrity of the virtual machine kernel control flow in a cloud computing environment. By skillfully utilizing the hardware characteristics of a modern processor, the real-time monitoring and the integrity verification of the control flow of the kernel of the virtual machine are realized, the control flow hijacking attack is effectively resisted on the premise of not influencing the normal operation of the virtual machine, and the safety of the virtual machine in the cloud computing environment is improved.

The hardware characteristics of the contemporary processor are skillfully combined, a pure hardware method is adopted to monitor the control flow generated when the kernel of the virtual machine runs, the security check is timely carried out, and the virtual machine is transparently protected. The specific implementation steps are as follows:

And S1, adding a probe into the KVM module. These probes (Prober) are triggered when the protected VM schedules an entry/exit for activating/deactivating the LBR and PMU. At the same time, allowing integrity verification of code residing within the host, the cloud provider can flexibly select control flow integrity check algorithms according to its own needs without modifying KvmCFI's kernel driver KvmMon. KvmMon derives a device file (in Linux systems, a device file is a special file that provides an interface for the user space program to communicate with the hardware device or virtual device) to the user space code (code residing in the host) through which the user space program can interact with the KvmMon module. Specifically, the user program may open the device file to obtain a file handle, and then call the system call for performing the read/write operation with the handle as a parameter. As described below, these system calls are all customized, translating into control of the KvmMon modules.

S2, using the mechanism provided by the Intel CPU, a virtual machine manager (Hypervisor, such as KVM) can configure the control, data and status registers of the LBR and PMU for the protected client by setting the VMCS hardware structure. The LBR is configured to record execution of the client kernel space intermediate control flow transfer ICFTRANSFER, for each execution, a pair of effective addresses < jump source address, jump destination address > is recorded to the LBR buffer. The PMU is used to monitor the LBR buffer for record insertion events, with its associated PMC increasing each time a record insertion event occurs. By initializing the PMC to the complement of the LBR stack size, it is ensured that when the LBR buffer is full, the PMC overflows, at which point the PMU will cause a hardware interrupt, resulting in a VM exit. The VMExit handler will check for the reason for the exit and if the VM exits due to PMC overflows, the KvmMon module will dump the LBR record to the cache buffer. In addition, to ensure security, control Registers (CR) that configure the LBR and PMU are not allowed to be accessed by the guest VM, and certain fields of the VMCS are set by the "vmread" and "vmwrite" instructions, making these registers invisible to the guest VM.

And S3, adding probes in the vCPU scheduling function vmx _ vCPU _entity_exit () of the KVM to acquire notification of VMenter and VMexit events. Each probe is associated with a function pointer that points to the function in KvmMon when the KvmMon modules are loaded. Meanwhile, a substructure named "cfi_ctrl" is added to a structure named "vCPU _ vmx" associated with the KVM configuration to track the CFI-related status of a given vCPU. One flag bit in cfi_ctrl is used for indicating whether the vCPU is in a monitoring state, and the flag bit also decides whether to call a probe function. When the vCPU is in the monitoring state, before the VM enters, the function pointed by the related function pointer is called for setting the VMCS hardware structure and enabling the LBR and PMU functions, and when the VM exits, the function pointed by the other function pointer is called for dumping ICFTRANSFER records in the LBR buffer area to the buffer area.

S4, to achieve safe and flexible integrity verification, kvmCFI transfers the integrity verification task to the host user space code. KvmMon all ICFTRANSFER execution records are cached using a kernel memory buffer, which is also mapped to user space using a "Double Mapping" mechanism (Double Mapping) (so that the contents of the same piece of physical memory can be seen by the kernel code and user program). At each VMExit KvmMon dumps LBR records into the cache buffer even if the cause of the VM exit is not LBR buffer full. For a guest VM with multiple vcpus, kvmMon maintains a pool of kernel cache buffers, with each monitored vCPU having its own cache buffer. The cache buffer is initialized when its associated vCPU is first executed and released when the vCPU stops monitoring. The buffer pool is created and destroyed when KvmMon modules load and unload, respectively. By means of a dual mapping mechanism provided by the Linux kernel, kvmMon exports the buffer to a host user space code, and the user space program can initiate a dual mapping request to the device file through mmap () system call, so that ICFTRANSFER records are accessed efficiently. mmap () is an important system call in the Linux operating system and is mainly used for mapping file contents to the virtual address space of a process. As described below, this patent customizes its functionality to enable dual mapping functionality.

S5, the prototype system (the patent realizes a preliminary and simplified system based on the KVM according to the design principle) exports the device file to the host user space code, and the host user space code can interact with the KvmMon module through system call. The custom mmap () system call realizes the dual mapping of the kernel buffer to the user space, and maps the buffer into a read-only memory area. The custom close () system call is used to free up the cache buffer and close the file. Custom ioctl () system calls are used to establish a communication channel between the host user space code and KvmMon drivers. Host user space code implements related functions in Linux kernel through custom mmap () system and ioctl () system, ioctl () allows user space code to operate device through custom command, and user space code can send information to KvmMon driver or retrieve data from by making communication protocol, such as inquiring the number of vCPU of protected VM and kernel buffer size.

S6 CFI scheme verifies the integrity by referring to a given CFG, the accuracy of which is critical to the validity of the scheme. KvmCFI support modularity, whose CFGs contain cross-module nodes. The present invention employs the policies CFIGuard and CFIMon to extract CFGs and develops a tool chain to organize CFGs into desired structures. Specifically, as with CFIGuard and CFIMon, a complete CFG control flow graph is obtained using static program analysis and dynamic run profiling. Each CFG stores offset values, which are the relative addresses of the code points in the current module. When loading a CFG file for CFI verification, all offset values will be relocated and set to the real (fixed) address of the referenced code point. The CFGs are organized in a two-level tree structure, the first level storing all valid ICFInstruction addresses and the second level storing the allowed jump target address for each ICFInstruction. For each ICFTRANSFER record, the record is valid if its source address can be located in the CFG to a valid ICFInstruction entry and the target address is a member of the corresponding permission set. For a transfer across modules, the source address and the destination address are located in different modules, and the source address needs to be ensured to be valid in the current CFG during verification, and the destination address is a valid entry in the destination CFG.

In the design, the interaction flow among the modules is carefully optimized to ensure the efficient transmission and processing of data, for example, in the interaction of the KVM probe and the kernel cache buffer, the monitoring data can be stored accurately.

When the KVM probe is triggered, the state of the vCPU is checked preferentially, so that the accuracy and the safety of operation are ensured, and the influence of misoperation on a system is avoided.

Regarding the kernel cache buffer, the size of the kernel cache buffer can be dynamically adjusted according to the scale of the protected VM and the number of vCPUs, so as to realize reasonable utilization of resources.

The user space API fully considers compatibility and expansibility in design, and can be conveniently integrated with different types of user space programs.

In the process of integrity verification, an incremental update strategy is adopted for updating the CFG, so that verification efficiency is improved and resource consumption is reduced.

When the LBR buffer area is full, the system can adopt a quick dump mechanism to ensure that data is not lost, and meanwhile, follow-up monitoring work is reasonably scheduled.

In the cooperative work of the KVM probe and the LBR+PMU module, the consistency of data is ensured by adding a synchronous mechanism.

When the dual mapping of the kernel cache buffer is abnormal, the system automatically performs remapping operation so as to ensure the access of the user space program to the data.

As shown in fig. 1, a system architecture diagram corresponding to the monitoring method of the present application is shown. In a cloud computing environment, a kernel of a host system integrates a KVM module, which provides convenient support for creating and running virtual machines. The control flow protection scheme proposed by this patent is composed of two parts in system implementation, both of which run in the host. And a part of functions are integrated in the LBR+PMU kernel module and are mainly responsible for controlling and managing the LBR and the PMU. The other part works in the host user mode and is mainly responsible for controlling the flow integrity check. After the virtual machine is started and operated, the LBR+PMU module tracks control flow transfer operated in the kernel of the virtual machine and dumps records to the kernel buffer. With the "dual mapping" mechanism, the kernel buffer is mapped to the host user state. Therefore, the user mode code of the host machine can use the CFG to develop accurate control flow integrity detection for the transfer record, so that the safety of the virtual machine is effectively ensured.

As shown in fig. 2, a system workflow diagram is shown, which shows further technical details. After the virtual machine is started, the host user mode program can communicate with the LBR+PMU module by means of open () and close () system call, and further control the LBR and PMU to operate according to the two CPU characteristics. In addition, the user mode program can also use mmap () system call to map the user kernel buffer storing the control flow transfer record to itself so as to carry out the control flow integrity check work.

As shown in fig. 3-6, the performance overhead of the system is very small when implementing control flow integrity protection for the virtual machine as shown in the test results in the figures. In the case of common virtual machine configurations such as 1 to 8 vcpus, only about 1.5% overhead is incurred for the throughput of servers such as sftp, http, and mysql. As shown in fig. 6, similar conclusions can be drawn on both the Windows virtual machine and the Linux virtual machine.

It is obvious that the specific implementation of the present invention is not limited by the above-mentioned modes, and that it is within the scope of protection of the present invention only to adopt various insubstantial modifications made by the method conception and technical scheme of the present invention.

Claims

1. A cloud security-oriented transparent virtual machine monitoring method, characterized by:

S1. Activation and deactivation of LBR and PMU: Add probes to the KVM module to activate or deactivate LBR and PMU when the protected virtual machine VM is scheduled to enter or exit.

S2. Integrity verification and user space interaction: The host user space code performs integrity verification, and the kernel driver KvmMon exports device files to the user space for operation by the user space code;

S3, hardware configuration and interrupt handling: configure the control, data and status registers of LBR and PMU by setting up the VMCS hardware structure;

S4. Kernel cache and user space dual mapping: KvmMon uses a kernel memory buffer to cache all ICFTransfer execution records and "double maps" this buffer to user space;

S5. User space interaction and CFI verification: The prototype system exports device files to the host user space. The user space code interacts with the KvmMon module through customized mmap(), close(), and ioctl() system calls to achieve double mapping and release of the kernel cache buffer and communication with the KvmMon driver.

2. A cloud security-oriented transparent virtual machine monitoring method as described in claim 1, characterized in that: a prototype system is implemented by extending the KVM hypervisor on the Linux kernel, and its functions include driving the LBR and PMU features to be implemented in the KvmMon module, as well as an integrity checker in the host user space; the KvmMon module is responsible for managing the configuration and data collection of hardware features, and the user space program is responsible for executing specific integrity verification logic.

3. The cloud security-oriented transparent virtual machine monitoring method according to claim 1, characterized in that:

When configuring LBR, instruction type filtering is implemented by configuring the MSR_LBR_SELECT register so that only indirect control flow transfers within the guest kernel space are recorded; the PMU's related PMC counter is increased each time a record is inserted into the LBR buffer. By initializing the PMC to the complement of the LBR stack size, it is ensured that the PMC overflows when the LBR buffer is full; at each VMEnter, KvmMon initializes and clears the related MSR registers of the LBR and PMU by setting the VMCS structure to ensure the correct operation of the hardware features.

4. A cloud security-oriented transparent virtual machine monitoring method as described in claim 1 is characterized in that: the original KVM module is minimally modified and only multiple probes are added, where the probes refer to function pointers; when the KvmMon module is loaded, the function pointers kvm_lbrpmu_setter() and kvm_lbrpmu_dumper() are initialized; when the vCPU is in the monitoring state, before the VM enters, the kvm_lbrpmu_setter() function will be called to set the VMCS hardware structure and enable the LBR and PMU functions; when the VM exits, the kvm_lbrpmu_dumper() function will be called to dump the ICFTransfer records in the LBR buffer to the cache buffer.

5. A cloud security-oriented transparent virtual machine monitoring method as described in claim 1 is characterized in that: during the initialization phase, the KvmMon module uses memory to create a cache buffer pool; each cache buffer consists of several consecutive physical memory pages, and the first page contains a header for managing the metadata of the current buffer; the buffer is used in a circular manner, and when the buffer overflows, the earliest record will be overwritten; through the API exported by the KvmMon module, the cache buffer and its header can be mapped to user space, and the user space program can read the ICFTransfer record in the buffer and obtain the buffer status by parsing the buffer header.

6. A cloud security-oriented transparent virtual machine monitoring method as described in claim 1, characterized in that: the prototype system exports device files to the host user space, the user space program implements dual mapping of the kernel cache buffer to the user space through a customized mmap() system call, and simultaneously utilizes the ioctl() system call to implement communication between the user space code and the KvmMon module.

7. A cloud security-oriented transparent virtual machine monitoring method as described in claim 1, characterized in that: a user space program is used for integrity verification, and the program adopts a multi-threaded approach to create a working thread for each monitored vCPU for verification.

8. A cloud security-oriented transparent virtual machine monitoring method as described in claim 1, characterized in that: in step S3, the LBR records the execution of ICFTransfers in the client kernel space, and the PMU monitors the record insertion events in the LBR buffer; when the LBR buffer is full, the PMU triggers a hardware interrupt, causing the VM to exit, and the VMExit handler checks the exit reason and dumps the LBR record to the cache buffer.

9. A cloud security-oriented transparent virtual machine monitoring method as described in claim 1, characterized in that: in step S5, the control flow graph CFG is extracted using the strategies of CFIGuard and CFIMon, and a tool chain is developed to organize the CFG into an expected structure; for each ICFTransfer record, if its source address can locate a valid ICFInstructions entry in the CFG and the target address is a member of the corresponding allowed set, then the record is valid.