CN108733455B - Container isolation enhancement system based on ARM TrustZone - Google Patents

Abstract

The invention provides a container isolation enhancement system based on ARM TrustZone, including: a container management client running on the user side; an untrusted operating system, an untrusted container management module and a trusted execution environment running in the common world of the server side ;Page table management module, register protection module, system call hijacking module, file system security enhancement module, execution stream synchronization service security enhancement module, inter-process communication service security enhancement module, trusted container image download running in the server-side security world module and secure container launch module. The invention safely runs the existing application program on the malicious operating system completely controlled by the attacker; different applications of different users in the container can perform secure communication and control flow synchronization; the user does not need to make any modification to the existing image.

Description

Container isolation enhancement system based on ARM TrustZone

technical field

The invention relates to the field of virtualization technology, in particular, to a container isolation enhancement system based on ARM TrustZone.

Background technique

Virtualization technology can simulate multiple virtual computers on one physical computer, thereby improving hardware utilization and facilitating multiple users to share the same physical device. Containers are a lightweight virtualization technology. Different containers will share the same operating system kernel, but each container has its own independent file system, user space, process space, etc. Compared to traditional virtualization technologies, containers have shorter startup times, faster performance, and easier deployment methods. Due to these significant advantages, containers have been widely used in the server field. Based on containers, cloud servers can quickly and easily create their own independent running environment for each user. More and more users also choose to store their data in containers located in the cloud, so as to obtain a faster and more convenient user experience.

While having faster performance and easier deployment methods, the security of containers has always been criticized. Different containers need to share the same operating system kernel. Once the kernel is compromised, the isolation between containers will be destroyed. At the same time, the operating system kernel has always had thousands of vulnerabilities due to its huge amount of code. In the cloud computing environment, once an attacker compromises the operating system kernel through a container, he will be able to control all the containers in the cloud.

At present, the ARM architecture is gradually being favored by the server field due to its better energy utilization and higher cost performance. A series of processor chips based on the ARM architecture already exist on the market. A new security problem is also introduced. Once the operating system in the ARM-based cloud server is compromised, how to ensure the security of the user container, and prevent attackers from controlling the execution flow of the application in the user container and stealing the user storage in the container. sensitive data, etc., has become a major problem facing this field.

After searching, it was found:

1. Owen et al. used hardware virtualization technology to design and implement a security system InkTag, which can protect applications from attacks by untrusted operating system kernels. InkTag provides an independent safe operating environment for each application, preventing the operating system from directly accessing the memory data in the safe operating environment, thereby protecting the application data from being stolen and the control flow from being tampered with. At the same time, InkTag will also check for services that partially depend on the completion of the operating system.

However, although InkTag prevents OS attacks on a single application, it is not suitable for container environments. First, the system cannot directly start existing container images, making it capable of protecting containers. Secondly, the system does not take into account the complex use environment of the container itself with multiple users and applications, and cannot guarantee the security of communication, information sharing, and permission control between different user applications inside the container.

2. In order to protect the security of containers, Arnautov and others used Intel's SGX technology to design and implement a secure container protection system SCONE. The system utilizes the trusted execution environment enclave provided by SGX technology to protect each individual container process. Since the hardware ensures that any software outside the enclave (including the operating system) cannot access the memory of the encalve and interfere with the execution flow inside the encalve, SCONE can effectively protect the security of a container application and prevent it from being attacked by an untrusted operating system. .

However, SCONE has two disadvantages: First, it can only support containers with single application and single process. That is to say, only one single-process application can run in a container, which greatly limits the application scenarios of containers. At the same time, SCONE does not consider the issue of how different applications work together safely in a multi-user and multi-application environment; secondly, SCONE requires modification of the original container image, so the original image cannot be directly run. This shortcoming makes it impossible for users to run millions of container images in Docker repositories, limiting the usage scenarios of containers.

In summary, how to use the characteristics of the ARM platform's own architecture to maintain any number of trusted execution environments for containers, how to rely on untrusted operating systems to provide services while preventing them from attacking container applications, and how to be compatible with existing Container images have become an urgent problem to be solved in this field.

At present, no description or report of similar technology to the present invention has been found, and similar materials at home and abroad have not been collected.

SUMMARY OF THE INVENTION

In view of the above deficiencies in the prior art, the purpose of the present invention is to provide a container isolation enhancement system based on ARM TrustZone. The system utilizes the characteristics of the ARM platform's own architecture to maintain any number of trusted execution environments for containers; while relying on untrusted operating systems to provide services, it can prevent them from attacking container applications; it is compatible with existing container images.

The present invention is achieved through the following technical solutions.

According to an aspect of the present invention, an ARM TrustZone-based container isolation enhancement system is provided, including:

A container management client running on the client side;

Untrusted operating system, untrusted container management module and trusted execution environment running in the normal world of the server side;

Page table management module, register protection module, system call hijacking module, file system security enhancement module, execution stream synchronization service security enhancement module, inter-process communication service security enhancement module, trusted container image download module running in the server-side security world And the secure container startup module;

in:

The container management client connects with any number of containers in the common environment of the server, and sends user instructions to the containers; each container runs in a trusted execution environment;

The untrusted operating system provides a running environment and required services for container applications;

The untrusted container management module provides basic container management operations;

The page table management module, the register protection module and the system call holding module jointly complete the maintenance of any number of trusted execution environments;

The file system security enhancement module, the execution stream synchronization service security enhancement module, and the inter-process communication service security enhancement module jointly complete the security enhancement of the untrusted operating system service, preventing the untrusted operating system from stealing the container by providing malicious system services. Process privacy data and/or tamper with container process control flow;

The trusted container image downloading module and the secure container startup module jointly complete the provision of secure container management functions.

Preferably, none of the hardware resources in the trusted execution environment can be directly accessed by an untrusted operating system.

Preferably, the page table management module exclusively manages all page tables of the entire untrusted operating system, and by controlling the page tables, protects the memory of the container process and restricts the untrusted operating system from accessing the memory of the container process.

Preferably, the page table management module searches all instructions for managing page tables in the untrusted operating system kernel, and replaces these with corresponding requests sent to the page table management module; the page table management module ensures that the physical memory pages storing the page tables are The common world is read-only, while ensuring that the code of the untrusted operating system kernel is not writable, and the page table management module implements the exclusive management of the page table.

Preferably, the register protection module ensures that the untrusted operating system cannot directly access or modify the register information of the trusted execution environment; wherein, the register protection module intercepts the switching between all user-mode processes and the kernel-mode untrusted operating system kernel; During the switching process, the register protection module is responsible for saving and restoring the registers of the trusted operating environment, thereby ensuring that the untrusted operating system cannot arbitrarily tamper and/or steal the register information of the trusted operating environment.

Preferably, the switching between the user mode process and the untrusted operating system kernel in kernel mode includes the following process:

All operations of entering the untrusted operating system kernel from user mode are realized by handling exceptions; handling of all exceptions is completed by the exception handling function stored in the exception vector table, and the address of the exception vector table is stored in the In the physical register; the register protection module replaces all instructions that modify the physical registers in the untrusted operating system by sending corresponding requests to the register protection module, and inserts a switching instruction (this instruction is the smc instruction) in the exception handling function maintained by the exception vector table. , is a standard instruction in the ARM processor, which can switch to the secure world and enter the register protection module), ensuring that all actions entering the untrusted operating system kernel will be intercepted by the register protection module;

Exit from the kernel state to the user state is completed by the eret instruction; the register protection module ensures that there is no instruction to exit the kernel state in the kernel code of the untrusted operating system, and all exit operations are forwarded to the register protection module to complete, so as to intercept all the exit cannot be completed. operation of the operating system kernel.

Preferably, the system call hijacking module implants a specific instruction (also called a specific hook, which refers to an smc instruction, which will enter the secure world) at the processing function for processing the exception triggered by the system call. , thus entering the system call hijacking module) to ensure that all system calls are handled by the system call hijacking module.

Preferably, the file system security enhancement module ensures the privacy and integrity of the container application file system, and at the same time ensures the access authority of the file system; wherein:

The file system security enhancement module utilizes the system call to hold the module, intercepts the file system access operations performed by all containers, and encrypts all file write operations; at the same time, for all file read operations, it performs corresponding decryption to ensure that the file is safe. privacy;

The file system security enhancement module maintains a piece of metadata for each container file, and the metadata includes the hash value and version number of the file content; each write operation to the file will update the hash value and version number, and The file system security enhancement module itself records the latest version number; when the container process performs a file read operation, the file system security enhancement module verifies the hash value and version number of the read content, and returns the read content to the container process to ensure that the integrity of the document;

The file system security enhancement module intercepts all system calls of the corresponding user of the modified container, and tracks the corresponding user of the current process; at the same time, additionally stores a permission information in the metadata of each container file, and for all intercepted file access operations, Permission verification is performed based on the permission information;

The trusted container image download module automatically creates the original data when downloading the image, and the file system security enhancement module modifies the automatically created original data according to the system call invoked by the user to form the metadata of each container file; these metadata use Each container is encrypted with a unique encryption key; the encryption key is protected by a file system security enhancement module and stored in a trusted storage medium before shutdown.

Preferably, the execution flow synchronization service security enhancement module controls the execution flow of the container process by intercepting the corresponding system calls when the execution flow synchronization service synchronizes different processes, analyzes the corresponding semantics, and prevents the execution flow from being tampered by an untrusted operating system.

Preferably, the inter-process communication service security enhancement module performs the following protection on the inter-process communication process according to the data transfer mode between the processes:

- For the direct data transfer method, the inter-process communication service security enhancement module identifies all communication channels by intercepting system calls, generates a communication key for each communication channel, and encrypts these communication channels during the communication process;

- For the data transfer method of shared memory, the inter-process communication service security enhancement module uses the page table management module to protect the shared memory; specifically, when the shared memory is established, the inter-process communication service security enhancement module informs the page table management module, Helps different container application processes to complete the establishment of shared memory; the page table management module ensures that no other process can map the physical memory page corresponding to the shared memory, thereby directly guaranteeing the security of the inter-process communication service based on shared memory.

Preferably, the trusted container image downloading module is used to verify the integrity of the downloaded container image, and at the same time initialize the container image to ensure the normal operation of the above-mentioned file system security enhancement module; specifically:

The trusted container image download module establishes a trusted network connection encrypted by the SSL protocol with the image repository, obtains the container image directly from the image repository, and verifies the integrity of the image by calculating the hash value; after the image download is complete, the trusted container The image download module performs the following initialization operations on the image:

Generate a unique image key for an image, and the image key is used to encrypt other keys in all images;

Traverse all files in the image, encrypt each file, calculate the hash value of the file content, and generate a metadata file; the processed container image is sent to the untrusted container management module for storage.

Preferably, the secure container startup module is used to ensure the security when the container is started, to ensure that the container runs correctly in a trusted execution environment, and to build a trusted communication channel for the container and the container management client; specifically:

When the container is started, the secure container startup module verifies the integrity of the boot image to ensure that the launched container application runs in a trusted execution environment; the secure container startup module will negotiate a communication key with the container management client, and use the communication key. The key encrypts all input and output of the container application.

Compared with the prior art, the present invention has the following beneficial effects:

1. The present invention can defend against all attacks from applications outside the container.

With the trusted execution environment provided by the present invention, any application outside the container cannot access the memory used by the application inside the container, and interferes with the execution state of the application inside the container. Therefore, the present invention can protect the application in the container and the privacy of the user, and is not infringed by the application outside the container.

2. The present invention can defend against all attacks from other containers.

The trusted execution environment of the present invention also isolates applications in different containers, ensuring that applications in different containers cannot access the memory, control state, etc. of other applications, thereby being able to defend against attacks from other containers.

3. The present invention can defend against all software attacks from untrusted operating systems.

The invention restricts the direct access of the untrusted operating system to the container application memory, registers, etc., thereby preventing the untrusted operating system from directly attacking the container application (directly stealing memory data, tampering with the application control flow, etc.).

At the same time, the present invention checks all the services that depend on the untrusted operating system, preventing the untrusted operating system from attacking the container application by providing malicious system services.

4. The present invention can defend against attacks from other user applications inside the container.

The present invention checks the communication and file sharing among different applications inside the container. In addition to preventing attacks from other user applications in the container, it can also prevent attacks on other user applications/data from collusion between applications in the container and untrusted operating systems.

5. The present invention proposes a method for constructing a trusted execution environment by utilizing the features of the ARM architecture, which can safely run existing application programs on a malicious operating system that is completely controlled by an attacker.

6. The present invention utilizes the aforementioned trusted travel environment, combined with the trusted service mechanism proposed by the present invention, so that different applications of different users in the container can perform secure communication and control flow synchronization.

7. The present invention automatically protects the container image from the Docker official warehouse, and the user does not need to make any modification to the existing image.

8. The present invention provides a security enhancement scheme based on TrustZone technology for the existing multi-user and multi-process container environment, which effectively prevents untrusted common world operating systems from attacking user containers. The modules such as the trusted execution environment and the secure file system mentioned in the present invention can also be used to enhance user program security in other environments.

Description of drawings

Other features, objects and advantages of the present invention will become more apparent by reading the detailed description of non-limiting embodiments with reference to the following drawings:

FIG. 1 is a system architecture diagram of an embodiment of the present invention;

FIG. 2 is a flow chart of downloading a trusted image according to an embodiment of the present invention;

FIG. 3 is a flowchart of a secure container startup according to an embodiment of the present invention;

FIG. 4 is a flowchart of a container process system call check according to an embodiment of the present invention;

FIG. 5 is a flowchart of a secure file system access according to an embodiment of the present invention;

FIG. 6 is a flow chart of using a security control flow synchronization service according to an embodiment of the present invention;

FIG. 7 is a flow chart of using a secure inter-process communication service according to an embodiment of the present invention.

Detailed ways

The embodiments of the present invention are described in detail below: This embodiment is implemented on the premise of the technical solution of the present invention, and provides detailed implementation modes and specific operation processes. It should be pointed out that for those skilled in the art, without departing from the concept of the present invention, several modifications and improvements can be made, which all belong to the protection scope of the present invention.

Example

This embodiment provides an ARM TrustZone-based container isolation enhancement system, which can effectively solve the problems encountered in the prior art, including:

How to use the characteristics of the ARM platform's own architecture to maintain any number of trusted execution environments for containers? ARM's TrustZone hardware technology provides only a single trusted execution environment called "Secure World". Simply running different container applications in the "secure world" cannot improve the security between containers. Therefore, how to use the "secure world" to provide an exclusive trusted execution environment for each container process is crucial.

How to prevent attacks on container applications while relying on untrusted operating systems to provide services? Applications (including applications in containers) rely on system calls provided by the operating system to complete many functions (such as file access, network communication, etc.). For a multi-user and multi-application container environment, different applications will also rely on the operating system for file sharing, execution flow synchronization, inter-process communication, and so on. An untrusted operating system may exploit these system services to steal private data in container applications, or manipulate the execution flow of container applications.

How to be compatible with existing container images? As a technology that has been widely used, there are millions of container images on the Internet. Users can easily download these images and start a container of their own. As the most widely used Docker container management tool, there are nearly one million Docker container images in the official Docker repository. Therefore, the designed container security system must be compatible with existing container images.

In order to prevent the untrusted operating system from directly accessing and tampering with the memory data of the application in the container and controlling the instruction flow of the application in the container, this embodiment will first create a trusted execution environment for each container application process. The TrustZone hardware technology of the ARM platform provides a trusted execution environment called "Secure World", the original ordinary operating environment is called "Ordinary World", and "Secure World" can access all hardware resources (memory, registers, external devices, etc.), while the "normal world" cannot access the hardware resources of the "secure world". Based on the "secure world" provided by the hardware, this embodiment first controls the page table mapping in the "normal world" to prevent the untrusted operating system from accessing the memory of the container application; then, this embodiment hijacks the relationship between the container application and the untrusted operating system switch, so as to ensure that the registers and control flow of the container application will not be tampered with. Through the above two methods, this embodiment can provide an exclusive trusted execution environment for each container application.

After that, this embodiment also needs to prevent the untrusted operating system from maliciously manipulating the system services provided to the container application, thereby stealing user privacy in the container application and tampering with the control flow of the container application. This embodiment mainly protects three types of system services:

File system: Container applications rely on file system services provided by the operating system. In this embodiment, all file system-related services called by the container application will be hijacked, and the files will be encrypted and hashed, so as to ensure the privacy and integrity of the files inside the container. At the same time, the operating system may also maliciously control file access rights, thereby leaking user privacy. In this embodiment, the file access rights of all container applications will also be reviewed.

Executing stream synchronization service: This embodiment considers a complex container environment with multiple users and multiple applications. In this environment, it is agreed that different applications inside the container need to rely on the services of the operating system to perform some cross-process execution flow synchronization operations. For example, by using IPC semaphore, different application processes will not access a resource at the same time, resulting in data race (data race), resulting in unpredictable execution results. An untrusted operating system can control these execution flow synchronization services, thereby interfering with the normal execution of applications in the container, and even further stealing the private data of applications in the container. This embodiment will hold and check the execution flow synchronization service called by the application in the container, including signal (signal) sending and receiving, semaphore, file lock, etc., to ensure the correctness of their semantics.

Inter-process communication service: Different processes in the same container can also communicate between application processes through pipes, IPC message queues, and IPC shared memory. Since most existing applications consider the operating system to be trusted, they rarely encrypt inter-process communication. This means that a compromised operating system can easily steal or even tamper with the content of any inter-process communication, thereby stealing data from the container process and hijacking the control flow of the container process. This embodiment will first hijack all inter-process communication. For the communication methods of "message passing" such as pipelines and IPC message queues, cryptographic methods are used to protect the communication content. As for the special communication mode of shared memory, this embodiment controls the page table to ensure that only legitimate memory sharing processes can access the corresponding shared memory, and an untrusted operating system cannot access any shared memory.

In addition to ensuring the security of the container runtime, this embodiment also needs to provide a secure container management function. Including two processes of trusted container image download and secure container startup. Trusted container image download first ensures the integrity of the container image downloaded from the official image repository, and also performs some initialization work for the downloaded container image. The secure container startup process will first verify the identity and integrity of the startup container, and negotiate a communication key with the container management client on the client side to ensure that after the container is started, the user can safely control his own container.

The technical solution of this embodiment will be further described in detail below with reference to the accompanying drawings.

A specific system embodiment of the present invention is shown in FIG. 1 . The client side runs a container management client to send commands to the container running on the server side. On the server side, ARM's TrustZone security extension divides the hardware into the normal world and the secure world. The normal world runs untrusted operating systems, untrusted container management modules, and any number of user containers. There may be multiple container processes in each container, and each container process runs in a trusted execution environment maintained by this embodiment.

In the security world, there are many modules that enhance security. The page table management module, the register protection module, and the system call hijacking module are responsible for maintaining multiple trusted execution environments. File system security enhancement module, execution stream synchronization service security enhancement module, and inter-process communication service security enhancement module are used to enhance the security of untrusted operating system services and prevent untrusted operating systems from providing malicious system services to steal container process private data, Tampering with container process control flow. The trusted container image download module and the secure container startup module provide secure container management functions.

The specific implementation of each module in this embodiment will be described in detail below:

【Container management client】

The container management application running on the user side is used to connect with the container on the server side and send user instructions to the container on the server side.

【Untrusted operating system】

Manage the hardware resources of the general world, and provide the running environment and required services for the application. Due to its huge amount of code, this module has many security holes, so it is more vulnerable to attack.

【Untrusted container management module】

It is used to perform basic management operations on containers, such as viewing CPU usage, starting containers, etc. Due to the untrustworthy operating system, this module is also vulnerable to attackers. Therefore, for the two key operations of container image download and container startup, the security enhancement is performed by the trusted container image download module and the secure container startup module in the secure world.

【Trusted Execution Environment】

In a trusted execution environment maintained by this embodiment, hardware resources such as memory and registers in the environment cannot be directly accessed by an untrusted operating system.

【Page table management module】

When the processor (CPU) accesses the memory, it needs to translate the virtual memory address to the physical memory address to read/write data from the physical memory. The page table is responsible for storing the mapping from virtual memory to physical memory, and the processor will automatically complete the conversion from virtual address to physical address according to the mapping relationship in the page table. Therefore, once you control the page table, you will also be able to control access to physical memory by any process (including the operating system). However, the page table management module of this embodiment first exclusively manages all page tables of the entire system, and then controls the page tables to protect the memory of the container process and restrict untrusted operating systems from accessing the memory of the container process.

In order to implement exclusive page table control by the page table management module, this embodiment needs to restrict the access of the untrusted operating system to the page table. On the ARM platform, the management of the page table (enabling a certain page table, turning off/on the page table function) needs to be implemented through some special instructions. This embodiment first searches for all these special instructions in the kernel of the untrusted operating system, and replaces them with sending corresponding requests to the page table management module. After that, the page table management module ensures that the page table page (the physical memory page where the page table is stored) is "read-only" to the normal world, while ensuring that the code of the operating system kernel is "not writable". So far, the page table management module has realized the "exclusive" management of the page table. For all page table modification operations, the untrusted operating system must forward it to the page table management module for completion.

After that, the page table management module will review all page table modification operation processes according to some security policies, such as: 1) All executable code in kernel mode must be marked as "unwritable"; 2) Anything that is mapped to a trusted execution environment The physical memory in the server cannot be mapped to the operating system; 3) Between different trusted execution environments, the corresponding physical page can be shared only after the shared memory operation completed by the "Interprocess Communication Service Security Enhancement Module".

【Register Protection Module】

The register protection module needs to ensure that the untrusted operating system cannot directly access or modify the register information of the trusted execution environment. First, the register protection module needs to intercept all switching between user mode processes and the untrusted operating system kernel in kernel mode. After that, during the switching process, the register protection module will be responsible for saving and restoring the registers of the trusted operating environment, so as to ensure that the untrusted operating system cannot arbitrarily tamper/steal the register information of the trusted operating environment.

For the two different switching operations of entering and exiting the untrusted operating system kernel, this module adopts different interception methods. First of all, under the ARM platform, all the operations of entering the untrusted operating system kernel from the user mode are realized by processing exceptions (execption). All exceptions are handled by exception handling functions stored in an exception vector table whose address is stored in a special physical register (VBAR). The register protection module first ensures that all instructions for modifying physical registers in the operating system are replaced by sending corresponding requests to the register protection module. After that, insert a switching instruction (the switching instruction refers to the smc instruction, which is a special instruction of the ARM processor) into the exception handling function maintained by the exception vector table, thereby ensuring that all actions entering the untrusted operating system kernel will be registered by the register. Protection module interception.

Exiting from kernel mode to user mode can only be completed by a few very limited instructions. Under the latest ARMv8 structure, an eret instruction is used to complete. The register protection module ensures that there is no instruction to exit the kernel state in the kernel code of the untrusted operating system, and all exit operations must be forwarded to the register protection module for completion. Thus, all operations that exit the untrusted operating system kernel can be intercepted.

【System call hijacking module】

The application uses the system services provided by the untrusted operating system through system calls. In order to audit these service processes, the system call hijacking module will ensure that all system calls initiated by the trusted execution environment will be captured by this module first.

Under the ARM platform, the system call is implemented through a specific "svc" instruction, and the execution will trigger a specific exception. This module will implant a specific hook (the specific hook refers to an smc instruction) at the exception handling function to ensure that all system calls will be processed by the system call hijacking module.

Note that the page table management module ensures that the code of the untrusted operating system cannot be modified, so the "hook" implanted in the exception handling function by the register protection module and the system call hijacking module cannot be tampered with by the untrusted operating system.

【File System Security Enhancement Module】

This embodiment protects three types of system services, namely, the file system, and implements a stream synchronization service and a set of inter-process communication services. The file system security enhancement module is responsible for ensuring the privacy and integrity of the file system of the container application, and at the same time ensuring the access rights of the file system.

In order to ensure the privacy of the file, this module uses the system call to hold the module, intercepts the file system access operations performed by all containers, and encrypts all file write operations. At the same time, for all file read operations, corresponding decryption will be performed.

In order to ensure the integrity of the file, this module maintains a metadata for each container file, which includes the hash value and version number of the file content. Each write operation updates the hash value and version number, and the module itself records the latest version number. When the container process performs a file read operation, this module first verifies the hash value and version number of the read content, and then returns the read content to the container process after the verification is passed.

In order to perform permission verification, this module first intercepts all system calls of the corresponding user of the modified container, and tracks the corresponding user of the current process. After that, this module will additionally store an additional permission information in the metadata of each file, and perform permission checking according to the permission information for all intercepted file access operations.

The metadata used in this module is automatically created by the trusted container image download module when downloading the image. At the same time, this module will also modify these metadata according to the system calls called by the user. This metadata will be encrypted with a container-unique encryption key and stored on the hard drive. The encryption key used will be protected by this module and stored in a trusted storage medium (eg RPMB provided by the ARM platform) before shutdown.

[Execution stream synchronization service security enhancement module]

The execution flow synchronization service is used to synchronize the execution flow between different processes, mainly including semaphores, locks, signals, etc. The execution flow synchronization service security enhancement module controls the execution flow of the container process by intercepting the corresponding system call and analyzing the corresponding semantics to ensure that it is not tampered with by an untrusted operating system.

For semaphores and locks, this module will intercept all corresponding system calls, including initialization, acquisition, and release of resources. For all acquisition operations, once the resources do not meet the requirements (the semaphore is insufficient or the lock is being occupied), this module will organize the current container process to continue to execute, thereby ensuring the semantic correctness of the stream synchronization service.

As for the special synchronization operation of signals, this module will intercept all signals inserted into the container process and check them. Each signal needs to have a corresponding trigger event before it can be inserted into the container process.

[Interprocess Communication Service Security Enhancement Module]

There are two main types of data transfer methods between processes. The first is direct data transfer, such as pipes and message queues. For this type of communication, this module uses encryption to protect. First, this module identifies all communication channels by intercepting system calls, then generates a communication key for each channel, and encrypts these channels in the subsequent communication process.

The second communication method is shared memory. Different processes can map their respective virtual memory spaces to the same physical memory space through system calls, so as to communicate based on the shared memory. Different from the inter-process communication method based on the communication channel, the shared memory only needs the help of the operating system when it is established, and does not need the intervention of the operating system in the communication. This means that it is difficult for this module to intercept the inter-process communication method based on shared memory.

Therefore, this module uses the page table management module to protect the shared memory. Specifically, when the shared memory is established, this module will inform the page table management module to help different container application processes complete the establishment of the shared memory. After that, the page table management module will ensure that no other process, including the operating system, can map the physical memory page corresponding to the shared memory. Thus, the security of the inter-process communication service based on shared memory is directly guaranteed.

[Trusted container image download module]

After ensuring the security of the container runtime, this embodiment still needs to ensure the security of the container image download and the container startup. The trusted container image download module is used to verify the integrity of the downloaded container image, and at the same time, initialize the container image to ensure that the above-mentioned file system security enhancement module can work normally.

This module first establishes a trusted network connection encrypted by the SSL protocol with the mirror warehouse, then directly obtains the container image from the mirror warehouse, and verifies the integrity of the mirror by calculating the hash value. After the image download is complete, this module initializes the image. First, an image-unique image key is generated, and the image key is used to encrypt other keys in all images. After that, traverse all the files in the image, encrypt each file, and calculate the hash value of the file content to generate a metadata file. The processed container image will be stored by the untrusted container management module of the ordinary world.

【Secure container startup module】

The secure container startup module is used to ensure the security of the container during startup, to ensure that the container runs correctly in the trusted execution environment maintained in this embodiment, and to build a trusted communication channel for the container and the container management client of the user.

When the container is started, this module will first verify the integrity of the startup image to ensure that the started container application runs in the trusted execution environment maintained by this embodiment. After that, this module will negotiate a communication key with the container management client, and use this key to encrypt all the input and output of the container application (the default input and output of the container application will be sent to the container management client)

The method embodiment of the present invention, based on the above-mentioned ARM TrustZone-based container isolation enhancement system, specifically includes: a trusted image download process, a secure container startup process, a container process system call checking process, a secure file system access process, and a security control flow synchronization process Service usage flow and secure inter-process communication service usage flow.

【Trusted image download process】

Step 1: Establish a trusted download channel. The trusted container image download module first establishes an SSL encrypted channel with the remote image repository to ensure the authentication of the image repository, exchange of channel encryption keys, and so on. Once the authentication of the image repository fails, the trusted image download process ends immediately.

Step 2: Download the container image. The trusted container image download module completes the download of the container image through the previously established trusted download channel.

Step 3: Verify image integrity. The integrity of the image can be verified by calculating the hash value of the image and comparing it with the image hash value obtained from the image repository. If the image completion is damaged, the download process will end immediately.

Step 4: Encrypt user files in the image. The trusted image download module first generates a unique encryption key for the image—the image key. This key is then used to encrypt all files inside the image.

Step 5: Generate file metadata. Traverse each file in the image, hash its content, and store metadata such as the hash value, initial version number, and file permissions in the corresponding metadata file. At the same time, the metadata file is encrypted with the mirror key. After the metadata is generated, the downloaded scene file will communicate with the container management module for normal initialization. This part of the work is a general technology of the existing container management tool, and does not belong to the innovative design of this embodiment, so it will not be described in detail here.

[Secure container startup process]

Step 1: Send a container start request. The container management client sends a container start request to the secure container start module through an SSL channel. The request includes startup instructions, startup image ID, startup application name, container application parameters, etc.

Step 2: Start preparations. The startup request is first forwarded to the untrusted container management module in the ordinary world, and the preparations for startup are completed, including: retrieving the startup container image, completing the initialization of the container namespace, and so on.

Step 3: Verify the boot image. After completing the preparations, the untrusted container management module will call a specific system call to start the first application of the container. At this time, the secure container startup module will intercept the system call to verify the integrity of the file system on which the current container is mounted. Once the current container image does not match the startup image specified by the user, or the files in the image are tampered with, the container startup process will be terminated immediately.

Step 4: Negotiate the communication key. After the image is verified, the secure container launch module negotiates a communication key with the container management client, and all interactions between the user and the container are encrypted using this key.

Step 5: Start the container application. After the secure container startup module, the container application will be run in the trusted execution environment maintained by this embodiment. This process includes page table initialization of the container application process, file system hanging, application code loading, and so on.

Step 6: The container startup is complete. Eventually, the operating system will switch to the execution entry of the container application and start executing the container application. The standard output of the container application will be sent to the container management client through the trusted channel. The default application standard input is also obtained from the container management client through a trusted channel.

[Container process system call check flow]

Step 1: The process initiates a system call. The container process invokes the system call by executing the "svc" instruction through the standard library function.

Step 2: The operating system catches the exception. This instruction triggers a characteristic exception that will trap into the operating system kernel. The kernel handles this exception through a specific exception handler.

Step 3: Switch to the interception module. In this embodiment, a "hook" is implanted at the entrance of the exception handling function of the untrusted operating system, which ensures that the exception will first be switched to the system call interception module for processing.

Step 4: Get the system call number. A system call uses a system call number to indicate the system function that the process wants to call. Under the ARM platform, the system call number is stored in the "X8" register. The system call interception module will obtain the system call number to determine the system call called by the process.

Step 5: Get system call parameters. According to different system calls, parse the parameters passed in by the system calls, and analyze the semantics of the parameters.

Step 6: Check for system calls. After obtaining the specific system call semantics, one-stage inspection of the system call can be performed. This stage includes checking whether the application has system call permission and recording the current system call.

Step 7: The kernel completes the system call. After the first-stage check is completed, switch to the untrusted operating system kernel to perform normal processing of system calls.

Step 8: Switch to the interception module. When the untrusted operating system kernel completes the system call and returns to the container process, the return operation is first intercepted by the system call interception module.

Step 9: Get the system call return value. After the interception module, the specific return value of the system call is obtained, and the type of the system call corresponding to the return value is determined according to the information obtained in the previous steps 4 and 5.

Step 10: Check the system call return value. Check whether the return value of the system call is legal according to different system call types.

Step 11: Return to the application process. After the check is completed, return to the application process, and this system call is completed.

【Secure file system access process】

File write operation process:

Step 1: Initiate a file write operation. The container application process invokes system calls to perform file write operations.

Step 2: Intercept the system call. The system call interception module intercepts the write operation, and first forwards the system call to the file system security enhancement module for processing.

Step 3: Access permission check. The file system security enhancement module first checks whether the current application process has the corresponding file access authority.

Step 4: Encrypt the write content. The file system security enhancement module first obtains the file encryption key corresponding to the target file. After that, the content of the file to be written is encrypted with the block as the granularity.

Step 5: Update the hash value. The file system security enhancement module calculates the hash value of the written file content with the block as the strength, and updates the corresponding hash value in the corresponding metadata file.

Step 6: Update the version number. The file system security enhancement module updates the current latest file version number, and writes the version number into the corresponding metadata file. At the same time, the file system security enhancement module itself will also save the version number.

Step 7: Complete the write operation. After updating the corresponding metadata, the encrypted file content is written to the target file, and the system call operation returns to the user process.

File read operation process:

Step 1: Initiate a file read request. The container application process initiates a file read system call to perform a file read operation.

Step 2: Access permission check. The system call interception module intercepts the system call, records the current system call, and forwards the current call to the file system security enhancement module to check the access authority.

Step 3: Read the encrypted file content. After the check is passed, the request is forwarded to the operating system for completion. The untrusted operating system reads the encrypted file and returns the read ciphertext to the container application process.

Step 4: Intercept the return of the system call. The system call interception module intercepts the return of the system call, and forwards the return to the file system security enhancement module for inspection.

Step 5: Decrypt the read content. The file system security enhancement module first decrypts the ciphertext of the read file according to the key corresponding to the file.

Step 6: Verify the hash and version number. The metadata corresponding to the file system security enhancement module reads the file, and compares whether the hash value of the decrypted file content is the same as the hash value in the file metadata. At the same time, compare whether the file version recorded in the file metadata is the same as the latest file version recorded by the file system security enhancement module.

Step 7: The read operation is complete. After both the hash value and the version number are verified, the decrypted original text of the file will be returned to the application process, and the file reading operation is completed.

【Use process of security control flow synchronization service】

Step 0: Create a control flow synchronization object. The container application first creates a control flow synchronization object through a system call. Afterwards, the control flow of different application processes is coordinated by acquiring/releasing the object.

Resource acquisition operation:

Step 1: Get the object resource. The application process initiates the control flow resource acquisition operation, which is completed through a special system call.

Step 2: Intercept the request. The system call hijacking module intercepts the system call and forwards it to the execution stream synchronization service security enhancement module for processing.

Step 3: Wait for the object resource. The security enhancement module that executes the stream synchronization service will maintain the corresponding resource values for different control stream synchronization objects (for example, mutex locks, the corresponding resource maximum value is 1, and semaphore, this value is the initial value of the semaphore). For each resource acquisition operation, the module judges whether the current resource value is sufficient.

Step 4: Successful get/wait. If the resource value is sufficient, execute the stream synchronization service security enhancement module minus the corresponding resource value to complete the acquisition request. If the current resources are insufficient, the fetch operation will enter the waiting queue until sufficient resources are available.

Step 5: Complete the acquisition operation. After the resource acquisition is successful, the application process will be returned, and the acquisition operation is completed.

Resource release operation:

Step 1: Release object resources. The container application process initiates a resource release operation through a system call.

Step 2: Intercept the request. The operation is first intercepted by the system call interception module, and forwarded to the security enhancement module of the execution stream synchronization service.

Step 3: Record the release operation. Executing the stream synchronization service security enhancement module will record this release operation, and release the resource value corresponding to the object. At the same time, a waiting resource acquisition process will also be selected, allowing it to continue the resource acquisition operation.

Step 4: Complete the release operation. After the resources are released, the application process will be returned, and the release operation is completed.

【Use flow of secure inter-process communication service】

Step 0: Create a communication channel. The container application makes a system call and first establishes a communication channel. For the special communication method of shared memory, once the channel is established, applications can communicate with each other without further steps. For communication methods such as pipes and message queues, the subsequent message sending and receiving processes are still required.

Step 1: Create a channel key. After the establishment of the communication channel is completed, the security enhancement module of the inter-process communication service maintains a unique communication key for the channel.

Message sending process:

Step 2: Send a message. The container application process sends messages through system calls.

Step 3: Intercept the request. The system call interception module intercepts the system call, and forwards the call to the interprocess communication service security enhancement module for processing.

Step 3: Encrypt message content. The security enhancement module of the inter-process communication service encrypts the communication content to be sent according to the key of the corresponding communication channel.

Step 4: Complete the send operation. The encrypted message content is sent by the untrusted operating system. After the sending is completed, return to the application process, and this message sending operation is completed.

Message acceptance process:

Step 2: Accept the message. The container application process accepts the content of messages sent by other processes through system calls.

Step 3: Accept the message ciphertext. Use the untrusted operating system to receive messages.

Step 4: The intercepted message is returned. After the message reception is completed, the system call return operation is intercepted, and the returned content is checked and processed by the inter-process communication service security enhancement module.

Step 5: Decrypt the message content. The interprocess communication service security enhancement module decrypts the received message.

Step 6: Finish accepting the request. The content of the disclosed message is sent to the application process, and the message acceptance process is completed.

The ARM TrustZone-based container isolation enhancement system provided in this embodiment:

1. It can defend against all attacks from applications outside the container.

With the trusted execution environment provided by this embodiment, any application outside the container cannot access the memory used by the application inside the container, and interferes with the execution state of the application inside the container. Therefore, the present invention can protect the application in the container and the privacy of the user, and is not infringed by the application outside the container.

2. It can defend against all attacks from other containers.

The trusted execution environment also isolates applications in different containers, ensuring that applications in different containers cannot access the memory, control state, etc. of other applications, so as to defend against attacks from other containers.

3. It can defend against all software attacks from untrusted operating systems.

Restrict direct access of untrusted operating systems to container application memory, registers, etc., thereby preventing untrusted operating systems from directly attacking container applications (directly stealing memory data, tampering with application control flow, etc.).

At the same time, all services that depend on untrusted operating systems are checked to prevent untrusted operating systems from attacking container applications by providing malicious system services.

4. It can defend against attacks from other user applications inside the container.

The communication between different applications inside the container, file sharing, etc. are checked. In addition to preventing attacks from other user applications in the container, it can also prevent attacks on other user applications/data from collusion between applications in the container and untrusted operating systems.

The technical scheme of this embodiment has brought the following beneficial effects:

1. Using the characteristics of ARM architecture, a method of constructing a trusted execution environment is proposed, which can safely run existing applications on a malicious operating system that is completely controlled by attackers.

2. Using the trusted travel environment described above, combined with the trusted service mechanism proposed by the present invention, different applications of different users in the container can perform secure communication and control flow synchronization.

3. Automatically protect container images from Docker's official repository, and users do not need to make any modifications to existing images.

In this example:

Use ARM TrustZone technology to create a trusted execution environment in the ordinary world that cannot be accessed by the operating system.

By removing key instructions, the ordinary world operating system is restricted from executing special privileged instructions, thereby realizing exclusive (exclusive) page table management.

Using the two different page table base address registers that come with the ARM architecture, isolate the virtual memory address space of the common world untrusted operating system and the easy application process, and prevent the common world operating system from accessing the physical memory pages allocated to the container application process .

Using TrustZone technology, it hijacks the switching between all container application processes and the ordinary world operating system, and further protects the context (registers, etc.) of the container application process, preventing the ordinary world operating system from hijacking the execution flow of the container application.

Utilize hardware features to ensure the security of the file system used by container applications.

By hijacking the switching between the container application process and the ordinary world operating system, it checks the access of all file systems, and encrypts and decrypts the reading and writing of files. At the same time, the hash tree ensures the integrity of the file content.

Use TrustZone to control file access permissions between different processes and different users in the container.

Use TrustZone technology to ensure the security of control flow synchronization services and inter-process communication services between different container application processes.

The control flow synchronization service system call initiated by the application process in the container is hijacked, and the TrustZone technology is used to provide a safe and reliable control flow synchronization service for the container.

It hijacks the inter-process communication service initiated by the application process in the container, binds encryption keys for different communication channels, ensures the security of communication data, and controls the access rights of communication channels.

A trusted container management service implemented with TrustZone technology.

Automated container image preprocessing method, compatible with existing images.

Use TrustZone technology to identify the container startup request initiated by the user, and verify the integrity of the container startup parameters to ensure that the started container is consistent with the user request.

Create a communication key for each individual container start request, and protect the security of the communication data between the two applications by hijacking the communication channel between the application and the user in the container.

The TrustZone technology is used to maintain a separate trusted execution environment for each container application process to protect the memory, registers and other data of the container application from being tampered with by ordinary world operating systems.

The implementation of the trusted file system includes hijacking the file system access operation initiated by the application, encrypting/decrypting the read and write content of the file, and controlling the access permission of the file in the container.

Trusted inter-process communication service and control flow synchronization service.

Secure container management services, including automated verification of the integrity of downloaded container images, automated construction of trusted container images, verification of the integrity of container startup parameters, and security protection of communications between users and containers.

Definitions of abbreviations and key terms in this example:

Containers: A lightweight virtualization technology. Using the interface provided by the operating system, a user-defined operating environment can be quickly started.

Docker: An open source container management tool that helps users manage container images and start, pause, and manage containers.

Docker image: It mainly contains a file system of a container, and Docker can start a container according to the Docker image.

Docker repository: A container image management platform, users can upload their own container images to the repository, and can also download container images shared by other users from the repository.

Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the above-mentioned specific embodiments, and those skilled in the art can make various variations or modifications within the scope of the claims, which do not affect the essential content of the present invention.

Claims (10)

1. A container isolation enhancement system based on ARM TrustZone is characterized by comprising:

the container management client runs on the client;

the system comprises an untrusted operating system, an untrusted container management module and a trusted execution environment, wherein the untrusted operating system, the untrusted container management module and the trusted execution environment run in the common world of a server side;

the system comprises a page table management module, a register protection module, a system call clamping module, a file system security enhancement module, an execution flow synchronization service security enhancement module, an interprocess communication service security enhancement module, a trusted container mirror image downloading module and a secure container starting module which run in a server-side secure world;

wherein:

the container management client is connected with any number of containers in the common environment of the server and sends user instructions to the containers; each container runs in a trusted execution environment;

the untrusted operating system provides a running environment and required services for the application program of the container;

the untrusted container management module provides basic container management operations;

the page table management module, the register protection module and the system calling clamping module jointly complete maintenance of any number of trusted execution environments;

the file system security enhancement module, the execution flow synchronization service security enhancement module and the interprocess communication service security enhancement module jointly complete the enhancement of the security of the services of the untrusted operating system, and prevent the untrusted operating system from stealing container process privacy data and/or tampering container process control flow by providing malicious system services;

the trusted container mirror image downloading module and the secure container starting module jointly complete the provision of the secure container management function.

2. The ARM TrustZone-based container isolation enhancement system of claim 1, wherein none of the hardware resources in the trusted execution environment are directly accessible to the untrusted operating system.

3. The ARM TrustZone-based container isolation enhancement system of claim 1, wherein the page table management module manages all page tables of the entire untrusted operating system exclusively, protects the memory of the container process by controlling the page tables, and restricts the untrusted operating system from accessing the memory of the container process;

the page table management module searches all instructions for managing the page table in the kernel of the untrusted operating system and replaces the instructions with the instructions for sending corresponding requests to the page table management module; the page table management module ensures that a physical memory page storing the page table is read only in the common world, and simultaneously ensures that a code of an untrusted operating system kernel is unwritable, so that exclusive management of the page table by the page table management module is realized.

4. The ARM TrustZone-based container isolation enhancement system of claim 1, wherein the register protection module ensures that untrusted operating systems cannot directly access or modify register information of trusted execution environments; the register protection module intercepts and captures switching between all user mode processes and a kernel of a kernel-mode untrusted operating system; in the switching process, the register protection module is responsible for finishing register storage and recovery work of the trusted operating environment, so that the untrusted operating system can not randomly tamper and/or steal register information of the trusted operating environment;

the switching between the user mode process and the kernel of the kernel mode untrusted operating system comprises the following processes:

all user states enter the operation of an untrusted operating system kernel, and are realized by processing the exception; processing all exceptions is done by an exception handling function stored in an exception vector table, the address of which is stored in a physical register; the register protection module replaces all instructions for modifying the physical registers in the untrusted operating system with instructions for sending corresponding requests to the register protection module, and inserts a switching instruction in an exception handling function maintained by an exception vector table to ensure that all behaviors entering the untrusted operating system kernel are intercepted by the register protection module;

exiting from the kernel state to the user state, and completing by adopting an eret instruction; the register protection module ensures that no instruction exiting the kernel state exists in the kernel code of the untrusted operating system, and all exiting operations are forwarded to the register protection module to be completed, so that all operations exiting the kernel of the untrusted operating system can be intercepted.

5. The ARM TrustZone-based container isolation enhancement system of claim 1, wherein the system call holding module implants a specific instruction at a processing function that handles exceptions triggered at the time of the system call, ensuring that all system calls are handled by the system call holding module.

6. The ARM TrustZone-based container isolation enhancement system of claim 1, wherein the file system security enhancement module ensures privacy and integrity of a container application file system, while ensuring access rights of the file system; wherein:

the file system security enhancement module intercepts file system access operations of all containers by using a system call clamping module, and encrypts all file write operations; meanwhile, all file reading operations are correspondingly decrypted, so that the privacy of the files is ensured;

the file system security enhancement module maintains a piece of metadata for each container file, wherein the metadata comprises a hash value and a version number of file contents; the hash value and the version number are updated every time the file is written, and the latest version number is recorded by the file system security enhancement module; when the container process reads the file, the file system security enhancement module returns the read content to the container process by verifying the hash value and the version number of the read content, so as to ensure the integrity of the file;

the file system security enhancement module intercepts system calls of users corresponding to all modification containers and tracks the corresponding users of the current process; meanwhile, additionally storing a piece of authority information in the metadata of each container file, and performing authority verification on all intercepted file access operations according to the authority information;

the trusted container mirror image downloading module automatically creates original data when downloading mirror images, and the file system security enhancement module modifies the automatically created original data according to a system called by a user to form metadata of each container file; these metadata are encrypted using an encryption key unique to each container; the encryption key is protected by the file system security enhancement module and is stored in the trusted storage medium before shutdown.

7. The ARM TrustZone-based container isolation enhancement system of claim 1, wherein the execution flow synchronization service security enhancement module analyzes corresponding semantics by intercepting corresponding system calls when the execution flow synchronization service synchronizes different processes, controls an execution flow of a container process, and prevents the execution flow from being tampered with by an untrusted operating system.

8. The ARM TrustZone-based container isolation enhancement system of claim 1, wherein the interprocess communication service security enhancement module performs the following protection on the interprocess communication process according to the data transmission mode between the processes:

for direct data transfer, the interprocess communication service security enhancement module identifies all communication channels by intercepting system calls, generates a communication key for each communication channel, and encrypts these communication channels during communication;

for the data transmission mode of the shared memory, the interprocess communication service security enhancement module protects the shared memory by means of the page table management module; specifically, when the shared memory is established, the inter-process communication service security enhancement module informs the page table management module to help different container application processes to complete establishment of the shared memory; the page table management module ensures that any other process cannot map the physical memory page corresponding to the shared memory, thereby directly ensuring the safety of the inter-process communication service based on the shared memory.

9. The ARM TrustZone-based container isolation enhancement system of claim 1, wherein the trusted container mirror image downloading module is configured to verify the integrity of a downloaded container mirror image and initialize the container mirror image to ensure that the file system security enhancement module operates normally; the method specifically comprises the following steps:

establishing a trusted network connection encrypted by an SSL (secure socket layer) protocol between a trusted container mirror image downloading module and a mirror image warehouse, directly acquiring a container mirror image from the mirror image warehouse, and verifying the integrity of the mirror image by calculating a hash value; after the mirror image downloading is completed, the trusted container mirror image downloading module performs the following initialization operations on the mirror image:

generating a mirror image key unique to the mirror image, wherein the mirror image key is used for encrypting other keys in all the mirror images;

traversing all files in the mirror image, encrypting each file, calculating a hash value of the file content, and generating a metadata file; and the processed container mirror image is delivered to the untrusted container management module for storage.

10. The ARM TrustZone-based container isolation enhancement system of claim 1, wherein the secure container start module is configured to ensure security of the container when starting, ensure that the container operates correctly in a trusted execution environment, and establish a trusted communication channel for the container and a container management client; the method specifically comprises the following steps:

when the container is started, the safety container starting module verifies the integrity of the starting mirror image and ensures that the started container application runs in a trusted execution environment; the secure container launch module will negotiate a communication key with the container management client and encrypt all inputs and outputs of the container application using the communication key.

Images