MEMORY PROTECTION FOR VIRTUAL MACHINES

[0001] This application claims the benefit of U.S. Provisional Application No. 63/123,046 filed on Dec. 9, 2020, the entirety of which is incorporated by reference herein.

[0002] Embodiments of the invention relate to a computing system that provides memory protection to virtual machines.

[0003] A hypervisor enables multiple operating systems to run in parallel on a single physical machine. These operating systems, referred to as “guest operating systems,” can include multiple instances of an operating system as well as different operating systems. Multiple virtual machines (VMs) can run on top of the hypervisor. Each VM runs a guest operating system to manage resource allocation for the VM. The hypervisor typically uses a memory management unit (MMU) to support address translation and memory protection for the VMs. In a multiprocessor system, each processor core has its own MMU.

[0004] An MMU is responsible for translating virtual addresses to physical addresses. The MMU may include one or more translation look-aside buffers (TLBs) to store a mapping between virtual addresses and their corresponding physical addresses. The MMU provides a two-stage memory translation mechanism. Every memory access from applications running on a VMs undergoes a two-stage translation in the MMU. A guest operating system configures first-stage translation tables that map a virtual address to an intermediate physical address. The hypervisor configures second-stage translation tables that map the intermediate physical address to a physical address. Thus, the two-stage translation enables a hypervisor to control the guests' view of the memory and to restrict the physical memory that a guest can access.

[0005] MMU hardware can be complex and costly. Management of the MMU often requires highly complex software and incurs performance overhead. Thus, there is a need for developing a low-complexity and low-overhead memory protection scheme for a virtual machine system.

[0006] In one embodiment, a system is provided for supporting virtual machines (VMs). The system includes a memory addressable by addresses within a physical address (PA) space, and one or more processors that perform operations of the VMs. The VMs are allocated with extended PA regions outside the PA space. The system further includes a memory interface controller coupled to the memory and the one or more processors. The memory interface controller receives a request for accessing an address in the extended PA regions from a requesting VM, and uses a remap circuit to map the address in the extended PA regions to a remapped address in the PA space. A memory protection unit (MPU) in the memory interface controller grants or denies the request based on stored information indicating whether the remapped address is accessible to the requesting VM.

[0007] In another embodiment, a method is provided for controlling access to a physical address (PA) space in a system that supports multiple VMs. According to the method, a request is received from a given VM for accessing an address in extended PA regions that are outside the PA space and are allocated to the VMs. A remap circuit maps the address to a re-mapped address within the PA space. The request is granted or denied based on stored information indicating whether the remapped address is accessible to the given VM.

[0008] Other aspects and features will become apparent to those ordinarily skilled in the art upon review of the following description of specific embodiments in conjunction with the accompanying figures.

[0015] In the following description, numerous specific details are set forth. However, it is understood that embodiments of the invention may be practiced without these specific details. In other instances, well-known circuits, structures, and techniques have not been shown in detail in order not to obscure the understanding of this description. It will be appreciated, however, by one skilled in the art, that the invention may be practiced without such specific details. Those of ordinary skill in the art, with the included descriptions, will be able to implement appropriate functionality without undue experimentation.

[0016] Embodiments of the invention provide a memory protection mechanism to maintain memory separation among virtual machines (VMs). The memory protection mechanism uses hardware circuits for address re-mapping and memory protection. The hardware circuits manage memory separation of VMs such that the complexity of memory management units (MMUs) can be reduced. The hardware-based mechanism is fast and efficient and has a small footprint in the circuit area.

[0017] FIG. 1 is a block diagram illustrating a system 100 according to one embodiment. The system 100 includes multiple processors 115 in one or more processor clusters, such as processing clusters 110a, 110b, and 110c. The processors 115 may include central processing units (CPUs), graphics processing units (GPUs), digital processing units (DSPs), multimedia processors, neural processing unit (NPU), and/or other general-purpose and/or special-purpose processing circuitry. In some embodiments, the processor 115 may be referred to as a processor core, a microprocessor, a core, a multi-core processor, a computing unit, an execution unit, processing circuitry, or the like. The processor clusters 110a, 110b, and 110c are collectively referred to as the processor clusters 110.

[0018] The processors 115 in the processor clusters 110 may have the same or compatible Instruction Set Architecture (ISA), which enables them to execute instructions from the same or compatible instruction set. In some embodiments, the processors 115 in different processor clusters 110 may have different microarchitecture in the implementation of the same or compatible ISA. Examples of compatible ISA may include the ISA of the same processor architecture series, such as the RISC-V® series, the ARM® series, the MIPS® series, or other processor architecture series. The processors 115 in the processor clusters 110 can execute instructions independent of each other. In one embodiment, the processor clusters 110 may be part of a System-on-a-Chip (SoC) platform. It is understood the embodiment of FIG. 1 is simplified for illustration purposes. Additional hardware components may be included, and some of the components shown in FIG. 1 may be omitted.

[0019] In one embodiment, the processors 115 have access to a memory 120. The memory 120 may be the system memory or the main memory of the system 100. The memory 120 may include Random Access Memory (RAM) devices such as a Dynamic Random Access Memory (DRAM) device, a flash memory device, and/or other volatile or non-volatile memory devices. The processors 115 may access the memory 120 via a bus or an interconnect 130. Access to the memory 120 is under the control of a memory interface controller 125. In one embodiment, the processors 115 are operative to execute instructions stored in the memory 120 to perform virtual memory management and memory separation for VMs.

[0020] In one embodiment, the processors 115 also have access to peripheral devices 140, also referred to as I/O devices such as a keyboard, a speaker, a microphone, a display, a camera, etc. The peripheral devices 140 may be accessed via a bus or the interconnect 130 under the control of a peripheral interface controller 145. The peripheral devices 140 may include I/O devices and may be memory-mapped. For example, the peripheral interface controller 145 may include or control a device controller that is mapped to a physical address range in which I/O data speed, format, etc. are passed between the processor 115 and the device controller. In one embodiment, the processors 115 are operative to execute instructions stored in the memory 120 to perform virtual memory management and memory seperation for VMs regarding the access to the peripheral devices 140.

[0021] In one embodiment, each processor 115 uses a memory management unit (MMU) 116 to perform two-stage address translations from a virtual address (VA) space to an intermediate physical address (IPA) space, and from the IPA space to a physical address (PA) space. In one embodiment, the PA space may be a memory address space in a DRAM (e.g., the memory 120). The mapping between the VA space to the IPA space is managed by a guest operating system that runs on a VM, and the mapping between the IPA space to the PA space is managed by a hypervisor or a host operating system that manages the hardware resources of the system 100. In one embodiment, the system 100 further includes an MMU 117 shared by direct memory access (DMA) devices 160 that, among other functions, perform data transfers between the devices in the system 100.

[0022] The memory interface 125 includes a re-map circuit 126 and a memory protection unit (MPU) 127. The remap circuit 126 remaps an extended PA region to a re-mapped PA region in the PA space, where the extended PA region is outside the PA space and is allocated to a VM. The MPU is a hardware circuit including a local memory to store the access rights information of each VM with respect to the PA space. The access rights information may be stored in data structures referred to as permission filters. The MPU may use different permission filters for different VMs. In one embodiment, a permission filter stores an indication of one or more address ranges for a corresponding VM, where the one or more address ranges contain the physical addresses accessible by the VM. This data structure for a given VM is referred to as the permission filter for the given VM. For each address range, the permission filter may indicate the types of access rights; e.g., read, write, execute, etc. The address range may be a fixed-sized address block or a configurable-sized address segment. The MPU grants or denies a VM's access requests to a physical address based on the information stored in the corresponding permission filter.

[0023] In one embodiment, the peripheral interface controller 145 also includes a remap circuit 146 and an MPU 147, which perform analogous functions to the remap circuit 126 and the MPU 127. In one embodiment, the remap circuit 146 may perform the second-stage MMU operations to map a VM's address in the IPA space to the PA space. The MPU 147 also uses permission filters to store access rights information for the VMs, where the access rights are with respect to regions in the PA space that are allocated to controllers of the peripheral devices 140.

[0024] Thus, the remap circuits 126, 146, and the MPU 127, 147 provide memory separation of VMs by restricting the physical memory and the memory-mapped resources that a VM can access. This hardware-based memory protection mechanism has low complexity and low overhead.

[0025] Referring also to FIG. 2, which illustrates a software structure supported by the system 100 according to one embodiment. The hardware components described in connection with FIG. 1 are collectively referred to as hardware resources 250 in FIG. 2. In one embodiment, the system 100 supports multiple VMs 230. Each VM 230 runs a guest operating system 220 that manages a VA space and the allocation of an IPA space for applications 225. A hypervisor 240 creates and runs the VMs 230; e.g., by managing the scheduling of the hardware resources 250 for the VMs 230. In one embodiment, the hypervisor 240 may run on top of a host operating system 245; alternatively, the hypervisor 240 may be part of the host operating system 245. The hypervisor 240 manages the mapping between the IPA space and an extended PA space for the VMs 230. As will be shown in FIG. 3, the extended PA space is the PA space of the memory 120 plus the extended PA regions allocated to the VMs 230. The hypervisor 240 may configure the remap circuits 126, 146 and the MPU 127, 147 according to the allocation of physical addresses to each VM 230.

[0026] FIG. 3 is a diagram illustrating multiple stages of address mapping according to one embodiment. As mentioned before, a first-stage MMU translates a virtual address to an intermediate physical address, and a second-stage MMU translates the intermediate physical address to a physical address. In the second-stage MMU translation, physical addresses allocated to VMs are mapped into extended PA regions 320. For example, a virtual address allocated to VM0 is mapped by the second-stage MMU to R0, which is an extended PA region 320 allocated to VM0. In an example of four VMs, each VM is allocated with one extended PA region, and different VMs are allocated with different extended PA regions. All of the extended PA regions 320 are non-overlapping and have the same size. The extended PA regions 320 are outside of the PA space 310; the PA space 310 and the extended PA regions 320 are collectively referred to as an extended PA space 350. It is understood that the memory protection mechanism described herein is applicable to any number of VMs.

[0027] In one embodiment, different extended PA regions have different offsets from a remapped PA region 330 in the PA space 310. The offsets are also referred to as base address offsets. For example, the extended PA region (R0) allocated to VM0 has an offset (S0) from the remapped PA region; more specifically, the base address of R0 is offset from the base address of the remapped PA region 330 by S0. The extended PA region allocated to VM1 may have an offset (S1) from the remapped PA region 330, where S1=S0+the size of R0. The offsets for the extended PA regions allocated to VM2 and VM3 can be similarly calculated. Taking R0 as an example, the remapping of R0 to the remapped PA region 330 shifts each address in R0 by S0. The remapping of other extended PA regions can be similarly calculated.

[0028] In one embodiment, the extended PA regions 320 allocated to the VMs may not be immediately adjacent to the PA space 310; alternatively, the extended PA regions 320 allocated to the VMs may be immediately adjacent to the PA space 310. In one embodiment, the extended PA regions 320 allocated to different VMs may be consecutive in the extended PA space 350 as shown in FIG. 3; alternatively, the extended PA regions 320 allocated to different VMs may be non-consecutive in the extended PA space 350. In one embodiment, the base address and/or the size of each extended PA region 320 may be configurable.

[0029] Allocating the extended PA regions 320 to VMs enables a system to use different page sizes for a guest operating system and a host operating system. Suppose that VM0-VM3 run on top of multiple instances of a guest operating system in a computing system. The guest operating system may use a first page size for virtual memory management. The host operating system of the computing system can use a second page size that is greater than the first page size for virtual memory management. A greater page size can reduce the overhead of host system operations. On the other hand, a smaller page size may be more suitable for VM operations. The MMU of each processor may perform address translations using the first page size for host system operations and the second page size for VM operations.

[0030] FIG. 4 is a diagram illustrating memory protection according to one embodiment. In this example, four VMs (VM0-VM3) are allocated with extended PA regions (e.g., R0, R1, R2, and R3, respectively). The four extended PA regions have the same size (e.g., K bytes) and are non-overlapping. The base address offsets for the four VMs are indicated as S1, S2, S3, and S4, respectively. The remap circuit (e.g., the remap circuit 126 in FIG. 1) can map all four extended PA regions into one remapped PA region. In one embodiment, the remap circuit calculates the remapped address of a given address in an extended PA region (e.g., R0) by subtracting the offset (e.g., S0) from the given address.

[0031] Within each extended PA region, the hypervisor or the host operating system may allocate one or more address blocks and/or address segments to the corresponding VM. The size of the blocks may be fixed (e.g., 2 megabytes each), and the size of the address segments may be configurable. In the example of FIG. 4, VM0 is allocated with address block D0, VM1 is allocated with address blocks D11 and D12, VM2 is allocated with address block D2, and VM3 is allocated with address segment D3. All of the allocated address blocks and segments are non-overlapping when R0-R3 are mapped to the remapped PA region. In one embodiment, each address segment can be individually switched on (i.e., activated) or off (i.e., deactivated) for allocation.

[0032] Referring also to FIG. 1, the memory interface controller 125 may receive a request from a requesting VM (e.g., VM0) for accessing a requested address in the PA space. From the request, the memory interface controller 125 identifies a domain ID identifying the requesting VM as VM0. The remap circuit 126 identifies the base address offset of the extended PA region containing the requested address, and removes the base address offset from the requested address to obtain a remapped address in the PA space. The MPU 127 checks the permission filter of the identified domain ID to determine whether the request should be granted or denied. The permission filter associates VM0's domain ID with a list of address blocks and/or address segments that VM0 is allowed or denied access.

[0033] The description in connection with FIG. 3 and FIG. 4 applies to access control and protection of the peripheral devices 140 in FIG. 1. In one embodiment, the peripheral interface controller 145 may receive a request from a requesting VM for accessing an address allocated to a peripheral device. The peripheral interface controller 145 may use a peripheral remap circuit (e.g., the remap circuit 146) to map the address to a remapped address in the PA space. The peripheral interface controller 145 may use a peripheral MPU (e.g., the MPU 147) to determine whether to deny or grant the request based on the remapped address and information indicating addresses in the PA space that are accessible to the requesting VM.

[0034] FIG. 5 is a flow diagram illustrating a method 500 for memory protection according to one embodiment. In one embodiment, the method 500 may be performed by a system that supports VMs, such as the system 100 in FIG. 1 and FIG. 2. The method 500 starts at step 510 when the system receives, from a given one of the VMs, a request for accessing an address in extended PA regions that are outside the physical address (PA) space and are allocated to the VMs. At step 520, a remap circuit maps the address to a re-mapped address within the PA space. At step 530, an MPU grants or denies the request based on stored information indicating whether the remapped address is accessible to the given VM.

[0035] In one embodiment, the stored information is stored in a permission filter used by the MPU to associate a domain ID identifying the requesting VM with addresses in the PA space accessible to the requesting VM. The addresses may be indicated by: one or more address blocks having the same block size, one or more address segments having configurable sizes, or a combination of the one or more address blocks and the one or more address segments.

[0036] In one embodiment, the extended PA regions have the same size and are non-overlapping, and each extended PA region is allocated to a different VM. The remap circuit maps all of the extended PA regions to a remapped PA region containing contiguous addresses in the PA space. Each extended PA region has a base address offset from the remapped PA region in the PA space. The remap circuit calculates the remapped address by removing the base address offset from the address in the request. The base address offset is an offset between an extended PA region containing the address and the remapped PA region in the PA space

[0037] The operations of the flow diagram of FIG. 5 have been described with reference to the exemplary embodiments of FIGS. 1 and 2. However, it should be understood that the operations of the flow diagram of FIG. 5 can be performed by embodiments of the invention other than the embodiments of FIGS. 1 and 2, and the embodiments of FIGS. 1 and 2 can perform operations different than those discussed with reference to the flow diagram. While the flow diagram of FIG. 5 shows a particular order of operations performed by certain embodiments of the invention, it should be understood that such order is exemplary (e.g., alternative embodiments may perform the operations in a different order, combine certain operations, overlap certain operations, etc.).

[0038] Various functional components or blocks have been described herein. As will be appreciated by persons skilled in the art, the functional blocks will preferably be implemented through circuits (either dedicated circuits or general-purpose circuits, which operate under the control of one or more processors and coded instructions), which will typically comprise transistors that are configured in such a way as to control the operation of the circuity in accordance with the functions and operations described herein.

[0039] While the invention has been described in terms of several embodiments, those skilled in the art will recognize that the invention is not limited to the embodiments described, and can be practiced with modification and alteration within the spirit and scope of the appended claims. The description is thus to be regarded as illustrative instead of limiting.