Hardware-Implemented Hypervisor for Root-of-Trust Monitoring and Control of Computer System

[0001] This invention relates to computer security, and more particularly to hardware modifications to a host computer system to provide root-of-trust monitoring and control.

[0002] The complexity and stealthy nature of today's malware attacks make the task of securing computer systems a daunting one. The increased availability of malware techniques on the Internet has rapidly increased the frequency, number, and intricacy of attacks. Additionally, computer platforms and their applications continue to grow in complexity. While this complexity provides increased capabilities for users, it inherently leads to security vulnerabilities.

[0003] One approach to computer security is the use of hardware-based solutions that assign security functions to dedicated hardware. Because the functions on hardware are typically not directly accessible, it is difficult to change the hardware's security characteristics. Additionally, hardware-based security can reduce the overall performance penalty of monitoring while allowing real-time monitoring.

[0004] There are two approaches to hardware-based security: passive and active. Passive hardware can be useful for concealing secrets while performing security functions, but must be queried by the main system to be functional. Active hardware attempts to independently track the state of the system, and to report and correct abnormal behavior.

[0005] At a high level, active hardware monitors can be referred to as coprocessor intrusion detection systems (IDSs), and can be categorized as either loosely-coupled coprocessor IDSs or tightly-coupled coprocessor IDSs. Loosely-coupled IDSs do not reside at the same logical level as the host (monitored) processor, limiting the intrusiveness of the system as well as the data that it can access (e.g. cache, internal data buses, etc.). Tightly-coupled IDSs attempt to address the shortcomings of loosely-coupled IDSs by residing at the same or higher logical level as the host processor, allowing them equal or greater privileges than the host processor.

[0006] IDSs typically record information related to observed events, notify security administrators of important observed events, and produce reports. Many IDSs can also respond to a detected threat by attempting to prevent it from succeeding. They use several response techniques, which may involve the IDS stopping the attack itself, changing the security environment (e.g., reconfiguring a firewall), or changing the attack's content.

[0007] Another approach to computer security is the use of virtual machines (VMs) and virtual machine monitors (VMMs). VM technology allows an entire operating system (OS) to be “sandboxed”, limiting the effects that a compromised OS can have on the entire system. A VMM has control over the OS, providing numerous opportunities for system monitoring and malware prevention. VMMs have been developed that run on popular commodity OSs. They create a layer of software below the OS and tightly control the interfaces between the OS and VMM, reducing the ability of an attacker to subvert an entire system.

[0008] One of the flaws typically present in a host-based IDS is that advanced malware can subvert the IDS, rendering it ineffective. Moving the IDS to a VMM enables the detection of malware that would otherwise remain hidden. However, the complexity of the VMM and the fact that it is implemented in software still leave the possibility of subverting the system.

[0013] The following description is directed to a “hardware hypervisor” that provides a host computer system with security from malware. At the hardware level, the hypervisor is isolated from the rest of the system while maintaining control of the system's resources and providing protection from tampering. It combines the secret-hiding capabilities of a passive hardware system with active tightly-coupled access control capabilities and integrity checking of software layers.

[0014] FIGS. 1A and 1B illustrate the general concepts of a hardware hypervisor 11 and a software hypervisor 21, respectively. The system being monitored, referred to herein as the “host” or “primary” system, includes conventional computer system elements, such as a central processing unit (CPU) and operating system (OS).

[0015] As indicated in FIG. 1A, the hardware hypervisor 11 resides in hardware below the CPU 13 of the primary system 10. Its monitoring and control functions, performed by a monitor processor 12, are isolated from the CPU 13 and operating system 14. The hardware hypervisor 11 provides control of access to the system's resources and protection from malware running on the primary system. In contrast, as illustrated in FIG. 1B, in a system 20 having a software hypervisor 21, the hypervisor 21 resides in software between the CPU 23 and the operating system 24.

[0016] For the hardware hypervisor 11, the monitoring and control functions are designed with the assumption that all software is untrusted until proven otherwise. This leads to a protection approach that quickly responds to anomalous system behavior, rather than relying on fixed signatures. The hardware hypervisor 11 is extensible, and its detection and response mechanisms can be configured in software to provide a more general solution as compared to approaches that handle only one class of vulnerability.

[0017] Hardware hypervisor 11 may be implemented without making any modifications to the primary system software. There is no channel for the primary system 10 to access what is stored on the hypervisor 11, providing protection from malware running on the primary system 10. This approach provides a true hardware-based root-of-trust for the computer system 10, ensuring that system integrity is maintained between system boot cycles and enforcing user-defined system policies.

[0018] FIG. 2 illustrates a processor system 200, having a monitoring system (hereafter referred to as the “hypervisor”) in accordance with the invention. The processor system 200 is shown with various peripherals, which except for the hardware hypervisor elements described below, are typical of today's processor chips and chip sets. FIG. 2 is intended to primarily depict the elements relevant to the present invention; other peripherals not shown may also be a part of the processor system 200.

[0019] As explained below, the hypervisor comprises an isolated monitor processor 211 and a set of hardware alterations to system 200. These hardware alterations include a hypervisor communications interface (HCI) 213, a hypervisor behavioral interface (HBI) 215, and hypervisor access controller (HAC) 216. The co-processor 211 has a narrow hardwired communication channel 212 to the host processor system 200 via the HCI 213.

[0020] In the example of this description, the monitoring system is used with a system 200 having a shared bus interconnection. This interconnection scheme is an alternative to more recently developed point-to-point interconnections. Examples of point-to-point interconnections are the Hypertransport links associated with AMD processors and the Quick Path Interconnect links associated with Intel processors. In general, the concepts described herein may be applied to systems having either point-to-point or shared bus interconnections.

[0021] For point-to-point interconnection systems, it is desirable to install monitoring and control functionality in a number of locations. At one extreme, these systems could be enhanced with distributed access control at each router to minimize overhead created by denied messages propagating over multiple links and to maximize system control. However, it may be more practical to install access control at only a few strategic nodes in the system, such as adjacent to the processor and network interface controller. For purposes of the present invention, the hypervisor is extensible, so that specific locations of monitoring and control components can vary between systems.

[0022] In the example of this description, the host processor system 200 has an Advanced Microcontroller Bus Architecture (AMBA), which is a commercially available on-chip bus for system-on-a-chip designs. The AMBA has an Advanced High-performance Bus (AHB) 240 and an Advanced Peripheral Bus (APB) 230. The AHB 240 provides communications between the CPU 205 and “master” and “slave” peripherals. The APB 230 is accessed using a bridge 218 on the AHB 240, and communicates with peripherals.

[0023] A slave interface supports programmed I/O, so that master interfaces can write/read I/O registers. A master interface can also perform DMA transactions to and from system memory without heavily loading the CPU 205.

[0024] Host system 200 has several AHB master and slave peripherals of relevance to the present invention. A Debug Universal Asynchronous Receiver-Transmitter (UART) 206 is an AHB master that provides a client with read and write access to all memory-mapped locations on the AHB bus. As explained below, for the present invention, UART 116 provides an interface 213 to the co-processor monitor 211. Like the CPU, UART 116 is a “bus master”. It may be referred to generically as a “debug interface unit”.

[0025] A Joint Test Action Group (JTAG) Test Access Port (TAP) 207 is an AHB master that provides access to AHB peripherals using the JTAG protocol. A Debug Support Unit 219 is an AHB slave that exports internal processor signals and records AHB transactions. It should be understood that the hypervisor could be implemented on a different processor system having equivalent components, although the nomenclature of the components may be different.

[0026] To implement communications channel 212, one hardware alteration to the host system 200 is a hypervisor communication interface (HCI) 213, which uses UART 206 to permit the monitor coprocessor 211 to indirectly access data from the host CPU 205 or its peripherals. Ideally, co-processor 211 should be able to access all peripherals that the CPU 205 can access as well as the peripherals that comprise hypervisor (HBI 215 and HAC 216). The HCI 213 is simple, and most of its complexity comes from the security components used to protect its communications to and from the monitor coprocessor 211. In a preferred embodiment, monitor coprocessor 211 may be synthesized adjacent to HCI 213, on the same silicon, obviating any need for cryptographic channel protections.

[0027] A second hardware addition to host system 200 is a slave peripheral, the hypervisor behavioral interface (HBI) 215. HBI 215 monitors internal processor control signals and transactions on system interconnects and exports that information to the monitor coprocessor 211 via the HCI 213. As explained below, the hypervisor requires access to certain processor control signals to effectively monitor and control its security-sensitive operations. However, the set of exported signals is kept small to minimize the area and wire overhead of the hypervisor elements.

[0028] A third hardware addition to host system 200 is a slave peripheral, the hypervisor access controller (HAC) 216. HAC 216 is used by co-processor 211 to selectively permit or deny system interconnect transactions in accordance with a policy specification. It may also be necessary to modify interconnect controllers to cooperatively block transactions in a graceful manner.

[0029] The two AHB slave peripherals, HBI 215 and HAC 216, support the hypervisor functionality. Both are accessible by monitor coprocessor 211 using the UART interface 206, because their configuration registers are accessible over the AHB 240.

[0030] Monitor coprocessor 211 operates independently from the host processor 205 in that it uses separate reset lines, clocks, memories, and other core system peripherals. However, it may be synthesized onto the same silicon wafer used by host processor 205, which simplifies interconnection between the two, may reduce the cost of the complete system, and increases resistance to physical attacks by rendering system interfaces physically inaccessible.

[0031] Monitor coprocessor 211 monitors and controls the operation of the host processor 205, using the hypervisor hardware modifications described above. It reports the state of the host processor 205. It is capable of performing computations in parallel with the host processor 205, but is permitted to operate more slowly than the host processor 205. This is important because many of its applications are not sufficiently sensitive from a security standpoint to justify the added expense of a second processor that is as capable as the host processor.

[0032] Security Methods; HBI

[0033] In the example of FIG. 2, HBI 215 has a Memory Management Unit (MMU) tracer 215a. MMU tracer 215a extends the functionality of the Debug Support Unit (DSU) 219 by recording transitions in MMU control signals. In this example, host processor 205 uses a context table in conjunction with a context number to manage page table descriptors. The context table pointer and the context number are both stored in registers of MMU 205a. Whenever a new process is scheduled by the operating system, the context number value is modified. By monitoring changes in that register, the MMU tracer 215a can monitor process transitions.

[0034] A useful system metric associated with this functionality is the amount of processor time consumed by each process (in thousands of cycles, to prevent overflow). Thus, the MMU tracer 215a is configured to also record the amount of user-space processor time consumed between each transition.

[0035] It is important to exclude time spent in supervisor mode, because some kernels use process page tables even when it is running in supervisor mode. Thus, the signals that are exported from host processor 205 to the MMU tracer 215a are the MMU control signals as well as the supervisor-mode bit. Each transition is recorded in a 2-ported circular buffer so that reading from the buffer does not interfere with recording simultaneous process transitions. In this manner, the MMU tracer 215a provides reports to the co-processor 211.

[0036] Security Methods; HAC

[0037] HAC 216 selectively permits or denies individual transactions on the AHB 240. HAC 216 operates by checking each transaction between an AHB master device and an AHB slave device against two policies stored locally in registers. The first is a Device Access Control List (DACL) policy, and the second is an AHB access control region policy.

[0038] FIG. 3 illustrates how the DACL may be implemented with a matrix of master indices vs. slave indices. This matrix is possible because each AHB device, master or slave, has an index number associated with a transaction. At each location in the matrix, two bits indicate whether read and/or write access must be blocked.

[0039] The AHB access control region policy permits protection regions to be defined in terms of a base address and mask used to restrict read and/or write access by AHB transactions whose starting addresses fall within those regions (note that the actual transaction may extend beyond the regions). All policy entries are evaluated in parallel, because policy evaluation is on the critical path of the AHB 240.

[0040] FIG. 4 illustrates the data flow for AHB access control, as implemented by HAC 216. The HAC 216 intercepts each bus request and either grants or denies the request according to an access control policy. The access control policy is implemented with access control data in the form of a DACL or address regions, or both, as described above.

[0041] If the request is denied, the slave input signals are configured so that no slaves process the request, and a denial signal is sent to the AHB bus controller 208. The AHB bus controller 208 responds to the denial signal by setting the master signals to idle values.

[0042] Monitor Coprocessor Software

[0043] In the example of this description, the operating system for host processor 205 is a general-purpose LINUX version. The monitor coprocessor 211 runs Angstrom, a full-featured Linux version for embedded systems. However, in a production environment, a more efficient and minimal operating system can be used for coprocessor 211.

[0044] The monitor coprocessor 211 runs a software environment that provides interfaces to the hypervisor functionality in such a way that individual security functions can be encapsulated into payloads that are concurrently executed on the monitor coprocessor 211 while still being isolated and protected from interference by other payloads. The software environment is centered on a hardware interface library and a kernel introspection library.

[0045] The hardware interface library contains classes that can be used to access functionality from the HIB 215 and HAC 216. Ultimately, each interface uses the UART interface 206 to read and write data to and from specific memory-mapped locations on the AHB bus, but the other classes still provide significant value by abstracting away the low-level details of the AHB memory map.

[0046] The kernel introspection library provides support for interpreting the data structures in the kernel's memory. It can be easily extended, but it includes the following functionality: (1) enumerate all processes on the system, (2) look up a process by the command used to start it, (3) look up a process by its page table descriptor, (4) get the page table descriptor of a process identified by its Process Identifier (PID), and (5) translate a kernel symbol name to a virtual or physical address.

[0047] There are two primary influences on system performance resulting from the hypervisor. First, the monitor coprocessor 211 halts CPU 205 for around 581 ms while it initializes. Second, some performance degradation results from bus contention induced by requests from the kernel introspection library. However, due to the low speed of the link between the monitor coprocessor 211 and the host 205, this effect is negligible.

[0048] Example Methods

[0049] To validate the functionality of the system, a number of demonstration attacks and corresponding defenses were developed.

[0050] A first attack scenario involves a kernel-level rootkit that replaces the getdents and getdents64 system call pointers with pointers to malicious replacement functions. Those system calls are used to generate listings of directory contents, so the malicious functions are designed to remove files that match a particular number from file listings. This effectively removes any process whose PID matches the specified number from the output of the process status command.

[0051] The hypervisor can be used to detect the presence of this rootkit or any other rootkit that modifies pointers in the system call table. It accomplishes this by using debug UART 206 to periodically read the entire system call table and generate its digest using a cryptographic hash function. If at any time, the digest does not match the digest value that was computed when the payload (i.e., the software process resident on the co-processor) was first started, the payload indicates to its administrator that the system call table has been compromised. If the detection payload is started after the kernel has fully booted and before the rootkit is installed, and if its sampling period is short enough that the rootkit is still installed by the time it reads the system call table data, it can reliably detect such intrusions. A typical sampling period is 305 ms, but this could be reduced by increasing the bandwidth of communication channel 212.

[0052] A second method of using the hypervisor is using HAC 216 to prevent an intrusion process from completing in the first place. It accomplishes that by defining a write-protected region on the AHB bus such that no portion of the system call table can be written by any AHB transaction issued by the processor. Whenever such an access is attempted, the system crashes. In some applications, this is a preferable outcome to continued system operation in a compromised state, specifically when data confidentiality and integrity are valued more highly than availability.

[0053] A third method of using the hypervisor is using HAC 216 and its Discretionary Access Control List (DACL) to prevent the JTAG TAP 207 from accessing or changing data. It simply sets all bits in the DACL corresponding to the JTAG TAP 207 so that no read or write access is permitted.

[0054] A fourth method of using the hypervisor is using HBI 215 and its MMU tracer 215a to monitor process transitions without cooperation from the host system. The monitor coprocessor 211 periodically reads any new entries from a buffer of MMU tracer 215a and uses them to update statistics about the total user-space CPU time consumed by each process. It also uses its introspection library to obtain additional information about the processes being monitored when possible, such as the commands used to invoke them.

[0055] A fifth method of using the hypervisor is using access libraries to perform tasks that are not directly security-related. Using kernel introspection in conjunction with direct AHB memory access permits code deployment to the host processor, which is useful when it is more convenient to deploy code through the monitor coprocessor 211 than using another interface, such as a network interface.

[0056] For this method, two additional programs were developed that run on the host processor. The first is a code deployment framework, and the second is a stub process that transfers data from the monitor coprocessor interface to the payload deployment framework. The code deployment framework uses a socket interface to receive executable code encapsulated in a Java JAR file, and the stub process uses that interface to send code to the framework. The stub process allocates an empty memory region at a well-known location in its virtual address space. The monitor coprocessor 211 sequentially loads data blocks directly into that memory region using the introspection library to obtain the physical address of that region. It uses a simple protocol based on word-length control codes to coordinate the transfer of data chunks.

[0057] The above-described hardware hypervisor design allows for tightly-coupled IDS solutions to be developed by exporting VMM-like functionality through well-designed hardware interfaces.