System and method for binding virtual machines to hardware contexts

[0001] 1. Field of the Invention

[0002] This invention relates to virtual machines of a computer processor such as a microprocessor. In particular, the invention relates to a virtual machine manager that may bind virtual machines to hardware contexts of a processor in a computer or other computing device.

[0003] 2. Background

[0004] An Operating System (OS) is a software program that controls physical computer hardware (e.g., a processor, memory, and disk and CD-ROM drives) and presents application programs with a unified set of abstract services (e.g., a file system). A Virtual Machine Manager (VMM) is also a software program that controls physical computer hardware such as, for example, the processor, memory, and disk drives. Unlike an OS a VMM presents programs executing within a Virtual Machine (VM) with the illusion that they are executing on real physical computer hardware that includes, for example, a processor, memory and a disk drive. Each VM typically functions as a self-contained entity, such that software executing in a VM executes as if it were running alone on a “bare” machine instead of within a virtual machine that shares a processor and other physical hardware with other VMs. It is the VMM that emulates certain functions of a “bare” machine so that software executing within a VM executes as if it were the sole entity executing on the computer.

[0005] In order to accomplish this emulation it is necessary for some operations within a VM to be trapped and emulated by the VMM. The VMM may perform a sequence of operations on simulated hardware resources in order to maintain the illusion that a VM is actually interacting with real hardware. Transitions from a VM to the VMM and back will occur with some frequency, depending upon the number of tasks which the VMM must emulate for the VM. For example, a VMM must trap and emulate attempts to configure hardware devices. This may be achieved by the VMM via simulated hardware registers in system memory.

[0006] When executing “real-time” applications, computations upon data that is available at one substantially predetermined time should be completed by another substantially predetermined time. An OS that schedules a real-time application with sufficient frequency and for sufficient duration that the real-time application is able to complete its computations before their respective deadlines is said to have received adequate scheduling Quality of Service (QoS). Similarly, a VMM that includes a VM and a real-time OS (RTOS) provides adequate scheduling QoS when real-time applications and the VMs complete execution before respective deadlines. OSs and VMMs should schedule the computing resources of their real or virtual machine in such a fashion as to ensure that real-time applications receive adequate scheduling QoS.

[0007] Many current generation microprocessors such as, for example, the Intel® Pentium® 3 and 4 microprocessors include superpipelined out of order machines where instructions need not be executed in strict program order. While such processors typically have a number of independent execution units, they only fetch instructions from a single instruction stream. Some microprocessors, including some future members of the Pentium® family of processors, will have the ability to simultaneously fetch instructions from two or more instruction streams. These instruction streams are generally called threads because they correspond to threads scheduled by the system software. Microprocessors that simultaneously fetch instructions from two or more instruction streams are variously referred to as “hyper-threaded”, “multi-threaded” or “symmetric multi-threaded.” On hyper-threaded microprocessors the instruction fetch units are controlled by “hardware contexts”, which include both a stack pointer and an instruction pointer, a set of standard processor registers plus any additional state information necessary such as, for Pentium® family processors, control registers and translation look-aside buffer (TLB) tag bits. On out-of-order processors the nominal processor registers will generally be dynamically allocated from a pool of renameable registers, so that the actual static hardware context may be little more than a stack pointer, instruction pointer and a few control registers.

[0012] In many processors used in various kinds of computing devices, including personal computers, a virtual machine manager (VMM) may be implemented and used to manage multiple virtual machines (VMs) that are used to provide an environment for multiple operating systems to concurrently execute. The VMM provides emulation assistance to the VMs so that software (e.g., an operating system) in each VM believes that it has sole access to and thus control of the entire computing device. In reality, the VMM shares the resources of the computing device (e.g., the processor, memory and disk drive(s)) among the multiple VMs based on the respective requirements of the software executing in each VM and the availability and suitability of system and processor resources. Thus, there may be significant overhead incurred in switching among the various VMs. According to a method described herein, the VMM may bind certain VMs to certain hardware contexts in a hyper-threaded processor. According to the methods described herein, in certain situations, it may be beneficial to bind one or more VMs, particularly real-time VMs, to one or more hardware contexts for reasons including, but not limited to, maximizing efficiency and minimizing latency and response time.

[0013] Real-time virtual machines (RTVMs) are virtual machines that contain software applications that perform computations that must complete before a deadline or deadlines as measured according to an external reference clock (i.e., a so-called “wall clock”). Among the applications that an RTVM might contain are streaming media applications, including audio and video applications, digital signal processing and communications applications, including modem and networking applications. RTVMs may also be used in other instances. Because of the fine grained machine switch requirements placed on a virtual machine system by RTVMs, an undesirable amount of overhead may be generated when the VMM switches between various VMs and RTVMs. By binding RTVMs, and, in some instances, non-real-time VMs, to dedicated hardware contexts on a hyper-threaded processor, the overhead associated with such switching is reduced and performance of the virtual machine system is improved. Although the term “binding” is used herein, binding includes and/or is synonymous with, for example, but not limited to, assigning, aggregating, designating, allocating, distributing, or otherwise establishing and/or managing the relationship between VMs and hardware contexts.

[0014] FIG. 1 illustrates a conceptual diagram of the components involved in the invention described herein. In one embodiment, one or more virtual machines 110 and one or more real-time virtual machines 120 are managed by VMM 130. VMM 130 may permanently allocate some of hardware contexts 140 to specific virtual machines 110 and/or real-time virtual machines 120, and may dynamically allocate other hardware contexts to other virtual machines. Generally, a hardware context may be defined by various data required by a processor to execute a software task. In one embodiment, hardware contexts 140 may include an instruction pointer, a stack pointer, control registers, a micro-architecture to fetch, a micro-architecture to claim renameable registers and a micro-architecture to dispatch to shared execution units. The shared execution units may be an adder, a floating point multiplier accumulator (FMAC), and other execution units.

[0015] FIG. 2 illustrates a computer system in which one embodiment of the invention can be practiced. In various embodiments, the methods described herein may be implemented in any processor in any type of computing device, including personal computers, personal digital assistants (PDAs), servers, workstations, cellular telephones, laptops, computing tablets, and the like. In one embodiment, computer system 200 includes processor 210, system memory 220, storage device 230, graphics controller 240 and Universal Serial Bus (USB) host controller 250 all coupled to bus 260. In various embodiments, one or more instances of these components, as well as other well-known components may be included in personal computer 200. Personal computer 200 may also include peripheral buses such as Peripheral Component Interconnect (PCI), accelerated graphics port (AGP), Industry Standard Architecture (ISA) bus, I.E.E.E. 1394 (aka, Firewire), etc. in addition to or in place of USB host controller 250. In one embodiment, storage device 230 may be any machine readable medium such as, for example, optical and magnetic disk drives such as floppy disks, hard disks, compact disk read-only memory (CD-ROM) readable and writeable compact disk (CD-RW), stick and card memory devices, read-only memory (ROM), random access memory (RAM), flash memory devices, and the like, whether internal, external, accessible locally or remotely via a network, and accessible via electrical, acoustic, optical or other forms of propagated signals (e.g., carrier waves, initiated signals, digital signals, etc.).

[0016] In various embodiments, the methods described herein may be implemented as software, such as VMM software 232 stored on storage device 230. When executed by processor 210, VMM software 232 is stored as VMM 222 in memory 220. When managing real-time virtual machines, the hardware contexts and the real-time virtual machines themselves may be stored as RTVM 226 in memory 220, and may be stored in and/or swapped out to storage device 230 as RTVM 236. As shown in FIG. 2, while some RTVMs (such as RTVM-2, RTVM-3, and others through RTVM-Q) may be stored in memory, other RTVMs (such as RTVM-1, RTVM-4, and others through RTVM-P) may be concurrently stored in a storage device, such as storage device 230.

[0017] In one embodiment, the software executing on personal computer 200 and in the virtual machines may have access to and communicate with users of personal computer 200 by presenting text and images via graphics controller 240 which is coupled to display 242. Similarly, a user of personal computer 200 may provide input to those programs executing as virtual machines via keyboard 252 and mouse 254 coupled to USB host controller 250.

[0018] The processor 210 represents a processing unit of any type, including embedded processors, micro-controllers, digital signal processors, superscalar computers, vector processors, single instruction multiple data (SIMD) computers, complex instruction set computers (CISC), reduced instruction set computers (RISC), very long instruction word (VLIW) computers, hybrid architecture processors, and the like. In one embodiment, the processor 210 is compatible with the 32 bit Intel Architecture (IA) processor specification, known as IA-32. In another embodiment, the processor may be compatible with other instruction sets and/or processor architectures, including the 64 bit Intel Architecture known as IA-64. For further information regarding IA-32 and IA-64 see IA-32 Intel Architecture Software Developer's Manual, rev. 2 (February 2001) and Intel IA-64 Architecture Software Developer's Manual, rev. 1.1 (July 2000) available from Intel Corporation of Santa Clara, Calif. The processor 210 may contain a number of control registers to support memory management tasks such as virtual memory and cache memory. These tasks may include paging and segmentation. In various embodiments, personal computer 200 may include two or more processors like processor 210.

[0019] FIG. 3 illustrates one embodiment of a flow of actions taken by a virtual machine manager. The VMM first evaluates the bandwidth and latency requirements of one or more real-time virtual machines, as shown in block 310. Bandwidth and latency may be described as resource requirements of a virtual machine. Bandwidth is referred to by the variable X, and latency is referred to as the variable Y. Bandwidth refers to the amount of computing resources such as memory and other devices that are required by the particular virtual machine, and latency refers to how frequently the virtual machine requires access to the processor for execution. In one embodiment, bandwidth may be a tuple of values between 0 and 1, inclusive, which represent the fraction of corresponding resources the VM requires. In one embodiment, the tuple may be a quad tuple in which each element represents the fraction or percentage of processor, memory, display and disk resources required. For example, the tuple (0.5, 0.25, 0, 0.1) could represent a VM which requires 50% of the processor cycles, 25% of the system memory, 0% of the display (i.e., this VM is non-interactive) and 10% of the system disk bandwidth. The size of the tuple may vary depending on the components of a particular system. The resources represented in the tuple are not limited and are based on the hardware components of a particular system. In one embodiment, latency may refer to a period such as, for example, five milliseconds, or fifty microseconds, etc. which represents the longest duration for which the VM can be “held off” or delayed from execution. The VMM determines whether any real-time virtual machines are candidates for binding to one or more hardware contexts, as shown in block 320. The VMM binds one or more of the binding candidates; that is, real-time virtual machines, to one or more hardware contexts, as shown in block 330. Although the VMM is described in this paragraph as functioning solely with real-time virtual machines, the VMM may apply this same flow of execution to non-real-time virtual machines. Such an application may be desired if performance gains are to be realized. This is both system and processor dependent.

[0020] FIG. 4 illustrates a more detailed flow of actions of one embodiment of a virtual machine manager. In one embodiment, the VMM obtains a first virtual machine from a list of virtual machines and sets it as a current virtual machine, as shown in block 400. The VMM evaluates the bandwidth X of the current virtual machine, as shown in block 410. The latency Y of the current virtual machine is evaluated, as shown in block 412. In one embodiment, both the processor bandwidth X and latency Y may be expressed as milliseconds (ms.), such that the current virtual machine may be thought of as executing for X ms. every Yms. In this way, the processor bandwidth X may be represented as a duration of execution, while latency Y may be though of as a period of execution, both measured in predetermined time units (e.g., milliseconds). In an alternate embodiment, the bandwidth and latency may both be values between 0 and 1, representing, respectively, a fraction of the processor cycles and a fraction of a second. There are various ways to evaluate the bandwidth and latency described herein. One such technique is detailed in U.S. patent application having Ser. No. 09/541,444.

[0021] The VMM evaluates the single context execution time X′ of the current virtual machine, as shown in block 414. The single context execution time X′ is a value that is smaller than the bandwidth X; X′ is also referred to as nominal execution time as it takes into account the savings implicit in executing the current virtual machine as a single hardware context. The VMM evaluates the efficiency of executing the current virtual machine as a single hardware context by evaluating X′/Y, as shown in block 416. That is, to determine the efficiency of executing the current virtual machine as a single hardware context, the maximal use of processor resources is evaluated by dividing the single context execution time by the latency of the current virtual machine. The VMM evaluates the execution footprint of the current virtual machine, as shown in block 418. In one embodiment, the footprint may be determined by evaluating the amount of memory required for executing the current virtual machine. The footprint may be used in further evaluation of the current virtual machine.

[0022] The VMM checks to determine if there are any further virtual machines on a virtual machine list to evaluate, as shown in block 420. If there are additional virtual machines to evaluate, the VMM sets the current virtual machine to be the next virtual machine, as shown in block 422. The flow of execution continues at block 410.

[0023] After all virtual machines have been evaluated by the VMM in blocks 410 through 418, when there are no additional virtual machines on a virtual machine list to evaluate, as shown in block 420, execution continues at block 430.

[0024] The VMM ranks all virtual machines on the virtual machine list based on the lowest latency, that is, the smallest Y, as shown in block 430. In one embodiment, the ranking may be on a numerical scale in which the smaller the latency, the larger the ranking. The VMM may rank all virtual machines based on the maximal use of the hardware contexts, that is, the most efficient virtual machines, namely, those machines having the largest X′/Y, as shown in block 440. In one embodiment, the ranking may be on a numerical scale in which the more efficient the VM, the larger the ranking. The latency and efficiency rankings may be assigned according to any scale. The numeral scale described is only presented as an example. Other scales, such as by letter, and by any other kind of designator may be used.

[0025] The VMM selects a group of candidates for binding to a hardware context based on the latency and/or the efficiency, as shown in block 450. In one embodiment, the group of candidates may be determined as those top group of virtual machines that are ranked highest as determined in blocks 430 and 440. In various embodiments, the group of candidates may be a group of predetermined size, such as, for example, a group of two, a group of ten, or some other number. The VMM binds each of the group of candidate real-time virtual machines to a single hardware context if the latency of the VM is less than a system specified maximum latency and the efficiency of the VM is greater than a system specified minimum efficiency, as shown in block 452. In this way, if the group of candidates do not meet system specified threshold requirements, they are not bound to the hardware contexts. In another embodiment, this binding may proceed if only one of the latency and the efficiency are within system mandated constraints. In one embodiment, both the system specified maximum latency and the system specified minimum efficiency are set based on the speed of the particular processor. The system specified maximum latency and minimum efficiency may also be influenced by on-chip cache memory size, the size of all of the data needed for a hardware context, and other processor related factors.

[0026] The VMM determines whether any virtual machines have execution periods that are multiples of each other (i.e., are harmonic) such that they may share a hardware context, as shown in block 460. VMs that have execution periods such that one VMs execution period is an integral multiple of another VM's execution period do not conflict with one another are considered harmonic and may share a hardware context if there is sufficient processor bandwidth available (i.e., given VM A and VM B, X′_A/Y_A+X′_B/Y_B<=1.0). If there are virtual machines that have execution periods that may share a hardware context, each group is added to a shared context candidate list, as shown in block 460. In this way, multiple virtual machines may share a single hardware context. That is, for example, if a virtual machine has a latency of once every three milliseconds, and another virtual machine has a latency of once every five milliseconds, these two virtual machines may efficiently share a single hardware context. The VMM evaluates virtual machine groups on the shared context candidate list and assigns each of one or more groups of virtual machines to single hardware contexts, as shown in block 470.

[0027] In addition, the VMM may bind groups of virtual machines having small footprints to single hardware contexts when the groups have latencies that are multiples of each other. That is, virtual machines that have latency periods that are multiples of each other may only share a hardware context when the total size of their footprints do not exceed the amount of resources available. In this way, both the resources required by a first virtual machine and a second virtual machine that share a hardware context may remain in memory, or otherwise be accessible, such that when they are alternately executed at their particular periods, no swapping or other related overhead is incurred. Other more complex sharing and scheduling schemes may be implemented.

[0028] In the foregoing specification, the invention has been described with reference to specific embodiments. It will be evident that various modifications and changes can be made thereto without departing from the broader spirit and scope of the invention as set forth in the appended claims. The specification and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense.