AUGMENTED ALLOCATION OF VIRTUAL MACHINES FOR APPLICATION

[0001] A virtual machine is code that hosts an operating system and potentially a set of one or more applications as well. The virtual machine executes in an environment provided by a host operating system on a host computing system (the “host”).

[0002] In operation, the virtual machine generates images and/or rendering instructions representing the user interface (such as a desktop or application user interface) to be rendered on the client, and causes the host to transmit the same to the client. The client receives the image and/or rendering instructions, and renders the user interface accordingly. The client also receives user input and transmits the same to the host. The virtual machine processes the input and changes the virtual machine state. If such change in state causes a change to the user interface, the changes are transmitted to the client for further rendering. This process repeats rather quickly, and the response time is sufficiently fast, that the user of the client might not be even able to distinguish whether the operating system and perhaps the associated applications are operated on the client or via virtual machine.

[0003] Alternatively, virtual machines may virtualize a computing system that does not include a user interface, in which case there is no such rendering of a user interface. For instance, virtual machines may represents computing nodes in a distributed application. In that case, a number of virtual machines may each represent computing nodes that each perform a certain role in the distributed application.

[0004] A host computing system (also referred to as a “host”) is typically capable of running a number of virtual machines. The host typically ensures isolation between the virtual machines. Each host has a hypervisor that uses underlying physical resources (such as network bandwidth, storage, memory, processing) to provide the appearance to the virtual machine as though the virtual machine has dedicated physical resources. This appearance is referred to as a virtual resource. Thus, each virtual machine may have virtual hard drives, virtual processors, virtual network bandwidth, virtual RAM and so forth.

[0005] At least one embodiment described herein relates to the starting up of an application involving multiple virtual machines. In response to a request to allocate a certain number of virtual machines corresponding to the application, an augmented number of virtual machines is determined to be prepared in response to the request. The augmented number includes both the initially requested number of virtual machines in addition to a surplus number of virtual machines. The augmented number of virtual machines are then caused to initiate startup. If there are already one or more virtual machines available for allocation, then the number of virtual machines to be started up may be offset by the number of virtual machines already started up.

[0006] As the virtual machines are started up, code is bound to the virtual machines. For instance, this code may be the customer code whereas the virtual machine is just a container or environment in which that customer code executes. Thus, because more virtual machines were initiated startup than are required for the application, the application code may be bound to all of the virtual machines in the application before all of the virtual machines have started up. This allows the application to be started up much more predictably as it is often the final few virtual machines that end up taking much longer than the mean (due to hardware and/or software flaws). The excess virtual machines may be allocated to other applications to fulfill subsequent requests.

[0007] This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.

[0013] In accordance with embodiments described herein, the overallocating of virtual machines for an application is described. In response to a request to allocate a certain number of virtual machines corresponding to the application, an augmented number of virtual machines is determined to be allocated in response to the request. The augmented number includes both the initially requested number of virtual machines in addition to a surplus number of virtual machines. The virtual machines are then caused to initiate startup if they are not already started up. Before all of the virtual machines are started up, and potentially as the virtual machines are started up, application code is bound to the virtual machines. Thus, because more virtual machines were initiated startup than are required for the application, the code may be bound to all of the virtual machines in the application before all of the virtual machine have started up. This result in much more predictable, and sometimes faster startup, of the application as a whole. The surplus virtual machines may be used for future application requests.

[0014] The binding may be performed first for virtual machines that are to perform more critical roles in the application. Furthermore, the amount of surplus virtual machines to be started up may be dependent upon the importance of the role to be played by the virtual machines.

[0015] Computing systems are now increasingly taking a wide variety of forms. Computing systems may, for example, be handheld devices, appliances, laptop computers, desktop computers, mainframes, distributed computing systems, or even devices that have not conventionally been considered a computing system. In this description and in the claims, the term “computing system” is defined broadly as including any device or system (or combination thereof) that includes at least one physical and tangible processor, and a physical and tangible memory capable of having thereon computer-executable instructions that may be executed by the processor. The memory may take any form and may depend on the nature and form of the computing system. A computing system may be distributed over a network environment and may include multiple constituent computing systems.

[0016] As illustrated in FIG. 1, in its most basic configuration, a computing system 100 typically includes at least one processing unit 102 and memory 104. The memory 104 may be physical system memory, which may be volatile, non-volatile, or some combination of the two. The term “memory” may also be used herein to refer to non-volatile mass storage such as physical storage media. If the computing system is distributed, the processing, memory and/or storage capability may be distributed as well. As used herein, the term “module” or “component” can refer to software objects or routines that execute on the computing system. The different components, modules, engines, and services described herein may be implemented as objects or processes that execute on the computing system (e.g., as separate threads).

[0017] In the description that follows, embodiments are described with reference to acts that are performed by one or more computing systems. If such acts are implemented in software, one or more processors of the associated computing system that performs the act direct the operation of the computing system in response to having executed computer-executable instructions. For example, such computer-executable instructions may be embodied on one or more computer-readable media that form a computer program product. An example of such an operation involves the manipulation of data. The computer-executable instructions (and the manipulated data) may be stored in the memory 104 of the computing system 100. Computing system 100 may also contain communication channels 108 that allow the computing system 100 to communicate with other message processors over, for example, network 110.

[0018] Embodiments described herein may comprise or utilize a special purpose or general-purpose computer including computer hardware, such as, for example, one or more processors and system memory, as discussed in greater detail below. Embodiments described herein also include physical and other computer-readable media for carrying or storing computer-executable instructions and/or data structures. Such computer-readable media can be any available media that can be accessed by a general purpose or special purpose computer system. Computer-readable media that store computer-executable instructions are physical storage media. Computer-readable media that carry computer-executable instructions are transmission media. Thus, by way of example, and not limitation, embodiments of the invention can comprise at least two distinctly different kinds of computer-readable media: computer storage media and transmission media.

[0019] Computer storage media includes RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store desired program code means in the form of computer-executable instructions or data structures and which can be accessed by a general purpose or special purpose computer.

[0020] A “network” is defined as one or more data links that enable the transport of electronic data between computer systems and/or modules and/or other electronic devices. When information is transferred or provided over a network or another communications connection (either hardwired, wireless, or a combination of hardwired or wireless) to a computer, the computer properly views the connection as a transmission medium. Transmissions media can include a network and/or data links which can be used to carry or desired program code means in the form of computer-executable instructions or data structures and which can be accessed by a general purpose or special purpose computer. Combinations of the above should also be included within the scope of computer-readable media.

[0021] Further, upon reaching various computer system components, program code means in the form of computer-executable instructions or data structures can be transferred automatically from transmission media to computer storage media (or vice versa). For example, computer-executable instructions or data structures received over a network or data link can be buffered in RAM within a network interface module (e.g., a “NIC”), and then eventually transferred to computer system RAM and/or to less volatile computer storage media at a computer system. Thus, it should be understood that computer storage media can be included in computer system components that also (or even primarily) utilize transmission media.

[0022] Computer-executable instructions comprise, for example, instructions and data which, when executed at a processor, cause a general purpose computer, special purpose computer, or special purpose processing device to perform a certain function or group of functions. The computer executable instructions may be, for example, binaries, intermediate format instructions such as assembly language, or even source code. Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the described features or acts described above. Rather, the described features and acts are disclosed as example forms of implementing the claims.

[0023] Those skilled in the art will appreciate that the invention may be practiced in network computing environments with many types of computer system configurations, including, personal computers, desktop computers, laptop computers, message processors, hand-held devices, multi-processor systems, microprocessor-based or programmable consumer electronics, network PCs, minicomputers, mainframe computers, mobile telephones, PDAs, pagers, routers, switches, and the like. The invention may also be practiced in distributed system environments where local and remote computer systems, which are linked (either by hardwired data links, wireless data links, or by a combination of hardwired and wireless data links) through a network, both perform tasks. In a distributed system environment, program modules may be located in both local and remote memory storage devices.

[0024] FIG. 2 abstractly illustrates an environment 200 in which the principles described herein may be employed. The environment 200 includes multiple clients 201 interacting with a system 210 using an interface 202. The environment 200 is illustrated as having three clients 201A, 201B and 201C, although the ellipses 201D represent that the principles described herein are not limited to the number of clients interfacing with the system 210 through the interface 202. The system 210 may provide services to the clients 201 on-demand and thus the number of clients 201 receiving services from the system 210 may vary over time.

[0025] Each client 201 may, for example, be structured as described above for the computing system 100 of FIG. 1. Alternatively or in addition, the client may be an application or other software module that interfaces with the system 210 through the interface 202. The interface 202 may be an application program interface that is defined in such a way that any computing system or software entity that is capable of using the application program interface may communicate with the system 210.

[0026] The system 210 may be a distributed system, although not required. In one embodiment, the system 210 is a cloud computing environment. Cloud computing environments may be distributed, although not required, and may even be distributed internationally and/or have components possessed across multiple organizations.

[0027] In this description and the following claims, “cloud computing” is defined as a model for enabling on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services). The definition of “cloud computing” is not limited to any of the other numerous advantages that can be obtained from such a model when properly deployed.

[0028] For instance, cloud computing is currently employed in the marketplace so as to offer ubiquitous and convenient on-demand access to the shared pool of configurable computing resources. Furthermore, the shared pool of configurable computing resources can be rapidly provisioned via virtualization and released with low management effort or service provider interaction, and then scaled accordingly.

[0029] A cloud computing model can be composed of various characteristics such as on-demand self-service, broad network access, resource pooling, rapid elasticity, measured service, and so forth. A cloud computing model may also come in the form of various service models such as, for example, Software as a Service (“SaaS”), Platform as a Service (“PaaS”), and Infrastructure as a Service (“IaaS”). The cloud computing model may also be deployed using different deployment models such as private cloud, community cloud, public cloud, hybrid cloud, and so forth. In this description and in the claims, a “cloud computing environment” is an environment in which cloud computing is employed.

[0030] The system 210 includes multiple host computing systems 211 (also referred to hereinafter as simply “hosts”), that are each capable of running virtual machines. Although the system 200 might include any number of hosts 211, there are three hosts 211A, 211B and 211C illustrated in FIG. 2, with the ellipses 211D representing that the principles described herein are not limited to the exact number of hosts that are within the system 210. There may be as few as one, with no upper limit. Furthermore, the number of hosts may be static, or might dynamically change over time as new hosts are added to the system 210, or as hosts are dropped from the system 210. Each of the hosts 211 may be structured as described above for the computing system 100 of FIG. 1.

[0031] Each host is capable of running one or more, and potentially many, virtual machines. For instance, FIG. 3 abstractly illustrates a host 300 in further detail. As an example, the host 300 might represent any of the hosts 211 of FIG. 2. In the case of FIG. 3, the host 300 is illustrated as operating three virtual machines 310 including virtual machines 310A, 310B and 310C. However, the ellipses 310D once again represents that the principles described herein are not limited to the number of virtual machines running on the host 300. There may be as few as zero virtual machines running on the host with the only upper limit being defined by the physical capabilities of the host 300.

[0032] During operation, the virtual machines emulates a fully operational computing system including an at least an operating system, and perhaps one or more other applications as well. Each virtual machine may be assigned to a particular client, and thus be responsible to support the desktop environment for that client. Each virtual machine might alternatively emulate a fully operational computing system that does not render a user interface. For instance, there may be a number of virtual machines that cooperatively interact to perform a distributed application in which each virtual machine performs a role in the application.

[0033] The host 300 includes a hypervisor 320 that emulates virtual resources for the virtual machines 310 using physical resources 321 that are abstracted from view of the virtual machines 310. The hypervisor 321 also provides proper isolation between the virtual machines 310. Thus, from the perspective of any given virtual machine, the hypervisor 320 provides the illusion that the virtual machine is interfacing with a physical resource, even though the virtual machine only interfaces with the appearance (e.g., a virtual resource) of a physical resource, and not with a physical resource directly. In FIG. 3, the physical resources 321 are abstractly represented as including resources 321A through 321F. Examples of physical resources 321 including processing capacity, memory, disk space, network bandwidth, media drives, and so forth. The host 300 may operate a host agent 302 that monitors the performance of the host, and performs other operations that manage the host.

[0034] Referring back to FIG. 2, the system 200 also includes services 212. In the illustrated example, the services 200 include five distinct services 212A, 212B, 212C, 212D and 212E, although the ellipses 212F represent that the principles described herein are not limited to the number of service in the system 210. A service coordination system 213 (or a “service fabric”) communicates with the hosts 211 and with the services 212 to thereby provide services requested by the clients 201, and other services (such as authentication, billing, and so forth) that may be prerequisites for the requested service. The service fabric 213 includes an overallocation policy engine 221, an allocator 222, and a binding engine 223, which may operate as described further below. Alternatively or in addition, one or more of the services 212 may perform some or all of the functions of some or all of the overallocation policy engine 221.

[0035] FIG. 4 illustrates a flowchart of a method 400 of starting virtual machines corresponding to an application. The method 400 will be described with frequent reference to the computing environment 200 of FIG. 2, and the host computing system 300 of FIG. 3.

[0036] The method 400 is initiated upon accessing a request to allocate a requested number of virtual machines corresponding to an application (act 401). For instance, referring to FIG. 2, the service fabric 213 may receive such a request from one of the clients 201 over the interface 202. The request might specify a number of attributes of the application including how many virtual machines are to be used in the application, as well as attributes of the virtual machines themselves. For instance, for each virtual machine, such attributes might include the size of the virtual machine, the fault domain of the virtual machine, the update domain of the virtual machine, and the code to which the virtual machine is to be bound.

[0037] Rather than initiate the startup of only the requested number of virtual machines that are to be ultimately used in the application, the method 400 determines an augmented number of virtual machines to allocate in response to the request (act 402). The augmented number is the sum of the requested number of virtual machines, in addition to a surplus number of virtual machines. For instance, if the requested number of virtual machines is to be 5 total virtual machines, it might be determined that 7 virtual machines are to be started up. Referring to FIG. 2, the overallocation policy engine 221 may determine the surplus number of virtual machines to be started up. The overallocation policy may take into consideration a number of factors including, for instance, a statistical history for time to start up a virtual machine of a particular type, an identity of a requestor, the number of available host computing systems, and percentage of utilization, and so forth. If there is not enough capacity in the datacenter, the number of surplus virtual machines can be reduced, including being fully removed. This avoids failing the customer deployment request of act 401 if there is not enough room to allocate the surplus.

[0038] The overallocation engine 221 may determine the number of virtual machines to overallocate depending on an importance of respective roles that are to be performed by the virtual machines. For instance, suppose that a distributed application is to be supported by 100 virtual machines (the example referred to hereinafter as the “primary example”). In this primary example, suppose that there is 1 virtual machine that is to serve in a critical coordination role that would render the application completely ineffective without that virtual machine. Now suppose there are 9 virtual machines in this primary example that are to serve in a role of data interpretation that have a medium level of criticality to the application. Suppose further that there are 90 virtual machines that are to serve as data mining virtual machines in this primary example. If all 90 data mining virtual machines are not operational, then perhaps that might influence the speed at which data is fed to the 9 data interpretation virtual machines, but otherwise, the functionality of the distributed application is not adversely affected.

[0039] Thus, in this primary example, there are three roles to be performed, a coordination role, a data interpretation role, and a data mining role. The role to be played by a virtual machine will be determined by the code that is bound to that virtual machine. For instance, a virtual machine to which coordination code is bound performs as a coordination virtual machine. A virtual machine to which data interpretation code is bound performs as a data interpretation virtual machine. A virtual machine to which data mining code is bound performs as a data mining virtual machine.

[0040] The overallocation engine 221 may be configured to take such underlying roles into account when determining how many surplus virtual machines to allocate. For instance, for the critical coordination virtual machines, even though only 1 virtual machine is needed, the overallocation engine 221 might allocate three virtual machines. Thus, the ratio of the surplus virtual machines to the requested virtual machines would be 2 to 1 in the case of the coordination virtual machines. For the medium criticality data interpretation virtual machines, even though only 9 virtual machines are needed, the overallocation engine 221 might allocate 15 virtual machines. Thus, the ratio of the surplus virtual machines to the requested virtual machines would be 6 to 9 (or two thirds) in the case of the data interpretation virtual machines. For the less critical data mining virtual machines, even though 90 virtual machines are needed, the overallocation engine 221 might allocate 100 virtual machines. Thus, the ratio of the surplus virtual machines to the requested virtual machines would be 10 to 90 (or one ninth) in the case of the data mining virtual machines.

[0041] Thus, the overallocation engine 221 may cause the extent of the overallocation to be determined in response to certain policy that takes into consideration the underlying role to be performed by the virtual machine. For instance, the ratio of the surplus number of the virtual machines to the requested number of the virtual machines may be determined based on a role to be played by at least one of the virtual machines. Such a ratio might be greater the more important or critical the underlying role of the virtual machine.

[0042] In some cases, it may be possible to bind different types of code to the same virtual machine. Thus, a particular virtual machine might play different roles depending on which of the multiple types of code is actually bound to the virtual machine. For instance, a virtual machine might be bound to either the coordination code or the data interpretation code if both code requires the same size of virtual machine and have other compatible requirements that allow for either code to be bound to that particular virtual machine. As an example, suppose that the 1 coordination virtual machine is to be 4 gigabytes, and be in certain fault domain and a certain update domain. Suppose further that the 9 data interpretation virtual machines are also to be of that same size, fault domain and update domain. Then, any virtual machines started up with that size, fault domain, and update domain, may be bound to either the coordination code or the data interpretation code.

[0043] In the case in which the same virtual machine may be bound to different code, the overallocation engine 221 might consider each of the multiple possible roles for a particular type of virtual machine in determining how many of those virtual machines to allocate. For instance, suppose that 10 virtual machines of 4 gigabytes size and the same fault and update domains are needed, 1 for the coordination virtual machine and 9 for the data interpretation virtual machines. The overallocation engine may determine that a surplus of 8 such virtual machines (or 18 virtual machines) is to be allocated. Note this would just be the sum of the individually calculated surpluses for each role (2 for the coordination role and 6 for the data interpretation role). However, the allocation engine 221 may take into consideration more complex calculations to determine the amount of surplus virtual machines to allocate.

[0044] Returning to FIG. 4, the method 400 then causes a particular number of virtual machines to initiate startup across a set of one or more host computing systems (act 403). For instance, in the case of FIG. 2, this allocation may be caused by the allocator 222 such that virtual machines are started up across one, some, or all of host computing systems 211. In the case of there not having any virtual machines already prepared, this particular number would be the same as the determined augmented number in act 402. However, there may already be virtual machines started up for reasons stated below, such as from a previous performance of method 400 in response to a previous application request. In that case, the particular number to be started up may be offset by the amount already started up.

[0045] For instance, in the primary example, suppose that coordination code may be bound to a one virtual machine of a type generically referred to as “type A” (e.g., a 4 gigabyte virtual machine of a particular fault and update domain), and that data interpretation code may be bound to 9 virtual machines of type A. In that case, the allocation engine 221 might have determined that the augmented number is to be 18 virtual machines of type A. If there are 3 type A virtual machines that are already started up, this act 403 would involve the startup of 15 additional type A virtual machines. Also, suppose that data mining code may be bound to a 90 virtual machines of a type generically referred to as “type B” (e.g., its size, fault domain and/or update domain is different than the type A virtual machines). In that case, the allocation engine 221 might have determined that the augmented number is to be 100 virtual machines of type B. If there are 5 type B virtual machines that are already started up, this act 403 would involve the startup of 95 additional virtual machines.

[0046] Note that binding of code to the virtual machines has not happened yet. The virtual machines of type A, and the virtual machines of type B are just initiated to start up. The binding occurs before all of the augmented number of virtual machines are started up. Thus, all virtual machines for the application may be bound to code, before all of the excess virtual machines have completed startup. Since there are often only a few stragglers that tend to take a lot longer than the majority of other virtual machines to start up, this results in a significant latency reduction between the initial request to set up the application, and the availability of the application to the requestor.

[0047] Each host agent may monitor startup completion for the various virtual machines on the corresponding host computing system, and provide notifications of completed startups to the binding policy engine 223. For instance, host agent 302 of FIG. 3 may monitor startup progress of virtual machines on the host 300 and report to the binding policy engine 223. In response to these notifications, the binding policy engine 223 binds code to started up virtual machines until the requested number of virtual machines have the appropriate code bound thereto to formulate the application. The binding decisions may be based on policy such as, for example, the criticality of the role, and the compatibility of the virtual machine with the role.

[0048] For instance, referring to FIG. 2, and applying the primary example, suppose that host 211A informs binding policy engine 223 that it has started up 1 virtual machine of type A and 40 virtual machines of type B. Since there is only 1 virtual machine of type A available (which could be bound to either coordination code or data interpretation code), and since the coordination role is more critical to the application than the data interpretation role, suppose the binding policy engine 223 binds the coordination code to the single available type A virtual machine. Thus, the binding policy engine 223 may prioritize in time the coordination virtual machine such that code is bound to create the coordination virtual machine before the less critical data interpretation virtual machines are created. Of course, there are not multiple roles competing for virtual machines of type B in the primary example as only data mining code may be bound to the virtual machines of type B in accordance with the customer request. Accordingly, the binding engine 223 may bind data mining code to the 40 virtual machines of type B to create 40 data mining virtual machines.

[0049] Suppose further that host 211B subsequently informs binding policy engine that it has started up 11 virtual machines of type A, and 30 virtual machines of type B. The binding policy engine 223 may then allocate data interpretation code to 9 of the virtual machines of type A. In addition, there are now 2 virtual machine of type A that is determined to be excess. Furthermore, the data mining code may be bound to the 30 virtual machines. Now, the highly critical coordination virtual machine is operational. In addition, the medium criticality data interpretation virtual machines are also all operational. Furthermore, 70 out of 90 of the less critical data mining virtual machines are operating. Thus, the application may actually be operational at this point, even though 20 more data mining virtual machines are yet to complete startup.

[0050] Now suppose that the host 211C notifies the binding engine 223 that 2 virtual machines of type A and 25 virtual machines of type B have been started up. The additional virtual machines of type A will not have code bound thereto, but may be registered as additional excess virtual machines of type A. Thus, there would now be recognized as being 4 excess virtual machines of type A. The allocation engine would allocate data mining code to 20 of the virtual machines of type B thus completing the 90 requested data mining virtual machines, and the excess 5 virtual machines of type B would be registered. At this point, the binding of code to the virtual machines have been completed (thereby completing act 404), and the application is fully functional for the client. Furthermore, there are 4 excess type A virtual machines that are not bound to any code, and 5 excess type B virtual machines that are not bound to any code.

[0051] Note that in this example, there were 18 virtual machines of type A that were initiated startup, and 14 virtual machines of type A have been started up. Thus, 4 virtual machines of type A have not yet finished starting up yet. Furthermore, in this example, there were 100 virtual machines of type B that were initially started up, and 95 virtual machines of type B have been started up. Thus, 5 virtual machines of type B have not yet finished starting up. Yet, even though some of the type A and type B virtual machines are still starting up, the application itself is complete, and may be made available to the requester. Since a small proportion of virtual machines take much longer than the majority (e.g., due to software and/or hardware issues), this means a significant reduction in latency between the time the application was requested, and the time that the application is available for use. As further notifications are received, the service fabric 213 may merely register such virtual machines as available to satisfy subsequent application requests.

[0052] Thus, a significant mechanism has been described in which complex applications may be made available more quickly after requesting. The present invention may be embodied in other specific forms without departing from its spirit or essential characteristics. The described embodiments are to be considered in all respects only as illustrative and not restrictive. The scope of the invention is, therefore, indicated by the appended claims rather than by the foregoing description. All changes which come within the meaning and range of equivalency of the claims are to be embraced within their scope.