TCP/IP offload engine virtualization system and methods

[0001] This application claims the benefit under 35 U.S.C. Section 119(e) of U.S. Provisional Application Ser. No. 60/773,895, filed on 15 Feb. 2006, to Mahalingam et al., entitled “TCP/IP Offload Engine Virtualization System And Methods,” which is incorporated herein in its entirety.

[0002] 1. Field of the Invention

[0003] The present invention is generally related to networked virtual computer systems and in particular to an architecture and methods of providing TCP/IP offload engine support in virtual computer systems.

[0004] 2. Description of the Related Art

[0005] Virtual computer systems are conventionally recognized as providing a variety of practical benefits, including more efficient use of hardware resources, improved opportunity for security and management control over executing applications, and the ability to support multiple discrete if not wholly independent execution environments. Consequently, interest in the architectural development of virtual computer systems, particularly in the areas of supporting new, performance-enhancing hardware, and extending those performance enhancements to the individual execution environments, is substantial and ongoing.

[0006] In summary, virtual computer systems are typically based on a conventional hardware platform providing one or more central processing units, a main memory, various persistent storage devices, and one or more network interface controllers (NICs), potentially of different design and functional capabilities. The hardware platform is used to support execution of a typically dedicated operating system kernel that, in turn, implements various virtualization drivers and services that enable multiple virtualization environments to be executed under the control of the virtualization kernel. A conventional host computer operating system can, in the alternative, be employed in place of the dedicated operating system kernel.

[0007] The virtualization environments supported by the kernel may be fully isolated execution spaces that, in turn, each encapsulate a network operating system instance and application program execution space. Each virtualization environment represents a discrete virtual machine (VM), and, as such, is often referred to as a guest computer system. Applications executed by the guest computer systems and their respective included guest operating systems are presented with the appearance and, in select circumstances, the fact of directly executing on the hardware platform. While vendors provide operating system drivers for the different, assembled components of the hardware platform, these drivers typically do not incorporate specific support for, or are capable of handling the complications arising from, potentially concurrent use by applications executing in multiple, independent virtualization environments. Therefore, the virtualization kernel is responsible for and generally implements the controls for coordinating access to the shared resources of the underlying hardware platform.

[0008] TCP/IP offload engines (TOEs) have been developed to improve the network access performance of computer systems in general. As the supported Ethernet network transmission speeds have increased to 1 Gbps and beyond, execution of the TCP/IP stack purely as a software component can impose a significant burden on the main central processing unit and restrict the actual network data throughput obtainable. TOEs typically implement a hardware TCP/IP protocol stack in combination with a hardware NIC as a platform pluggable hardware adapter. Recent generations of TOEs are nominally capable of supporting session establishment and a significant degree of error-handling services independent of the main central processing unit. Characteristically, however, TOE implementations must rely on a standard software TCP/IP stack, as implemented in a conventional operating system, as a fall-back to handle operating conditions—specifically complex protocol and error conditions—that are otherwise beyond the nominal capabilities of the particular TOE hardware implementation.

[0009] A variety of TOE-to-software TCP/IP stack interfaces are known to exist. In most cases, TOE vendors provide proprietary drivers and operating system service modules that will enable a specific TOE adapter to be utilized by a conventional, network-capable operating system, as typified by the major Linux® and Microsoft® operating system variants. Additionally, Microsoft has proposed a defined API, code-named Chimney, to support and define the fall-back coupling between a TOE and operating system kernel-based software TCP/IP stack. See, Scalable Networking: Network Protocol Offload—Introducing TCP Chimney, www.microsoft.com/whdc/device/network/TCP_Chimney. In all, the TOE drivers and service modules enable common network connections and data flows to be conducted through the TOE between an offload target, typically the TOE embedded NIC, and a transport driver, socket, or equivalent layer interface. Where the connection setup and data transport are without exception, utilization of the main central processing unit is minimal. Whenever a unhandleable TOE exception occurs, a protocol object representing the state of the connection and any in-transit data is transferred from the TOE hardware to a corresponding layer level within the associated software TCP/IP stack. This effectively transfers the exception condition to the full software stack for handling and recovery.

[0010] Conventional TOE driver and associated service module support is difficult in the context of virtual computer systems. While the more recent TOE implementations are capable of independently handling a wide range of protocol conditions and exceptions, the TOE functions must still be closely coordinated with and backed by a full capability software-based network stack. While the individual guest computer systems typically implement a full network stack as part of the guest operating system, there are practical performance constraints that limit use of these stacks in support of TOE implementations. “TCP/IP Offloading for Virtual Machines,” U.S. patent application Ser. No. 10/741,244, which is assigned to the assignee of the present application and hereby expressly incorporated by reference, describes an effective approach to supporting TOE adapters in a virtual computer system. There, each TOE implementation provided as part of the hardware platform is supported by a virtualization kernel-based network stack. In turn, each guest computer system implements a guest network stack bypass that enables direct communications with an assigned TOE implementation and virtual kernel stack. A common socket connection space is defined for the guest stack, assigned virtualization kernel stack and TOE implementation to establish and ensure the path integrity of network session connections. This defined relation is effectively required by the fact that a conventional TOE implementation cannot multiplex between the separate socket spaces that would need to be presented to different guest computer systems.

[0011] Although fully functional, the system described in “TCP/IP Offloading for Virtual Machines” may not make optimum use of the TOE adapters provided as part of the hardware platform, particularly subject to dynamically changing operating conditions. Some guest computer systems may require only a fraction of the bandwidth provided by an assigned TOE implementation while others would be best served in a virtualization environment that supports dynamic aggregation of multiple TOE implementations. Consequently, there is a need for a TOE virtualization system and methods of integrating one or more TOE adapters into a virtual computer system that enables concurrent use of TOE implementations by multiple guest computer systems and, further, adaptability for dynamically changing operating conditions.

[0012] Thus, a general purpose of the present invention is to provide an efficient mechanism for enabling dynamic, aggregate use of multiple TCP/IP offload engines (TOEs) among multiple virtual machines within virtual computer system architectures.

[0013] This is achieved in the present invention by providing a virtual computer system employing one or more TOEs for use by guest computer systems hosted on a virtual computer system. Each of the guest computer systems is provided with an offload selection switch. An associated virtual machine monitor includes a first virtual context component. Second virtual context components are associated with the TOEs and further associated virtual kernel-based TCP/IP stacks. The first virtual context components interoperate with the first virtual context components to establish a virtual routing of network connections between the offload selection switches and the TOEs. The virtual context mapping retains the initially requested network connection information as well as the resolved virtual network connection established, thereby allowing the initial network connection request to be internally reapplied as required to accommodate dynamic changes in the network protocol parameters of the TOEs.

[0014] An advantage of the present invention is that the guest computer systems have virtualized access to any and all of the TOEs provided as part of the hardware platform. Bandwidth requirements and hardware utilization as between the guest computer systems can be readily optimized.

[0015] Another advantage of the present invention is that the TOE virtualization provided by the present invention reduces the software virtualization overhead otherwise incurred in the execution of the virtual machines. With network virtualization implemented in the virtual machines, the TOE hardware is leveraged to implement most layer two processing and packet routing.

[0016] A further advantage of the present invention is that the virtualization of the TOEs is fully compliant with the standard network interfaces used by conventional application programs. Guest computer system-executed applications and their guest operating systems are able to transparently utilize the full available set of hardware platform TOEs. Required system administration to enable use of the TOEs by the individual guest computer systems is minimal.

[0017] Still another advantage of the present invention is that the virtualization of the TOEs fully accommodates point-to-point, broadcast and listener network sessions. All standard socket configurations and uses are supported. In support of TOE virtualization, the present invention further enables dynamic reconfiguration of the internal virtual connections between the guest computer systems and TOEs in response to, for example, dynamic IP changes. The parameters used in initially requesting the establishment of a network session are recorded as provided to allow reevaluation in response to dynamic changes to the networking protocol parameters assigned to the TOEs. The sessions request parameters may also be persistently stored to enable support for suspended and moved guest computer systems.

[0026] In enabling the virtualization of TOE adapters for use in a virtual computer system, the present invention fundamentally enables any guest computer system to access and fully utilize any TOE adapter incorporated as part of the virtual computer system. This is accomplished through a complete virtualization of the set of TOE adapters made available to the guest computer systems, whereby the TOE adapters are functionally marshaled as a single aggregated resource available to each of the guest computer systems. Furthermore, this virtualization of the TOE adapters is implemented while maintaining conformance with the standard network APIs implemented by conventional application programs' network operating systems.

[0027] The virtualization of TOE adapters provided by the present invention is essentially independent of the detailed implementation of the TOE adapters, which may vary in multiple ways. Different conventional TOE adapter implementations will support varying degrees of TCP/IP offload functionality and can require different degrees of support by an associated software-based network stack. The TOE adapters and associated vendor-supplied TOE drivers and can integrate in different ways with the associated software-based network stack and preferentially operate through any of the conventional network APIs, including the Transport Driver Interface (TDI) and the various interfaces provided by Microsoft Chimney, Windows Sockets Direct, and standard Winsock drivers. Additionally, physical TOE adapters may and often will host multiple TOE cores that logically operate as parallel arrays of single TOE adapters. For purposes of this description, an unqualified reference to a TOE adapter will refer to a single logical adapter as viewed by the virtual computer system. Thus, while the invention will be described with specific reference to single TOE adapters and the TDI interface, it should be understood that the breadth of the present invention includes the various alternatives outlined above.

[0028] Referring now to FIG. 1, the architecture of a virtual computer system 10 incorporating the present invention is shown. The virtual computer system 10 hosts any number of guest computer systems 12_1-N, each nominally including a virtual machine (VM) 14 and an associated interface software layer or component such as a virtual machine monitor (VMM) 16, which, in some literature or implementations, is referred to as, or as part of, a “hypervisor.” In the preferred embodiment of the present invention, the guest computer systems 12_1-N are hosted by a virtual machine kernel (VMKernel) 18 on a conventional hardware platform 20, including central processing, memory, and persistent storage units (not shown). The platform 20 also may include any number of conventional network interface controllers (NICs) 22 and TOE adapters 24_1-X. Characteristically, functionally conventional NICs 22′ are incorporated as embedded components of the TOE adapters 24_1-X. These embedded NICs 22′ can appear as conventional NICs 22 to the VMKernel 18 while simultaneously supporting the hardware TCP/IP stack implemented by the embedding TOE adapter 24_1-X. In each case, the NICs 22, 22′ ultimately serve as the connection interface to conventional communications networks 26, 28, 30, 32.

[0029] In the preferred embodiments of the present invention, each of the virtual machines 14 hosts a guest operating system 34 including a local network stack 36, NIC driver 38 and virtual NIC 40. Application programs 42 executed within the user space of the guest operating system 34 as well as components of the guest operating system 34 nominally utilize the stack 36 to establish network communications sessions. The virtual NIC 40 operates to establish a data packet transfer path to and through a NIC driver module 44 loaded within the VMKernel 18 to the NIC 22.

[0030] A virtual TOE switch 46 is implemented in each of the virtual machines 12_1-N, preferably in the form of a network component attached to the stack 36 using a standard interface. In a preferred embodiment of the present invention, the virtual TOE switch 46 is implemented as a network driver at the level of the TDI interface to allow evaluation and selective diversion of socket calls and related data transfer operations. Each call received through the TDI interface to create a socket is evaluated based on the provided socket number, the port number, and the identity of the requesting application or operating system component to determine whether the network session will be established through the stack 36 or routed through a virtual path to a TOE adapter 24_1-X connected to an appropriate network 26, 28, 30, 32. The criteria for diversion can be established administratively in the form of conventional network properties associated with the stack 36. The criteria can be specified differently for different guest computer systems 12_1-N. Thus, the switch 46 operates to logically divide the socket space presented to the applications 42 and operating system 34 between un-accelerated NIC network connections and TOE accelerated network connections.

[0031] For diverted socket creation requests, a network session will be established utilizing a virtual path connection from the switch 46 initially through a virtual TOE device implemented in part in the virtual machine monitor 16 as a virtual context component 48, in accordance with a preferred embodiment of the present invention. The virtual context component 48 is responsible for maintaining the appearance of a standard network session connection to the requesting application program 42 or operating system 34 component. The virtual context component 48 interoperates with the balance of the virtual TOE device, as implemented in the VMKernel 18 as a corresponding virtual context component 50_1-N, to establish a virtual connection path suitable to support the originally requested network session.

[0032] In the preferred embodiments of the present invention, the virtual content components 48 serve largely as a connection conduit to the corresponding virtual context components 50_1-N that, in turn, implement the driver functions of the virtual TOE devices. Specifically, the virtual context components 50_1-N are responsible for selecting and establishing a route to the TOE adapter 24_1-X capable of servicing a given session request. A hardware abstraction layer 52 operates as a function dispatch table to enable connections, through defined APIs, between the virtual context components 50_1-N and vendor-supplied TOE drivers 54_1-X. Various circumstances exist where an initially diverted network session must be routed through a conventional NIC 22. In these circumstances, the applicable virtual context component 50_1-N operates to route the connection through a TCP/IP stack 56 (not shown) resident in the VMKernel 18 to effect a software-emulated TOE capable of transparently supporting the diverted network session.

[0033] A detailed view 70 of the architecture 10 is provided in FIG. 2. As generally shown, the TOE intercept switch 46 operates to functionally implement one or more network call interfaces and divert selected network sessions, depending on management-determined criteria, to use a TOE accelerated network session path. In general, network sessions can be established in a number of different configurations, though typically configured to establish either a point-to-point communications path or to establish a listener open to receiving connections from any number of different clients. In both cases, a client or server application 42 initially requests, in conventional manner, creation of a socket through which to conduct the corresponding network session. Additional conventional calls to set desired socket optional parameters can and typically are then made. A bind or connect call is also made that provides a concrete IP specification and port request. Calls to listen and accept network connections further determine the particular behaviors implemented by the socket.

[0034] In accordance with the present invention, the virtual context component 48 receives and responds to the diverted application 42 provided socket calls consistent with the appearance of immediately establishing a network session. Configuration and routing information necessary to respond to the socket calls is preferably retrieved, as needed, from the local network stack 36. The socket call parameters asserted to define an application context for the requested network session are progressively accumulated in an internal context table 72, 74. These collected application context entries are preferably held until the corresponding network session is terminated.

[0035] The context table 72, 74 is preferably structured as an upper context table 72 that allows the applications 42 to call and query the parameters of a requested network connection as appropriate to support the conventional and expected behavior of a local network stack. The lower context table 74 functionally stores a mapped context and potentially multiple mapped contexts for each of the application context entries. A mapped or lower context entry is established when the parameters accumulated into an application or upper context can be sufficiently resolved to determine the required routing through and among the set of TOE adapters 24_1-X as ultimately necessary to establish the application requested network session. In the preferred embodiments of the present invention, each of the virtual context components 48 interoperates with their corresponding virtual context component 50_1-N to evaluate the currently available external routes supported by the TOE adapters 24_1-X and define an internal or virtual routing necessary to functionally connect the application context with an appropriate TOE adapter 24_1-X. The resolved routing information is stored as part of the corresponding lower context 74 entry and applied in the routing of data packets to and from the socket connected application 42. Preferably, a lower context entry includes an IP specification, port number, and MAC address, thereby allowing the virtual context components 48 to map between the socket space of the individual guest computer systems 12_1-N and the socket space supported by the TOE adapters 24_1-X as needed to avoid conflicts in the separate and concurrent use of the TOE adapters 24_1-X.

[0036] The context table 72, 74 is preferably shared with the virtual context components 50_1-N. In the establishment of a virtual path, a virtual context component 50_1-N is provided with the IP specification for the requested network session. For purposes of the present invention, an IP specification is defined as an IP address, IP address range, or INADDR_ANY IP identifier. This requested IP specification is stored as an entry in a TOE context table 76 and used to determine the nature of the corresponding network session created by the TOE adapters 24_1-X. Thus, for a given guest computer system 12_1-N that may be administratively associated with a subset of the TOE adapters 24_1-X, a restricted IP specification, reflecting the subset selection, is defined for the lower context entry. The original IP specification for the requested network session is instead used in creating the TOE network sessions on the subset of included TOE adapters 24_1-X. The virtual context component 50_1-N thereafter uses the contents of the TOE context table 76 to maintain the multiplexing of packet data paths to the subset selection of TOE adapters 24_1-X.

[0037] The context table 72, 74 is also preferably shared with the virtual context components 50_1-N to support persistence of the current network session state of the guest computer systems 12_1-N. A copy of the context table 72, 74 is preferably maintained in a state storage table 78. In response to periodic checkpoint events generated internally by the virtual machine monitors 16 or VMKernel 18, or in response to a suspend event signal provided from a persistence manager 80, resident internal to a virtual machine monitor 16, the contents of the state table 78 are transferred to persistent storage 82 provided by or through the hardware platform 20. In the preferred embodiment of the present invention, persistent storage 82 is a secure file on a local disk drive unit. On restoration of a checkpointed network session state, the TOE context 76 is reconstructed utilizing the context table 72, 74 data. To preserve the integrity of the context table 72, 74, diverted network sessions no longer routable through a TOE 24 are instead routed through the VMKernel 18 resident TCP/IP stack 56 to effect a software emulated TOE 24.

[0038] A preferred process of establishing a network session, in accordance with the present invention, is shown in FIG. 3. A conventional series of calls are made by an application program 42 to create 92 a communications session through a request-specified socket. The TOE intercept switch 46 determines the network interface and the related network parameters appropriate for use in creation of the network connection. The local stack 36 may be consulted 94 to determine an otherwise unused port value within the range assigned to the corresponding guest computer system 12_1-N. Set socket option calls are asserted to configure 96 the communications session. A bind socket call is then made by the application program 42 to establish 98 the session.

[0039] When the bind socket call is received, sufficient information has been accumulated by the TOE device virtual context components 48, 50_1-N to resolve 100 a virtual connection to an appropriate TOE adapter 24_1-X determined from the network routes handled by the set of TOE adapter 24_1-X administratively assigned or bound to the network session requesting guest computer system 12_1-N. Once the identity of the portal TOE adapter(s) 24_1-X is determined 100, the accumulated application context information is applied 102, preferably by replaying in order as socket calls to the participant portal TOE adapter or adapters 24_1-X. A corresponding network endpoint will thereby be established 104 by the participating TOE adapters 24_1-X. The network endpoint creation status is then returned 106 to the application program 42. The extension of the requested network endpoint through the virtual communications path created by the present invention to enable aggregate use of the set of TOE adapters 24_1-X is thus entirely transparent to the application programs 42.

[0040] The present invention also supports dynamic alteration of the virtual network communications paths, particularly in response to protocol parameter changes applied or enforced on any of the TOE adapters 24_1-X. Where, for example, the IP address of a TOE adapter 24_1-X is established using the conventional dynamic host configuration protocol (DHCP), a new IP address may be dynamically assigned to a TOE adapter 24_1-X while the adapter is actively hosting one or more network sessions.

[0041] In response to a changed network protocol parameter that would affect the integrity of a virtual path, the present invention implements a dynamic virtual path reconfiguration process 110, as illustrated in FIG. 4A. Conventional network stacks natively implement a standard interface that provides notices of such changes. The network stacks 36, 38, 40 resident in the guest computer systems 12_1-N will report network protocol changes relative to the NICs 22, 22′ configured to be visible to a corresponding guest computer system 12_1-N. These notices are propagated to the TOE device virtual context components 50_1-N for evaluation. Where the network protocol parameter change affects an existing virtual network connection path, a determination 112 is made to reestablish the network session consistent with the changed protocol parameter. Where the changed parameter is the IP address assigned to a TOE adapter, which is the predominant case and generally illustrative, the lower mapping is re-resolved 114 to reflect the changed IP address. In the typical case, only the endpoint IP address defined for the network session needs to be changed. Provided the corresponding vendor TOE driver supports dynamic updating of the network session control blocks, the vendor TOE driver is directed 116 to implement the change. Normal use of the network session then continues.

[0042] In cases where the vendor TOE driver does not support dynamic changes to network session control blocks or a dynamic updating of any one or more of the changed protocol parameters, the process 130 shown in FIG. 4B is implemented. In connection with the re-resolving of the virtual network connection paths 132, the changes to the protocol parameters are incorporated into the virtual context table 72, 74 mapping. The affected network sessions are then closed and, based on the virtual context mapping table 72, 74, updated network sessions are established 134. Again, the operation of the present invention in adapting to protocol changes enforced on the TOE adapter 24_1-X maintains the aggregate use of the set of TOE adapters 24_1-X entirely transparent to the application programs 42.

[0043] FIG. 5 illustrates the establishment 140 of a virtual network connection path for a point-to-point network session in accordance with the present invention. The initial request to establish the network session defines the target IP address and port. The target port may be privileged or unprivileged. The port opened to the guest operating system 12_1-N may be any available unprivileged port where the application 42 is administratively assigned for diversion. Alternately, the opened port may be one in a range of unprivileged ports administratively assigned for diversion. The TOE device virtual context components 48, 50_1-N function to determine and apply a mapping that enables an internal, virtual re-targeting of the connection request. The connection request is internally targeted to any available unprivileged port of a TOE adapter 24_1-X that supports a route to the original target IP address. Where multiple TOE adapters 24_1-X support a usable target IP route, the virtual TOE components 48, 50_1-N elect one as the virtual target. Conventional techniques can then be used for locating an acceptable unprivileged port for use as the virtual target port. When the TOE adapter 24_1-X and port are identified, the mapping is completed and the network session is created on the identified TOE adapter 24_1-X using the application context to provide the target IP address and port.

[0044] Establishment of a multiplexed virtual network connection path 150 is shown in FIG. 6. Typically, an application program 42, to operate as a server, will open a local privileged port for listening with a target IP specification of INADDR_ANY. For example, the application program 42 may operate as a HTTP server on privileged port 80. In accordance with the present invention, the TOE device virtual context component 48, 50_1-N supporting the virtual machine 12_X containing the application program 42 creates multiplexed virtual connection paths to the service participating TOE adapters 24_1-X; that is, to the TOE adapters 24_1-X assigned or otherwise accessible by the virtual machine 12_X. In creating these paths, the TOE device virtual context component 48, 50_1-N resolves and establishes the individual paths using the IP address assigned to the individual participating TOE adapters 24_1-X. Map entries are created in the context table 72, 74 identifying each of the participating TOE adapters 24_1-X, the IP address of the virtual machine 12_X hosting the application program 42, and the port number of the opened local privileged port. Each of the participating TOE adapters 24_1-X is then directed to listen for network sessions requests on each of the privileged port instances. Where the port is already open on any of the participating TOE adapters 24_1-X, the port is opened in a shared mode. An external client access through any of the networks connected to the participating TOE adapters 24_1-X is then effectively routed over a branch of the multiplexed virtual paths to the server application 42. The client access appears as an ordinary network connection on the local privileged port.

[0045] A preferred embodiment of the present invention enables transportability of network sessions through a process of persistently storing and restoring the mappings contained in the context tables 72, 74. A advantageous feature of virtual computer systems is the ability to suspend and restore the working state of individual guest computer systems 12_1-N. The present invention allows the state of a guest computer system 12_1-N, specifically in regard to network sessions, to be suspended even in the presence of active network sessions. As indicated in FIG. 7, the current state of a guest computer system 12_1-N is preferably checkpointed 162 periodically and in response to the operation of the persistence manager 80. When a checkpoint 162 occurs, the virtual TOE components 48, 50_1-N interoperate 164 to update 166 the state store 78 and then persist the mapping 168 information.

[0046] On a restore 170 from a suspended state, as shown in FIG. 8, at a checkpoint 172 defining a consistent system state pending the reactivation of a suspended a guest computer system 12_1-N, the corresponding virtual TOE components 48, 50_1-N are selected 174, and the context table 72, 74 mapping information retrieved 176. Each of the network sessions identified by an application context in the context table 72, 74 mappings is then reestablished 178. Since the application contexts include the complete set of socket configuration calls and related parameter data, new corresponding virtual connection paths appropriate to the then current set of TOE adapter 24_1-X can be resolved and established, with the result that the full state of the network sessions can be restored transparent to the applications 42.

[0047] The restore 170 from a suspended state may occur following a transfer and hosting of a guest computer systems 12_1-N by a different hardware platform 20. In this case, there is no assurance that the various network sessions can be reestablished utilizing the same or directly equivalent NICs 22 and TOE adapters 24_1-X. Preferably, the application contexts are further used to store identifications of the network session types. A network session previously using NIC 22 will be reestablished, depending on network routing requirements, through a NIC 22, 22′ on the new hardware platform 20. Network sessions previously utilizing the TOE adapters 24_1-X will be reestablished directly through TOE adapters 24_1-X on the new hardware platform 20 or through software emulated TOE adapters using the kernel TCP/IP stack 56 and a routing appropriate NIC 22.

[0048] Thus, a system and methods for enabling the separate and aggregate use of TOE adapters in the context of a virtual computer system have been described. In view of the above description of the preferred embodiments of the present invention, many modifications and variations of the disclosed embodiments will be readily appreciated by those of skill in the art. It is therefore to be understood that, within the scope of the appended claims, the invention may be practiced otherwise than as specifically described above.