SYSTEM AND METHOD TO CHAIN DISTRIBUTED APPLICATIONS IN A NETWORK ENVIRONMENT

[0001] This application claims the benefit of priority under 35 U.S.C. §119(e) to U.S. Provisional Application Ser. No. 62/081,457, entitled “SYSTEM AND METHOD TO CHAIN DISTRIBUTED APPLICATIONS IN A NETWORK ENVIRONMENT” filed Nov. 18, 2014, which is hereby incorporated by reference in its entirety.

[0002] This disclosure relates in general to the field of communications and, more particularly, to a system and method to chain distributed applications in a network environment.

[0003] Data centers are increasingly used by enterprises for effective collaboration and interaction and to store data and resources. A typical data center network contains myriad network elements, including hosts, load balancers, routers, switches, etc. The network connecting the network elements provides secure user access to data center services and an infrastructure for deployment, interconnection, and aggregation of shared resource as required, including applications, hosts, appliances, and storage. Improving operational efficiency and optimizing utilization of resources in data centers are some of the challenges facing data center managers. Data center managers want a resilient infrastructure that consistently supports diverse applications and services while minimizing the potential for overloads of links between network elements. A properly planned and operating data center network provides application and data integrity while optimizing application and service availability and performance.

[0011] A method is provided in one example embodiment and may include communicating information between a plurality of network function virtualized (NFV) based applications; and creating at least one service chain using at least two of the plurality of NFV-based applications based, at least in part, on the information communicated between the plurality NFV based applications. In some instances, the information can be communicated using border gateway protocol (BGP) exchanges between the plurality of NFV-based applications.

[0012] In some instances the information can include next-hop address information for one or more ingress points of a particular NFV-based application. In other instances, the information can include one or more capabilities by which a particular NFV-based application can receive data on one or more ingress points. In yet other instances, the information can include a method by which one or more egress points of a previous NFV-based application in a particular service chain is to perform load balancing for a subsequent NFV-based application in the particular service chain.

[0013] In some cases, each of the plurality of NFV-based applications can include a data plane load balancer embedded at one or more egress points. In other cases, a particular NFV-based application can include functionality associated with mobile core applications. In still other cases, a particular NFV-based application can include functionality associated with a Gi-LAN virtual service area.

[0014] As used herein, the terms ‘network function virtualized (NFV)-based application’, ‘NFV application’ and ‘virtual machine’ encompass an emulation of a computer system, operating based on the computer architecture and functions of a real or hypothetical computer, with particular implementations involving specialized hardware, software, or a combination of both. An NFV-based application or virtual machine may execute via a hypervisor-based virtualization or a container-based virtualization of a server (e.g., blade server, rack server, stand-alone server) using the server's hardware (e.g., processor and memory element) and/or operating system. In various embodiments, an NFV-based application, NFV application, or virtual machine can be configured to carry out one or more network functions, which can be chained together to create one or more service chains.

[0015] Turning to FIG. 1A, FIG. 1A is a simplified block diagram illustrating a communication system 10 to facilitate chaining distributed applications in a network environment according to one embodiment of the present disclosure. FIG. 1A illustrates a network 30 comprising a plurality of switches and routers, for example, one or more router(s) 32 and one or more switch(es) 34. Router(s) 32 and switch(es) 34 may comprise identical or similar network elements and may differ merely in the nature of devices attached to their respective ports: whereas router(s) 32 connect with switch(es) 34 on their respective ports, a plurality of servers 36 may be connected to switches 34 on their respective ports. In various embodiments, network 30 may implemented in a data center.

[0016] Each server 36 may host one or more Network Function Virtualized (NFV) based applications 40. In general, NFV-based applications 40 can be networked and distributed applications that can be configured to perform various telecommunication functions. Note the term ‘NFV-based application’ may also be referred to more generally using the term ‘NFV application’ throughout this Specification.

[0017] Each server 36 may be configured with overlay or underlay software network switches, for example, an overlay switch 38 that forms part of a distributed overlay switch 58 across network 30. As shown, each overlay switch 38 enables NFV applications 40 to connect to network 30 through any suitable overlay or underlay software network architectures, such as, for example FabricPath, virtual Extensible Local Area Network (VXLAN), Transparent Interconnection of Lots of Links (TRILL) overlay, Network Service Header (NSH) [as prescribed by the Internet Engineering Task Force (IETF) NSH working group], differentiated services code point (DSCP) markings, combinations thereof or the like. In various embodiments, overlay switches 38 may operate in a virtual machine (VM), in a container and/or in a host kernel (e.g., via each server 36). A central NFV orchestrator and manager 26 enables configuration and instantiation of NFV applications 40 and overlay switches 38 for connectivity in network 30 for service chains, such as, for example, a service chain 56 shown in FIG. 1A. In general, service chains (e.g., service chain 56) of NFV applications 40 may be provided to facilitate unique and customizable services to one or more nodes in communication system 10. A node may be any electronic device (e.g., UE 14), client, server, peer, service application, access point name (APN) or other object capable of sending, receiving, or forwarding information over communications channels in a network. A NFV database 28 may include information pertaining to NFV applications 40, which are described in further detail in FIG. 1B.

[0018] FIG. 1B illustrate example details of a given server 36 according to a certain embodiment of the present disclosure. In at least one embodiment, server 36 may include overlay switch 38 and a given NFV application 40 instantiated in a hypervisor 50 in server 36. Sever 36 may include a memory element 52a and a processor 54a to facilitate the chaining operations described herein.

[0019] Each NFV application 40 is comprised of three stages: one or more ingress point(s) 42 at which packets can be received from either an external network or a preceding NFV-capable application (e.g., another NFV application 40); one or more application part(s) 44 potentially maintained in a service chain, such as service chain 56; and one or more egress point(s) 46 at which packets can be transmitted to the signaled next application's ingress or an external network. A data plane load balancer 48 can be embedded in egress point(s) 46.

[0020] Each of the stages has a specific role or function in a packet processing pipeline (e.g., service chain 56). In various embodiments, NFV applications 40 may combine these functions in aggregate functions, and some of the NFV applications may keep these functions separate. In various embodiments NFV applications 40 may split these functions onto individual virtual machines (not shown) instantiated in servers 36. The software/functional decomposition of NFV applications is an ongoing study area within the European Telecommunications Standards Institute (ETSI) NFV Working Group.

[0021] FIG. 1C illustrates other example details of a given server 36 according to a certain embodiment of the present disclosure. In particular, FIG. 1C illustrates NFV application 40 virtualized in container-based environment. In at least one embodiment, server 36 may include overlay switch 38 and a given NFV application 40 virtualized in a container 62 of an operating system 60 for server 36. Sever 36 may further include memory element 52a and processor 54a to facilitate the chaining operations described herein. NFV application 40 can include ingress point(s) 42, application part(s) 44 and egress point(s) 46. Data plane load balancer 48 can be embedded in egress point(s) 46. Accordingly, the system and method provided by communication system 10 can provide for different virtualization architectures in order to facilitate chaining distributed applications in a network environment.

[0022] For purposes of illustrating the techniques of communication system 10, it is important to understand NFV-based applications as generally operated in commercial architectures. The following foundational information may be viewed as a basis from which the present disclosure may be properly explained. Such information is offered earnestly for purposes of explanation only and, accordingly, should not be construed in any way to limit the broad scope of the present disclosure and its potential applications. As noted previously, NFV-based applications are networked and distributed applications that may perform telecommunications functions, such as, for example, line termination and policy enforcement functions.

[0023] Within the telecommunications industry, there is an ongoing effort to commoditize and standardize telco applications and host these applications in data centers. The reason for this “virtualization” is the desire by operators to reduce both operational and capital expenditures by standardizing common hardware and software.

[0024] One key consideration for the telecommunications industry is the ability to mix and match “telco” applications from many different application manufacturers. Mobile and non-mobile service providers typically obtain subcomponents, or “applications”, of their “telco” solutions from many different manufacturers for a variety of reasons, such as, but not limited to, risk and vendor management. Service providers prefer to obtain the “best of breed” solutions from a set of manufacturers and combine the parts into their specific solution to address their specific business needs. Risk mitigation plays an important role here to avoid a service provider to be locked into the solutions of a single manufacturer.

[0025] Thus, there exists a need for “universal glue” between the individual NFV-based components that are hosted in data centers. This glue can enable the creation of a combined “virtualized telco” solution comprised of one or more NFV-based applications hosted in one or more data centers. To tie together a set of NFV-based applications to form service chains, a method is needed where the egress point(s) of one of these “telco” NFV applications can directly and efficiently route traffic to the appropriate ingress point(s) of a subsequent application. It should be recognized that given that NFV-based applications are inherently distributed in nature, each of the applications potentially has many ingress and egress points or locations.

[0026] As network traffic is routed between egress point(s) of one NFV-application and ingress point(s) of the next NFV-application, there is likely to be an impedance mismatch. Performance on each NFV-application is based on a number of factors, including, but not limited to instance size, software efficiency, software function, etc., which often results in numerous internal and external load distribution methods being employed to manage network traffic between NFV applications. It should also be noted that ingress point(s) may be provided by hardware load balancers, and egress point(s) may also route packets outside the data center. It should also be further noted that the ingress and egress locations may be either layer-2 (L2) adjacent or layer-3 (L3) adjacent.

[0027] All NFV-based applications are to be hosted in data centers, yet, given that these NFV-based applications are running in network-challenged data center locations, such as, for example, the data center servers, it is important to consider efficient if not optimal routing between applications. The server blades in a data center are typically connected by way of a 1-10 gigabit per second (Gb/s) Ethernet link to one or more top-of-rack (ToR) switches, end-of-row (EoR) switches and the data center router. The links between EoR, ToR and servers are considered at risk for overload since these links are configured with statistical multiplexing in mind to optimize data center expenditures. This means that egress point(s) of one application can connect to ingress point(s) of another application through these potentially oversubscribed links.

[0028] However, if external load balancers are used, a mechanism to automate the provisioning of the load balancer is required. A generic mechanism to add functions to a load balancer pool does not exist today. This elasticity requirement thus implies that from an application connectivity perspective, a method is needed that dynamically tracks ingress points as these are also created dynamically. While a data-plane load balancer can potentially be built for NFV-based applications, doing so would (1) lead to aggregation of traffic and thus potential bottlenecks, (2) to address such a bottleneck, it would imply building a special-purpose data-plane load balancer close to the data-center router and thus (3) inefficient use networking resources in a data center and (4) inefficient use of Wide Area Network (WAN) resources when NFV-based applications exist across multiple data centers.

[0029] Communication system 10 is configured to address these issues (and others) in offering a system and method for chaining distributed applications in a network environment. The methods described herein may be executed by respective hardware processors of servers 36 in network 12. In general, communication system 10 provides a system and method to host distributed NFV applications 40 in a service chain (e.g., service chain 56) by way of leaking routes between routed areas and to “steer” all traffic between these routed areas through service applications. In embodiments described herein, communication system 10 may provide a system and method by which one distributed NFV-based application can signal another distributed NFV-based application of (1) a method by which the connectivity between NFV-applications can be established, (2) the method of signaling by which one NFV-based application can provide data-plane load distribution for the next application, (3) a method whereby one NFV-based application can share intimate application details with the next and (4) the method by which dynamic changes can be signaled to previous applications. Additionally, the procedures can also be used to signal external routing sources of connectivity into a data center including communication system 10. One or more service chain(s) 56 may serve as the primary method for providing connection distributed, data-center hosted, networked applications in data centers with distributed routes. Note NFV-based applications 40 in a service chain may be referred to as “hops” herein in this Specification.

[0030] More specifically, such distributed NFV applications 40 can signal connectivity between ingress point(s) 42 and egress point(s) 46 using a routing protocol, such as for example, Border Gateway Protocol (BGP) [as defined in RFC 4271 and RFC 4364], application-specific routing information beneficial for applications further in the chain. This construct can then be used to route traffic between multiple, independently operating, NFV-capable applications in data centers and cloud infrastructures. In various embodiments, routing techniques can include tunneling over Multi-Protocol Label Switching in Internet Protocol (MPLS-in-IP), Network Service Header (NSH) tunneling and Equal Cost Multipath Routing (ECMP) based load balancing, combinations thereof or the like. Communication system 10 may provide NSH endpoints of ingress points of NFV applications 40 to be signaled through the BGP protocol (referred to herein as a BGP/NSH interconnection) in addition to more traditional methods such as MPLS, MPLS-over-IP, General Routing Encapsulation (GRE), GPRS Tunneling Protocol (GTP) encapsulation, VXLAN or others. Overloading risks for links within communication system 10 are addressed by providing distributed data plane load balancer 48 directly embedded in the egress point(s) 46 of NFV applications 40.

[0031] BGP generally announces routable addresses and next hop information to all interested parties. Typically, a BGP speaker maintains a mesh of connections and floods all routers in a network with network routing information. During operation, communication system 10 may provide a similar technique in network 30 (e.g., within a data center) to create service chain 56 of NFV-based applications 40. For example, as shown in FIG. 1A, (first) NFV application (NFV App1) and (seventh) NFV application (NFV App7) may be chained together to form service chain 56. It should be noted the order and number of NFV applications 40 for service chain 56 is shown for illustrative purposes only and is not meant to limit the broad scope of the many possibilities that may be configured for chaining together NFV applications 40 to form service chains.

[0032] In various embodiments, NFV applications 40 may announce, via BGP:

[0033] 1) One or more next-hop addresses of NFV-based application routers, (e.g., ingress point(s) of a subsequent NFV application);

[0034] 2) The capabilities by which an NFV-based application can receive data on ingress point(s). In various embodiments, these capabilities can include the tunneling techniques used (e.g. NSH-based tunneling, encrypted tunnel, or more traditional information) and, if needed, additional application-specific parameters such that one application can “help” the next application(s) to perform better (e.g., subscriber information, or other contextual information); and/or

[0035] 3) The method by which a previous hop's egress is to perform load balancing. For instance, ECMP and/or 5-tuple route leaking (e.g., source IP, destination IP, source port, destination port, protocol) are examples by which a previous hop's egress can make a next hop selection and BGP can be used to instruct the previous hop of parameters that are needed for ECMP (e.g., hashing technique, number of bins, “leg” (e.g., each ECMP load balanced endpoint), etc.). Further, in some instances where contextual information is already stored at the NFV-application, generic decisions on load distribution may not be made, but instead may additional require “stickiness” parameters. In various embodiments, “stickiness” parameters (e.g., source IP, destination IP, 5-tuple, subscriber information, etc.) can also be communicated. It should be noted that egress load balancing is distinct from load balancing at the ingress, as is typically used for Web based application architectures.

[0036] In at least one embodiment, NFV-based applications 40 can include, but not be limited to, virtual complex mobile core applications. In at least one embodiment, NFV-based applications 40 can be associated with a Gi-LAN virtual service area and/or a service platform. It should be understood, however, that NFV-based applications 40, as described in the present disclosure do not need to adhere to any specific configuration of such functions.

[0037] While the connectivity between NFV applications 40 can be established by way of BGP, it should be recognized that this connectivity may lead to imperfect results. It should be understood, therefore that within BGP, convergence of routes may not be immediate, and that previous hops may send data to incorrect next hops. It is assumed in certain embodiments described herein that NFV-based applications 40 can route packet flows internally to address these imperfections.

[0038] Typically in an NFV-based environment, traffic is sent bi-directionally. Care should be taken that load balancing decisions in the forward link are reversible for return traffic. Firewall load balancing techniques have been used in the past to accomplish this, but obviating the need for a forward-path load balancer also obviates the need for a reverse-path load balancer. Also, in case next hops fail, it should be understood that application-specific routing on egress removes routing opportunities accordingly.

[0039] In some instances where failures result in loss of contextual information in a next-hop NFV Application, generic failover rules may not be applied. In these instances, multiple decisions may result, including the dis-association of all flows destined to the failed NFV-application instance. The scenarios and implications of contextual loss during failure are outside scope of the present disclosure. In terms of establishing adjacency between applications, embodiments disclosed herein assume provisioning to take place between NFV applications 40 to exchange BGP signaling end-points and/or the establishment of NFV-based route reflection. Without restraining the solution, output-route filtering (ORF) as described in RFC 5291 may be applied.

[0040] In essence, the communication system 10 offers a system and method for connecting NFV-based applications 40, which can be hosted in a data center, such that efficient and potentially even optimal routing can take place between such NFV applications 40 given the constraints of data-center designs. Although solutions provided by communication system 10 may use standard BGP for establishing connectivity, specific additions can be provided for signaling application-specific parameters and load-balancing procedures. Various advantages provided by using BGP in such environments may include: (1) it is an open standard, (2) it has shown to be resilient against partial failures in networks, (3) it can provide efficient and optimal routes, and (4) it can be extended to support application-specific messaging that can be useful for NFV-based applications 40.

[0041] Note that the number of switches, servers, NFV applications, virtual machines, service chains and other elements of communication system 10 shown in the FIGURE is merely for ease of illustration, and is not intended to be limiting in any sense. Any number of such and other elements may be included within communication system 10 within the broad scope of the embodiments.

[0042] In at least one embodiment, NFV database 28 can comprise a standalone server or a local configuration storage in switches 34 and/or in servers 36 that maps each server 36 to the configurations, hop addresses, capabilities, etc. of its instantiated NFV applications 40 and any corresponding service chains (e.g., service chain 56). Routers 32 and switches 34 comprise hardware switches configured to forward packets in a network environment. In at least one embodiment, distributed overlay switch 58 comprises overlay switches 38 with separate instances on each of a hypervisor of servers 36 in network 30. Each instance of overlay switch 38 may comprise packet forwarders, virtual ports, and other virtual switch components instantiated and executing in servers 36 (e.g., in a VM, container or kernel) and may perform Layer 2 (L2) switching and advanced networking functions, such as applying quality of service (QoS) settings, access control lists, etc. on inbound and outbound traffic. In at least one embodiment, NFV orchestrator and manager 26 comprises an application executing in a suitable server or other network element in network 30 to facilitate instantiation and maintenance of NFV applications 40 and one or more service chain(s) 56. In at least one embodiment, each server 36 can include a corresponding NFV orchestrator and manager 26 for NFV applications instantiated on a corresponding server. In at least one embodiment, NFV orchestrator and manager 26 enables configuring and managing virtualization hosts, networking, and storage resources in a virtualized data center environment, for example, to create and deploy NFV-based applications 40 and services to private clouds.

[0043] Turning to the infrastructure of communication system 10, the network topology can include any number of servers, virtual machines, switches (including distributed overlay switches), routers, and other nodes inter-connected to form a large and complex network. A node may be any electronic device, client, server, peer, service, application, or other object capable of sending, receiving, or forwarding information over communications channels in a network. Elements of FIG. 1A may be coupled to one another through one or more interfaces employing any suitable connection (wired or wireless), which provides a viable pathway for electronic communications. Additionally, any one or more of these elements may be combined or removed from the architecture based on particular configuration needs. Communication system 10 may include a configuration capable of TCP/IP communications for the electronic transmission or reception of data packets in a network. Communication system 10 may also operate in conjunction with a User Datagram Protocol/Internet Protocol (UDP/IP) or any other suitable protocol, where appropriate and based on particular needs. In addition, gateways, routers, switches, and any other suitable nodes (physical or virtual) may be used to facilitate electronic communication between various nodes in the network.

[0044] Note that the numerical and letter designations assigned to the elements of FIG. 1A do not connote any type of hierarchy; the designations are arbitrary and have been used for purposes of teaching only. Such designations should not be construed in any way to limit their capabilities, functionalities, or applications in the potential environments that may benefit from the features of communication system 10. It should be understood that communication system 10 shown in FIG. 1A is simplified for ease of illustration.

[0045] The example network environment may be configured over a physical infrastructure that may include one or more networks and, further, may be configured in any form including, but not limited to, local area networks (LANs), wireless local area networks (WLANs), virtual local area networks (VLANs), metropolitan area networks (MANs), wide area networks (WANs), VPNs, Intranet, Extranet, any other appropriate architecture or system, or any combination thereof that facilitates communications in a network. In some embodiments, a communication link may represent any electronic link supporting a LAN environment such as, for example, cable, Ethernet, wireless technologies (e.g., IEEE 802.11x), Asynchronous Transfer Mode (ATM), fiber optics, etc. or any suitable combination thereof. In other embodiments, communication links may represent a remote connection through any appropriate medium (e.g., digital subscriber lines (DSL), telephone lines, T1 lines, T3 lines, wireless, satellite, fiber optics, cable, Ethernet, etc. or any combination thereof) and/or through any additional networks such as a wide area networks (e.g., the Internet).

[0046] Turning to FIG. 2, FIG. 2 is a simplified block diagram illustrating interactions between NFV applications 40 for service chain 56 according to one embodiment of communication system 10. FIG. 2 illustrates communication interactions 70 between each NFV application 40 (e.g., NFV App1 and NFV App7), which can be used to create service chain 56. In various embodiments, communication interactions 70 may include NFV applications 40 communicating using BGP as a signaling mechanism to set up ECMP for load balancing purposes. In at least one embodiment, communication interactions 70 may be exchanged not only between NFV applications 40 but also other elements of network 30 and/or external networks. It should be noted that the order of NFV applications 40 in service chain 56 should not be construed to indicate an order of operations for service chain 56, but rather is provided for illustrating various features of communication system 10. As shown in FIG. 2, communication interactions 70 may include next-hop address information 72, NFV ingress capability information 74 and previous egress load balancing information 76 for each NFV application 40 (e.g., NFV App1 and NFV App7).

[0047] In various embodiments, next-hop address information 72 can include address information for ingress point(s) 42 for each NFV application 40. In various embodiments, NFV ingress capability information 74 can include the tunneling techniques used (e.g. NSH-based tunneling, encrypted tunnel, or more traditional information) and, if needed, additional application-specific parameters such that one application can “help” the next application(s) to perform better (e.g., DPI information, subscriber information, or other contextual information). In various embodiments, previous egress load balancing information 76 can include the method by which a previous hop's egress is to perform load balancing. For instance, ECMP is an example by which a previous hop's egress can make a next hop selection and BGP can be used to instruct the previous hop of parameters that are needed for ECMP (e.g., hashing technique, number of bins, “leg”, etc.) and “stickiness” parameters (e.g., source IP, destination IP, 5-tuple, subscriber information, etc.).

[0048] It should be understood that the number of NFV applications included in a service chain can vary, depending on configuration. The number of NFV applications shown in service chain 56 is provided for illustrative purposes and is not meant to limit the broad scope of the teachings of the present disclosure.

[0049] Turning to FIG. 3, FIG. 3 is a simplified flow diagram illustrating example operations 300 associated with chaining distributed applications in a network environment in accordance with one embodiment of communication system 10. In various embodiments, operations 300 can be performed using one of more NFV based applications (e.g., any of NFV based applications 40).

[0050] At any time, a service chain can be formed using NFV-based applications. At 302, the operations can include communicating information between a plurality of NFV-based applications. At 304, the operations can include creating at least one service chain using at least two of the plurality of NFV-based applications based, at least in part, on the information communicated between the plurality of NFV-based applications and the operations may end. In various embodiments, the information can be communicated using BGP exchanges between the plurality of NFV-based applications.

[0051] In at least one embodiment, the information can include next-hop address information for one or more ingress points of a particular NFV-based application. In certain embodiments, the information can include one or more capabilities by which by which a particular NFV-based application can receive data on one or more ingress points. In various embodiments, the information can also include a method by which one or more egress points of a previous NFV-based application in a particular service chain is to perform load balancing for a subsequent NFV-based application in the particular service chain.

[0052] Turning to FIG. 4, FIG. 4 is a simplified block diagram 400 illustrating example details associated with an example use case in which communication system 10 is provided in data centers 20. As shown in FIG. 4, an example communication network 12 may include users operating user equipment (UE) 14, one or more radio access networks (RANs) 16, one or more mobile core networks 18, one or more data centers 20 (each including communication system 10), one or more packet data networks 22 and one or more cloud applications/services 24.

[0053] Each of the elements of communication network 12 may communicate and/or exchange information through simple interfaces or through any other suitable connection (wired or wireless), which provides a viable pathway for network communications. Additionally, any one or more of these elements may be combined or removed from the architecture based on particular configuration needs. For example, communication network 12 may include a configuration capable of TCP/IP communications for the transmission or reception of packets in a network. Communication network 12 may also operate in conjunction with a user datagram protocol/IP (UDP/IP) or any other suitable protocol where appropriate and based on particular needs.

[0054] In various embodiments, RANs 16 can include 3rd Generation Partnership Project (3GPP) access networks and non 3GPP access networks. In various embodiments, 3GPP access networks can include legacy access networks such as Global System for Mobile communications (GSM) Enhanced Data rates for GSM Evolution (EDGE) Radio Access Network (GERAN), Universal Mobile Telecommunications System (UMTS) Terrestrial Radio Access Network (UTRAN), generally referred to as 3G, and/or Long Term Evolution (LTE) access networks such as Evolved UTRAN (E-UTRAN), generally referred to as 4G/LTE/LTE-Advanced (LTE-A). In various embodiments, non-3GPP access networks can include, but not be limited to digital subscriber line (DSL), Cable, WLAN (e.g., WiMAX, WiFi) or the Internet.

[0055] In at least one embodiment, UE 14 is a mobile device having multi-mode capabilities and is able to simultaneously communicate using one or more mobile wireless connections such as 2G/3G/4G/LTE/LTE-A connections and/or one or more wireless LAN (WLAN) connections such as WiFi and/or Worldwide Interoperability for Microwave Access (WiMAX), Bluetooth™, combinations thereof or the like. In accordance with various embodiments, UE 14 can be associated with clients or customers wishing to initiate a flow in communication network 12 via some network. The terms ‘user equipment’, ‘mobile node’, ‘end user’, ‘and ‘subscriber’ are inclusive of devices used to initiate a communication, such as a computer, a personal digital assistant (PDA), a laptop or electronic notebook, a cellular telephone, an i-Phone®, i-Pad®, a Google® Droid® phone, an IP phone, or any other device, component, element, or object capable of initiating voice, audio, video, media, or data exchanges within communication network 12 using multiple access technologies.

[0056] UE 14 may also be inclusive of a suitable interface to the human user such as a microphone, a display, a keyboard, or other terminal equipment. UE 14 may also be any device that seeks to initiate a communication on behalf of another entity or element such as a program, a database, or any other component, device, element, or object capable of initiating an exchange within communication network 12. Data, as used herein in this document, refers to any type of numeric, voice, video, media, or script data, or any type of source or object code, or any other suitable information in any appropriate format that may be communicated from one point to another. In certain embodiments, UE 14 may have a bundled subscription for network access and application services (e.g., data, voice over LTE (VoLTE), etc.).

[0057] In at least one embodiment mobile core networks 18 can include one or more elements comprising an Evolved Packet Core (e.g., Mobility Management Entities (MMEs), serving gateways (SGWs), packet data network (PDN) gateways (PGWs), etc.) to support the transmission and/or exchange of information and/or data between UE 14 (communicating via RANs 16) and data centers 20, packet data networks 22 and/or cloud applications/services 24. In various embodiments, packet data networks 22 can include the Internet, service provider IP services, combinations thereof or the like. In certain embodiments, policy and charging services and applications can be included in mobile core networks 18, data centers 20 (including communication system 10), cloud applications/services 24 and/or packet data networks 22.

[0058] As shown in FIG. 4, data centers 20 can be provided with communication system 10. Referring to communication system 10, in one example implementation, routers 32, switches 34, servers 36 and NFV orchestrator and manager 26 are network elements, which are meant to encompass network appliances, servers, routers, switches, gateways, bridges, loadbalancers, firewalls, processors, modules, or any other suitable device, component, element, or object operable to exchange information that facilitates or otherwise helps coordinate the operations described herein. In other embodiments, these operations and/or features may be provided external to these elements, or included in some other network device to achieve this intended functionality. Alternatively, one or more of these elements can include software (or reciprocating software) that can coordinate in order to achieve the operations and activities outlined herein.

[0059] Note that in this Specification, references to various features (e.g., elements, structures, modules, components, steps, operations, characteristics, etc.) included in ‘one embodiment’, ‘example embodiment’, ‘an embodiment’, ‘another embodiment’, ‘some embodiments’, ‘various embodiments’, ‘other embodiments’, ‘alternative embodiment’, and the like are intended to mean that any such features are included in one or more embodiments of the present disclosure, but may or may not necessarily be combined in the same embodiments. Note also that an ‘application’, an ‘NFV application’ and/or an ‘NFV-based application’ as used herein this Specification, can be inclusive of an executable file comprising instructions that can be understood and processed on a computer, and may further include library modules loaded during execution, object files, system files, hardware logic, software logic, or any other executable modules.

[0060] In example implementations, at least some portions of the activities outlined herein may be implemented in software in, for example, servers 36 (including NFV applications 40). In some embodiments, one or more of these features may be implemented in hardware, provided external to these elements, or consolidated in any appropriate manner to achieve the intended functionality. The various network elements (e.g., servers 36 (including NFV applications 40), NFV orchestrator and manager 26, switches 34, routers 32) may include software (or reciprocating software) that can coordinate in order to achieve the operations as outlined herein. In still other embodiments, these elements may include any suitable algorithms, hardware, software, components, modules, interfaces, or objects that facilitate the operations thereof. This may be inclusive of appropriate algorithms and communication protocols that allow for the effective exchange of data or information.

[0061] Furthermore, servers 36 (including NFV applications 40) described and shown herein (and/or their associated structures) may also include suitable interfaces for receiving, transmitting, and/or otherwise communicating data or information in a network environment. Additionally, some of the processors and memory elements associated with the various nodes may be removed, or otherwise consolidated such that a single processor and a single memory element are responsible for certain activities. In a general sense, the arrangements depicted in the FIGURES may be more logical in their representations, whereas a physical architecture may include various permutations, combinations, and/or hybrids of these elements. It is imperative to note that countless possible design configurations can be used to achieve the operational objectives outlined here. Accordingly, the associated infrastructure has a myriad of substitute arrangements, design choices, device possibilities, hardware configurations, software implementations, equipment options, etc.

[0062] In some of example embodiments, one or more memory elements (e.g., memory element 52a) can store data used for the operations described herein. This includes the memory element being able to store instructions (e.g., software, logic, code, etc.) in non-transitory media, such that the instructions are executed to carry out the activities described in this Specification. A processor, including a hardware processor, can execute any type of instructions associated with the data to achieve the operations detailed herein in this Specification. In one example, hardware processors (e.g., processor 54a) could transform an element or an article (e.g., data) from one state or thing to another state or thing. In another example, the activities outlined herein may be implemented with fixed logic or programmable logic (e.g., software/computer instructions executed by a processor) and the elements identified herein could be some type of a programmable processor, programmable digital logic (e.g., a field programmable gate array (FPGA), an erasable programmable read only memory (EPROM), an electrically erasable programmable read only memory (EEPROM)), a digital signal processor (DSP), an application specific integrated circuit (ASIC) that includes digital logic, software, code, electronic instructions, flash memory, optical disks, CD-ROMs, DVD ROMs, magnetic or optical cards, other types of machine-readable mediums suitable for storing electronic instructions, or any suitable combination thereof.

[0063] These devices may further keep information in any suitable type of non-transitory storage medium (e.g., random access memory (RAM), read only memory (ROM), field programmable gate array (FPGA), erasable programmable read only memory (EPROM), electrically erasable programmable ROM (EEPROM), etc.), software, hardware, or in any other suitable component, device, element, or object where appropriate and based on particular needs. The information being tracked, sent, received, or stored in communication system 10 could be provided in any database, register, table, cache, queue, control list, or storage structure, based on particular needs and implementations, all of which could be referenced in any suitable timeframe. Any of the memory items discussed herein should be construed as being encompassed within the broad term ‘memory element.’ Similarly, any of the potential processing elements, modules, and machines described in this Specification should be construed as being encompassed within the broad term ‘processor.’

[0064] It is also important to note that the operations and steps described with reference to the preceding FIGURES illustrate only some of the possible scenarios that may be executed by, or within, the system. Some of these operations may be deleted or removed where appropriate, or these steps may be modified or changed considerably without departing from the scope of the discussed concepts. In addition, the timing of these operations may be altered considerably and still achieve the results taught in this disclosure. The preceding operational flows have been offered for purposes of example and discussion. Substantial flexibility is provided by the system in that any suitable arrangements, chronologies, configurations, and timing mechanisms may be provided without departing from the teachings of the discussed concepts.

[0065] Note that with the examples provided above, as well as numerous other examples provided herein, interaction may be described in terms of one, two, three, or four network elements. However, this has been done for purposes of clarity and example only. In certain cases, it may be easier to describe one or more of the functionalities by only referencing a limited number of network elements. It should be appreciated that communication system 10 (and its teachings) are readily scalable and can accommodate a large number of components, as well as more complicated/sophisticated arrangements and configurations. Accordingly, the examples provided should not limit the scope or inhibit the broad teachings of communication system 10 as potentially applied to a myriad of other architectures.

[0066] Although the present disclosure has been described in detail with reference to particular arrangements and configurations, these example configurations and arrangements may be changed significantly without departing from the scope of the present disclosure. For example, although the present disclosure has been described with reference to particular communication exchanges involving certain network access and protocols, communication system 10 may be applicable to other exchanges or routing protocols. Moreover, although communication system 10 has been illustrated with reference to particular elements and operations that facilitate the communication process, these elements, and operations may be replaced by any suitable architecture or process that achieves the intended functionality of communication system 10.

[0067] Numerous other changes, substitutions, variations, alterations, and modifications may be ascertained to one skilled in the art and it is intended that the present disclosure encompass all such changes, substitutions, variations, alterations, and modifications as falling within the scope of the appended claims. In order to assist the United States Patent and Trademark Office (USPTO) and, additionally, any readers of any patent issued on this application in interpreting the claims appended hereto, Applicant wishes to note that the Applicant: (a) does not intend any of the appended claims to invoke paragraph six (6) of 35 U.S.C. section 112 as it exists on the date of the filing hereof unless the words “means for” or “step for” are specifically used in the particular claims; and (b) does not intend, by any statement in the specification, to limit this disclosure in any way that is not otherwise reflected in the appended claims.