Dynamic links in content-based networks

[0001] This application claims the benefit under 35 USC 119(e) or prior U.S. provisional application Ser. No. 60/530,676 filed Dec. 19, 2003, the contents of which are incorporated herein by reference.

[0002] This invention relates to content-based networks, in particular, but not exclusively, to Extensible Markup Language (XML) routing in such networks.

[0003] Content-based networks are described in A Carzaniga, M. J. Rutherford, A. L. Wolf, A routing scheme for content-based networking, Department of Computer Science, University of Colorado, June 2003, the contents of which are incorporated herein by reference.

[0004] In Extensible Markup Language (XML) routing, a network of XML routers exists as an overlay on top of a traditional IP routed network. The XML routers communicate with each other via Transmission Control Protocol (TCP) sockets. A socket in this context is an endpoint of a two-way communication link between two programs running on the network. These TCP sockets are formed between XML routers that are considered “neighbors”. The neighbor relationships are created by the network administrator, by configuring the IP address and TCP port number of the adjacent XML router. The communication path (i.e. TCP socket) between neighbors is named a “static XML link”.

[0005] FIG. 1 shows a sample content routed network 1. The network 1 consists of a plurality of publishers 2, 3 and 4, a plurality of subscribers 5, 6 and 7, 16 and a plurality of content routers 8, 9, 10, and 11. A content router is a digital communications networking device which forwards content based on inspection of the contents of a message or document, rather than on an explicit destination address in the networking header of a packet or frame. An example of such a device is the XCR 3200 XML content router from Solace Systems, Inc. A publisher is a computer, user or device that can insert content into the network. Another name commonly used in the literature is an event source or a producer. A subscriber is a computer, user or device that has expressed interest in some specific content. Another name commonly used in the literature is event displayers or consumers. In network 1, routers 8 and 9 are neighbors, via the static XML link 12. Similarly, router 8 and router 11 are neighbors via static link 15, routers 9 and 10 are neighbors via static link 13, and routers 10 and 11 are neighbors via static link 14.

[0006] The XML Link State Protocol (XLSP) is a link state protocol designed specifically for XML routed networks. It operates between XML routers, and is used to discover and resolve the routed topology consisting of XML routers and XML static links. The XML Subscription Management Protocol (XSMP) is used to propagate and manage content subscriptions throughout the XML routed topology. After subscription information has been propagated, the XSMP utilizes the topology discovered by the XLSP to create a “subscription tree”. The subscription tree defines the path that XML documents will take through the network from a given publisher to a set of subscribers. This subscription tree always traverses a set of XML routers and XML static links discovered by the XLSP. XLSP and XSMP are fully described in co-filed application Ser. No. 60/530,615, the contents of which are incorporated herein by reference.

[0007] For information on XML, refer to “Extensible Markup Language (XML) 1.0 (Third Edition)”, W3C Recommendation Feb. 4, 2004, W3C (World Wide Web consortium) and “Extensible Markup Language (XML) 1.1”, W3C Recommendation Apr. 15, 2004, W3C. For content routing with XML, an example of suitable subscription syntax is XML Path Language (XPath) (refer to reference “XML Path Language (XPath) Version 1.0”, W3C Recommendation Nov. 16, 1999, W3C (Word Wide Web Consortium)).

[0008] The methods described above allow the network administrator to build a highly scalable, efficient and robust XML routed network. However, each XML router incurs delay and in some circumstances may present a performance bottleneck.

[0009] The invention offers a more efficient method for delivering some traffic using of “dynamic XML links”. The dynamic XML links can be used to reduce the number of XML router hops the customer data traverses, thereby reducing delay and increasing overall network throughput.

[0010] In a broad aspect therefore the invention provides a method of managing a content-based network including a plurality of content routers, comprising at an originating router examining an incoming document to identify a destination router for said document, said originating router and said destination router forming a pair of routers; establishing at least one dynamic link between said pair of routers when said destination router is not a direct neighbor of said originating router; and transferring documents over said at least one dynamic link.

[0011] It will be understood that the term “document” in the context of this application is used in the most general senses and includes any entity containing content, for example, including multimedia content, that is capable of being published to subscribers.

[0012] Embodiments of the invention permit the network administrator to control customer traffic such that some class of identifiable XML documents can be passed through the network with a minimal number of XML router hops (i.e. higher priority traffic which requires reduced delay); some portion of XML traffic can be directed through a path other than that which is resolved by the XLSP/XSMP, for the purpose of off-loading a congested node or avoiding a “network hot spot”; and some TCP connections can be established only on an “as needed” basis, so that network resources are minimized when dealing with intermittent or sporadic traffic patterns.

[0013] Embodiments of the invention provide a set of techniques for maintaining dynamic links between XML routers, including procedures for establishment; procedures for glare handling; and procedures for aging and/or tear-down.

[0014] Embodiments of the invention also provide a method for provisioning and configuring dynamic XML links and a set of techniques for ensuring that dynamic and static XML links can co-exist in a common XML routed network.

[0015] In another aspect the invention provides a content routed network, comprising a plurality of content routers interconnected by static links; one or more publishers connected to said content routed network; a plurality of subscribers connected to said content routed network; and said content routers being configured to establish a dynamic link between a pair of said content routers which are not direct neighbors wherein documents received from said one or more publishers can be transferred over said dynamic link.

[0020] Dynamic XML links are communication paths established between XML routers which are not direct neighbors. They can be used to reduce the delay between a given publisher and subscriber, perform traffic engineering by bypassing some percentage of traffic away from network congestion points, and increase the scalability of the network by reducing the number of TCP sockets which need be open at a single time.

[0021] A dynamic XML link can be considered a “shortcut” between two XML routers which are not directly connected by a static XML link. XML documents traversing a dynamic XML link bypass one or more XML routers in the overlay topology.

[0022] Consider the static XML network depicted in FIG. 1. If publisher 2 is publishing data into the network which subscriber 6 has subscribed to, the path would be from the publisher 2 to router 8, over link 15 to router 11, over link 14 to router 10, and finally to subscriber 6. This example path is selected by the XML routing protocols referenced above. An alternate path would be publisher 2 to router 8, over link 12 to router 9, over link 13 to router 10, and finally to subscriber 6. The method of choosing the path to use is disclosed in the above-referenced filing Ser. No. 60/530,615.

[0023] By adding a dynamic XML link configuration at router 8 for some set of XML documents forwarded from publisher 2 to subscriber 6, router 8 will establish a direct TCP connection to router 10 on an as needed basis.

[0024] FIG. 2 shows the same elements as FIG. 1, with common elements sharing the same label. In FIG. 2, there is an additional direct Dynamic XML Link 20 shown between router 8 and router 10. When such a link is established, content published by publisher 2 which has been subscribed to by subscriber 6 can follow the short-cut path: publisher 2 to router 8, over dynamic XML link 20 to router 10, and finally to subscriber 6. Such a path avoids the XML traffic from having to be transited by router 9 or router 11.

[0025] For proper network operation, the mechanisms for establishment, maintenance and tear-down of dynamic XML links must be well defined.

[0026] Two options are supported for the establishment of dynamic XML links: a) configuration driven establishment, or b) data driven establishment. When configuration driven establishment is used, the TCP connection is established as soon as the network administrator creates and configures it. For data driven establishment, the router establishes the TCP connection after it receives the first XML document which is required to traverse the dynamic XML link. While the connection is being established, the XML document which-triggered it, and any subsequent documents which are destined for the same dynamic link, must be buffered by the XML router. FIG. 3 shows a typical content router 8 containing a processor 80, routing tables 81, and a buffer 82.

[0027] It is possible for two routers to have symmetrical dynamic XML links configured for each other. That is, two routers may need to asynchronously establish TCP connections to each other. For example, in FIG. 2, router 8 may be configured to establish a dynamic XML link to router 10, and router 10 may be configured to establish a dynamic XML link to router 8. If the establishment of these connections occur at well spaced times, then the second connection can be avoided simply by re-using the first connection in both directions. This falls within the spirit of reducing the number of TCP connections in the network. However, it is possible that both routers may be triggered to establish the dynamic link to each other at close time intervals, and create what is known as a “glare” condition. Glare handling of dynamic links is similar to that which is used in other connection oriented networking technologies; which is to elect one of the TCP connections for use, and disconnect the other. The two routers must implement the same election algorithm for determining which connection to retain; otherwise both connections may be torn down. For the dynamic XML links, the TCP connection of the router with the numerically lower IP address is the one which will be torn down. It is the responsibility of the router that established the connection to subsequently tear it down.

[0028] The decision to tear down the TCP connection for a dynamic link also has two options: a) aging after lack of use for some time interval T, or b) automatic disconnection after each XML document. It is a configuration option for the network administrator which method of dynamic link tear down is used.

[0029] In the first case, aging of the connection is the responsibility of the router which established the connection. If after T seconds, no traffic has been exchanged between the two routers across the dynamic link, the establishing router disconnects it. The dynamic link would then be re-established at a later time as per the establishment rules described above.

[0030] In the second case, the TCP connection associated with the dynamic link is torn down after each XML document is sent. The connection would then be re-established at a later time as per the establishment rules described above.

[0031] There are two phases to creation and configuration of a dynamic XML link. The first phase consists of the creation of the dynamic link destination; the second phase consists of the creation of the XML filtering rule(s) to identify traffic destined for that link.

[0032] The Command Line Interface (CLI) format for creating the dynamic link is:

router [no] link <link_id> address <ip>:<tcp_port> [dscp <dscp>] [AUTO_ESTABLISH] [aging <aging_time>]

[0033] where items in square brackets [ ] indicates optional items, items in angle brackets < > indicates parameters, and the optional “no” keyword indicates that the dynamic link configuration is to be deleted. Table 1 below describes the configurable parameters supplied by the network administrator when creating the dynamic link.

[0034]

TABLE I
Dynamic Link Configuration Parameters
	Required/
Parameter	Optional	Description
<link_id>	Required	A unique identifier for the dynamic
		link.
<dst_router_ip>	Required	The IP address of the XML router
		at the remote end of the dynamic
		link. This will be used in the
		establishment of the TCP connection.
<dst_tcp_port>	Required	The TCP port number to use in the
		establishment of the dynamic link
		TCP connection.
<dscp>	Required	Differentiated Services Code Point
		to be used for the TCP connection.
		This allows the TCP connection to
		utilize the quality of service
		capabilities of the underlying IP or
		MPLS network, as described in
		co-filed patent application
		60/588,797, the contents of which
		are incorporated herein by reference.
AUTO_ESTABLISH	Optional	Token specifying that the TCP
		connection for the dynamic link
		should be automatically established
		after configuration. If omitted, then
		the TCP connection will only be
		established when data destined for the
		dynamic link is to be forwarded.
<aging_time>	Optional	Time in seconds that the TCP
		connection must be idle before it will
		be torn down. If omitted, then the
		TCP connection is automatically torn
		down at the end of transmission of
		each XML document.

[0035] When creating the XML filtering rules for traffic to traverse that link, the user must enter the rule in the form of an XPath Expression (XPE). XPath is described in the above-referenced XPath specification. The CLI format is:

router xsmp [no] dynamic-link-filter xpe <xpe> link <link_id>

[0036] where the optional “no” keyword indicates that the dynamic link filter rule is to be deleted, <xpe> is the XPath expression string, and <link_id> indicates a dynamic link which was configured using the command defined earlier. It is important to understand that there can be a many-to-one relationship between the filter rules established in step (2) and the dynamic link created in step (1). That is, multiple filter rules can reference the same dynamic link identified by <link_id>.

[0037] Clearly, in a single XML routed network, both static and dynamic XML links can and must be allowed to co-exist. For proper network behavior, a number of simple design rules must be followed to allow this co-existence:

• • [0038] 1. Dynamic XML links are not affected by or visible to the operation of the XLSP in any way. The consequence of this is that dynamic links are not present in the topology database created by the XLSP. This is done to avoid short-duration dynamic links from causing churn in the topology seen by the routers discovering and monitoring the network topology via XLSP.
  • [0039] 2. As a result of 1, dynamic links are not utilized when matching subscriptions advertised via XSMP. Only static XML links, which are present in the XLSP topology database, are used by the XSMP.
  • [0040] 3. The Filter rules created to reference dynamic XML links take preference in the classification of XML documents over those learned via the XSMP.

[0041] In copending patent application 60/588,797, the contents of which are incorporated herein by reference, the scenario of using HTTP over TCP as the method of communicating between content routers is described. When HTTP over TCP is utilized, a given TCP connection between a pair of content routers can only carry documents in one direction. For example, in FIG. 2, a TCP connection from router 8 to router 9 is used to carry HTTP requests (containing the document to be sent) from router 8 to router 9, and to carry HTTP responses back from router 9 to router 8. A separate TCP connection is established from router 9 to router 8, and is used to carry documents from router 9 to router 8 in an HTTP request, and an HTTP response is sent back over the same TCP connection from router 8 to router 9. In this scenario of using HTTP over TCP, an XML link, such as link 12 between content routers 8 and 9, is actually composed of a pair of TCP connections. Moreover, additional TCP connections can be used, with the concept of document priority, to effect quality of service in a content-routed network. When HTTP over TCP is used and a TCP connection is only used to carry documents in one direction (and responses in the opposite direction), the issue of glare handling described above is not present, and each router can independently set up a TCP connection for a dynamic XML link independently to each other.

[0042] When document priorities are utilized as described in 60/588,797, document priorities can also be used as a trigger to set up a dynamic link to another content router. Instead of, or in addition to, using an XPath expression filter to trigger the use of a dynamic XML link as defined above, the document priority assignment algorithm defined in 60/588,797 can be used as a trigger to set up a dynamic link. The CLI format for defining this operation is:

router xsmp [no] priority-filter xpe <xpe> priority <priority> [dynamic-link <link_id>]

[0043] where items in square brackets [ ] indicates optional items, items in angle brackets < > indicates parameters, and the optional “no” keyword indicates that the priority filter configuration is to be deleted. <priority> is a document priority in the range of 0 (lowest) to 3 (highest) as defined in 60/588,797; <xpe> is an XPath expression used to match received documents to priorities as explained in 60/588,797; and <link_id> refers to a dynamic XML link configuration as described above. For example, a dynamic XML link can be defined, and use of this link can be specified for highest-priority documents (e.g. priority 3) as determined by an XPath expression. The dynamic XML link will then be used when the document priority classification scheme classifies a document as priority 3. Note that multiple dynamic XML links can be defined to a given destination router, with each link being used by different priorities. Note also that a given dynamic link can be used by more than one document priority, since multiple filters can be assigned to use the same <link_id>. As explained in 60/588,797, a link between a pair of content routers can be made up of a plurality of TCP connections in order to support quality of service. Dynamic links support this; however, each dynamic link represents one such TCP connection so that they can be set up and torn down independently as needed.

[0044] As described in the above-referenced filing 60/530,615, the XSMP routing protocol allows a router to know of a summarized view of subscriptions for each other content router in the same routing area, independent of the topology of the routing area. For example, in FIG. 2, content router 8, through XSMP, will discover the summarized subscription set for each of content routers 9, 10 and 11. Because of this scheme, a content router knows of the precise destination content router in a routing area, as opposed to other schemes which summarize subscriptions among content routers and thus this information is not known. For example, with the XSMP algorithm, when a document arrives to content router 8 from publisher 2, content router 8 can determine that the document is required by at least one subscriber on router 9 and on router 10. In other routing schemes, the router would only know that the document is required by router 9 or some unknown set of downstream routers. The XSMP routing algorithm allows for another novel use of dynamic XML links in content routed networks. A content router can measure the document bandwidth (in kilobits per second) independently to each non-neighbor destination content router, and determine if the bandwidth is above a threshold BW setup. If so, a dynamic XML link can be set up directly to that content router. When the measured bandwidth falls below BW tear-down, then the dynamic XML link can be torn down. This allows dynamic XML links to set up shortcuts to destination content routers based on the traffic flows seen, with hysteresis provided by the configurable setup and tear-down bandwidth thresholds. Note that the bandwidth measurement above is measuring the amount of document bandwidth destined to each content router, and not the actual bandwidth used on XML links. For example, with the static XML links used in FIG. 2, when content router 8 has a document that is destined to both content routers 9 and 10, a single copy of the document can be sent over link 12 to content router 9, and content router 9 then sends a copy over link 13 to content router 10. However, router 8 can separately track the document bandwidth that it has sent to each content router, independently of the actual path taken.

[0045] To enable bandwidth measurement, the dynamic link configuration of Table 1 above is augmented with the additional parameters specified in Table 2 below (and these parameters are added to the CLI command as optional parameters, and if not present, the bandwidth measurement algorithm is not active for that destination content router).

[0046]

TABLE 2
Additional Dynamic Link Configuration Parameters
	Required/
Parameter	Optional	Description
<BW_setup>	Optional	The bandwidth threshold, in kilobits per
		second, for setting up the dynamic XML
		link.
<BW_teardown>	Optional	The bandwidth threshold, in kilobits
		per second, for tearing-down the
		dynamic XML link. This value must
		be less than the <BW_setup>

[0047] In the above description of dynamic links, the published documents are in the form of XML documents, and the subscriptions and dynamic link and priority filters are in the form of XPath expressions. However, this invention is applicable to content routing for non-XML data formats. For example, in the above-referenced “A routing scheme for content-based networking”, a published document (also referred to as a message or an event) is in the form of a structured set of attribute/value pairs, and a subscription (also referred to as a selection predicate) is in the form of a logical disjunction of conjunctions of elementary constraints over the values of individual attributes. The above description of dynamic links is applicable to such content-routing networks, with the specification of XPath Expressions above replaced by the specification of selection predicates. In addition, the dynamic links established would carry messages in the form of a structured set of attribute/value pairs instead of in the form of XML documents.

[0048] The invention has been described with reference to an exemplary embodiment. It will be appreciated by one skilled in the art that many variants are possible within the scope of the invention.

[0049] All references mentioned above are herein incorporated by reference. Reference has been made herein to copending provisional applications, which are incorporated by reference. Such incorporation by reference should also be taken to include the non-provisional applications based thereon whose serial numbers will be inserted when they become available.