US 6,760,331 B1Grant
Multicast routing with nearest queue first allocation and dynamic and static vector quantization
Issue Date:2004-07-06
•83 Claims
•6 Drawing Sheets
Abstract
The invention includes a way to route multicast traffic through a switch or other device that uses input queuing. Basis vectors are associated with each virtual queue, which in a preferred embodiment are multicast virtual output queues (MVOQs). Each incoming flow is allocated to the MVOQ whose basis vector is closest to the destination vector of the incoming flow, creating queues whose contents are distinct from each other in terms of destination addresses of the flows in the respective queue. To optimize for traffic encountered basis vectors can be chosen using vector quantization methods. Basis vectors can be set statically or can be selected and updated dynamically, responsive to the traffic encountered or as set by an operator. The invention can reduce the number of virtual queues required for a given incremental improvement in performance or can improve performance for a given number of virtual queues.
Metadata
Assignee
- Cisco Technology, Inc.
Inventors
- Farshid Moussavi
- Dhaval N. Shah
Application Information
Application Number:US 09/283,109
Filing Date:1999-03-31
Priority Date:1999-03-31
Art Unit:7
Classifications
IPC:
H04L 1228H04L 1256
Field of Search:
370389370390370428370429
Patent Drawings (6 sheets)
Description
RELATED APPLICATIONS
[0002] This invention relates to U.S. patent application: application Ser. No. 09/282,539, in the name of Farshid Moussavi and Dhaval N. Shah, titled “Multicast Routing With Multi-cast Virtual Output Queues And Shortest Queue First Allocation,” assigned to the same assignee. The application is hereby incorporated by reference as if fully set forth herein.
[0003] It is respectfully suggested that it may be appropriate for the same examiner to examine both applications.
BACKGROUND OF THE INVENTION
[0004] 1. Field of the Invention
[0005] This invention relates to multicast routing.
[0006] 2. Related Art
[0007] Communication on a computer network is accomplished by sending messages. Messages can include one or more data packets. Packets can be of fixed or variable lengths. Cells are packets having a fixed length.
[0008] Messages have a source and at least one destination address. A computer network includes devices that direct traffic towards the destination address. A switch is one such device.
[0009] Switches have multiple input interfaces and multiple output interfaces, which may be connected in a variety of ways. A cross bar switch is designed so that every input interface can be connected to every output interface.
[0010] There are two types of network traffic. In the first type a message has only one final destination address. This is known as unicast traffic. One use of unicast traffic is point to point communication between two computers. In the second type of traffic, called multicast, a message is sent to multiple destinations. One use of multicast transmissions is when a computer user wishes to send a message over the Internet to many individuals wishing to receive the message. A switch can have both unicast input interfaces as well as multicast input interfaces; often an interface handles both unicast as well as multicast traffic.
[0011] Memory Bandwidth Limitations
[0012] There are two main types of schemes for storing messages in the known art, which are input queuing and output queuing. (Combinations are also possible). In input queuing, a packet is queued before it enters the crossbar switch, and waits in line to arrive at the head of the input queue and be sent onward to its destination across the crossbar fabric. In output queuing, packets are forwarded onto the crossbar fabric from the input interface immediately, and queued up as they arrive at their destination output.
[0013] Output queuing hits limitations in memory speed faster than does input queuing because output queuing requires a memory at the output which is capable of momentarily receiving traffic from multiple inputs (in the worst case, all inputs), and sending out traffic at the output line rate. This means the memory in an output queued scheme must be faster than the memory in an input queued scheme by a factor equal to the number of interfaces.
[0014] In order to improve efficiency and thus the general performance in the case of high performance systems, it is preferable to use input queuing to accommodate the limited memory speeds available. The following description of the Head of Line Blocking problem assumes an input queued system.
[0015] Head of Line Blocking Problem
[0016] A unicast message, having only one destination, only needs to be routed to one output interface of a switch. Messages may be simply queued in the order received until they can be transmitted through the selected output interface. A problem in the known art occurs when the message at the head of a first queue is to be sent to an output interface that is not available due to a message from another queue using the output interface. The first queue is blocked until the particular output interface is available; no messages from this queue can be sent until the first element in the queue, or “head element”, is cleared by being sent across the switch to the output interface. (The queue may be implemented with each element being a single packet or cell, or may be implemented with each element including all the packets or cells that make up a single message). If the output interface is busy for an extended period, several queues may become blocked. This is known as the Head-of-Line blocking (“HOL blocking”) problem.
[0017] A known technique for approaching the HOL blocking problem for unicast traffic is the use of virtual output queues (“VOQs”). VOQs are virtual (logical) queues maintained in software or hardware; each VOQ is associated with a physical interface. There is a one-to-one correspondence between the VOQs and possible input/output combinations. The number of VOQs needed scales arithmetically as M×N, where M and N are the number of output interfaces and input interfaces, respectively. For a crossbar switch with 16 unicast input interfaces and 16 output interfaces, commonly called a “16×16” switch, 256 VOQs are need. If a particular output interface is tied up, a virtual queue associated with some other output interface can still send messages to that other output interface.
[0018] It should be noted that although the term used is “virtual output queue”, the method is actually an input queued method, as the queues are maintained for each input interface.
[0019] The VOQ method has the drawback that it only applies to unicast routing. VOQs cannot be applied to multicast routing because for multicast the number of VOQs needed to accommodate all possible input/output combinations is prohibitively large, growing exponentially as 2M×N, where M is the number of output interfaces and N is the number of multicast input interfaces. For a switch with 2 input interfaces and 16 output interfaces, a total of 2×216(approximately 130,000) virtual queues would be required to implement VOQs. Very large numbers of VOQs use valuable resources such as memory and chip real estate, and likely cannot fit on a single chip using current technology.
[0020] For multicast routing, it is disclosed in the above referenced, co-pending application, “Multicast Routing with Multicast Virtual Output Queues and Shortest Queue First Allocation”, to use multicast virtual output queues (“MVOQs”) to reduce Head-of-Line blocking and improve performance. MVOQs are virtual queues maintained for an input interface having multicast capability. The number of MVOQs is fewer than the 2Mpossible destination vectors for a multicast message. Because of this lack of a one-to-one correspondence between MVOQs and destination vectors, it is not possible to simply maintain dedicated queues as in the unicast/VOQ case where, for each input interface, a VOQ is dedicated to each output interface. Consequently, the problem of assigning incoming flows of messages to the limited number of MVOQs available must be solved.
[0021] The use of Shortest Queue First (SQF) Allocation to assign flows is also disclosed in the above referenced, co-pending application, “Multicast Routing with Multicast Virtual Output Queues and Shortest Queue First Allocation”. SQF allocation is intelligent and provides improved performance. However, SQF does not always provide optimal performance. SQF is responsive to the length of the virtual queues but is not directly responsive to, and cannot be customized for, characteristics of the traffic encountered or characteristics of the flows that are already in the queues.
[0022] Allocation of flows in such a manner as to improve performance for some kinds of traffic, such as non-uniformly distributed, long-lived, and bursty flows, continues to be a problem. Allocation of flows that can be customized to the traffic is desirable. Multicast transmissions are increasingly common and HOL blocking is an ongoing problem. It will become increasingly common for traffic on networks to be bursty and non-uniform due to increasing popularity of videoconferencing, IP telephony, and other long-lived communications.
[0023] Accordingly, it would be advantageous to be able to route multicast transmissions using virtual queues in a manner that further reduces HOL blocking, improves system performance, and can be adjusted for different traffic patterns.
[0024] This advantage is achieved in the invention in which vectors, called basis vectors, are associated with virtual queues and multicast flows are assigned to the nearest queue first, producing virtual queues that are distinct from each other in terms of the characteristics of the elements in the queue. Distance is measured as described in the Detailed Description below. In a preferred embodiment, vector quantization methods are used to determine the basis vectors to optimize for the traffic pattern encountered. One embodiment of the invention uses an extension of the Lloyd Max Quantizer to carry out the vector quantization.
SUMMARY OF THE INVENTION
[0025] The invention includes a way to route multicast traffic through a switch or other device that uses input queuing. Basis vectors are associated with each virtual queue, which in a preferred embodiment are multicast virtual output queues (MVOQs). Each incoming flow is allocated to the MVOQ whose basis vector is closest to the destination vector of the incoming flow, creating queues whose contents are distinct from each other in terms of destination addresses of the flows in the respective queue. To optimize for traffic encountered basis vectors can be chosen using vector quantization methods. Basis vectors can be set statically or can be selected and updated dynamically, responsive to the traffic encountered or as set by an operator. The invention can reduce the number of virtual queues required for a given incremental improvement in performance or can improve performance for a given number of virtual queues.
BRIEF DESCRIPTION OF THE DRAWINGS
[0026] FIG. 1 is a diagram of a crossbar switch with 2 input interfaces and 4 output interfaces, with messages in a queue for each input interface.
[0027] FIG. 2 is a diagram of a flow table.
[0028] FIG. 3A is a process flow diagram of queuing a cell.
[0029] FIG. 3B is a process flow diagram of updating the basis vectors for a window of incoming cells.
[0030] FIG. 3C is a process flow diagram of calculating the adjustment to a basis vector from one cell.
[0031] FIG. 4 is a process flow diagram of selecting the nearest MVOQ.
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT
[0032] In the following description, a preferred embodiment of the invention is described with regard to preferred process steps and data structures. Embodiments of the invention can be implemented using general purpose processors or special purpose processors operating under program control, or other circuits, adapted to particular process steps and data structures described herein. Implementation of the process steps and data structures described herein would not require undue experimentation or further invention.
[0033] Related Applications
[0034] Inventions described herein can be used in combination or conjunction with inventions described in the following patent application:
[0035] application Ser. No. 09/282,539, in the name of Farshid Moussavi and Dhaval N. Shah, titled “Multicast Routing With Multicast Virtual Output Queues And Shortest Queue First Allocation,” assigned to the same assignee. The application is hereby incorporated by reference as if fully set forth herein.
[0036] Overview of the Invention
[0037] Packets, which are cells in the preferred embodiment, arrive at an input interface of a switch. The switch routes network traffic to the appropriate destination(s). FIG. 1 is a diagram 100 of a cross bar switch having two input interfaces 111, 112 and four output interfaces 151-154. Input queues 120, 130 contain messages for respective input interfaces 111, 112. For convenience, in FIG. 1 each message is depicted as a single entry in the queue although each cell in a message is processed and queued separately, as discussed below, and the cells of different messages may arrive at the input interleaved. The messages in input queues 120, 130 are destined for various combinations of output interfaces. Each message may include one or more packets. The destination output interfaces for each message in the input queues 120, 130 are indicated by the sets of numbers shown in the schematic depiction of the entries in the queues 121-123, 131-134.
[0038] For a switch having M output interfaces, there are 2Mpossible destination address combinations. But, the number of MVOQs available to a multicast input interface is limited and is less than 2M. An allocation policy maps any one of the 2Mdestination vectors to one of the MVOQs.
[0039] It is desirable for the set of multicast groups of the messages in each MVOQ to have as few destinations in common as possible with the set of multicast groups of the messages in other MVOQs, thereby reducing delays due to conflicts at the destinations.
[0040] In the invention, a vector, called a basis vector, is associated with each MVOQ. The first aspect of the invention is that incoming flows are assigned to the queue whose basis vector is closest to the flow's destination vector, measured as described below. This is called Nearest Queue First (“NQF”) allocation.
[0041] The basis vectors can be chosen by any means; however, a second aspect of the invention is to choose the basis vectors to minimize the sum of the distances for all queues between the destination vectors of the cells in the incoming flows that are assigned to each queue and the respective queue's designated basis vector (which is usually reflective of the destination vectors of cells in that queue so far.) In a preferred embodiment, vector quantization methods are used to select the best basis vectors.
[0042] Cell-based Network with Crossbar Switch, non-Global MVOQs, and Modified Lloyd-Max
[0043] For clarity, the invention is described as used in a crossbar switch having one multicast input interface and 16 output interfaces, in a network using cell-based messages. It is to be understood that the invention may be applied to sizes other than 16-output interface crossbar, to switches having both unicast as well as multicast input interfaces, and to switches other than cross bar switches. It is further to be understood that the invention may be applied to data transmissions other than cell-based transmission.
[0044] The invention is described for MVOQs associated with a single multicast input interface. It is to be understood that the invention can be practiced with multiple multicast input interfaces. In addition, the invention can be practiced with global MVOQs (associated with multiple multicast input interfaces) which are disclosed in the above-referenced, co-pending application, “Multicast Virtual Output Queued Multicast Routing With Shortest Queue First Allocation”. With global MVOQs, for a message arriving at a multicast physical input interface, the Nearest Queue First allocation and if desired the vector quantization analysis is performed across all MVOQs and the choice of MVOQ is made based on that analysis. If the desired MVOQ is associated with a second multicast physical input interface, the message may be sent to that input interface to be queued in that MVOQ.
[0045] The invention can be applied to any input-queued system, such as those not using crossbar switches, and these alternatives are within the scope and spirit of the invention.
[0046] The invention is described using vector quantization to determine the basis vectors. Vector quantization is used because it optimizes (by minimizing) the sum of distance of the incoming destination vectors from the basis vector associated with the MVOQ to which each incoming destination vector is assigned. However, basis vectors can be chosen by any means. The basis vectors preferably span the space spanned by the destination vectors, but need not do so.
[0047] An extension of the Lloyd Max quantizer is used to perform the quantization. It is to be understood that the invention can be practiced with other methods of quantizing. Such methods are known in the art of mathematics. It is further to be understood that the invention can be practiced with combinations of allocation policies, such as in combination with Shortest Queue First allocation.
[0048] The invention is described with the basis vectors in the continuous bitmap subspace, as described below. However, the invention can also be practiced with the basis vectors in the discrete bitmap subspace.
[0049] The preferred embodiment is described with dynamic updating of the MVOQ basis vectors and a fixed value for the centroid weight and fixed window length. Alternatively the invention can be practiced with the centroid being implemented by updating each element of the basis vector to be the centroid of the corresponding element of the basis vector and the element of each incoming destination vector in the window. Calculating a centroid is known in the art of linear algebra. Alternatively and additionally, the window length can be varied, such as by action by an operator or in response to traffic.
[0050] The invention alternatively can be practiced with static basis vectors or with basis vectors that are updated according to some external criteria. For example, an operator can set a basis vector to a specific destination address, in response to the information that a large volume of traffic is expected for that address. The basis vectors can be initialized to have predetermined values that are known to perform well for expected traffic patterns.
[0051] Implementation of dynamic updating of basis vectors is described below. To implement static basis vectors, the steps related to updating the basis vectors are skipped. In some embodiments static basis vectors are preferred due to the increased ease of implementation.
[0052] As noted in the above referenced, co-pending application, “Multicast Virtual Output Queued Multicast Routing With Shortest Queue First Allocation”, MVOQs can be used in conjunction with VOQs.
[0053] Cell-based means messages consist of one or more data packets of a fixed length.
[0054] Terminology
[0055] A multicast message is directed to multiple destinations. The “multicast destination vector” or “destination vector” stores the output interfaces for a given message. M is the number of output interfaces in the switch and the vector can be implemented as an M-bit bitmask. Each destination address combination can be represented by an M-bit vector; setting a bit to 1 indicates the message is to be sent to the corresponding output interface. (A broadcast message is treated as having all bits set.) For instance, a switch having 4 output interfaces has 16 possible destination address combinations; a destination vector of [0,1,1,0] means that the message should be sent to output interfaces 1 and 2, but not to output interfaces 0 or 3, in a switch where the four output interfaces are numbered 0, 1 ,2 , and 3.
[0056] A broadcast message is a multicast message. The broadcast message may have a special destination address, or an address where all bits in the destination vector are set. A broadcast message is treated as though all bits in the destination vector are set.
[0057] The “residue” of two destination vectors is the intersection of their set of outputs; that is, the destination output interfaces they have in common.
[0058] A “multicast flow” is a stream of packets, with the same multicast destination vector. A flow is defined as a “new flow” if no packets with that flow's destination vector exist in any of the multicast virtual output queues at the time of its arrival. Note this is not the same as a TCP flow or other flow pertaining to higher layers. The term “flow” used by itself below refers to a multicast flow.
[0059] Two flows are “non overlapping flows” if their destination vectors do not both have a bit set in any location of their destination bitmask. That is, the flows do not share any common output interface.
[0060] Relation of Message to Flows
[0061] A message includes one or more data packets, which are preferably cells (fixed-length packets). A message can be one flow. If the transmission of a message is interrupted for a length of time, a message can give rise to multiple multicast flows, as defined above. However, at the time of handling by the switch, a flow that comprises a complete message is handled the same as a flow that is one of several flows comprising a message.
[0062] Grouping of Flows
[0063] The cells in a multicast flow are preferably assigned to the same multicast virtual output queue.
[0064] Assigning the cells in one multicast flow to different multicast virtual output queues would increase the chance that the head elements of two queues will be the same, resulting in HOL blocking, and is therefore undesirable in most circumstances. In addition, assigning a flow to one queue assures that the cells in the flow retain their ordering; that is, the cells will not arrive at the destination out of order, as might happen if flows were split between different queues. Note that through its lifetime a TCP flow could be assigned to different queues.
[0065] Number of MVOQs
[0066] As disclosed in the above-referenced, co-pending application, “Multicast Routing With Multicast Virtual Output Queues and Shortest Queue First Allocation”, for a switch with 16 outputs, there are preferably eight MVOQs. We describe the allocation method of the current invention for 8 MVOQs in a switch having 16 outputs.
[0067] Distance Between Two Vectors, Discrete Bitmap Space, Continuous Bitmap Space
[0068] In traditional vector quantization, distance corresponds to distortion of a signal. In the current invention, the distance between two vectors is related to residue.
[0069] Distance is the square of the traditional Euclidean |E1| distance. The traditional Euclidean |E1| distance is the square root of the sum of the squares of the difference for each element, i.e. for vectors {right arrow over (a)} and {right arrow over (b)} of length N the Euclidean |E1| distance is
[0070] Alternatively, the Euclidean |E1| distance could be used, but it requires the taking of a square root, which is computationally expensive. Because the parameter of interest is the relative distance we instead use the square of the Euclidean |E1| distance. Other measures of relative distance can be used, such as those known in the art of signal processing. Distance is preferably calculated in a Content Addressable Memory (CAM).
[0071] A discrete bitmap vector is a vector where every element is either 0 or 1. The discrete bitmap subspace is the subspace spanned by all discrete bitmap vectors. Because every element is 0 or 1, the distance between two discrete bitmap vectors turns out to be the number of positions where their elements are different. For a discrete bitmap vector of length L, the largest possible distance is L. For instance, the distance between the 6-bit vectors [1,0,0,0,0,0] and [0,1,0,0,0,0] is 2 because the vectors differ in the first and second positions. As another example, the distance between [1,1,0,0,0,0] and [1,1,1,1,1,1] is 4, because the vectors differ in positions 2-5 (and are the same in positions 0 and 1).
[0072] Destination vectors are discrete bitmap vectors. In an alternative embodiment, basis vectors are also discrete bitmap vectors.
[0073] The continuous bitmap subspace is the subspace spanned by vectors where every element is in the interval [0,1]. That is every element is equal to or greater than 0 and equal to or less than 1. Again, the maximum distance between two vectors of length L, is L.
[0074] Basis vectors are in the continuous bitmap subspace in the preferred embodiment.
[0075] Note that distance is not the scalar product or the overlap: every vector would have maximum overlap with the broadcast vector if this were the measure.
[0076] Choice of Basis Vectors
[0077] In the art of mathematics it is known to map from J-dimensional space to K-dimensional space. Mapping associates any point in J-space (i.e. the endpoint of a vector of dimension J) with a point in K-space (i.e. the endpoint of a vector of dimension K).
[0078] The K-subspace is spanned by k basis vectors. The set of k basis vectors is called a codebook. Methods for codebook design are known in the art of mathematics. At any given time, in the invention the incoming destination vectors received span k dimensions, with k<=16 (the number of output interfaces); and the MVOQ basis vectors span at most 8-dimensions (the number of basis vectors). Depending on the circumstances, at any given point in time, the 8 basis vectors may not necessarily span 8-dimensions. For instance, if all initial vectors are only to output interfaces 1 and 2, the basis vectors may only span 2-space.
[0079] A better codebook results in better grouping of cells based on their destination vectors and ultimately smaller residue.
[0080] Static basis vectors can be chosen in numerous ways. One way to choose basis vectors would simply be to choose the 8 most frequently encountered destination vectors.
[0081] The 8 preferred basis vectors may be orthogonal or non-orthogonal. In the preferred embodiment the vectors are allowed to be non-orthogonal. Non-orthogonality leads to more flexible response to traffic patterns and also enables recalculation of basis vectors to occur independently of each other. That is, each MVOQ basis vector can be updated independently.
[0082] It is desired to minimize the sum of the distances of the incoming vectors from the basis vector of the queue to which each incoming vector is assigned. Choosing the best set of basis vectors, or codebook, by examining every combination of basis vectors against an incoming training stream of destination vectors is impractical due to the large number of possible combinations. However, heuristic codebook design, which is faster than an exhaustive search, and which results in near optimal code books, is known in the art of mathematics.
[0083] In the preferred embodiment, determination of basis vectors is done using the Lloyd Max Quantizer method of vector quantization, modified as described below. The use of the Lloyd Max Quantizer to heuristically determine a codebook is known in the art of mathematics.
[0084] In standard Lloyd Max quantizing, the codebook is restricted so that each component of the basis vectors is in the closed interval [−R,R]. In the preferred embodiment the Lloyd Max Quantizer is modified in the following ways: the codebook is restricted so that each component of every vector is in the closed interval [0,1]; i.e. each component is greater than or equal to 0 and less than or equal to 1. We refer to such vectors as being in the continuous bitmap subspace. In the preferred embodiment a vector is stored as an array of non-negative integers. These integers are normalized to be in the interval [0,1] and quantized for storage, to the precision available, as a fixed point number. (Alternatively the vectors can be stored as arrays of fixed point or floating point numbers, or by other known means.)
[0085] In an alternative embodiment the codebook is restricted to that each component of every vector is in the set {0,1}, i.e. every component is either 0 or else is 1 (and none are negative). One way to implement this embodiment is to take the vector in the preferred embodiment (i.e. the continuous bitmap subspace), and for each element of the vector round the value down to 0 if the element is less than 0.5 or up to 1 if it is greater than or equal to 0.5.
[0086] Queuing a Cell
[0087] In the preferred embodiment, for each queue, counts are kept of the number of cells having a specific destination vector for each destination vector for which there is a cell in the queue. An entry showing the output queue to which the destination vector is assigned is also kept for each destination vector in the queue. This information can be kept in a flow table. A diagram of an example flow table is shown in FIG. 2. In the preferred embodiment, the table is implemented in lookup table in a Content Addressable Memory (CAM).
[0088] FIG. 3A is a process flow diagram 300A for queuing a cell. Once a multicast message is received 310, an MVOQ is selected. Preferably, VOQs are maintained for unicast messages. However, a unicast message may be treated as multicast with a single destination. To select an MVOQ, it is determined whether the cell is part of an existing multicast flow 320 as defined above. This is done by checking the multicast destination vector against the entries in the flow table.
[0089] If a matching entry is found, the cell is assigned to the same MVOQ as the entry and the counter in the flow table for that destination vector is incremented 322. If a matching entry for that destination vector is not found in the flow table, the cell is the first cell in a new flow as defined above. The MVOQ with the basis vector closest to the destination address (Nearest Queue First) is selected 324. A new entry in the flow table is created for this destination vector with the cell count for the entry equal to unity 326.
[0090] Once the MVOQ is selected, in a preferred embodiment the cell is queued 330 by causing the tail element of the queue to point to the cell. The cell is set to have a null pointer in the node that is reserved to point to the next element. MVOQs are preferably maintained as linked lists with a node pointing to the next cell in the queue, with the last cell in the queue having a null pointer. There is a pointer to the head of each queue. A pointer to the tail of the queue is also maintained. In a preferred embodiment, these queues are implemented in hardware.
[0091] Updating the Basis Vectors
[0092] In the preferred embodiment the MVOQ codebook (set of 8 basis vectors), are updated dynamically. The basis vectors can be adjusted in response to the traffic encountered. A process flow diagram 300B for updating the set of basis vectors is shown in FIG. 3B.
[0093] At an initial time not shown in FIG. 3B, preferably the 8 basis vectors are initially set to be an arbitrary subset of the destination vectors of incoming flows, such as the first 8 unique destination vectors. They can alternately be set to arbitrary values, random values, predetermined values, or other values. The initial values of the basis vectors are stored in an array, and a copy of the initial basis vectors are stored in a second storage array.
[0094] Also at an initial time not shown in FIG. 3B, a counter that keeps track of the number of cells processed is initialized to a value of zero (called “window_count” in FIG. 3B).
[0095] After a cell is queued 330, the adjustment to the basis vector for the selected MVOQ due to this cell is calculated and the adjusted basis vector stored in the second storage array 345. The counter of cells processed (window_count) is incremented 350. If the counter indicates the window length (100 in the preferred embodiment) has been reached since the basis vectors were last updated 355, all the basis vectors are updated, by copying the adjusted basis vectors stored in the second storage array onto the basis vector array 370, and the window_count counter is reset to zero 370. The process continues on to process the next cell 360.
[0096] In an alternative embodiment, the MVOQ vectors are not updated but have static values. The static values for the various MVOQ vectors are preferably set to values that are known to perform well with the expected traffic or are otherwise desirable. For example, the 8 static basis vectors can be chosen to be the 8 basis vectors resulting from a dynamic training exercise on simulated or real traffic. Or, one of the basis vectors can be set to be the broadcast vector.
[0097] Updating an MVOQ vector for one input interface 345, using a fixed centroid weight, is illustrated in the pseudo code below and in FIG. 3C. Using a fixed centroid weight reduces calculation time and controls the rate at which change occurs.
| UpdateMVOQVector(MVOQ_vectors, received_vector, input, MVOQ): |
| At each arrival: { |
[0098] where the variable “MVOQ” is the index of the virtual queue to which the incoming message is assigned. The variable “received_vector” is the destination vector of the incoming message, and the multiplication of the received vector by the centroid weight indicates multiplication of every element of the destination vector by the scalar centroid weight 380. The value 1+centroid_weight is similarly a scalar value and the division by this value indicates division of every element of the array in the numerator by this value 385. Vector addition is otherwise implied. This pseudo code applies to the preferred continuous bitmap embodiment and also can be applied to the alternative discrete bitmap embodiment.
[0099] Alternatively, instead of using a fixed centroid weight the actual centroid of the MVOQ basis vector and the destination vector can be used. Calculation of the centroid of two vectors is known in the art of mathematics.
[0100] In the preferred embodiment the centroid weight is fixed to be 0.1. The window size is fixed at 100. These numbers work well for bursty traffic with geometrically distributed burst lengths with an average length of 10 packets, which is similar to traffic currently encountered on the Internet.
[0101] Choosing the Closest MVOQ
[0102] Each incoming new flow to the input interface is assigned to the MVOQ having the basis vector closest to the destination vector of the flow. The steps taken to choose an MVOQ are shown in process flow diagram 400 and can be illustrated in pseudo code. This pseudo code applies to the preferred embodiment, and also can be applied to the discrete bitmap alternative embodiment.
| Allocate(new_vector, queue_vectors) { |
[0103] where J is the number of output interfaces. The minimum distance, min_distance, is initialized to be the furthest possible distance, J 410. The furthest possible is 16 in the preferred embodiment.
[0104] In the preferred embodiment, when there is more than one MVOQ having the same smallest distance, the MVOQ is chosen randomly from among these equidistant and closest MVOQs. The choice is randomized by initializing the index of the starting MVOQ index randomly 420.
[0105] A place-holding, current index is initialized to be the index of the first MVOQ 430. The distance between the basis vector of this current MVOQ indicated by the current index, and the destination vector of the cell is calculated 440, and if the distance is less than the current minimum distance found (min_distance) 450, the nearest queue variable (result_queue) is set to be this queue and the current minimum distance is set to be this distance 460. The distance is calculated for all queues 440-470, and the final result_queue that is the nearest queue is returned 480.
[0106] Sending a Message
[0107] Messages are sent as disclosed in the copending, above-referenced application, “Multicast Routing With Multicast Virtual Output Queues and Shortest Queue First Allocation”, which is incorporated by reference above.
Alternative Embodiments
[0108] Although preferred embodiments are disclosed herein, many variations are possible which remain within the concept, scope, and spirit of the invention, and these variations would become clear to those skilled in the art after perusal of this application.
Claims
What is claimed is:
1. A method for handling multicast flows in a switching device comprising:
receiving a multicast message at an input interface, where said multicast message has an associated destination vector;
associating a basis vector with each multicast virtual output queue of a set of multicast virtual output queues, said set of multicast virtual output queues having more than one and less than 2Nindividual multicast virtual output queues for said input interface, where N equals the number of individual output interfaces to which the said multicast message could be directed;
selecting a basis vector, as a selected basis vector, by comparing said destination vector of said multicast message with each said basis vector; and
assigning said multicast message to a multicast virtual output queue associated with said selected basis vector.
2. A method as in claim 1 wherein said comparing involves determining a distance measure between said destination vector and each said basis vector.
3. A method as in claim 2 wherein said distance measure is a square of a Euclidean distance.
4. A method as in claim 2 further comprising:
calculating said distance measure using a content addressable memory.
5. A method as in claim 2 further comprising:
calculating said distance measure for destination vector {right arrow over (D)} and basis vector {right arrow over (B)} where both are of N dimensions, as
6. A method as in claim 1 further comprising:
computing an updated basis vector in response to said destination vector of said multicast message.
7. A method as in claim 6 further comprising:
saving updated basis vectors in memory; and
substituting said updated basis vectors for use as basis vectors in said associating step in response to expiration of a window count.
8. A method as in claim 6 further comprising:
where {right arrow over (UB)} is said updated basis vector, {right arrow over (SB)} is said selected basis vector, {right arrow over (D)} is said destination vector, and CW is a centroid weight.
computing said updated basis vector by the expression
9. A method as in claim 8 where CW=0.1.
10. A method as in claim 8 where CW equals the centroid of said selected basis vector and said destination vector.
11. A method as in claim 1 further comprising:
sending a head element of a multicast virtual output queue of said set of multicast virtual output queues to output interfaces.
12. A method of routing a packet having an associated destination vector, said packet arriving at an input interface of a switching device, including steps for:
establishing a set of multicast virtual output queues for said input interface;
determining whether said packet is part of an existing flow;
if said packet is part of an existing flow, assigning said packet to one of said set of multicast virtual output queues associated with said existing flow;
if said packet is not part of an existing flow, assigning said packet to one of said set of multicast virtual output queues responsive to said destination vector; and
queuing said packet in said one multicast virtual output queue.
13. A method as in claim 12 wherein said step of assigning includes steps of:
if said packet is part of an existing flow, updating a flow table entry associated with said destination vector;
if said packet is not part of an existing flow, creating a flow table entry associated with said destination vector and said one multicast virtual output queues.
14. A method as in claim 13 including further steps of:
adjusting a copy of said basis vector associated with said one multicast virtual output queues;
incrementing a counter of a number of packets processed; and
updating each said basis vector associated with each of said set of multicast virtual output queues in response to a comparison between said counter and said selected window length.
15. A method as in claim 12, including steps of:
updating said basis vector associated with said one multicast virtual output queueS.
16. A method as in claim 15 wherein said step of updating includes steps of:
replacing said basis vector with the centroid of said basis vector and said destination vector.
17. A method as in claim 15 wherein said step of updating includes steps of:
adjusting said basis vector by an amount calculated using a fixed centroid weight and said destination vector.
18. A method as in claim 17 wherein said basis vector includes elements and said destination vector includes elements, and wherein said step of adjusting includes steps for:
adding to each element of said basis vector the product of the respective each element of said destination vector and said fixed centroid weight;
dividing each element of said basis vector by the sum of unity and said fixed centroid weight.
19. A method as in claim 18 wherein said packet includes a single cell.
20. A method for routing a packet arriving at a multicast input interface of a switching device, said packet having an associated destination vector, including steps for:
associating a basis vector with each multicast virtual output queue of a set of multicast virtual output queues to create a set of basis vectors;
selecting a multicast virtual output queue responsive to a comparison of said destination vector with said set of basis vectors; and
queuing said packet in said multicast virtual output queue.
21. A method as in claim 20 further comprising:
computing an updated basis vector in response to said destination vector of said multicast message.
22. A method as in claim 21 further comprising:
where {right arrow over (UB)} is said updated basis vector, {right arrow over (S)} B is said selected basis vector, {right arrow over (D)} is said destination vector, and CW is a centroid weight.
computing said updated basis vector by the expression
24. A method as in claim 22 further comprising:
setting CW equal the centroid of said selected basis vector and said destination vector.
25. A method as in claim 21 further comprising:
saving updated basis vectors in memory; and
substituting said updated basis vectors for use as basis vectors in response to expiration of a window count.
26. A method as in claim 20 further comprising:
determining a distance measure between said destination vector and each said basis vector in said comparing said destination vector with each said basis vector.
27. A method as in claim 36 further comprising:
computing distance measure as a square of a Euclidean distance.
28. A method as in claim 26 further comprising:
calculating said distance measure for destination vector {right arrow over (D)} and basis vector {right arrow over (B)} where both are of N dimensions, as
29. A method as in claim 26 further comprising:
calculating said distance measure using a content addressable memory.
30. A method for operating a switching device, said switching device having an input interface, said input interface having multicast capability, comprising:
establishing a set of multicast virtual output queues for said input interface;
choosing a set of basis vectors for said set of multicast virtual output queues;
associating each basis vector of said set of basis vectors with a multicast virtual out-put queue of said set of multicast virtual output queues;
receiving a message having an associated direction vector at said input interface;
queuing said message in a multicast virtual output queue in response to a comparison between said direction vector and said set of basis vectors.
31. A method as in claim 30 wherein said set of basis vectors are in a continuous bitmap subspace.
32. A method as in claim 30 wherein said set of basis vectors are in a discrete bitmap subspace.
33. A method as in claim 30 wherein said set of basis vectors are substantially orthogonal.
34. A method as in claim 30 wherein the sum of a distance measure between said set of basis vectors and a set of destination vectors is minimized.
35. A method as in claim 30 wherein at least one of said basis vectors includes a broadcast vector.
36. A method as in claim 30 wherein said step of choosing includes using vector quantization methods.
37. A method as in claim 30 wherein said step of choosing includes using a modified Lloyd Max quantizer.
38. A method as in claim 30 wherein said step of choosing includes setting at least one of said basis vectors to be a predetermined vector.
41. An apparatus having at least one processor and at least one memory coupled to said at least one processor, said apparatus comprising:
a reception module configured to receive a multicast message at an input interface, where said multicast message has an associated destination vector;
an association module configured to associate a basis vector with each multicast virtual output queue of a selected set of multicast virtual output queues, said set of multicast virtual output queues having more than one and less than 2Nindividual multicast virtual out-put queues for said input interface, where N equals the number of individual output interfaces to which the said multicast message could be directed;
a selection module configured to select a basis vector, as a selected basis vector, by comparing said destination vector of said multicast message with each said basis vector; and
an assignment module to assign said multicast message to a multicast virtual output queue associated with said selected basis vector.
42. The apparatus of claim 41 wherein said comparing involves determining a distance measure between said destination vector and each said basis vector.
43. The apparatus of claim 42 wherein said distance measure is a square of a Euclidean distance.
44. An apparatus of claim 41 further comprising:
a send module configured to send a head element of a multicast virtual output queue of said set of multicast virtual output queues to output interfaces.
45. An apparatus for routing a packet having an associated destination vector arriving at an input interface of a switching device, said input interface having at least one processor and at least one memory coupled to said at least one processor, said apparatus including:
a set of multicast virtual output queues for said input interface;
a determination module configured to determine whether said packet is part of an existing flow;
a first assignment module configured to assign said packet to one of said set of multicast virtual output queues associated with said existing flow responsive to a positive determination by said determination module;
a second assignment module configured to assign said packet to one of said set of multicast virtual output queues responsive to said destination vector responsive to a negative determination by said determination module; and
a queuing module configured to queue said packet in said one multicast virtual output queue.
46. The apparatus of claim 45 wherein said first assignment module includes:
an update module configured to update a flow table entry associated with said destination vector responsive to a positive determination by said determination module;
a second update module configured to create a flow table entry associated with said destination vector and said one multicast virtual output queue responsive to a negative determination by said determination module.
47. The apparatus of claim 45 further including:
an update module configured to update said basis vector associated with said one multicast virtual output queue.
48. The apparatus of claim 45 wherein said update module is further configured to replace said basis vector with the centroid of said basis vector and said destination vector.
49. The apparatus of claim 45 wherein said update module further includes:
an adjustment module configured to adjust said basis vector by an amount calculated using a fixed centroid weight and said destination vector.
50. An apparatus having at least one processor and at least one memory coupled to said at least one processor for operating a switching device, said switching device having an input interface, said input interface having multicast capability, said apparatus comprising:
a set of multicast virtual output queues for said input interface;
a choose module configured to choose a set of basis vectors;
an association module configured to associate each basis vector of said set of basis vectors with a multicast virtual output queue of said set of multicast virtual output queues, responsive to said choose module;
a reception module configured to receive messages at said input interface; and
a queue module configured to queue said messages in a multicast virtual output queue of said set of multicast virtual output queues.
51. The apparatus of claim 50 wherein said choose module is configured to choose such that the sum of a distance measure between said set of basis vectors and a set of destination vectors is minimized.
52. The apparatus of claim 50 wherein one of said basis vectors is a broadcast vector.
53. The apparatus of claim 50 wherein said choose module is configured to choose using vector quantization.
54. A computer program product comprising:
a computer usable storage medium having computer readable code embodied therein for causing a computer to route a packet arriving at a switching device, said packet being associated with a set of destinations, said computer readable code includes:
computer readable code configured to receive a multicast message at an input interface, where said multicast message has an associated destination vector;
computer readable code configured to associate a basis vector with each multicast virtual output queue of a set of multicast virtual output queues said set of multicast virtual output queues having more than one and less than 2Nindividual multicast virtual output queues for said input interface, where N equals the number of individual output interfaces to which the said multicast message could be directed;
computer readable code configured to select a basis vector, as a selected basis vector, by comparing said destination vector of said multicast message with each said basis vector; and
computer readable code configured to assign said multicast message to a multicast virtual output queue associated with said selected basis vector.
55. The computer program product of claim 54 wherein said comparing involves determining a distance measure between said destination vector and each said basis vector.
56. The computer program product of claim 55 wherein said distance measure is a square of a Euclidean distance.
57. A computer program of claim 54 further comprising:
computer readable code configured to send a head element of a multicast virtual output queue of said set of multicast virtual output queues to output interfaces.
58. A computer program product comprising:
a computer usable storage medium having computer readable code embodied therein for operating a switching device, said switching device having an input interface, said input interface having multicast capability, said input interface having a set of multicast virtual output queues, said computer readable code includes:
computer readable code configured to choose a set of basis vectors;
computer readable code configured to associate each basis vector of said set of basis vectors with a multicast virtual output queue of said set of multicast virtual output queues;
computer readable code configured to receive messages at said input interface; and
computer readable code configured to queue said messages in a multicast virtual output queue of said set of multicast virtual output queues.
59. A computer program product comprising:
a computer data signal embodied in a carrier wave having computer readable code embodied therein for causing a computer to route a packet arriving at a switching device, said packet being associated with a set of destinations, said computer readable code includes:
computer readable code configured to receive a multicast message at an input interface, where said multicast message has an associated destination vector;
computer readable code configured to associate a basis vector with each multicast virtual output queue of a set of multicast virtual output queues said set of multicast virtual output queues having more than one and less than 2Nindividual multicast virtual output queues for said input interface, where N equals the number of individual output interfaces to which the said multicast message could be directed;
computer readable code configured to select a basis vector, as a selected basis vector, by comparing said destination vector of said multicast message with each said basis vector; and
computer readable code configured to assign said multicast message to a multicast virtual output queue associated with said selected basis vector.
60. A computer product of claim 59 further including:
computer readable code configured to send a head element of a multicast virtual output queue of said set of multicast virtual output queues to output interfaces.
61. A switching device, comprising:
means for receiving a multicast message at an input interface, where said multicast message has an associated destination vector;
means for associating a basis vector with each multicast virtual output queue of a set of multicast virtual output queues, said set of multicast virtual output queues having more than one and less than 2Nindividual multicast virtual output queues for said input interface, where N equals the number of individual output interfaces to which the said multicast message could be directed;
means for selecting a basis vector, as a selected basis vector, by comparing said destination vector of said multicast message with each said basis vector; and assigning said multicast message to a multicast virtual output queue associated with said selected basis vector.
62. A switching device as in claim 61 further comprising:
means for sending a head element of a multicast virtual output queue of said set of multicast virtual output queues to output interfaces.
63. A switching device as in claim 61 further comprising:
means for determining a distance measure between said destination vector and each said basis vector in said comparing said destination vector with each said basis vector.
64. A switching device as in claim 63 further comprising:
means for computing said distance measure as a square of a Euclidean distance.
65. A switching device as in claim 63 further comprising:
means for calculating said distance measure for destination vector {right arrow over (D)} and basis vector {right arrow over (B)} where both are of N dimensions, as
66. A switching device as in claim 63 further comprising:
means for calculating said distance measure using a content addressable memory.
67. A switching device as in claim 61 further comprising:
means for computing an updated basis vector in response to said destination vector of said multicast message.
68. A switching device as in claim 67 further comprising:
where {right arrow over (UB)} is said updated basis vector, {right arrow over (SB)} is said selected basis vector, {right arrow over (D)} is said destination vector, and CW is a centroid weight.
means for computing said updated basis vector by the expression
70. A switching device as in claim 68 further comprising:
means for setting CW equal the centroid of said selected basis vector and said destination vector.
71. A switching device as in claim 68 further comprising:
means for saving updated basis vectors in memory; and
means for substituting said updated basis vectors for use as basis vectors in response to expiration of a window count.
72. A switching device, comprising:
means for receiving a packet at a multicast input interface of a switching device, said packet having an associated destination vector;
means for associating a basis vector with each multicast virtual output queue of a set of multicast virtual output queues to create a set of basis vectors;
means for selecting a multicast virtual output queue responsive to a comparison of said destination vector with said set of basis vectors, and
means for queuing said packet in said multicast virtual output queue.
73. A switching device as in claim 72 further comprising:
means for sending a head element of a multicast virtual output queue of said set of multicast virtual output queues to output interfaces.
74. A switching device as in claim 72 further comprising:
means for determining a distance measure between said destination vector and each said basis vector in said comparing said destination vector with each said basis vector.
75. A switching device as in claim 74 further comprising:
means for computing said distance measure as a square of a Euclidean distance.
76. A switching device as in claim 74 further comprising:
means for calculating said distance measure for destination vector {right arrow over (D)} and basis vector {right arrow over (B)} where both are of N dimensions, as
77. A switching device as in claim 74 further comprising:
means for calculating said distance measure using a content addressable memory.
78. A switching device as in claim 72 further comprising:
means for computing an updated basis vector in response to said destination vector of said multicast message.
79. A switching device as in claim 78 further comprising:
where {right arrow over (UB)} is said updated basis vector, {right arrow over (SB)} is said selected basis vector, {right arrow over (D)} is said destination vector, and CW is a centroid weight.
means for computing said updated basis vector by the expression
81. A switching device as in claim 79 further comprising:
means for setting CW equal the centroid of said selected basis vector and said destination vector.
82. A switching device as in claim 78 further comprising:
means for saving updated basis vectors in memory, and means for substituting said updated basis vectors for use as basis vectors in response to expiration of a window count.
83. A method as in claim 20 further comprising:
sending a head element of a multicast virtual output queue of said set of multicast virtual output queues to output interfaces.
Patent Citations (216)
| Patent | Date | Inventor | Cited By |
|---|---|---|---|
| US4131767(A) | 1978-12-01 | Weinstein | Applicant |
| US4161719(A) | 1979-07-01 | Parikh et al. | Applicant |
| US4316284(A) | 1982-02-01 | Howson | Applicant |
| US4397020(A) | 1983-08-01 | Howson | Applicant |
| US4419728(A) | 1983-12-01 | Larson | Applicant |
| US4424565(A) | 1984-01-01 | Larson | Applicant |
| US4437087(A) | 1984-03-01 | Petr | Applicant |
| US4438511(A) | 1984-03-01 | Baran | Applicant |
| US4439763(A) | 1984-03-01 | Limb | Applicant |
| US4445213(A) | 1984-04-01 | Baugh et al. | Applicant |
| US4446555(A) | 1984-05-01 | Devault et al. | Applicant |
| US4456957(A) | 1984-06-01 | Schieltz | Applicant |
| US4464658(A) | 1984-08-01 | Thelen | Applicant |
| US4499576(A) | 1985-02-01 | Fraser | Applicant |
| US4506358(A) | 1985-03-01 | Montgomery | Applicant |
| US4507760(A) | 1985-03-01 | Fraser | Applicant |
| US4532626(A) | 1985-07-01 | Flores et al. | Applicant |
| US4644532(A) | 1987-02-01 | George et al. | Applicant |
| US4646287(A) | 1987-02-01 | Larson et al. | Applicant |
| US4677423(A) | 1987-06-01 | Benvenuto et al. | Applicant |
| US4679189(A) | 1987-07-01 | Olson et al. | Applicant |
| US4679227(A) | 1987-07-01 | Hughes-Hartogs | Applicant |
| US4723267(A) | 1988-02-01 | Jones et al. | Applicant |
| US4731816(A) | 1988-03-01 | Hughes-Hartogs | Applicant |
| US4750136(A) | 1988-06-01 | Arpin et al. | Applicant |
| US4757495(A) | 1988-07-01 | Decker et al. | Applicant |
| US4763191(A) | 1988-08-01 | Gordon et al. | Applicant |
| US4769810(A) | 1988-09-01 | Eckberg, Jr. et al. | Applicant |
| US4769811(A) | 1988-09-01 | Eckberg, Jr. et al. | Applicant |
| US4771425(A) | 1988-09-01 | Baran et al. | Applicant |
| US4819228(A) | 1989-04-01 | Baran et al. | Applicant |
| US4827411(A) | 1989-05-01 | Arrowood et al. | Applicant |
| US4833706(A) | 1989-05-01 | Hughes-Hartogs | Applicant |
| US4835737(A) | 1989-05-01 | Herrig et al. | Applicant |
| US4879551(A) | 1989-11-01 | Georgiou et al. | Applicant |
| US4893306(A) | 1990-01-01 | Chao et al. | Applicant |
| US4903261(A) | 1990-02-01 | Baran et al. | Applicant |
| US4922486(A) | 1990-05-01 | Lidinsky et al. | Applicant |
| US4933937(A) | 1990-06-01 | Konishi | Applicant |
| US4960310(A) | 1990-10-01 | Cushing | Applicant |
| US4962497(A) | 1990-10-01 | Ferenc et al. | Applicant |
| US4962532(A) | 1990-10-01 | Kasiraj et al. | Applicant |
| US4965772(A) | 1990-10-01 | Daniel et al. | Applicant |
| US4970678(A) | 1990-11-01 | Sladowski et al. | Applicant |
| US4979118(A) | 1990-12-01 | Kheradpir | Applicant |
| US4980897(A) | 1990-12-01 | Decker et al. | Applicant |
| US4991169(A) | 1991-02-01 | Davis et al. | Applicant |
| US5003595(A) | 1991-03-01 | Collins et al. | Applicant |
| US5014265(A) | 1991-05-01 | Hahne et al. | Applicant |
| US5020058(A) | 1991-05-01 | Holden et al. | Applicant |
| US5033076(A) | 1991-07-01 | Jones et al. | Applicant |
| US5040176(A) | 1991-08-01 | Barzilai et al. | Examiner |
| US5054034(A) | 1991-10-01 | Hughes-Hartogs | Applicant |
| US5059925(A) | 1991-10-01 | Weisbloom | Applicant |
| US5072449(A) | 1991-12-01 | Enns et al. | Applicant |
| US5088032(A) | 1992-02-01 | Bosack | Applicant |
| US5095480(A) | 1992-03-01 | Fenner | Applicant |
| USRE33900(E) | 1992-04-01 | Howson | Applicant |
| US5115431(A) | 1992-05-01 | Williams et al. | Applicant |
| US5128945(A) | 1992-07-01 | Enns et al. | Applicant |
| US5136580(A) | 1992-08-01 | Videlock et al. | Applicant |
| US5166930(A) | 1992-11-01 | Braff et al. | Applicant |
| US5199049(A) | 1993-03-01 | Wilson | Applicant |
| US5206886(A) | 1993-04-01 | Bingham | Applicant |
| US5208811(A) | 1993-05-01 | Kashio et al. | Applicant |
| US5212686(A) | 1993-05-01 | Joy et al. | Applicant |
| US5224099(A) | 1993-06-01 | Corbalis et al. | Applicant |
| US5226120(A) | 1993-07-01 | Brown et al. | Applicant |
| US5228062(A) | 1993-07-01 | Bingham | Applicant |
| US5229994(A) | 1993-07-01 | Balzano et al. | Applicant |
| US5237564(A) | 1993-08-01 | Lespagnol et al. | Applicant |
| US5241682(A) | 1993-08-01 | Bryant et al. | Applicant |
| US5243342(A) | 1993-09-01 | Kattemalalavadi et al. | Applicant |
| US5243596(A) | 1993-09-01 | Port et al. | Applicant |
| US5247516(A) | 1993-09-01 | Bernstein et al. | Applicant |
| US5249178(A) | 1993-09-01 | Kurano et al. | Applicant |
| US5253251(A) | 1993-10-01 | Aramaki | Applicant |
| US5255291(A) | 1993-10-01 | Holden et al. | Applicant |
| US5260933(A) | 1993-11-01 | Rouse | Applicant |
| US5260978(A) | 1993-11-01 | Fleischer et al. | Applicant |
| US5268592(A) | 1993-12-01 | Bellamy et al. | Applicant |
| US5268900(A) | 1993-12-01 | Hluchyj et al. | Applicant |
| US5271004(A) | 1993-12-01 | Proctor et al. | Applicant |
| US5274631(A) | 1993-12-01 | Bhardwaj | Applicant |
| US5274635(A) | 1993-12-01 | Rahman et al. | Applicant |
| US5274643(A) | 1993-12-01 | Fisk | Applicant |
| US5280470(A) | 1994-01-01 | Buhrke et al. | Applicant |
| US5280480(A) | 1994-01-01 | Pitt et al. | Applicant |
| US5280500(A) | 1994-01-01 | Mazzola et al. | Applicant |
| US5283783(A) | 1994-02-01 | Nguyen et al. | Applicant |
| US5287103(A) | 1994-02-01 | Kasprzyk et al. | Applicant |
| US5287453(A) | 1994-02-01 | Roberts | Applicant |
| US5291482(A) | 1994-03-01 | McHarg et al. | Applicant |
| US5299260(A) | 1994-03-01 | Shaio | Applicant |
| US5303383(A) | 1994-04-01 | Neches et al. | Examiner |
| US5305311(A) | 1994-04-01 | Lyles | Applicant |
| US5307343(A) | 1994-04-01 | Bostica et al. | Applicant |
| US5309437(A) | 1994-05-01 | Perlman et al. | Applicant |
| US5311509(A) | 1994-05-01 | Heddes et al. | Applicant |
| US5313454(A) | 1994-05-01 | Bustini et al. | Applicant |
| US5313582(A) | 1994-05-01 | Hendel et al. | Applicant |
| US5317562(A) | 1994-05-01 | Nardin et al. | Applicant |
| US5319644(A) | 1994-06-01 | Liang | Applicant |
| US5327421(A) | 1994-07-01 | Hiller et al. | Applicant |
| US5331637(A) | 1994-07-01 | Francis et al. | Applicant |
| US5345445(A) | 1994-09-01 | Hiller et al. | Applicant |
| US5345446(A) | 1994-09-01 | Hiller et al. | Applicant |
| US5359592(A) | 1994-10-01 | Corbalis et al. | Applicant |
| US5361250(A) | 1994-11-01 | Nguyen et al. | Applicant |
| US5361256(A) | 1994-11-01 | Doeringer et al. | Applicant |
| US5361259(A) | 1994-11-01 | Hunt et al. | Applicant |
| US5365524(A) | 1994-11-01 | Hiller et al. | Applicant |
| US5367517(A) | 1994-11-01 | Cidon et al. | Applicant |
| US5371852(A) | 1994-12-01 | Attanasio et al. | Applicant |
| US5386567(A) | 1995-01-01 | Lien et al. | Applicant |
| US5390170(A) | 1995-02-01 | Sawant et al. | Applicant |
| US5390175(A) | 1995-02-01 | Hiller et al. | Applicant |
| US5394394(A) | 1995-02-01 | Crowther et al. | Applicant |
| US5394402(A) | 1995-02-01 | Ross | Applicant |
| US5400325(A) | 1995-03-01 | Chatwani et al. | Applicant |
| US5402415(A) | 1995-03-01 | Turner | Applicant |
| US5408469(A) | 1995-04-01 | Opher et al. | Applicant |
| US5416842(A) | 1995-05-01 | Aziz | Applicant |
| US5422880(A) | 1995-06-01 | Heitkamp et al. | Applicant |
| US5422882(A) | 1995-06-01 | Hiller et al. | Applicant |
| US5423002(A) | 1995-06-01 | Hart | Applicant |
| US5426636(A) | 1995-06-01 | Hiller et al. | Applicant |
| US5428607(A) | 1995-06-01 | Hiller et al. | Applicant |
| US5430715(A) | 1995-07-01 | Corbalis et al. | Applicant |
| US5430729(A) | 1995-07-01 | Rahnema | Applicant |
| US5442457(A) | 1995-08-01 | Najafi | Applicant |
| US5442630(A) | 1995-08-01 | Gagliardi et al. | Applicant |
| US5452297(A) | 1995-09-01 | Hiller et al. | Applicant |
| US5473599(A) | 1995-12-01 | Li et al. | Applicant |
| US5473607(A) | 1995-12-01 | Hausman et al. | Applicant |
| US5477541(A) | 1995-12-01 | White et al. | Applicant |
| US5485455(A) | 1996-01-01 | Dobbins et al. | Applicant |
| US5490140(A) | 1996-02-01 | Abensour et al. | Applicant |
| US5490258(A) | 1996-02-01 | Fenner | Applicant |
| US5491687(A) | 1996-02-01 | Christensen et al. | Applicant |
| US5491804(A) | 1996-02-01 | Heath et al. | Applicant |
| US5497368(A) | 1996-03-01 | Reijnierse et al. | Applicant |
| US5504747(A) | 1996-04-01 | Sweasey | Applicant |
| US5509006(A) | 1996-04-01 | Wilford et al. | Applicant |
| US5517494(A) | 1996-05-01 | Green | Applicant |
| US5519704(A) | 1996-05-01 | Farinacci et al. | Applicant |
| US5519858(A) | 1996-05-01 | Walton et al. | Applicant |
| US5526489(A) | 1996-06-01 | Nilakantan et al. | Applicant |
| US5530963(A) | 1996-06-01 | Moore et al. | Applicant |
| US5535195(A) | 1996-07-01 | Lee | Applicant |
| US5539734(A) | 1996-07-01 | Burwell et al. | Applicant |
| US5541911(A) | 1996-07-01 | Nilakantan et al. | Applicant |
| US5546370(A) | 1996-08-01 | Ishikawa | Applicant |
| US5555244(A) | 1996-09-01 | Gupta et al. | Applicant |
| US5561669(A) | 1996-10-01 | Lenney et al. | Applicant |
| US5583862(A) | 1996-12-01 | Callon | Applicant |
| US5592470(A) | 1997-01-01 | Rudrapatna et al. | Applicant |
| US5598581(A) | 1997-01-01 | Daines et al. | Applicant |
| US5600798(A) | 1997-02-01 | Cherukuri et al. | Applicant |
| US5604868(A) | 1997-02-01 | Komine et al. | Applicant |
| US5608726(A) | 1997-03-01 | Virgile | Applicant |
| US5617417(A) | 1997-04-01 | Sathe et al. | Applicant |
| US5617421(A) | 1997-04-01 | Chin et al. | Applicant |
| US5630125(A) | 1997-05-01 | Zellweger | Applicant |
| US5631908(A) | 1997-05-01 | Saxe | Applicant |
| US5632021(A) | 1997-05-01 | Jennings et al. | Applicant |
| US5634010(A) | 1997-05-01 | Ciscon et al. | Applicant |
| US5638359(A) | 1997-06-01 | Peltola et al. | Applicant |
| US5644718(A) | 1997-07-01 | Belove et al. | Applicant |
| US5659684(A) | 1997-08-01 | Giovannoni et al. | Applicant |
| US5666353(A) | 1997-09-01 | Klausmeier et al. | Applicant |
| US5673265(A) | 1997-09-01 | Gupta et al. | Applicant |
| US5678006(A) | 1997-10-01 | Valizadeh et al. | Applicant |
| US5680116(A) | 1997-10-01 | Hashimoto et al. | Applicant |
| US5684797(A) | 1997-11-01 | Aznar et al. | Applicant |
| US5687324(A) | 1997-11-01 | Green et al. | Applicant |
| US5689505(A) | 1997-11-01 | Chiussi et al. | Examiner |
| US5689506(A) | 1997-11-01 | Chiussi et al. | Applicant |
| US5694390(A) | 1997-12-01 | Yamato et al. | Applicant |
| US5724351(A) | 1998-03-01 | Chao et al. | Applicant |
| US5748186(A) | 1998-05-01 | Raman | Applicant |
| US5748617(A) | 1998-05-01 | McLain, Jr. | Applicant |
| US5754547(A) | 1998-05-01 | Nakazawa | Applicant |
| US5802054(A) | 1998-09-01 | Bellenger | Applicant |
| US5835710(A) | 1998-11-01 | Nagami et al. | Applicant |
| US5854903(A) | 1998-12-01 | Morrison et al. | Applicant |
| US5856981(A) | 1999-01-01 | Voelker | Applicant |
| US5892924(A) | 1999-04-01 | Lyon et al. | Applicant |
| US5898686(A) | 1999-04-01 | Virgile | Applicant |
| US5898687(A) | 1999-04-01 | Harriman et al. | Applicant |
| US5903559(A) | 1999-05-01 | Acharya et al. | Applicant |
| US5959989(A) | 1999-09-01 | Gleeson et al. | Applicant |
| US5974501(A) | 1999-10-01 | Shaver et al. | Applicant |
| US6049546(A) | 2000-04-01 | Ramakrishnan | Examiner |
| US6055364(A) | 2000-04-01 | Speakman et al. | Applicant |
| US6115462(A) | 2000-09-01 | Servi et al. | Applicant |
| US6188698(B1) | 2001-02-01 | Galand et al. | Applicant |
| US6212181(B1) | 2001-04-01 | Divivier | Applicant |
| US6212182(B1) | 2001-04-01 | McKeown | Applicant |
| US6216167(B1) | 2001-04-01 | Momirov | Examiner |
| US6243360(B1) | 2001-06-01 | Basilico | Examiner |
| US6295285(B1) | 2001-09-01 | Whitehead | Applicant |
| US6320861(B1) | 2001-11-01 | Adam et al. | Examiner |
| US6335939(B1) | 2002-01-01 | Hanna et al. | Applicant |
| US6351466(B1) | 2002-02-01 | Prabhakar et al. | Examiner |
| US6449283(B1) | 2002-09-01 | Chao et al. | Examiner |
| US6501754(B1) | 2002-12-01 | Ohba et al. | Examiner |
| US6603772(B1) | 2003-08-01 | Moussavi et al. | Examiner |
| US2001/0050916(A1) | 2001-12-01 | Krishna et al. | Applicant |
| EP384758(A2) | 1990-08-01 | Applicant | |
| EP431751(A1) | 1991-06-01 | Applicant | |
| EP567217(A2) | 1993-10-01 | Applicant | |
| WO93/07569 | 1993-04-01 | Applicant | |
| WO93/07692 | 1993-04-01 | Applicant | |
| WO94/01828 | 1994-01-01 | Applicant | |
| WO95/20850 | 1995-08-01 | Applicant |
Non-Patent Literature (12)
- Moussaui and Shah, “Improving Multicast Performance for Input Queued Crossbar Cell Switches,” Jun. 4, 1998.Applicant
- Allen, M., Novell IPX Over Various WAN Media (IPXW AN), Network Working Group, RFC 1551, Dec. 1993, pp. 1-22.Applicant
- Becker, D., “3c589.c: A 3c589 EhterLink3 ethernet driver for linux,” [email protected], May 3, 1994, pp. 1-13.Applicant
- Chowdhury, et al., “Alternative Bandwidth Allocation Algorithms for Packet Video in ATM Networks,” INFOCOM 1992, pp. 1061-1068.Applicant
- Doeringer, W., “Routing on Longest-Matching Prefixes,” IEEE/ACM Transactions in Networking, vol. 4, No. 1, Feb. 1996, pp. 86-97.Applicant
- Esaki, et al., “Datagram Delivery in an ATM-Internet,” 2334b IEICE Transactions on Communications, Mar. 1994, No. 3, Tokyo, Japan.Applicant
- IBM Corporation, “Method and Apparatus for the Statistical Multiplexing of Voice, Data and Image Signals,” IBM Technical Disclosure Bulletin, No. 6, Nov. 1992, pp. 409-411.Applicant
- Pei, et al., “Putting Routing Tables in Silicon,” IEEE Network Magazine, Jan. 1992, pp. 42-50.Applicant
- Perkins, D., “Requirements for an Internet Standard Point-to-Point Protocol,” Network Working Group, RFC 1547, Dec. 1993, pp. 1-19.Applicant
- Simpson, W., “The Point-to-Point Protocol (PPP),” Network Working Group, RFC 1548, Dec. 1993, pp. 1-53.Applicant
- Tsuchiya, P.F., “A Search Algorithm for Table Entries with Non-Contiguous Wildcarding,” Abstract, Bellcore.Applicant
- Zhang, et al., “Rate-Controlled Static-Priority Queuing,” INFOCOM 1993, pp. 227-236.Applicant