WDM optical network with passive pass-through at each node

[0001] The present invention relates to a method and apparatus for establishing communication over an optical network employing wavelength division multiplexing.

[0002] The ever-increasing demand for bandwidth has spurred the growth of high speed transport networks. Currently, the key standard for use in such networks is SONET, which is an acronym for Synchronous Optical Network. The SONET standard defines a hierarchy of optical transmission rates over point-to-point and ring network topologies. For example, the SONET optical carrier-level 3 (OC-3) transmits at 155 Mb/s and OC-12 transmits at 622 Mb/s.

[0003] SONET was developed to provide a survivable transport infrastructure that could carry a range of different payload protocols and payload bit rates.

[0004] Survivability is achieved in SONET using a ring topology with a SONET-defined standard protocol for coordinating traffic restoration in the event of a failure. On a ring, there are always two diverse routes that can be used to interconnect any two nodes on the ring. In the event of a failure of one of those routes, spare capacity on the other route is used to restore the traffic affected by the failure. In SONET, every node must terminate the entire optical signal in order to be able to access every payload, even though typically, a node would only access a small subset of the payloads and pass the rest of them downstream to other nodes. Termination of the entire optical signal is also required to give each node access to an automatic protection switching (APS) protocol that coordinates access to the spare capacity by the node during failure events. Unfortunately, this requirement of SONET to terminate the entire optical signal at every node makes upgrading the capacity of the ring a slow and costly process, because every node must be upgraded even though it may be that only one node requires the additional capacity.

[0005] In order to carry a variety of payloads and payload bit rates, the SONET standard defines a payload envelope structure into which all payloads must be mapped. This envelope is then carried within one timeslot within the time division multiplexed SONET signal. Although this provides a SONET network with the ability to carry a variety of payloads, a new payload cannot be transported until a mapping is defined and the interface circuit is developed and deployed. In addition, if there is insufficient spare capacity in the network to handle the new payload bit rate, then the entire network may have to be upgraded. Thus, SONET networks are not responsive to the needs of today's services, which are demanding greater capacity and introducing a wide range of protocols.

[0006] The networks of today's telecommunications carriers typically consist of an access portion that connects end-users to the carrier's network, and a transport portion (sometimes called backbone or core network) that provides the interconnection between the access networks. The access portion of the network is under pressure to provide a greater variety of signal types such as asynchronous transfer mode (ATM), asynchronous digital subscriber loops (ADSL), and SONET, to handle the emerging diversity of services. These new payloads also tend to require greater bit rates to support the underlying services. Transport networks are under pressure to provide more capacity due to the higher bit rate services coming out of the access networks as well as the growth in the number and size of the access networks reflecting the growth in the number of end-users.

[0007] An object of the invention is to alleviate the limitations in SONET-based networks.

[0008] According to the present invention there is provided a communications network employing wavelength division multiplexing, comprising a plurality of nodes; an optical transmission medium interconnecting said nodes, said transmission medium being capable of a carrying a plurality of wavelengths organized into bands; and an interface at each node for dropping a band associated therewith, adding a band carrying traffic for another node, and passively forwarding other bands; whereby communication can be established directly between a pair of nodes in said network sharing a common band without the active intervention of any intervening node.

[0009] A network in accordance with the invention is protocol and bit rate independent and is therefore more responsive than SONET to the demands placed on the access and transport networks of telecommunications carriers. Each payload is carried on separate optical wavelengths and payloads are multiplexed using wavelength division multiplexing techniques.

[0010] A band consists of a group of closely spaced wavelengths. A guard space normally exists between the bands to allow for simple filtering of a band out of the overall spectrum. For example, a band may consist of four wavelengths spaced at 1.6 nm intervals giving a total width for the band of 41.66.4 nm. With a guard band of 3.2 nm, the overall band spacing would be 6.43.29.6 nm.

[0011] A band is associated with the connection between two nodes, such that if, for example, node A wishes to communicate with node C via intervening node B, both node A and node C must access (add/drop) the same band, say band X. Node A would transmit to node C on band X, which would be passively forwarded by intervening node.

[0012] The use of bands as distinct from discrete wavelengths allows the filter specifications to be relaxed in the area of sideband roll-off slope since there are cascaded filters involved at each node. A primary (or band) filter discriminates a band of wavelengths. Further sub-division into specific channels is done with a narrow width filter(s) that is sub-tended after the band filter.

[0013] The use of a multi-level filtering approach is more energy efficient than other arrangements for ring networks. This is due to the fact that the band filter is the primary filter element that is repeated around the ring. As nodes are added to the ring, the attenuation loss due to the band filter element does not rise as fast as the case where individual wavelengths are filtered out at a node with the residual band being passed on.

[0014] The interface is typically in the form of a filter which separates out the band to be dropped and forwards the other bands by reflection. The filter acts as a multiplexer/demultiplexer which drops and adds the band associated with the node from the transmission medium.

[0015] The filter is preferably an interference filter with minimal loss, preferably less than 1 dB, and typically 0.5 dB. The division of the wavelength spectrum into bands, each associated with a node, is an important factor in reducing the loss at the passive filter. If individual wavelengths were employed, losses in the order of 3 to 6 dB could be expected, and the maximum size of the network would be very much restricted.

[0016] An important advantage of the present invention is that each wavelength essentially provides a protocol independent high speed bit pipe between a pair of nodes with minimal loss.

[0017] A node in one embodiment may also include a cross connect switch for changing wavelengths. For example, if a path is established between node A and node C over band C, and between node C and node F over band f, and no path exists between node A and node F, node A can send traffic for node F first to node C, which drops the band c, detects that the traffic is for node F, and passes it through the cross connect to forward the traffic in band f, which will be dropped by node F.

[0018] The optical path for the network is thus passive except for nodes where wavelengths are add/dropped. The system also has low overall loss in any wavelength path so that no optical amplifiers need be employed to achieve a 30 km ring circumference. The overall power loss budget is estimated at 30 dB.

[0019] In a typical maximum configuration system, approximately of the optical loss is in the fiber (9 dB) and approximately the loss is in the optical add/drop filters (16 band filters 0.5 db8 dB). The remainder of the 30 dB optical power budget is reserved for connector loss, splices and aging of components.

[0020] According to another aspect of the invention there is provided a method of establishing communication over an network employing wavelength division multiplexing and having a plurality of nodes interconnected by an optical transmission medium capable of carrying a plurality of wavelengths organized into bands, the method comprising the steps of sending traffic destined for a remote node in a band associated with the remote node; passively forwarding said band at any intervening nodes; and dropping said band at said remote node to extract said traffic destined therefor.

[0021] The invention still further provides an interface device for use in an optical network employing wavelength division multiplexing, comprising a demultiplexer for dropping a predetermined band of wavelengths from the network at a node, means for converting optical input signals from said demultiplexer to electrical output signals, means for generating optical output signals from electrical input signals, a multiplexer for adding said optical output signals in a predetermined band to the network, said demultiplexer and multiplexer being arranged to forward passively optical signals in bands other than said band that is dropped.

[0022] In another aspect the invention provides a fiber optic wavelength division multiplexed ring comprising a plurality of switching nodes, means for generating at plurality of wavelengths organized in to bands on said ring, and means for transmitting maintenance channel data on at least one of said wavelengths as a pilot tone.

[0023] The maintenance channel can conveniently be injected by modulating the bias current of the device generating the wavelengths, normally a laser.

[0024] The invention will now be described in more detail, by way of example only, with reference to the accompanying drawings, in which:

[0025] FIG. 1 is a block diagram showing the physical layout of a wavelength division multiplexed (WDM) bi-directional ring network;

[0026] FIG. 2 is a chart of the bands of wavelengths employed in a typical system in accordance with the invention;

[0027] FIG. 3 is a block functional diagram of a network node;

[0028] FIG. 4 is a block diagram network node with an optical cross connect switch;

[0029] FIG. 5 illustrates a ring showing the waveband connections;

[0030] FIG. 6 illustrates a ring showing protection switching;

[0031] FIG. 7 is an example of a hubbed connection pattern;

[0032] FIG. 8 is an example of a meshed connection pattern;

[0033] FIG. 9 illustrates a ring showing examples of payload signals;

[0034] FIG. 10 shows a bit rate consistency monitor;

[0035] FIG. 11 is a block diagram of a maintenance channel signal driver;

[0036] FIG. 12 is a graph showing the spectral density of each wavelength; and

[0037] FIG. 13 is a block diagram of a distributed ATM switch.

[0038] Referring now to FIG. 1, a WDM (Wavelength division Multiplexed) ring network generally referenced 1 consists of two counter rotating rings 2, 3 containing a plurality of nodes 4, 5, 6, 7, 8 providing interfaces to the rings 2, 3. It will be understood that FIG. 1 shows the physical layout of the network. The rings 2, 3 physically consist of optical fibers, which are capable of carrying multiple wavelengths generated by lasers in the nodes. The interconnectivity between the nodes is provided by WDM connections in a manner to be described.

[0039] Each ring may carry, for example, 16 or 32 wavelengths divided into eight bands, which provide the interconnectivity between the nodes. Typically there are either two or four wavelengths per band. With eight bands, there are therefore a total of 16 wavelengths per ring for two wavelengths per band or 32 wavelengths in the case of four wavelengths per band, for example, spaced at 1.6 nm with a guard band of 3.2 nm for a total spacing of 9.6 nm per band. FIG. 2 shows a typical distribution of bands and wavelengths. Typically the maximum number of nodes is eight, assuming eight bands.

[0040] Each node 4, 5, 6, 7, 8, adds/drops a predetermined band of wavelengths specific to that node. For example, node 6 might drop the 1.52411 to 1.51948 m band, which includes wavelengths at 1.52411, 1.52256, 1.52102, and 1.51948 m. In order to establish a path between node 4 and node 6, node 4 transmits to node 6 in this band on both counter rotating rings 2, 3. This band is passively reflected by nodes 5, 7, 8 and forwarded to node 6, which drops the band and extracts the traffic. In accordance with the principles of the invention, the bands of wavelength thus permit direct, protocol independent connections to be made between any nodes on the ring without the intervention of any intermediate node. The nodes on the ring can be interconnected in any traditional manner, for example, in star or mesh connections, by establishing waveband connections between desired pairs of nodes.

[0041] A typical node with a wavelength conversion function will be described in more detail with reference to FIG. 3.

[0042] Demultiplexers 10 and multiplexers 11 are shown connected into the fiber optic rings 2, 3. Demultipexers 10 drop, and multiplexers 11 add, a specific band of wavelengths associated with the node. Physically the MUX/DEMUX 10, 11 each consist of a single high performance optical interference filter that transmits the selected band to be dropped/added and passively reflects the remaining bands. The filters can be, for example, dichroic filters, Bragg grating filters based on circulators, and Bragg grating based on fused biconic taper. A suitable filter is made by JDS Fitel of Ottawa, Canada. Such filters offer low through-path loss (<1 dB), and reasonable (<2 db) add/drop loss. The bands 13 not dropped by the demultiplexers 10 are passively forwarded through the node.

[0043] In physical terms, the bands 13 of wavelengths that are not destined for a particular node are passively reflected on to the next node and so on until they reach the destination node where they are dropped in the manner described.

[0044] The dropped band 12 from each ring 2, 3 is passed to a second fine optical filter 19, which separates the dropped band into the individual wavelengths. The subdivided wavelengths are passed to electro-optic converters 14, which generate electrical signals from the optical inputs. The electric signals are then passed to digital cross connect switch, which connects them to payload interface devices 16 providing access to the network. Alternatively, cross connect 15 permits switching between different wavelengths or bands. In the latter case, the cross connect 15 can be connected to additional MUX/DEMUX filters (not shown) provided at the same node for adding/dropping different bands either on the same ring or a different ring.

[0045] The adding of a band works in the same way as the dropping of a band in reverse. Electrical signals are converted to optical form in electro-optic converters 14 and passed to fine channel filters 18, which combine the specific band of wavelengths that it is desired to add. The output 18 of these filters is passed to MUX 11 and combined with the forwarded bands 13. In physical terms, the added band(s) is/are transmitted through the optical filter and combined with the forwarded bands 13, which are passively reflected.

[0046] The optimum construction for a node filter is a 4, 6 or 10 port device having an in port, out port and 1, 2 or 4 drop ports and 1, 2 or 4 add ports. As there are eight bands, there will be eight varieties of the device, one per band. One such device is used where ever a band is to be add/dropped. Each port corresponds to a specific wavelength.

[0047] The filter is a highly integrated passive optical device. The design and construction of the filter is such that 1 wavelength (approx. 1 nm Bandwidth) is available from a drop port and 1 wavelength (approx. 1 nm Bandwidth) is added to an add port. By using identical band filters at two points on the ring, 1, 2 or 4 wavelengths can be used to implement a bi-directional communications pipe between those points. These pipes are independent of any other wavelengths on the fiber ring so long as no other nodes use the same band filters. Each wavelength used in the system in effect serves as a transparent digital bit pipe. Any specific formatting of data in a wavelength is to be carried out by sub-tending equipment. The system is not protocol dependent. The present invention employs bands to provide protocol independent direct connections between nodes on a physical ring, which in physical terms need not be adjacent.

[0048] FIG. 4 shows a similar arrangement to FIG. 3, except that the electro-optical converters have been omitted and the cross connect switch 115 is an optical switching device that performs optical switching and provides optical outputs to optical interfaces 116. With optical switching, wavelength conversion can be realized optically. Electro-absorption devices and/or semiconductor optical amplifiers (SOAs) may be used to perform the conversion.

[0049] Each node typically has at least one band filter, at least one laser diode, driver and MTCE (maintenance channel modulator), at least one PIN diode receiver, transimpedance amplifier, limit amplifier and MTCE demodulator, fine optical filters, a maintenance channel control processor, with Ethernet port and node control HW, and a per wavelength interface to subtending equipment or test data generator (option module). Optionally, a PLL data regenerator and cross-connect matrix can be provided. A low frequency bandpass filter picks off the maintenance channel data and it is demodulated by a PLL type FSK demodulator. The data stream then goes to the maintenance processor.

[0050] Each wavelength is driven by a DFB (Distributed Feedback) laser or Bragg grating reflector laser at a specific wavelength chosen to match the filter characteristics. The output power of the laser is in the range of 3 dbm to a possible high of 6 dbm. Laser components may require thermal stabilization (also providing a means of micro-tuning to specific wavelengths). The laser is driven by a single chip control circuit that combines the monitor diode and bias requirements of the device. Typically these devices have a PECL differential input.

[0051] FIG. 5 shows one example of a connectivity diagram of a network operating in accordance with the invention. The nodes are physically interconnected in a ring by counter-rotating optic fiber rings as described with reference to FIG. 1 The bands of wavelengths provided direct WDM protocol independent connections between non adjacent rings. In FIG. 5, band X connects node 5 to node 8 in both directions. This means that node 5 and 8 add and drop band X., that is band X is passively reflected by the interference filter at node 4.

[0052] The invention also allows protection switching, with the bands acting as direct connections between the nodes. FIG. 6 is an example of protection switching.

[0053] In FIG. 6, a band connects X two nodes via two diverse paths on opposite arcs 40, 41 of the ring 1. One of these arcs can be used to provide a restoration path for all of the wavelengths in the band in the event of a failure of the other path. In FIG. 6, a band connects nodes A and C. The arc 40 via node B is used normally and the arc via nodes D-Z is spare. Node A and node C monitor the quality of the signals dropped from the band at each end of the connection. In the event of a failure of the connections via node B, nodes A and C re-establish the connections via nodes D-Z.

[0054] The drop nodes may use optical power measurements on each wavelength as a quality measure. If the optical power drops below a preset threshold, then a protection switch is triggered. This measurement is not dependent on the protocol or bit rate of the information carried on the wavelength.

[0055] Another quality measure that is protocol and bit-rate independent is a Bit Rate Consistency Monitor. The drop node counts the number of bits received over a given unit of time (long relative to the bit time of the lowest expected bit rate) and records the value of this count. If the value varies by more than some nominal amount, it is an indication that the channel is carrying noise and has therefore failed.

[0056] An example of a Bit Rate Consistency Monitor is shown in FIG. 10. Incoming serial data 50 is fed to edge counter 51, which inputs an M-bit count to register 53. A reference clock, which has a repetition rate low relative to the minimum serial data rate, is input to register 53 and 54 to latch the count samples n and n1 and also to the counter to reset it between samples. Sample n from register 53 and sample n1 from register 54 are compared in comparator 55, which generates a true/output depending on the consistency of the incoming bit rate.

[0057] In order to coordinate the switching of the traffic, the nodes at each end of the connection must communicate directly For example, in FIG. 6, if a failure of the connection is observed only at node C, then node C may have to communicate with node A to get the wavelengths that were being sent across the band via node B to be sent across the band via node D-Z. This can be accomplished using one or more of the maintenance channels that are carried on a pilot tone on each wavelength.

[0058] FIG. 7 shows a hubbed connection pattern as a further example of the manner in which the nodes can be interconnected in accordance with the principles of the present invention. In FIG. 7, node C (5) acts as the hub from which connections are established to other nodes over dedicated wavebands extending between the hub node 5 and the remaining nodes.

[0059] FIG. 8 shows a meshed arrangement, where the nodes are connected in the form of a mesh pattern. In all cases the wavebands act as bit pipes establishing protocol independent high speed connections directly between nodes, which may be non-adjacent.

[0060] FIG. 9 shows examples of a signal payload that may be carried by a ring operating in accordance with the invention. In FIG. 9, band 30 establishes a protocol independent connection between nodes 4 and 7. This connection can carry SONET OC-3 traffic and Fiber channel traffic directly between the nodes. The system architecture does not need to know anything about the protocols. The band 7 merely delivers a high speed bit stream at node 7, which can be resolved into SONET and Fiber channel streams at the far end node.

[0061] FIG. 9 also shows Gigabit Ethernet and SONET OC-48 traffic between sent between nodes 5 and 8. Again the ring is indifferent to the protocols involved. The data is merely transported as a high speed bit stream over the carrier wavelength without regard to the underlying protocol.

[0062] It is of course possible to cascade multiple rings, interconnecting them at common nodes. However, if many rings are cascaded, dispersion effects and jitter effects on the electrical/optical signals may accumulate. In order to compensate for this, a regeneration stage may need to be added to the cross-connect matrix at selected interconnect points. This re-generation device is a wide range PLL (phase-locked loop) that locks onto the incoming digital data stream, recovers a clock and users the clock to regenerate the digital stream. It is not necessary for the re-generator to know the specific format of the data in the stream, only to be able to lock onto the digital transitions and clean up the edges with respect to dispersion and jitter.

[0063] If the PLL requires setting for lock range or center frequency, this can be accommodated by maintenance channel configuration messages that are directed to the hardware that needs to be controlled.

[0064] A variety of external data sources can be connected to the data path for each wavelength. This can include OC-3, OC-12, a proprietary interface such as Newbridge ISL (Inter Shelf Links) and possibly Gigabit Ethernet sources.

[0065] As mentioned above, the wavelengths carry a maintenance channel, which is driven by an FSK modulator originating directly from the maintenance channel data stream.

[0066] In a multi-node WDM ring network the opportunity exists for overall optimization economies that can be facilitated by the individual nodes being able to communicate with each other to exchange information that is used to control the laser device by adjusting the fundamental operating parameters of the device and other optical elements that may be used in such a network. Methods of local optimization of laser parameters have been discussed in the literature. However, this approach uses and end-to-end approach which is more complete in terms of being able to adjust for network operating parameters. It also allows consideration to be given for specification tolerance reduction of other network elements e.g. filter roll off. centre frequency position and gain elements that may bc present in such a ring configuration.

[0067] For each wavelength operating between two points in the WDM ring, there is a laser source, an add filter (or portion of an add/drop filter), fibre transmission media a drop filter (or portion of an add/drop filter), an optical detector and ancillary receive electronics to route the signal to other portions of the system.

[0068] The laser source is controlled by setting a laser current threshold, modulation current level and operating temperature. The operating wavelength is adjusted (by temp control) so as to provide maximum signal energy to the detector at the far end. This procedure aligns the emitted wavelength with the combined cascade filter response so as to minimizes the losses due to individual component tolerances. It also has the benefit of any medium to long term wavelength variation of the laser from the system loss plan calculations.

[0069] The peak optical power and the extinction ratio (ER) is regulated and controlled by special electronic circuits or by an embedded microcontroller. Laser slope efficiencies at bias level and at peak level can be measured by varying bias current and peak current in very small steps respectively. Such measurement allows the laser ER and peak power to be frequently monitored and controlled.

[0070] Wavelength stability is attained by adjusting the laser operating wavelength (e.g. adjusting laser operating temperature) while monitoring the received power level at the receiving node. Since the WDM filter modules have a narrow pass band (approximately 1 nm) for each wavelength channel and possesses other optical characteristics, it is possible to precisely re-align the laser operating wavelength on a regular basis.

[0071] In certain circumstances when operating the same wavelength on two different segments of the ring it may be necessary to set the operating wavelengths at slightly different points so as to minimize beat noise (a coherent interference effect between optical sources). This noise factor is overcome by having the operating wavelengths separated by come small amount (0.05 nm to 0.2 nm).

[0072] The other parameters of laser operation (threshold) and modulation depth are controlled end-to-end in such a way as to optimize the receive eye signal for a given data rate. The maintenance signal that is superimposed on the optical wavelength gives a means of measuring the error performance of the mtce channel (which is directly proportional to the error rate of the main data channel on the optical beam.). By means of message feedback, the received signal condition state can be sent back to the laser transmitter so that corrective actions or stabilization routines can be run. The specific control routines are software algorithms that run on the embedded processor that is associated with the laser control circuitry. These control algorithms will include both wavelength stabilization routines and received eye signal optimization.

[0073] In a network of WDM laser sources and receivers (including drop/add filters) it may be necessary to add amplification to individual wavelengths or groups of wavelengths so as to obtain sufficient optical energy to achieve a desired bit error rate at the receiver. In a known topology situation a number of EDFA elements could be added to the system to overcome transmission losses due to fibre and filter loss. Due to the topology of the ring and the fact that it is a ring configuration, fixed gain blocks such as EDFA's may bc difficult to specify and may in fact impair the performance of some wavelengths in the system.

[0074] A solution to the problem exists in the form of a SOA element combined with an electrically programmable attenuator element. This technology can be obtained in discrete element form or integrated onto a silicon waveguide structure. The SOA provides the pure gain required. the programmable attenuator allows for signal level optimization on a node to node basis, independent of the levels required for any other node to node level on the ring. The SOA/Attenuator combination may be applied at the laser source (as a post-amplifier), the receiver (as a pre-amplifier) or both.

[0075] In the event that it was desired to use EDFA elements as the gain block in a fibre based ring system, the programmable attenuator could be used to optimize the receive signal level at the receiver. It is felt that this would not provide as flexible a solution as the SOA/attenuator system but would overcome the problem of individual channels gain adjustment that is required in such an amplified add/drop system.

[0076] In all cases the key to the system optimization is the ability for the system to communicate on the mtce channels between source/receiver pairs and to optimize via control algorithms the operating level of the signal in such a way as to obtain the best end-to-end performance and WDM network management.

[0077] The maintenance and control processor of each node is a small computer board that contains processor, ram, flash memory for program and application store and several serial interfaces (one per MTCE link). The processor has an embedded DOS that is augmented with a TCP/IP routing and control module (Flexcom router and control switch). Since the Flexcom product is actually a multi-tasking O/S operating in conjunction with the embedded DOS, several monitor and control functions specific to laser operation and maintenance are integrated into this SW. A status and reporting function is also incorporated.

[0078] By means of this switch, all nodes in a system may be controlled and monitored by a remote PC that operates a Telnet session to each processor. Maintenance traffic may also be routed through a maintenance processor to other nodes or subtending equipment.

[0079] FIG. 11 shows the arrangement for creating the maintenance channel. Driver 90 for laser 91 has a bias input 92 that is modulated by FSK modulator 93 receiving at its input the maintenance channel 95. The data channel, which is all digital, is applied to the main input of the driver 90.

[0080] The arrangement shown in FIG. 11 embeds a pilot tone on each wavelength, which may be of low bit rate (<256 kbps). This pilot tone is injected into the wavelength channel by modulation of the bias current by the FSK modulator 93 that modulates the MTCE channel data stream into a sequence of tones. Other carrier modulation systems such as QAM-64 or QAM-256 or OFDM may be used. The level of the pilot tone is approximately 20 dB below the main data path. The effect of the pilot tone on the BER of the main data channel, which is purely digital, is minimal since it is carried on a portion of the spectral distribution curve well outside the portion carrying the high bit rate data (see FIG. 12).

[0081] The MTCE channel modulation ensures wavelength integrity between nodes, provides a power level estimate of wavelength link, provides node status and monitoring (SNMP, RMON type messaging), distribution of network level timing synchronization, and SW & FW downloads for node processors and sub-tended equipment.

[0082] Since the MTCE channel is modulated independently from the regular data pipe channel, the MTCE does not need to know what the format of data in the main channel is. This is extremely important in allowing format independence of end user applications and access.

[0083] The described arrangement provides a network capable of carrying data in the terabit/sec range over distances of up to 30 km, and more when cascaded rings are provided. It also allows the components of high speed switches, such as ATM switches, to be distributed over a campus wide network, resulting in substantial savings in trunk cards and processors. FIG. 13 is an example of such a distributed switch. Switch components 100 are interconnected over ring 1 using nodes 110 interconnected in the manner described above.