Credit sharing for fibre channel links with multiple virtual channels

[0001] 1. Field of the Invention

[0002] This invention relates generally to network switching devices and more particularly to Fibre Channel switching devices and the credit sharing for Fibre Channel links with multiple virtual channels.

[0003] 2. Description of the Related Art

[0004] The Fibre Channel family of standards (developed by the American National Standards Institute (ANSI)) defines a high speed communications interface for the transfer of large amounts of data between a variety of hardware systems such as personal computers, workstations, mainframes, supercomputers, storage devices and servers that have Fibre Channel interfaces. Use of Fibre Channel is proliferating in client/server applications which demand high bandwidth and low latency I/O such as mass storage, medical and scientific imaging, multimedia communication, transaction processing, distributed computing and distributed database processing applications. U.S. Pat. No. 6,160,813 to Banks et al. discloses one of the state of art Fibre Channel switch systems, which is hereby incorporated by reference.

[0005] One or more interconnected switches form a network, called a fabric, which other devices, such as mass storage devices, servers or workstations, can be connected to Any devices connecting to a fabric can communicate with any other devices connected to the fabric. A direct connection between two devices is a link. An interface on a device for connecting another device is a port. A non-switch device connecting to a fabric is a node on the network or fabric. A port on a non-switch and non-hub device is an N-port. A port on a switch may be an E-port, for connection to another switch port, an F-port, for connection to an N-port, an FL port for connection to an FC-AL loop or any combination of the above. A link between two switches is an inter-switch link (ISL).

[0006] Each port has a transmitter and a receiver. The transmitter sends out information and the receiver receives incoming information. There are buffer memories associated with each port, either the transmitter or the receiver, to temporarily store the information in transit, before the information is confirmed to be transmitted towards its destination by a switch, or to be stored or used by a device at its destination. The buffer memory can be in the actual port or, preferably, may be centralized, as shown in U.S. Pat. No. 6,160,813. The buffer memory is broken down into units. One unit of buffer memory, which can hold one frame, is represented by one buffer-to-buffer credit or one credit. A frame is a unit of information transmitted, which comprises a header portion and a payload portion. The header portion identifies the frame, including a Source Identification (SID) and a Destination Identification (DID). The payload portion contains the data being transmitted. A frame payload may be 2112 data bytes long, which, plus header, CRC, EOF totals 2148 bytes.

[0007] In the prior art, a receiver on a port is allocated a fixed amount of buffer space to temporarily store received frames, represented by a fixed number of buffer-to-buffer credits. The receiver controls the allocation of the buffer space. At the initial configuration when two switches connect, the receivers advertise to the transmitters the amount of buffer space represented by the number of credits available. The transmitters initialize their credit counters to the number of credits advertised by the receivers. Both the transmitting port and receiving port keep track of the use of the buffer space using the number of credits and credit counters. Each time a frame is received by the receiver, the frame is stored in a buffer space and the number of credits residing in the receiver is increased by one. The transmitting port keeps track of this by reducing its transmitter credit counter, which indicates how many more frames can be sent, while the receiver increments its receiver credit counter, which indicates how many frames are stored in the buffer space. Once the frame is confirmed to have been retransmitted by a transmitter on the receiving switch, or used by a device, then the buffer space is free to be used to store a new frame. At that time, a credit is returned by a transmitter on the receiving port to a receiver on the transmitting port and the receiver credit counter in the receiving port is decreased by one. When the transmitting port receives the credit, the transmitter credit counter in the transmitting port is increased by one.

[0008] Even though frames travel through the fiber optics at the speed of light, it still takes time for frames to move from one device to another. It also takes time for a device to receive a frame; process it or retransmit it; and then return a credit, i.e. a confirmation of receipt, back to the transmitting port. It takes some more time for the credit traveling through the optical fiber to reach the transmitting port. During the turn-around time between when the transmitting port sends out a frame and the transmitting port receives a credit, the transmitting port may have sent out a number of frames at its transmitting speed if the transmitting port has available credits. When the transmitting port has at least a minimum number of credits to allow the transmitting port to continue transmitting until it receives credits back from the receiving port, the effective frame transmission rate is the highest. If the transmitting port does not have that minimum number of credits, then it has to temporarily stop sending frames when all the credits are used and wait until the credits return. Due to this stoppage, the effective frame transmission rate may be substantially lower than the actual transmission rate. That minimum number of credits depends on the turn-around time and the frame transmitting speed. The longer the transmission line, or the faster the transmitting speed, the more frames that may be in transit. At a fixed transmitter speed, the more credits a port can have, the longer the transmission distance can be while the link still maintains the full effective transmitter speed.

[0009] In certain switches in the prior art, particularly those using a shared buffer space, there may be a pool of free buffer space which can be used by any ports on the switch needing additional buffer space. A receiving port requests an additional buffer from the pool and transmits an additional credit to the linked transmitting port before the frame is retransmitted from the receiving switch, thus allowing the switch to buffer an additional frame. The credits in this port-level pool are not advertised to the transmitting port. These credits are unknown to the transmitting ports and can only be utilized by the receiving port. When the frame is confirmed out of the receiver, its buffer is released and a credit is returned to the pool, thus allowing other ports on the switch to use the pool buffer space. The port-level pool would be utilized when the frames received from an upstream device are more than the frames retransmitted to a downstream device, for example when the downstream transmission speed is slower than the upstream transmission speed. Another example is when there is a blockage somewhere in the fabric, such as a device holding a loop open, so that frames cannot be delivered.

[0010] A switch may have many ports, e.g. 8, 16 or 64 ports etc., to interconnect many devices. Every device connecting through a single switch to many other devices can communicate with each one of them at the full transmission speed of a port-to-port link if there are enough credits on the ports. Two devices may be connected to two different switches and the two switches are directly connected, then the two devices can communicate with each other through the three links in series: device-to-switch, switch-to-switch (ISL) and switch-to-device. Two devices may also communicate with each other through more intermediate ISLs if the two switches they directly connect to do not share a common ISL. When more than one pair of devices communicates through the same ISL, conflicts may arise. For example, one pair of devices may utilize or hog all of the buffers, or other resources, on the ISL, blocking the other pairs of devices from utilizing the ISL.

[0011] To address this problem, Brocade Communications Systems Fibre Channel switches logically split the physical ISL into a series of virtual channels (VCs). The data flow can then be segmented among the VCs. The buffer space or credits available to the receiver is thus allocated among the VCs so that the blocking problem discussed above does not occur. Each channel will have at least some buffer space available, and so will have some ongoing data flow, albeit potentially at a reduced rate. VCs are also described in U.S. patent application Ser. No. 60/286,213, entitled “Quality of Service Using Virtual Channel Translation,” by David C. Banks and Alex Wang, filed Apr. 24, 2001, which is hereby incorporated by reference.

[0012] In some prior art switches, the buffer space allocation among the VCs has been fixed. The fixed VC credit allocation may result in performance problems, though not as severe as would be present without the use of VCs. If a VC is not in use, then the buffer space allocated to it by the receiver or the credits advertised by the receiver for this VC to the transmitter will not be used. The VCs in use may have their effective transmission rates reduced substantially due to the lack of credits.

[0013] Besides allocating all credits to VCs as in some prior art switches, in short distance applications, a receiving port may reserve some credits in a pool, a VC-level pool, available to any of the VCs in the port and evenly allocate the remaining credits to VCs. These credits in the pool are not advertised to the transmitter. When a VC in use is busy, a credit from the pool may be returned to the transmitting port before a buffer for a frame for the VC is released, i.e. ready to be used by next frame. This effectively increases the credits available for a particular VC in use and reduces credits available for a VC not in use. These credits may also be used to smooth the frame/credit flow due to frame traffic congestion, blockage etc. While the VC level pool on a receiver may provide many benefits, it has a big drawback. Due to the reserved credits in the pool, there are less credits advertised overall and less credits advertised for each individual VC to a transmitting port, so the distance concerns are actually exacerbated.

[0014] While VCs in the prior art are very beneficial in many situations, they do not help in many long distance situations. As noted above, the VC-level credit pooling actually makes the long distance problem worse. Basically the long distance problem of the prior art is that a receiver must advertise a minimum number of credits, otherwise, the transmitter has to stop sending frame and wait periodically between the time it uses up all the advertised credit until it receives a returned credit. The more credits advertised to a transmitter, the less the long distance problem. The more credits actually available to a transmitter, the less the long distance problem. If the credits advertised or available to a transmitter are less than the needed minimum, the effective transmission rate drops substantially, as described above. The use of VCs, and to a greater extent VCs with a credit pool, actually reduces the total credits available to a transmitting port and the credits available to VCs. This long distance problem is illustrated in more detail below.

[0015] This problem could be resolved by providing additional buffer space for each receiver, but in many cases the additional buffer space would not be utilized, thus wasting scarce resources on the ASICs which comprise the receivers. Therefore, it is desirable to reduce the long distance problem but at the same time conserve ASIC resources. It is also desirable not to cause new problems.

[0016] The present invention utilizes virtual channels and a common pool of credits to maximize the efficiency and throughput across an ISL. The control and management of the credits available on a receiver is transferred to a transmitter from a receiver during the initial port connection and configuration. Only the total credits on the receiver, not the credits for each of the VCs, are advertised by the receiver to the transmitter. Each virtual channel is allocated a minimum number of credits and the remaining credits of the ISL are held in a common pool for all the virtual channels in the ISL by the transmitter. When needs arise, the transmitter allocates or reallocates the credits in the pool among the virtual channels to make the best use dynamically.

[0026] FIG. 1 depicts a typical Storage Area Network (SAN) 20 utilizing a Fibre Channel network. A fabric 32 may comprise one or more switches 30. Three switches 30 are shown. Many devices or nodes, such as a general storage device 24, a server 26, database storage devices 28 and a loop 22 (itself comprised of devices, not shown) are connected to the fabric 32. Any devices on the fabric 32 can communicate to any other devices on the fabric 32.

[0027] FIG. 2 is a block diagram showing more details of several particular interconnections in a portion of a SAN 40. Two switches 72 and 74 are shown, together with two servers 52, 54 and two storage devices 62, 64. The N-ports 102, 104, 106 and 108 on storage devices 52 and 54, and servers 62 and 64, and their corresponding F-ports 66, 68, 76 and 78 on the switches 72 and 74 are linked by links 82, 84, 86 and 88. The two switches 72 and 74 have one E-port each, 44 and 46, which are connected through ISL 90. Each switch may also have a central memory 56 and 58 in switches 72 and 74, for buffering information in transit and a control module (not shown) for managing the information flow, such as directing flows of information and managing the buffer space. Each port has a transmitter (T) and a receiver (R). The available buffer spaces in the central memory are allocated among the ports on the switch. The operation of the buffer-to-buffer credit which represents the buffer space is as described above. As above, the transmitter can only send a frame when it has at least one credit for the VC of the frame. When the credits run out, the transmitter has to wait for the credits to be returned by the receiving port on the other end of the link.

[0028] When more than one pair of devices is connected through a common ISL, conflict may arise as shown in FIG. 3. In the simplest example of the problem, in Class 1 operation, when a first pair of devices 52 and 62 establishes a connection 82, 86 and through ISL 90 in the middle, no other devices can use the ISL 90. A second pair of devices 64 and 54 cannot communicate to each other even though both of them are free and ready to communicate, because the ISL 90 is not available. In the much more common Class 2 or 3 operations, similar blockages may also happen. When a first pair of devices uses up all the credits or other resources of the ISL 90, then other pairs have to wait until the credits are returned from the receiving device, which can then be used by the other pairs. So in class 2 or 3 operation the blockage occurs due to credit hogging on the ISL, as opposed to the more easily visualized class 1 link availability hogging. But the end result is the same: one pair of devices hogs the ISL at the expense of the other device pairs.

[0029] Time multiplexing has been used to avoid such blocking, as showing in FIG. 4. The frames may be segregated into separate frame flows or virtual channels with a priority scheme. As described above, on a single ISL, there may be several virtual channels with each virtual channel having its own priority level and queuing mechanism. A single ISL has two directional information flow paths. Each flow path can be operated independently but in the same fashion. Discussion of one information flow path is sufficient. A flow path is a path where data frames transfer from the transmitter on one E-port to the receiver on the other E-port of the ISL.

[0030] As also described above, the credits are allocated to each virtual channel, such that each virtual channel operates as a real link. When there are credits available, the virtual channel can send frames. When the allocated credits are used up, the virtual channel has to wait until a credit is returned by the receiving port on the other end of the link.

[0031] FIG. 5 shows one prior art implementation of the virtual channel (VC) in a switch. There is one ISL between E-ports 44 and 46. The ISL is divided into eight (8) virtual channels, VC0, 1, 2, 3, 4, 5, 6 and 7. On the transmitter of E-port 44, they are represented as channels 110 through 117 and on the transmitter of E-port 46, they are similarly represented as channels 120 through 127. In this prior art example, each port on the switch was allocated 28 credits. Each VC then has its own priority, its own credits and flow control. In the prior art implementation, the VCs were assigned as the following:

[0032] VC0 was for inter-switch traffic, Class F;

[0033] VC1 could be used for carrying Class 2 F_BSY and F_RJT frames generated by a fabric element;

[0034] VC2 through VC5 were four unicast data carrying virtual channels; and

[0035] VC6 and VC7 were used for data frames for multicast and broadcast traffic only.

[0036] The credits were allocated to virtual channels according to the following: (1, 1, 6, 6, 6, 1, 1), for VC0 through VC7, assuming the total number of credits is 28. The virtual channels VC2-5 needed more credits because they carried most of the data transfer traffic.

[0037] The assignments of VC2-5 to a particular port connected to a switch may be determined by various methods or factors. The major factors are the frame SID and DID and the routing functions. The actual assignment of VCs to physical ports servicing the data transmission is outside the scope of the present invention but the following assignment will be used as an example. Each switch has number 0 through number 7 physical ports. The number 0 port is the E-port of the ISL in the following discussion. For simplicity and clarity of the discussion of the present invention, the VC assignment may be determined by a predetermined routing table as in Table 1.

[0038]

TABLE 1
A simple Routing Table
	F-port, DID	Assigned Virtual Channel
	0	(the E-port on ISL)
	1		VC2
	2		VC2
	3		VC3
	4		VC3
	5		VC4
	6		VC4
	7		VC5

[0039] One drawback of this prior art scheme is that the credits assigned to each virtual channel are fixed. However, in an actual fabric, frequently, not all ports are connected to devices, so some of the virtual channels used for “connecting” those physical ports will not be utilized. The credits allocated to those VCs not in use are wasted, while other virtual channels in use may not have enough credits to achieve the full transmission speed.

[0040] FIGS. 6A-C illustrate another drawback of prior art VCs with fixed number of credits for each VC, as described in the background. Using the same example just above, the total credits of the ISL are 28 credits. Each of the unicast data frame carrying VCs has 6 credits. In a long haul or long distance scenario, assume that during the time a frame travels from transmitting port 132 to a receiving port 144, the transmitter can send out 8 frames. In FIG. 6A, the transmitting port 132 can send out six (6) frames without waiting. After the transmitter in the transmitting port 132 sends out six (6) frames, the transmitter runs out of credits and has to stop. All the six frames are still in the ISL. After two more credit time periods, the first frame reaches the receiving port 144. After receiving frames from the ISL, the receiving port 144, through a transmitter on the same receiving port, starts returning credit to the transmitting port 132, through a receiver on the same transmitting port 132, while the transmitter on the transmitting port 132 is still waiting, as shown in FIG. 6B. Assuming the turn-around time within the receiving port, which is time between the receiving port 144 receives a frame until the transmitter on the same receiving port 144 sends out a credit, is negligible. After sixteen (16) credit time periods, the first credit reaches the transmitting port 132, as shown in FIG. 6C. Then the transmitting port 132 can send more frames to the receiver again. During the sixteen (16) credit time periods, the transmitter is sending frames during only six (6) credit time periods, the same number of credits as the transmitter has. That is 6/16, or less than 40% of the bandwidth being utilized.

[0041] Continuing using the similar numbers, the following example illustrates the situation where a VC level credit pooling is used. Still assuming that the total credits of the ISL are 28 credits. Instead of allocating all credits to all VCs, about half of the total credits are reserved in a VC pool, and half of the credits are advertised to a transmitting port for each individual VCs, i.e. (1, 1, 3, 3, 3, 3, 1, 1), or 16 credits for the eight VCs, and 12 credits are reserved in a pool. So when the VC level pool is used, only 16 credits, out of 28 credits are advertised. Even though the receiving port now has a large pool of credits from which it can borrow credits, i.e. the receiving port can return a credit as soon as a frame reaches the receiving port, without waiting for the frame to be cleared out of its buffer, the receiving port still cannot return a credit before the transmitting port sends out a frame. If only VC2 is in use, then the transmitting port has to stop after sending out 3 frames. The receiving port with its large pool of credits cannot do anything until the receiving port receives the first frame, about 8 credit time periods later. It may still take another 8 credit time periods for first returned credit to reach the transmitting port. During the 16 credit time periods, the transmitting port can send only 3 frames for VC2, i.e. 3/16 or less than 20% of the bandwidth of the ISL is used.

[0042] If the VC has sixteen (16) credits instead of six (6) or three (3), then the transmitter can send sixteen (16) frames before it needs credits back from the receiver to continue sending frames. Because it takes 16 credit time periods for the first returned credit to reach the transmitter, by the time the transmitter finishes sending out the 16^th frame, the first credit should arrive at the transmitter. With that credit, the transmitter can send out another frame. This will establish a steady state, where the transmitter receives a credit and sends out a frame. There is no extra credit on the transmitter, but there is no waiting for credit either. So the transmitter can fully utilize the bandwidth on the ISL. If the VC has more than sixteen (16) credits, then there will be some extra or reserve credits on the transmitter as an additional buffer in case there is glitch on the credit return, or if the frames are not all full size. As a note, it is understood that frame sizes can vary and that may change specific examples given, but for purposes of the explanations, full size frames have been used in the examples. Thus, if the frame size were smaller, the credit problem would worsen.

[0043] It is now apparent that the amount of credits available for a VC can have a direct impact on the effective transmission speed, or the bandwidth utilization of the ISL. The VCs may solve the blockage problem, traffic congestion problem etc, but they may cause the under-utilization of the available bandwidth in an ISL.

[0044] One of the embodiments of the current invention advantageously fully utilizes the available bandwidth, while avoiding the blockage problem. In one embodiment according to the current invention, the virtual channels are assigned minimum credits initially, such as 0, 1 or a predetermined minimum, and the remaining credits in the port are assigned to a common virtual channel pool for the port. Since the virtual channels are not identical, the minimum credits for one VC may be different from the minimum credits for another VC, depending on the type of the VC. The credits in the VC pool are then allocated to the virtual channels that are actually in use according to certain criteria or formula. For example, the numbers of credits can be allocated to maximize the total throughput of the transmission through the ISL. Another example may be that the numbers of credits are allocated to make all VCs in use having the same throughput, so as not to shut out or severely penalize the VCs servicing slow physical links.

[0045] In one embodiment of the current invention, the credit allocation to VCs is dynamic, so it is necessary to have a manager to control or manage the credit allocation. Since the transmitter knows how many frames are waiting to be transmitted for each VC at any given point in time, it is better to have the transmitter manage the credit allocation. In this embodiment, when two E-ports are connected and negotiate the initial configuration, the transmitters negotiate and acquire the control of VC credit allocation, while the receivers advertise the total credits available for the whole port, rather than for each individual VC. Once the transmitter acquires the control of the VC credit allocation, it will divide the ISL into several VCs and assign the initial minimum credit allocation to each VC.

[0046] The following illustrative examples will exemplify this embodiment according to the current invention. For ease of comparison with the prior art virtual channel implementation, still assume that there are eight (8) VCs in the transmitter of an E-port and that the total credits of the E-port is 28. So the initial credits allocation by the transmitter may be: (1, 1, 1, 1, 1, 1, 1, 1) in VC0-7, with 20 in the pool.

[0047] FIGS. 7A-7D illustrate a similar situation as in FIGS. 6A-6C, except having the embodiment of the current invention. In this embodiment, the transmitter controls the allocation of the credits to each VC. The initial allocation is (1, 1, 1, 1, 1, 1, 1, 1) in VC0-7, with 20 in the pool. Assume that only one frame carrying VC, e.g. VC4 is in use. So the transmitter could allocate one or more of the credits in the pool to VC4, i.e. (1, 1, 1, 1, 21, 1, 1, 1) in VC0-7 and 0 in the pool. VC4 has 21 credits, more than the 16 credits needed to achieve the full speed. After the transmitter fills up the pipe line with 8 frames and 8 credits, the transmitter has 5 credits in its reserve. The transmitter can continuously send frames to the receiver without any waiting for return credits.

[0048] In a second scenario, as shown in FIGS. 8A-C, VC2 is used by a link to a device 64. Two frame-carrying VCs are in use. The transmitter may alter the credit allocation to (1, 1, 11, 1, 11, 1, 1, 1) in VC0-7 and 0 in the pool. There will be 22 credits available to VC2 and VC4 and they may send 16 out of 22 frames between VC2 and VC4 to fill up the ISL to achieve a steady state. During the steady state, the ISL is fully loaded and running at its full speed. Depending on the VC setup, VC2 and VC4 may alternate sending frames through the shared ISL. As the example shown in FIG. 8A, VC2 sends 4 frames with 7 credits left, then VC4 sends 4 frames with 7 credits left. The first frame reaches the receiver and then the receiver, through a transmitter on the receiving port, starts to return credits. VC2 sends 4 more frames, with 3 credits left; and VC4 sends 4 more frames with 3 credits left. At this time, the first credit reaches the transmitting port and returns to VC2, as in FIG. 8B. After this time, for every frame sent out, there is credit returned. A steady state is achieved, as in FIG. 8C. Both VC2 and VC4 are using the ISL. There is no blockage. The ISL is at the full speed. There is no reduction of bandwidth utilization.

[0049] The transmitter knows if there are frames in a queue in a particular VC waiting for transmission. Once the queue for a VC is clear, the transmitter can reallocate the credits for the VC to other VCs in need or back to the pool. Since there is a minimum number of credits for each VC initially, even before the transmitter allocates more credits to a particular VC, any VC is not completely shut out of using the ISL, unless, of course, if its minimum value is zero. The credit allocation or reallocation is carried out by the transmitter dynamically when the credits are returned from the receiving port. Whenever the need of credits for all the VCs in use changes, the transmitter can allocate the returned credits from the receiving port to the VC in need, according to a predetermined formula, such as maximum throughput through the ISL or uniform throughput through each VC in use.

[0050] In another embodiment of the present invention, the allocation or reallocation of credits may be completely and continuously dynamic. After the initial minimum credits allocation for each VC, all the remaining credits are allocated to the VC level pool. All the returned credits from the receiving port are also reserved into the pool, except that the minimum number of credits for each VC is maintained first. The transmitting port control module allocates each and every credit from the pool to a VC in use and in need credit by credit, frame by frame. Regardless how many VCs are in use, all of the credits in a receiving port will be advertised to the transmitting port. All the credit advertised to the transmitting port will be allocated by the transmitting port, i.e. all the credits minus the minimum credits for each VC, which could be zero, are allocated to the pool. When a frame needs to be transmitted through a particular VC, the transmitting port control module may allocate one credit from the pool to that particular VC for transmitting that particular frame. Since the transmitting port “knows” where each frame goes through which VC, the transmitting port may assign a priority to only the VCs that are in use, i.e. the VCs that have frames to send. Each VC may be assigned a priority by the control module through a predetermined priority scheme, such as round-robin, so that no one VC in use will be shut out. Since VCs not in use have no frames to send, VCs not in use do not participate in the priority scheme. VCs not in use cannot waste credits that they do not need. In this embodiment, essentially a frame level credit allocation, all of the credits are utilized, so that the bandwidth of the ISL may be utilized to the maximum. The number of VCs in use is irrelevant in this embodiment.

[0051] This continuous credit allocation scheme may be illustrated by the following numeric example using similar numbers as above. Initially, the credit allocation may be (0, 0, 0, 0, 0, 0, 0, 0) and 28 in the pool. Here the minimum number of credits has been selected to be zero. If there are frames needed to be sent through VC2, then for each frame, a credit is taken from the pool and assigned to VC2, so that VC2 can send the frame. After 16 frames are sent by VC2, the credit allocation becomes (0, 0, 16, 0, 0, 0, 0, 0) and 12 in the pool. When a credit received by the transmitting port, as would be the case in this example with VC2 having sent 16 frames, the credit is returned to the pool and the credits allocated to VC2 decreased by one. If more frames need to be sent through VC2, then more credits are allocated to VC2, one by one. So, using the values in the examples, the allocation would stabilize at VC2 having 16 credits. If besides frames for VC2, there are frames for VC3, then credits allocation may be alternated between VC2 and VC3 when round-robin priority scheme is used, i.e. one credit for VC2, then one for VC3, then one for VC2, one for VC3, and so on. The credit allocation, frames and credits in transit may become: (0, 0, 8, 8, 0, 0, 0, 0) and 12 in the pool. If besides frames for VC2 and VC3, there are frames for VC4, then credit allocation may be alternated among VC2, VC3 and VC4, i.e. one credit for VC2, one for VC3, one for VC4, and so on. The credit allocation and frames and credits in transit may be: (0, 0, 6, 5, 5, 0, 0, 0) and 12 in the pool.

[0052] Embodiments according to the present invention make the allocation of buffer credits in a Fibre Channel dynamically based on the need of each virtual channel in use, rather than statically based on the total number of virtual channels in an ISL. This operation is applicable regardless of the ways the virtual channels are assigned in respect to the physical port connections. The particular way of assigning a virtual channel to service a particular physical communication path is immaterial to the present invention.

[0053] The present invention may be implemented in many ways. For example, the present invention may be implemented in a software format, as a machine readable, machine executable program. The software program executing the present invention can be loaded into a processor or control module on a switch, or a buffer credit management module on a switch, during a power up initialization or a later set-up. This presumes that the hardware portions of the buffer credit logic are sufficiently programmable or are made sufficiently programmable to handle the changing credit allocations per VC.

[0054] The present invention may also be implemented in hardware format, as a new switch having the new credit sharing scheme built into the hardware.

[0055] While illustrative embodiments of the invention have been illustrated and described, it will be appreciated that various changes can be made therein without departing from the spirit and scope of the invention.

[0056] Titles and subtitles used in the text are intended only as focal points and an organization tool. These titles are not intended to specifically describe the applicable discussion or imply any limitation of that discussion.