Protecting data transactions on an integrated circuit bus

[0001] This application is related to U.S. patent application Ser. No. ______ entitled, “Method and Apparatus for Communicating Between an Agent and a Remote Management Module in a Processing System,” filed on Mar. 4, 2005, hereby incorporated by reference in its entirety.

[0002] Embodiments of the present invention relate to storage systems, and in particular, to protecting data transactions on storage systems.

[0003] The Open System Interconnection (OSI) model is a well-known seven-layer framework for defining communication protocols. For example, one communication protocol, such as Transmission Control Protocol/Internet Protocol (TCP/IP), is mapped to the transport layer and network layer of the OSI model, respectively. Specifically, the transport layer ensures that packets transmitted from a source computer to a destination computer are received by the destination computer in the correct order. The network layer determines the packet format and the addressing scheme of the source and destination computers.

[0004] The lower layers of the OSI model include the data link layer and the physical layer. A communication protocol corresponding to the data link layer defines how the packets are decoded and encoded into bits. Further, a communication protocol corresponding to the physical link layer defines the electrical and mechanical specifications for transmitting a bit stream. From a bottom-up viewpoint of the OSI model, the OSI model includes the physical, data link, network, and transport layers. Accordingly, the source and destination computers that implement the communication protocols of the bottom four layers of the OSI model can communication information.

[0005] When communicating a bit stream of information between the source and destination computers, or from one source component to a destination component within a single computer, the bit stream may be affected by magnetic fields and other natural phenomena, thereby potentially causing a bit flip. The bit flip results in erroneous information received at a destination computer or destination component.

[0006] To alleviate bit flip problems, some data link layer communication protocols include error checking capabilities. However, other data link layer communication protocols are not capable to alleviating bit flip problems. An exemplary data link layer communication protocol that does not solve such problems is a communication protocol operating over an integrated circuit bus between components of a motherboard of a computer.

[0007] A solution to protect against such problems is to provide error checking microcode that is stored in a component of the motherboard. However, providing the microcode requires extra circuitry on the motherboard, which may lead to extra processing cycles by the destination component and an overall increase in the power consumption of the motherboard. The extra circuitry also increases the overall complexity of the motherboard design. For example, by adding extra circuitry, motherboard production costs increase, which leads to an overall increase of the cost of the computer. Finally, complex microcode incurs a software development cost that increases over time because of the need to maintain the microcode.

[0008] Thus, what is needed is a solution addressing the communication of erroneous information over an integrated circuit bus of a computer that is simple to implement and maintain while having low power consumption.

[0009] Embodiments of the present invention provide a method and a system for protecting data transactions on an integrated circuit bus. Such data protection can be implemented by a data integrity protocol having a data protection format herein described. It should be appreciated that the present invention can be implemented in numerous ways, such as a process, an apparatus, a system, a device or a method on a computer readable medium. Several inventive embodiments of the present invention are described below.

[0010] In an embodiment of a method of formatting data on an integrated circuit bus, the embodiment includes initiating communication between a first controller and a second controller. Further, the embodiment includes communicating protected data between the first controller and the second controller. The first and the second controller are capable of communicating the protected data by using the integrated circuit bus. The protected data includes a first data chunk and a second data chunk such that the second data chunk is a complement of the first data chunk.

[0011] In an embodiment of a method of communicating data, the embodiment includes initiating communication between a first component and a second component and transmitting data and complemented data on an integrated circuit bus from the first component to the second component. The data and the complemented data are compared by the second component to determine an error.

[0012] In an embodiment of a system for protecting data communicated over an integrated circuit bus includes a first controller of a system board operating as a master for transmitting data and complemented data and a second controller of the system board. The second controller is connected to the first controller via the integrated circuit bus, such that the second controller operates as a slave for receiving the data and the complemented data. Further, the data and the complemented data are compared to detect errors.

[0013] In an embodiment of a system for formatting data on an integrated circuit bus, the embodiment includes means for initiating communication between a first controller and a second controller means for communicating protected data between the first controller and the second controller. The first and the second controller are capable of communicating the protected data by using the integrated circuit bus, such that the protected data includes a first data chunk and a second data chunk. The second data chunk is a complement of the first data chunk.

[0014] In an embodiment of a computer readable medium having stored thereon instructions capable of being executed by a computer system, the embodiment includes instructions for communicating a first data chunk over an integrated circuit bus from a first controller to a second controller. Further, the embodiment includes instructions for communicating a complemented second data chunk over the integrated circuit bus from the first controller to the second controller. The complemented second data chunk is compared with the first data chunk to determine whether an error occurred during the communication.

[0015] Other aspects of the invention will become apparent from the following detailed description taken in conjunction with the accompanying drawings, which illustrates by way of example, the principles of the invention.

[0016] FIG. 1 is a diagram illustrating a networked storage system, in accordance with an embodiment of the invention;

[0017] FIG. 2 is a diagram illustrating a system board of a storage system, in accordance with an embodiment of the invention;

[0018] FIG. 3 is a diagram illustrating a data protection format, in accordance with an embodiment of the invention;

[0019] FIG. 4 is a diagram illustrating an address field of the data protection format, in accordance with an embodiment of the invention;

[0020] FIG. 5 is a diagram illustrating a write data transaction communicated with a data protection format, in accordance with an embodiment of the invention;

[0021] FIG. 6 is a diagram illustrating a read data transaction communicated with a data protection format, in accordance with an embodiment of the invention; and

[0022] FIG. 7 is a flowchart diagram of operations for protecting data transactions on an integrated circuit bus, in accordance with an embodiment of the invention.

[0023] The following embodiments describe a method and a system for protecting data transactions on an integrated circuit bus. Such data protection can be implemented by a data integrity protocol having a data protection format herein described. It will be apparent, however, to one skilled in the art, that embodiments of the present invention may be practiced without some or all of these specific details. In other instances, well known process operations have not been described in detail in order not to obscure the embodiments of the present invention described herein.

[0024] FIG. 1 is a diagram illustrating a networked storage system, in accordance with an embodiment of the invention. A storage system 100, such as devices commercially available from Network Appliance, Inc of Sunnyvale, Calif. can be connected to other storage systems 100 and at least one client 170. Devices, such as the storage system 100 and clients 170 are computers that communicate with one another via a connection system 160. Further, the teachings of this invention can be adapted to a variety of storage system architectures including, but not limited to, a network-attached storage environment (NAS), storage area network (SAN), and disk assembly directly-attached to the client 170. The term “storage system” should therefore be taken broadly to include such arrangements in addition to any subsystems configured to perform a storage function and associated with other equipment or systems.

[0025] The connection system 160 is implemented with any type of connectivity method, such as a wire-based connection, such as copper, fiber optic, or other suitable media. Alternatively, the connectivity method can be a wireless connection, such as infrared, Bluetooth, or Wi-Fi. Thus, any suitable connection method between the storage system 100 and the client 170 is possible, as long as communication is enabled between the devices of the computing environment.

[0026] Exemplary architectures enabling communication on the connection system 160 include Ethernet or Fibre Channel (FC). Each client 170 may communicate with the storage system 100 over the connection system 160 by exchanging discrete frames or packets of data, according to multiple protocols, such as Transmission Control Protocol/Internet Protocol (TCP/IP) and the like. As previously described, the storage system 100 may be described as a computer that is configured to execute software applications. Alternatively, the storage system 100 can be described as an appliance, filer, storage server, or similar device that manages stored data. For example, the client 170 may interact with the storage system 100 in accordance with a client/server model of information delivery.

[0027] Specifically, the client 170 may request the services of the storage system 100, and the storage system 100 may return the results of the services requested by the client 170, by exchanging packets over the connection system 160. The client 170 may issue packets including file-based access protocols, such as the Common Internet File System (CIFS) protocol or Network File System (NFS) protocol, over TCP/IP when accessing information in the form of files and directories. Alternatively, the client 170 may issue packets including block-based access protocols, such as Small Computer Systems Interface (SCSI) protocol encapsulated over TCP (iSCSI) and SCSI encapsulated over Fibre Channel (FCP), when accessing information in the form of blocks.

[0028] The storage system 100 is coupled to any number of disks (not shown). Further, the storage system 100 provides storage services relating to the organization of information on the disks. The disks that store information may be any suitable writeable storage device media such as tape, optical media, magnetic media, electronic random access memory, micro-electro mechanical, and any other similar media adapted to store information, including data and parity information. The disks can connect to the storage system 100 using any suitable connection method, such as a Fibre Channel (FC) serial link topology. Further, the disks are logically grouped within a logical volume/file system, such that each group may be operated as a Redundant Array of Independent (or Inexpensive) Disks (RAID). The storage system 100 further includes various components (not shown), such as at least one processor, a network adapter, and a storage adapter that are connected together. It should be appreciated that the memory can include Random Access Memory (RAM), Non-Volatile Random Access Memory (NVRAM), and caches.

[0029] As illustrated in FIG. 1, the storage system 100 includes a system board 110, sometimes referred to as a motherboard. Other than the components described above, the system board 110 includes a Remote Management Module (RMM), such as Remote LAN Module (RLM) 120. The RLM 120 is coupled to an agent 130 via an integrated circuit bus. For example, an integrated circuit bus provides communications capability between components, such as any of a variety of physical interconnects, whether implemented with copper, fiber, or other signaling media. In one exemplary embodiment, the RLM 120 and the agent 130 include hardware components, but also include software components, whether stored as microcode on hardware, or operating as software of the operating system 150. Specifically, the agent 130, utilizing the data integrity protocol described herein includes a software component that logically compares data values and complemented data values, which will be later described. However, in other exemplary embodiments, the RLM 120 and the agent 130 can be implemented only with hardware. The agent 130 is further coupled to multiple sensors 140, such as sensor A, sensor B, and sensor C. The sensors 140 are used to detect changes in the state of various environmental variables in the storage system 100, such as temperatures, voltages, binary states, and the like. The RLM 120, agent 130, and sensors 140 are described in U.S. patent application Ser. No. ______ entitled, Method and Apparatus for Communicating Between an Agent and a Remote Management Module in a Processing System, filed on Mar. 4, 2005, hereby incorporated by reference in its entirety.

[0030] Embodiments of the present invention contemplate the data integrity protocol implemented over the integrated circuit bus of the system board 110. Thus, in one embodiment of the present invention, data transactions occurring between the RLM 120 and the agent 130 are protected by the use of the data integrity protocol. In one exemplary embodiment, the RLM 120 and the agent 130 can be referred to as a first controller and a second controller, respectively. Such controllers can be implemented as hardware that is a component of the system board 110, or as a hardware component of the system board 110 in combination with software, as long as data is protected during communication over the integrated circuit bus. Thus, it should be appreciated that the data integrity protocol can protect data transactions on an integrated circuit bus between any components of the motherboard. Further examples of such components will be described with respect to FIG. 2.

[0031] The storage system 100 is managed by an operating system 150, which communicates with components of the system board 110, such as NetApp® Data ONTAP™ operating system, which is available from Network Appliance, Inc., of Sunnyvale, Calif. The exemplary storage operating system from NetApp implements a Write Anywhere File Layout (WAFL™) file system. However, it is expressly contemplated that any appropriate operating system may be enhanced for use in accordance with the inventive principles described herein. As such, where the terms “Data ONTAP” or “WAFL” is employed, it should be taken broadly to refer to any operating system that is otherwise adaptable to the teachings of this invention.

[0032] FIG. 2 is a diagram illustrating the system board 110 of the storage system 100, in accordance with an embodiment of the invention. The system board 110 includes a variety of components that may be modified in other embodiments of the present invention. Such modifications include adding or removing components. Further, it should be appreciated that the components can be coupled together in any configuration, as long as the configuration implements an integrated circuit bus. In one exemplary embodiment, at least one processor 200 is coupled to a northbridge 210, which connects memory 220 to other components that are coupled to a southbridge 230. The southbridge 230 thus makes accessible to the processor 200, a Peripheral Component Interconnect (PCI) connector 240, the RLM 120, an interface 260 having a serial interface 270, sensors 140, the agent 130, and an I/O controller 280. Further, a power supply 290 is connected to the system board 110 to provide power to operate the components of the system board 110. The power supply 290 continues to power the RLM 120 and the agent 130 even when the remainder of the system board 110 components receive no power. Thus, the RLM 120 and the agent 130 can continuously perform RMM functions, such as turning off and turning on the storage system 100.

[0033] On various components, such as the RLM 120 and interface 260, the components include an Inter-Integrated Circuit (I²C) interface 250. In an exemplary embodiment of the present invention, the I²C interfaces 250 are used to connect components of an I²C bus to a connector 255, well known to those of ordinary skill in the art. However, it is expressly contemplated that any integrated circuit bus, such as I²C, or other communication protocol, can implement the data integrity protocol to protect data communicated over the integrated circuit bus.

[0034] In the embodiment of FIG. 2, the connectors 255 between any two I²C interfaces 250 implement the I²C bus. Specifically, the I²C bus permits components connected to the same bus to control the transmission of data by behaving as masters and slaves during data transactions. For example, when the RLM 120 and the agent 130 communicate data during a data transaction, such as during read data transactions or write data transactions, the RLM 120 can be the master and the agent 130 can be the slave, which can each control the transmission of data by the indication of various bits of a data protection format. Thus, in an exemplary embodiment of the present invention, the data integrity protocol can be implemented in a single master, multiple slave environment. In another exemplary embodiment, the data integrity protocol can be implemented in a multiple master, multiple slave environment.

[0035] FIG. 3 is a diagram illustrating a data protection format, in accordance with an embodiment of the invention. The data protection format is implemented according to the data integrity protocol referred to in FIGS. 5 and 6. Specifically, the data protection format includes a start (S) bit 310, an address field 320, an acknowledgement (A) bit 330, data 340, complemented data 350, and a stop (P) bit 360. The data 340 can further include a command without a parameter, represented as “command( )” or command with a parameter, represented as “command (parameter).” Further, the completed data 350 is a complement of the “commando” and not “command(parameter).” For example, the data 340 can represent commands, such as “turn off storage system” or “turn on storage system.” The data 340 can also represent other commands with or without parameters, such as reboot storage system, read(register), and enable or disable the sensors 140. Moreover, the commands and parameters are definable and permit the granular control of and access to the components of the system board 110, as long as the integrated circuit bus connects the components and implements the data integrity protocol. Further, as an example, it should be appreciated that without the implementation of the data integrity protocol having commands and complemented commands described herein, if an erroneous bit flip occurs within the data 340, then the storage system 100 may be inadvertently shutdown.

[0036] In exemplary embodiments of the present invention, the commands and parameters are implemented as bytes. Correspondingly, the complemented commands are implemented as bytes. If the command and parameter are implemented as bytes, then each is capable of 256 commands and 256 parameters, respectively. Of course, other implementations than bytes are possible. For example, groups of bytes permitting more commands and parameters are possible, as long as the command is complemented for transmission during the data transaction over the integrated circuit bus.

[0037] In between data 340 and complemented data 350, is an acknowledgement bit 330 or a non-acknowledgement (N) bits (not shown), depending on the data transaction. Specifically, in certain embodiments of the present invention, the non-acknowledge bit can indicate when a data transaction fails instead of using acknowledgement bits 330.

[0038] Further, FIG. 4 is a diagram illustrating the address field of the data protection format, in accordance with an embodiment of the invention. The address field 320 includes class code 410 bits that are used for any addressing scheme to identify components. In addition to the class code bits 410, the address field 320 includes preamble bits which are programmable (e.g. via pin strapping), such as an A-line bit 420, 1-line bit 430, and 2-line bit 440. The address field 320 also includes a read/write (R/W) bit 450. The A-line bit 420, 1-line bit 430, and 2-line bit 440 correspond to pins (not shown) of the I²C interface 250 (FIG. 2). Further, the R/W bit 450 indicates whether the data transaction is a read or write request. For example, 1 indicates read data transactions and 0 indicates write data transactions. Of course, alternative embodiments can reverse the bits to indicate 1 for write data transactions and 0 for read data transactions. Thus, it should be appreciated that any examples described herein with bit values are exemplary.

[0039] The data integrity protocol of the embodiments of the present invention operates by the master or slave supplying acknowledgement bits 330 depending on the type of data transaction. For example, during a write data transaction, the slave, such as the agent 130, supplies the acknowledgement bits 330. Alternatively, during a read data transaction, the master, such as the RLM 120, supplies the acknowledgement bits 330. Thus, it is illustrative to consider FIG. 5 and FIG. 6 to describe the operation of read and write data operations over the integrated circuit bus implementing the transmission of data and complemented data to protect data from problems such as erroneous bit flips.

[0040] FIG. 5 is a diagram illustrating a write data transaction communicated with a data protection format, in accordance with an embodiment of the invention. Specifically, in an exemplary embodiment, a master 510 initiates a write command, as indicated by the R/W bit 450. Specifically, the write command is directed to a slave address. If any errors occur in the start bit 310 or the first 7 bits of the address field, then the correct slave will never receive the write command. Thus, the correct slave 520 will not respond with an acknowledgement bit 330 and the write data transaction ends. Alternatively, to avoid incorrectly communicating the write command to a slave, careful selection of the slave addresses during the design phase of the system prevents corruption from accessing the incorrect slave. For example, if there are two slave devices in the system, then the first slave address can be “0100.” Further, the second slave address can be “0111.” Accordingly, if there is a single bit error in the address of the second slave, then the first slave will probably not receive the write command.

[0041] However, if only the R/W bit 450 is incorrectly changed into a read command, then the receiving slave 520 understands that the master 510 should supply acknowledgements. Specifically, because the master 510 is waiting for the slave 520 to acknowledge and vice versa, neither the master 510 or the slave 520 supplies an acknowledgment and the transaction will timeout. Consequently, the write data transaction will be reinitiated (i.e. the original write data transaction is voided).

[0042] Errors can occur in the data 340 and the complemented data 350 transmission. However, upon receipt of the data 340 and the complemented data 350, logical operations such as XOR or some other suitable logical operation, determines whether there was a change in either the data 340 or complemented data 350. Specifically, if “data XOR complemented data” does not result in one, then there was an error during transmission. For example, data 340 can be “10001111” and complemented data 350 can be “01110000,” such as the 1's complement of the data 340. However, it should be appreciated that any suitable number representation is possible, as long as the number representation of the data 340 and complemented data 350, when operated upon by a suitable logical comparison, permits the detection of differences. For example, the logical comparison using XOR results in “11111111,” indicating no errors. Of course, it should be appreciated that an error could have occurred in the data 340 and the complemented data 350, thus leading to an erroneous result of “11111111.” However, triple redundancy, or more, whereby data 340 is transmitted twice along with the complemented data 350, and then logically compared with one another can indicate such errors.

[0043] It should be appreciated that the logical comparison can be performed on the fly, such as the transmission of each data 340 and complemented data 350, or after the collection of the completed transmission all data 340 and complemented data 350 in a buffer (not shown) of a component. Alternative embodiments to protect data transmitted on an integrated circuit bus also include the transmission of a Cyclic Redundancy Check (CRC) value in addition to the transmission of complemented data. However, the transmission of a CRC value adds complexity to the detection of an error. Thus, it should be appreciated that the any implementation of the data integrity protocol over an integrated circuit bus is possible, as long as errors are detected using minimal complexity.

[0044] FIG. 6 is a diagram illustrating a read data transaction communicated with a data protection format, in accordance with an embodiment of the invention. Specifically, a master 610 initiates a read command, as indicated by the R/W bit 450, which is incorrectly determined by the slave 620 as a write command. In this example, the master 610 supplies acknowledgements, except for the address field 320. Specifically, the slave 620 will acknowledge the address because it erroneously received a write command. If there is an error in the start bit 310 or the first 7 bits of the address field, then the intended slave 620 will never receive the read command. To prevent the erroneous communication of the read command to an unintended slave, careful selection of the addresses of the slaves during the establishment of the system lowers the likelihood that a single bit error in the address bits causes the communication to an unintended slave. For example, if there are two slave devices in the system, then the first slave address can be “0100.” Further, the second slave address can be “0111.” Accordingly, if there is a single bit error in the address of the second slave, then the first slave will probably not receive the read command. Thereafter, if errors occur in the data 340, then once again, comparison with the complemented data 350 transmitted after the data 340 protects the data transaction, similar to the implementation of logical operations described above with respect to FIG. 5.

[0045] FIG. 7 is a flowchart diagram of operations for protecting data transactions on an integrated circuit bus, in accordance with an embodiment of the invention. Specifically, two components, such as a first controller and a second controller of a system board are connected by an integrated circuit bus. When the first controller initiates a data transaction with the second controller, the data integrity protocol begins with operation 710. Specifically, the data transaction would be a read or write data transaction that includes the transmission of at last one command and the complement of the command. In operation 720, the first controller transmits the command as a data command. The data command, which is the first chunk of the protected data is later compared with the second chunk of the protected data. In particular, in operation 730, the first controller generates the complement of the data command, which is the second data chunk. Then, in operation 740, the first controller transmits the complemented data command to the second controller. Consequently, in operation 750, whether the data commands and complemented data commands are all received and stored in a buffer, or if processed on the fly (i.e. the receipt of each pair of first and second chunks), the second controller verifies each data command with the associated complemented data command. Thereafter, the operations end.

[0046] With the above embodiments in mind, it should be understood that the invention can employ various computer-implemented operations involving data stored in computer systems. These operations are those requiring physical manipulation of physical quantities. Usually, though not necessarily, these quantities take the form of electrical, magnetic, or optical signals capable of being stored, transferred, combined, compared, and similarly manipulated.

[0047] Any of the operations described herein are useful machine operations. The invention also relates to a device or an apparatus for performing these operations. The apparatus can be specially constructed for the required purpose, or the apparatus can be a general-purpose computer selectively activated or configured by a computer program stored in the computer. In particular, various general-purpose machines can be used with computer programs written in accordance with the teachings herein, or it may be more convenient to construct a more specialized apparatus to perform the required operations.

[0048] The invention can also be embodied as computer readable code, such as instructions, on a computer readable medium. The computer readable medium is any data storage device that can store data, such as disks, which can be thereafter be read by a computer system. Examples of the computer readable medium include hard drives, network attached storage (NAS), Storage Area Networks (SANs), read-only memory, random-access memory, CD-ROMs, CD-Rs, CD-RWs, magnetic tapes and other optical and non-optical data storage media. The computer readable medium can also be distributed over a network-coupled computer system, such as a storage cluster, so that the computer readable code is stored and executed in a distributed fashion. The computer readable medium can also be distributed using a switching fabric, such as used in Linux® compute farms.

[0049] Although the foregoing invention has been described in some detail for purposes of clarity of understanding, it will be apparent that certain changes and modifications can be practiced within the scope of the appended claims. Accordingly, the present embodiments are to be considered as illustrative and not restrictive, and the invention is not to be limited to the details given herein, but may be modified within the scope and equivalents of the appended claims.