MULTI-FREQUENCY DEBUG NETWORK FOR A MULTIPROCESSOR ARRAY

[0001] This application is a continuation-in-part of co-pending U.S. application Ser. No. 10/871,347, filed Jun. 18, 2004, entitled DATA INTERFACE FOR HARDWARE OBJECTS, which in turn claims benefit of U.S. provisional application 60/479,759, filed Jun. 18, 2003, entitled INTEGRATED CIRCUIT DEVELOPMENT SYSTEM. Additionally, this application is a continuation-in-part of co-pending U.S. application U.S.______, filed______(attorney reference no 1436-0046 P114US), entitled INTERACTIVE DEBUG NETWORK FOR MULTIPROCESSOR ARRAY. Further this application claims the benefit of U.S. provisional application 60/790,912, filed Apr. 10, 2006, entitled MIMD COMPUTING FABRIC, of U.S. provisional application 60/836,036, filed Aug. 20, 2006, entitled RECONFIGURABLE PROCESSOR ARRAY, and of U.S. provisional application 60/850,078, filed Oct. 8, 2006, entitled RECONFIGURABLE PROCESSOR ARRAY AND DEBUG NETWORK. The teachings of all of these applications are explicitly incorporated by reference herein.

[0002] This disclosure relates to a system debugger, and, more particularly, to a system debugger structured to operate on a multiprocessor platform in which individual processors run at different frequencies.

[0003] Debugging software that executes on hardware systems is the process of recognizing, identifying and fixing or isolating software and/or hardware errors. An error occurs when an actual result does not match an expected result, and can be caused by errors in the software and/or hardware.

[0004] Developing software applications for a new computer processor typically uses a software simulation of the new processor design where software being developed is run on the software model of the chip being developed.

[0005] Debugging mixed software/hardware systems is easier if either the software or the hardware has been verified as accurate in another system. Debugging mixed software/hardware systems where neither has been verified to be accurate is difficult, and this difficulty scales as the number of interrelated processes increases.

[0006] The difficulty of debugging a software/hardware system that is based on an architecture of dozens or hundreds of individual processors does not scale linearly from experience in single processor systems. Not only must the operation of each processor be verified, but communication paths and buffers between the processors must be exposed for analysis. No tool currently exists that provides the type of exposure, depth, and flexibility necessary to adequately debug such multi-processor systems.

[0007] Embodiments of the invention address these and other limitations in the prior art.

[0008] FIG. 1 is a block diagram of an integrated circuit platform formed of a central collection of tessellated operating units surrounded by I/O circuitry according to embodiments of the invention.

[0009] FIG. 2A is a block diagram illustrating several groups of processing units used to make the operating units of FIG. 1 according to embodiments of the invention.

[0010] FIG. 2B is a block diagram illustrating a group of processing and memory units operating in two different clock domains.

[0011] FIG. 3 is a block diagram of a data/protocol register used to connect various components within and between the processing units of FIG. 3.

[0012] FIG. 4 is a block diagram of details of an example compute unit illustrated in FIG. 2A according to embodiments of the invention.

[0013] FIG. 5 is a block diagram of an example processor included in the compute unit of FIG. 4.

[0014] FIG. 6 is a functional block diagram of an example debug network according to embodiments of the invention.

[0015] FIG. 7 is a functional block diagram of an example debug slave according to embodiments of the invention.

[0016] FIG. 8 is an example flow diagram illustrating operation of a debug slave according to embodiments of the invention.

[0017] FIG. 9 is a functional block diagram of example pipeline stages in a processor according to embodiments of the invention.

[0018] FIG. 10 is a block diagram illustrating an example debug network operative in multiple different clock domains according to embodiments of the invention.

[0019] FIG. 11 is a block diagram of a clock crossing architecture according to embodiments of the invention.

[0020] FIG. 1 illustrates an example tessellated multi-element processor platform 100 according to embodiments of the invention. Central to the processor platform 100 is a core 112 of multiple tiles 120 that are arranged and placed according to available space and size of the core 112. The tiles 120 are interconnected by communication data lines 122 that can include protocol registers as described below.

[0021] Additionally, the platform 100 includes Input/Output (I/O) blocks 114 placed around the periphery of the platform 100. The I/O 114 blocks are coupled to some of the tiles 120 and provide communication paths between the tiles 120 and elements outside of the platform 100. Although the I/O blocks 114 are illustrated as being around the periphery of the platform 100, in practice the blocks 114 may be placed anywhere within the platform 100. Standard communication protocols, such as Peripheral Component Interface Express (PCIe), Dynamic Data Rate Two Synchronous Dynamic Random Access Memory interface (DDR2), or simple hardwired input/output wires, for instance, could be connected to the platform 100 by including particularized I/O blocks 114 structured to perform the particular protocols required to connect to other devices.

[0022] The number and placement of tiles 120 may be dictated by the size and shape of the core 112, as well as external factors, such as cost. Although only sixteen tiles 120 are illustrated in FIG. 1, the actual number of tiles placed within the platform 100 may change depending on multiple factors. For instance, as process technologies scale smaller, more tiles 120 may fit within the core 112. In some instances, the number of tiles 120 may be purposely be kept small to reduce the overall cost of the platform 100, or to scale the computing power of the platform 100 to desired applications. In addition, although the tiles 120 are illustrated as being equal in number in the horizontal and vertical directions, yielding a square platform 100, there may be more tiles in one direction than another, and may be shaped to accommodate additional, non tiled elements. Thus, platforms 100 with any number of tiles 120, even one, in any geometrical configuration are specifically contemplated. Further, although only one type of tile 120 is illustrated in FIG. 1, different types and numbers of tiles may be integrated within a single processor platform 100.

[0023] Tiles 120 may be homogeneous or heterogeneous. In some instances the tiles 120 may include different components. They may be identical copies of one another or they may include the same components packed differently.

[0024] FIG. 2A illustrates components of example tiles 210 of the platform 100 illustrated in FIG. 1. In this figure, four tiles 210 are illustrated. The components illustrated in FIG. 2A could also be thought of as one, two, four, or eight tiles 120, each having a different number of processor-memory pairs. For the remainder of this document, however, a tile will be referred to as illustrated by the delineation in FIG. 2A, having two processor-memory pairs. In the system described, there are two types of tiles illustrated, one with processors in the upper-left and lower-right corners, and another with processors in the upper-right and lower-left comers. Other embodiments can include different component types, as well as different number of components. Additionally, as described below, there is no requirement that the number of processors equal the number of memory units in each tile 210.

[0025] In FIG. 2A, an example tile 210 includes processor or “compute” units 230 and “memory” units 240. The compute units 230 include mostly computing resources, while the memory units 240 include mostly memory resources. There may be, however, some memory components within the compute unit 230 and some computing components within the memory unit 240. In this configuration, each compute unit 230 is directly attached to one memory unit 240, although it is possible for any compute unit to communicate with any memory unit within the platform 100 (FIG. 1).

[0026] Data communication lines 222 connect units 230, 240 to each other as well as to units in other tiles. Detailed description of components with the compute units 230 and memory units 240 begins with FIG. 4 below.

[0027] FIG. 2B illustrates an example tile 260, which could be the same or similar to one of the tiles 210 of FIG. 2A. The tile 260 includes two interconnected pairs of processor/memory groups. Specifically, one pair includes a processor group 272 coupled to a memory group 274 through a set of communication channels 262. The channels 262 in FIG. 2A are functional illustrations and there may be any number of physical or logical communication channels between the processor group 272 and the memory group 274. A second pair includes a processor group 282 coupled to a memory group 284.

[0028] The first pair, 272, 274 operates in a first clock domain, domain A, while the second pair 282, 284 operates in a second clock domain, domain B. Operating in different clock domains means that a local clock may have a different operating frequency than another domain'S local clock. For instance, a system clock 290 may be modified by a clock rate controller, 292, 294, to create a different local clock for the clock domains A and clock domain B. For example, a system clock may operate at 600 MHz, while the clock rate controller 292 generates a local clock of 300 MHz for the clock domain A, and the clock rate controller 294 generates a local clock of 123 MHz for clock domain B.

[0029] In a multiprocessing system, individual processor groups or even individual processors may operate at different clock rates. Typically the clock rate is based on the function of the processors, how quickly it can be performed, and how the performance time relates to the requirements of passing data to other processors. In this embodiment, the local clock rate is controlled by a value set in a particular register. Thus the register can be set once at initialization: for instance, if a first set of processors is performing a complex filtering procedure, the processors may need to run at a high clock rate; but processors in a second group that manage the filtering procedure may only require a much lower rate. The register may also be written during operation of a processor, allowing the code to run at different speeds; for instance a tight loop may need to run very fast, but error handling code may only require a much lower rate. Lowering the operating frequency of any circuitry, even for a limited period of time, has many benefits, such as reducing dissipated power and reducing the need for external buffers to store data that has been produced that cannot yet be consumed.

[0030] As described above, communication lines 262 couple a processor 272, 282 to its associated memory block 274, 284. Because the components coupled to the lines 262 synchronously using the same clock, no special clock considerations are necessary. Communication lines 264 are coupled between processors 272, 282 that may operate in different clock groups, and therefore may require special logic to ensure data is neither lost nor duplicated when crossing into a different clock rate. Clock crossing logic is covered in detail below.

[0031] FIG. 3 is a block diagram illustrating a data/protocol register 300, the function and operation of which is described in U.S. application Ser. No. 10/871,347, referred to above. The register 300 includes a set of storage elements between an input interface and an output interface.

[0032] The input interface uses an accept/valid data pair to control the flow of data. If the valid and accept signals are both asserted, the register 300 moves data stored in sections 302 and 308 to the output datapath, and new data is stored in 302, 308. Further, if out_valid is de-asserted, the register 300 continues to accept new data while overwriting the invalid data in 302, 308. This push-pull protocol register 300 is locally self-synchronizing in that it only moves data if the data is valid and the subsequent register is ready to accept it. Likewise, if the protocol register 300 is not ready to take data, it de-asserts the in accept signal, which informs the previous stages that the register 300 cannot take the next data value.

[0033] In some embodiments, the packet id value stored in the section 308 is a single bit and operates to indicate that the data stored in the section 302 is in a particular packet, group or word of data. In a particular embodiment, a LOW value of the packet_id indicates that it is the last word in a message packet. All other words in the packet would have a HIGH value for packet_id. Thus the first word in a message packet can be determined by detecting a HIGH packet_id value that immediately follows a LOW value for the word that precedes the current word. Alternatively stated, the first HIGH value for the packet_id that follows a LOW value for a preceding packet_id indicates the first word in a message packet.

[0034] The width of the data storage section 302 can vary based on implementation requirements. Typical widths would include powers of two such as 4, 8, 16, and 32 bits.

[0035] With reference to FIG. 2A, the data communication lines 222 could include a register 300 at each end of each of the communication lines. Because of the local self-synchronizing nature of register 300, additional registers 300 could be inserted anywhere along the communication lines without changing the operation of the communication.

[0036] FIG. 4 illustrates a set of example elements forming an illustrative compute unit 400 which could be the same or similar to the compute 230 of FIG. 2A. In this example, there are two minor processors 432 and two major processors 434, The major processors 434 have a richer instruction set and include more local storage than the minor processors 432, and are structured to perform mathematically intensive computations. The minor processors 432 are more simple compute units than the major processors 434, and are structured to prepare instructions and data so that the major processors can operate efficiently and expediently.

[0037] In detail, each of the processors 432, 434 may include an execution unit, an Arithmetic Logic Unit (ALU), a set of Input/Output circuitry, and a set of registers. In an example embodiment, the registers of the minor processors 432 may total 64 words of instruction memory while the major processors include 256 words, for instance. Additionally, a debug unit (DB) may be instanced in each of the processors 432, 434.

[0038] Communication channels 436 may be the same or similar to the data communication lines 222 of FIG. 2A, which may include the data registers 300 of FIG. 3.

[0039] In embodiments of the invention, the processors 432, 434 may operate at any clock rate, even rates different from those processors adjacent to one another. Communication channels that cross different clock domains require special clock crossing circuits to ensure that data is neither lost nor duplicated. A specific example of a clock crossing circuit is described in detail below.

[0040] FIG. 5 illustrates an example processor 500 that could be an implementation of either the minor processor 432 or major processor 434 of FIG. 4.

[0041] Major components of the example processor 500 include input channels 502, 522, 523, output channels 520, 540, an ALU 530, registers 532, internal RAM 514, and an instruction decoder 510. The ALU 530 contains functions such as an adder, logical functions, and a multiplexer. The RAM 514 is a local memory that can contain any mixture of instructions and data. Instructions may be 16 or 32 bits wide, for instance.

[0042] The processor 500 has two execution modes: Execute-From-Channel (channel execution) and Execute-From-Memory (memory execution), as described in the '036 application referred to above.

[0043] In memory execution mode, the processor 500 fetches instructions from the RAM 514, decodes them in the decoder 510, and executes them in a conventional manner by the ALU 530 or other hardware in the processor 500. In channel execution mode, the processor 500 operates on instructions sent to the processor 500 over an input channel 502. A selector 512 determines the source of the instructions for the processor 500 under control of a mode register 513. A map register 506 allows any physically connected channel to be used as the input channel 502. By using a logical name for the channel 502 stored in the map register 506, the same code can be used independent of the physical connections.

[0044] Numeric operations are performed in the ALU 530, and can be stored in any of a set of registers 532. One or two operands may be sent to the ALU 530 from the selectors 564 and 566. Specialized registers include a self-incrementing register 534, which is useful for counting, and a previous register 526, which holds the output from the previous ALU 530 computation.

[0045] Input channels 522, 523 supply data and/or instructions for the processor 500.

[0046] To ease the difficulty of design, registers or other components that are clocked within the processor 500 typically use the same local clock signal and therefore run synchronously. The local clock is known as the “processor clock”.

[0047] A debug slave 570 is an independently operating unit that may be included in each processor and memory of the entire system 100, including the core 112 and I/Os 114 (FIG. 1). Including an interactive debug network on the system 100 allows software to thoroughly examine and test the hardware as it runs. The debug slave 570 is unique in that it performs most operations at the speed of the debug network and only some operations at the local speed of its processor when required to do so, such as stepping an instruction and having to wait for a result value. Detailed description of the debug slave 570 and how it relates to an entire debug system follows.

[0048] FIG. 6 illustrates a debug network 600 according to embodiments of the invention. The debug network includes a debug system controller 610, which directly controls a debug master controller 612 and communicates with a debug slave controller 614.

[0049] A debug datapath 622 connects through a series of slaves 620 in a ring. In one embodiment, the datapath 622 is formed of data/protocol registers 300 illustrated in FIG. 3. In the embodiment described with reference to FIG. 6, the width of the data register 302 can be a single data bit. Additionally, the valid, accept, and packet-id registers 304, 306, 308, can also be a single bit wide. Thus, the datapath 622 can be four bits wide. The master controller 612 and slave 614 control the valid and accept signals for the datapath 622 as described with reference to FIG. 3 above. By controlling these valid and accept signals, data is sent around the datapath 622 in a predetermined manner. In an example embodiment, the entire datapath 622 is formed of a single shift register, where the output of one slave 620 is bitwise shifted to the next location in the datapath. Each slave 620 is identified with a particular bit location in the shift register. By stepping the exact number of times as there are slaves 620 on the network 600, a data bit makes a complete circuit around the datapath 622. In the illustrated embodiment, the master controller 612 generates data to be placed on the datapath 622 and the slave controller 614 removes data from the datapath after it has completed the entire datapath 622 ring. In other embodiments, the master controller 612 performs both functions of placing data on the datapath 622 ring as well as removing data from the ring. In embodiments of the invention, the datapath 622 is a standalone datapath that no other object within the system 100 uses for any other purpose.

[0050] The slaves 620 may be resident in the processors, as illustrated above, or may otherwise may be connected to retrieve data from and write data to the processors. In an example embodiment, there is only one debug master controller 612 active for each datapath 622, although more than one master could be operating if desired. For example, one master controller 612 could be monitoring a particular aspect of all the slaves 620, while another master controller could be controlling the ring. In other embodiments, each system 100 could include several or even many separate datapaths 622, each under the control of at least one master controller 612. Further, in some embodiments, the datapath 622 may include dedicated portions of data communication lines 222 of FIG. 2. In addition, to being resident in processors, the slaves 620 can be resident in any piece of state logic, such as a memory controller. For instance, with reference to FIG. 2, several slaves 620 (not shown) may be coupled to or resident in processors in the processor group 230, while one or more slaves may also be coupled to or resident in memory controllers in the memory group 240.

[0051] Still referring to FIG. 6, a debugger 640 can reside separate from the core 112 (FIG. 1), and can connect to it through a host to chip communication interface 630, which may be embodied by one of the I/Os 114 of FIG. 1. For instance, the communication interface could be embodied by a Joint Test Action Group standard, JTAG), also known as IEEE 1149.1, PCI, general I/O, or other acceptable interface as is known in the art. The debugger 640 could be a hardware and/or software process running on conventional hardware.

[0052] The debugging network 600 of FIG. 6 is straightforward to implement on a multiprocessing platform, such as the platform 100 of FIG. 1. In this embodiment, one or more processors in the core 112, such as a major or minor processor 434, 432 of FIG. 4 could operate as the system controller 610, the master controller 612, and the slave controller 614. In other embodiments, these duties could be shared across one or more processors in the platform 100. In some embodiments, each processor 432, 434 (FIG. 4) includes hardware to be the system controller 670 and the master controller 672, as well as a slave 676, and any of the processors could be selectively driven to perform any or all of the functions. Because this would be expensive to implement in hardware, other embodiments include specialized hardware in only some of the processors, such as one or two processors in every tile 210 (FIG. 2). Partitioning the components of the debug network 600 is left to the implementation engineer.

[0053] One way to implement the debug system controller 610 is to use an operating kernel that accepts commands from the off-chip debugger 640 or on-chip debugger 650. The commands are translated into one or more debug packets according to a predetermined protocol used by the master controller 612, slaves 620, and the slave controller 614. The system controller 610 generates the debug packets and the master controller 612, and places them on the datapath 622. After one of the slaves 620 responds to the request from the debug packet, the slave controller 614 (or master controller 612) removes the packet from the datapath 622 and transfers it to the system controller 610 for further analysis and operation. In the event that no slave 620 responds, e.g., the packet comes back unchanged, the system controller 610 can determine that no slave 620 had a valid response.

[0054] In another embodiment, the debugger 640, 650 itself generates and interprets the debug packets, which would make the system controller 610 easier to implement, at the expense of a more complicated debugger.

[0055] FIG. 7 is a functional block diagram of a slave 700, which is an example embodiment of the slave 620 of FIG. 6 or slave 570 of FIG. 5. Of course, other embodiments are possible.

[0056] The slave 700 couples to the debug network 622 (FIG. 6), referred to here as the debug channel. The slave 700 is divided into two clock domains: a debug clock domain 730 and a processor clock domain 720. In the debug domain 730, all read and write actions occur synchronously to the debug system clock. In particular, local state such as the address register 712 or other state information 714, such as various processor states, and the import/export to the debug channel all occur using the debug clock. Similarly, all reads and writes within the processor logic occur synchronously in the processor clock domain 720. In particular, the processor (not shown in FIG. 7) can read and write all the register sets 722, 728, 724, 740, 742 and 744 synchronously without hazard.

[0057] The processor register sets, 722, 728, 724, 740, 742 and 744 are also accessed by the debug controller 710. An instruction register 728, a previous result register 724, and a control register set 722 have both read and write access by the debug controller 710, and so are accessed through bidirectional clock-crossing channels. A watchdog register 740, a status register set 742, and an exception register set 744 only has read access to the debug controller, and so registers 740, 742 and 744 are simply sampled asynchronously.

[0058] Typically, the debug channel has a much simpler functionality than a processor, and so the logic will operate at a faster clock rate than the processor clock. Thus splitting the clock domains 720 and 730 as shown in FIG. 7 allows debug operations in domain 730 that do not have to cross into the processor clock domain 720 to execute faster. In particular, debug commands not used by the particular slave controller will be quickly passed on without affecting the execution of the associated processor.

[0059] An address unique to each slave 700 is stored in an address register 712. A channel controller 710 accepts debug packets (described below) from the debug channel, determines the requested information or action, performs the request (or denies the request based on a state of the processor), and places packets back on the channel.

[0060] An instruction register 728 may be the same as the instruction register 511 of FIG. 5 or a slaved copy. If requested to do so, the controller 710 can load an instruction into the instruction register 728, or read an instruction from the instruction register 728. This read or write must be synchronous to the processor clock domain 720, and thus occurs at the clock rate of the attached processor. In one embodiment the clock-crossing channel is 1-bit wide to reduce hardware, and so loading and unloading is performed by a serial bit shift register.

[0061] Similar to the instruction register 728, the previous result register may be the same as the previous register 526 of FIG. 5. Similarly, any reads or writes by the controller 710 to the previous result register 724 also must happen at the clock rate of the attached processor.

[0062] The single-bit operation of loading or unloading the instruction register 728 and previous result register 724 matches the bit-wise operation of the datapath 622 of FIG. 6, so that the debug network and slaves are always synchronized.

[0063] The slave 700 also includes specialized data storage, which is used to control or read relevant data from its host processor. A watchdog bit 740 can be written by the host processor when instructed to do so. The watchdog bit 740 is initialized to zero on startup of the host processor. Executing a watchdog command in the processor writes a 1 in the watchdog bit 740. The debug network can then query the watchdog bit 740 and report it over the debug channel to the debug system controller 610. If the watchdog bit 740 contains a 1, the debug master determines that the host processor is operational, or at least has executed the watchdog command since startup or the last time it was reset. The watchdog bit 740 can also be cleared by the debug system controller 610 by sending an appropriate debug message to the particular slave 700, as described below.

[0064] A set of control data is stored in a control register 722 used by the slave 700 to control its host processor. Similar to an instruction register 728 and a previous result register 724, a control register 722 is accessed via a clock-crossing channel and so reads and writes are accessed at the slowest speed of the two clock domains 720 and 730. For instance, a “keep” command is effected by storing a “1” in the K register of the control register 722. Other commands include “step” (S), “execute” (E), and “abort” (A). These commands and their operation are described below.

[0065] A set of status information in a status register 742 provides status information about particular data and control signals on the host processor. For example, status information can include whether particular flags are asserted, if a conditional branch is present, whether any of the input or output channels of the processor have stalled, whether there is an instruction in the instruction register 728 or if the execution of the processor is blocked. Additional status information can include the mode the processor is operating in, such as memory execution or channel execution mode. A copy of the program counter (such as the program counter 508 in FIG. 5) may also kept in the status register 728.

[0066] Exception information is stored in an exception register 744. Exceptions occur when particular instructions or behavior is executed on the host processor. For example, when a trap instruction is executed by the host processor, relevant trap data is stored in a trap ID section of the exception register 744. Channel identification and exception identification can also be stored upon similar commands. Description of commands to store and use such data follows.

[0067] All information that is stored or retrieved from the status register 742, the watchdog bit 740, and the exception register 744 can be accessed and updated without crossing the processor clock domain 720. Thus, reading this data occurs at the full rate of the clock signal of the debug clock domain 730.

[0068] FIG. 8 is a flow diagram illustrating an example operation flow 800 of the slave 700 according to embodiments of the invention. Debug message packets can be addressed to specific slaves 700, or, in some cases, sent out generally for any slave to answer. The first bit of the debug message packet indicates whether the message is destined for a particular processor or is sent out to all slaves. Each slave 700 matches its own address to the data stored in its address register 712. In the embodiment described, each slave 700 need not know the address of any other slave.

[0069] In a process 810, the first slave 700 downstream of the master controller 612 of the debug system 600 inspects the global bit of the current debug packet. If the global bit is set and the slave 700 has a response that can be given in response to the global request, the process 800 exits the query 814 in the YES direction. Then, a process 820 de-asserts the global bit 820 and overwrites the address portion of the debug packet with its own address, so that no subsequent slave 700 can respond. Next, in a process 824, the slave 700 modifies the debug packet with the response data. The modification can be simply changing bits in the existing debug packet, or can involve appending data generated by the slave 700 to the end of the original debug packet as its response. Some processes of the slave 700 can be performed within the debug clock domain 730, while others must cross into the processor clock domain 720. After the modification to the current debug packet has been made, the process 800 transmits the debug packet to the next stage out on the debug channel.

[0070] If the global bit of the current debug packet is not set (or the slave 700 has no response to give to a global inquiry), the slave 700 reads the debug packet destination address in a process 830. If the current debug packet is not addressed to the particular slave 700 in inquiry 834, or if the slave does not have a response to the debug packet in inquiry 844, the slave simply sends the debug packet, with no modification, out onto the debug channel to the next slave 700.

[0071] If instead the current debug packet matches the local slave address 712 and the slave has a response in the inquiry 844, the flow 800 proceeds back to the process 824 to modify the debug packet with the appropriate response.

[0072] Once the slave 700 has completed the debug packet in the process, the flow 800 returns to the process 810 and the slave 700 waits to receive the next debug packet.

[0073] FIG. 9 illustrates how the debug slave 700 can dynamically change the operation of its host processor while the system 100 (FIG. 1) is in operation. FIG. 9 illustrates a typical operating flow in processors, such as the processor 500 of FIG. 5. The operating flow of a processor 900 is divided into three main stages, a fetch stage 910, a decode stage 940, and an execute stage 970.

[0074] Between each stage is a set of data/protocol registers, such as the register 300 of FIG. 3. Each register 300 is a master-slave register and at any instant in time holds two (possibly different) values. The data width of the registers 300 can depend on their application. With reference to FIG. 9, an instruction register can be 16 or 32 bits wide, as can a branch register. A flag register can be as wide as necessary depending on the number of flags used for the particular processor. Because each register comprises two values, separate labels for each side of each register are illustrated in FIG. 9. For instance, the instruction register has a side that is in the fetch stage 910, referenced as register 912, and a side that is in the decode stage 940, referenced as register 942. Similar references are made to the branch and flag registers to denote which side is being referred to. Likewise, a decoded instruction register is 962/972, and two operand registers 964/974 and 966/976 are illustrated.

[0075] Feeding the instruction register 912 is a selector 930, which determines whether the processor is in memory execution mode or channel execution mode, as described above. The selector 930 receives its channel input from an input channel 902 and its memory input from RAM 924. Another selector 922 feeds the RAM 924 with the normal incrementing program counter 920 or one from a value generated by a branch decoder 952 in the decode branch 940. Also within the decode branch is a, decoder 950, which may be identical to the decoder 510 of FIG. 5. In the execute stage 970, an ALU 980 receives instructions from the instruction register 972 and is connected to two operand registers 974, 976. The output of the ALU is fed to an output register 984, which further feeds the output channel 904.

[0076] In operation, the flow illustrated in FIG. 9 begins at the fetch stage, where an instruction is sent to the instruction register 912, either from the input channel 902 or from the RAM 924. Because the instruction register comprises two values, a first instruction is propagated from the instruction register 912 to the instruction register 942 when there is a valid instruction in the register 912 and the register 942 is accepting. Thus the instruction register 912/942 can be holding 0, 1, or 2 instructions, Further, if the instruction register 912/942 holds a single instruction, the instruction can be stored in either the instruction register 912 or 942. Precise control of the location of instructions in the instruction register 912/942 gives the debug network 600 the power to easily control the processor 900.

[0077] Such precise control could also be exercised on the border between the decode stage 940 and execute stage 970, but in this embodiment such fine control is typically unnecessary for operation of the debug network 600.

[0078] The debug network 600 can change the operation of the processor 900 under its control by extracting instructions from the instruction register 942 and writing new instructions into register 942. Recall in the description with reference to FIG. 7, that the slave 700 can remove instructions from, or can insert instructions into the instruction register 728 one bit at a time, at the speed of the slowest clock domain 720 or 730. The same is true for the instruction register 942 of FIG. 9. Similarly, the slave 700 can extract from and load to the previous result register 724. Although no analogue to the previous result register 724 is illustrated in FIG. 9, it would be located in the execute stage 970. Interaction with the previous result register would also be governed by which clock domain 720 or 730 is slowest.

[0079] If such an extracted instruction is stored where it can be accessed by the debug network 600, such as in the debug system controller 610 (FIG. 6), the debug network could re-insert the extracted instruction back into the instruction register 942 when it concluded its operations. Thus, the debug network 600 is able to stop a processor from executing, store the processor's current state, execute its own instructions for testing/verifying/debugging and then, when the debug network has finished, replace the processor to its original state and re-start the processor.

[0080] Operation of the debug network 620 will now be described with reference to FIGS. 5-11.

[0081] A master controller 612 generates debug packets and places them on the debug datapath 622. The debug packets could be any length, but is convenient to make them equal the lowest common multiple of instruction width, 16-bits in this embodiment.

[0082] A debug packet is delimited by the packet_id of FIG. 3. In one embodiment, the packet_id is set to “1” for each bit in the debug packet except for the last bit. When a receiver, such as the controller 710 of FIG. 7, detects the packet_id changing from “1” to “0”, it knows it has received the last bit in the debug packet. Because debug packets are always received and sent on (sometimes with modifications), debug packets cannot ever become blocked because slaves are waiting. Keeping the debug channel free from blockage ensures that all packets can get access to the destination slave 620.

[0083] The debug packet includes a header, which identifies the packet as a global packet (which any slave 620 can answer) or includes a destination address for a particular slave. Other fields in the packet include a command field and an indication of how detailed of a response it is requesting. For example, the debug packet may instruct that the slave 620 simply acknowledge the receipt of the command. Alternatively, the slave 620 may be requested to append a result, copy status bits, or include other information. Additionally, the debug packet may include data, for example values to be loaded into specific registers of the processor. In most cases the packet requests data about the host processor of the slave 620, such as operating state, or the packet simply requests that the slave 620 acknowledge that it has received the command. In some embodiments, the global packet is limited only to particular debug commands. As described with reference to FIG. 7, if the packet only requests state that is stored in the debug clock domain 730 or can be asynchronously read (register sets 740, 742 and 744), the answer can be generated and returned at the speed of the debug network. If instead the packet requests interaction with the processor, through an instruction register 728, a previous result register 724, or direct control using a control register 722, the slave 620 interacts with those registers at the slowest clock rate of the two domains 720 and 720, typically the processor clock.

[0084] All debug packets are returned by the slave 620 over the debug datapath 622 to the slave controller 614 for transfer to the debug system controller 610. In some embodiments, a slave cannot create a packet and can only modify the received packet. The slave 620 can append data by simply changing the packet_id of the former last bit of the current debug packet to “1,” appending the data from the slave, and then inserting a “0” as the packet_id of the new last bit. When the slave controller 614 receives the new packet, it continues to process until it recognizes the 0 as the packet_id, thus operating on the whole length of the new packet.

[0085] Commands are broadly split into two groups: those that are guaranteed to produce a result (so long as the debug network 600 is operational), and those with contingent success. Those commands that are guaranteed to respond are those that can be completed within the debug clock domain 730. The commands that are not guaranteed to complete are those that cause the slave to interact with the attached processor, having to cross into the processor clock domain 720, and are thus dependent on a state of the processor. The guaranteed success actions include “watchdog,” “slot”, and “set-state.”

[0086] The watchdog command from the debug network 600 is used in conjunction with a watchdog instruction executed by the processor. At any time a processor can execute a watchdog instruction, which sets to “1” the watchdog bit in the watchdog register 740 of its attached slave 700 (FIG. 7). At any time the debug controller system 610 (FIG. 6) can send a watchdog command to a specific processor by sending it in a debug packet. When responding to the command, a slave 700 reports the status of its watchdog bit in the watchdog register 740, and resets the bit value in the watchdog register 740 to “0.”

[0087] The slot command from the debug network is used in conjunction with a trap instruction executed by the processor. The trap instruction stops the processor pipeline by not allowing the next instruction to execute. The processor on which the trap instruction just completed notifies its attached slave 700 that the trap has occurred, such as by sending an “except” signal. This causes the slave 700 to load a trap ID into the exception information register 744 (FIG. 7). Then, when the debug system controller 610 issues a slot command, which may be a globally issued or directed to a specific slave 700, the slave appends the trap ID from register 744 in response to the request.

[0088] Channel exceptions follow the same pattern. A channel exception occurs when the processor decodes an instruction that is scheduled to receive data from, or output data to, a channel that is on an exception list stored by the processor. If such an exception occurs, similar to how traps are handled above, the processor notifies its slave 700, which causes the slave to store (in its exception register 744) the channel ID that caused the exception. If more than one channel could have caused the exception, only the highest priority channel ID is loaded into the exception register 744. Also similar to the procedure above, when the debug system controller 610 issues the slot command, the slave 700 answers by appending the exception ID from register 744 to the requesting debug packet.

[0089] The “set-state” command is used to set or clear the state of the information in the control register 722 (FIG. 7). Recall that the control register 722 stores states for “keep,” “step,” “execute,” and “abort,” which control operation of the slave 700, and thus the attached processor. The states may be set or cleared by sending appropriate debug commands, through the debug network 600 to the appropriate slave 700. There are also debug commands that may not be guaranteed to complete successfully, which include “load-previous,” “extract,” “insert,” and “insert-execute.”

[0090] Because they depend on the state of the processor when the command is received and attempted, the commands below are not guaranteed to complete. With reference to FIG. 7, the load-previous command causes the slave 700 to load data from the debug channel into the previous result register 724. Recall that in some embodiments loading the previous result register occurs one bit at a time, in the processor clock domain 720. Similarly, the insert command loads data from the debug channel into the instruction register 728, also in the processor clock domain 720. The insert-execute command first loads data from the debug channel into the instruction register 728, then causes the loaded instruction to execute. The extract command loads data from the instruction register 728 back onto the debug channel.

[0091] During operation, the processor can be in one of several states. For example, the processor can be running, or it can be blocked. A block occurs when the protocol signals prevent data from being fetched, decoded, or executed (FIG. 9). For instance, if an object connected to the output channel 904 is not receiving data, the ALU 980 cannot process further data. If the ALU 980 cannot process data, then the instructions and operands fill the registers 962, 972, 964, 974, 966, and 976. This, in turn, causes the decode stage 940 to stall, which backs up instructions in the instruction register 912/942.

[0092] For effective debugging to occur, it is best to have the decode and execute stages 940, 970 empty (or know they will be empty), and an instruction held in the instruction register 942. This is denoted a “clean-halt” state, which means the processor 900 is ready to be controlled by the debug system 600. The instruction in the register 942 can be extracted by using the extract command, as described above. Also as described above, at the conclusion of the debugging, the previously extracted instruction can be replaced using the insert command, which would place the processor in the original state.

[0093] With reference to the control register 722 of FIG. 7, the Keep command halts the processor at the fetch stage and attempts to allow the processor to clear. The Step command allows one fetched instruction to decode and execute. The Execute command allows an inserted instruction to decode and execute, while the Abort input is used to nullify and remove results from the decode 940 and execute stages 970, thereby clearing the processor for operation. In the debug system, using the Abort command is usually a non-preferred choice because it removes data from a processor in an attempt to gain control when the processor is fatally blocked. By combining these commands, a rich dialog can be built between the debug master controller 612 (FIG. 6), the debug slave 620, and the processor attached to the slave.

[0094] In principle, the debug network 600 first uses Keep to stop the processor pipeline at the input of the decode stage 940 (FIG. 9), by de-asserting the valid signal to the instruction register 942. Instructions in the execute stage 970 are not affected and are completed normally. Operation of the fetch stage 910 will eventually stop because of de-asserted accept signals flowing back from the instruction register 942.

[0095] The debug network 600 can query the slave 700 to send the value of its status register 742, which indicates whether there is an instruction waiting and/or the execution is blocked, as described above.

[0096] Once the pipeline is put into a clean-halt state, instructions may be single stepped by executing them from the instruction register 942 one at a time using the Step control. The slave 700 could insert its own instruction into the instruction register 942, as described above, or can allow the instruction presently stopped in the register 942 to continue. If the slave 700 inserts its own instruction into the instruction register 942, the instruction stored in the instruction register 912 remains undisturbed.

[0097] After executing the desired instruction, the debug network 600 could request that the slave 700 send a copy of its status registers 742, which allows the debug system master 610 to determine how the processor is operating. Also, the debug network 600 could request that the slave 700 send the previous result register 724. The system master 610 would need to recognize that the previous result register 724 is potentially invalid until the processor has completed a number of cycles because of the pipeline created by the execution logic.

[0098] The debug system master 610 can cause the processor to execute many different instructions by using the insert-execute command. When the system master 610 is ready to return the processor to the original instruction stream, it can put the saved instruction back into the instruction register 728, then cause it to execute, returning the processor back to its original condition before debug started.

[0099] If instead the processor is not in the clean-halt state, any attempt at executing the commands insert, insert-execute, and extract will be unsuccessful. The slave 700 indicates this by modifying a bit in the debug packet containing the instruction before sending it along the debug network to the debug system master 610.

[0100] FIG. 10 is a block diagram that illustrates a debug network 1000 for a multiprocessor system that includes multiple clock domains according to embodiments of the invention. A backbone debug network 1010 operates at the system clock, which in this example can be 1 GHz. A debug domain 1020 is similar to the debug domain 730 of FIG. 7, and includes a controller 1012. The debug domain 1020 is coupled through a clock crossing network (not shown) to a processor domain 1022, which is similar to the processor clock domain 720 of FIG. 7. The processors may operate at various frequencies as illustrated, such as 123 MHz, 221 MHz, 400 MHz, and 1 GHz, etc. The particular clock frequencies driving the pairs 120 is unimportant, other than for illustration.

[0101] In operation, the backbone debug network 1010 transmits a debug message packet to the first local controller 1012 in the network 1000 at the debug clock frequency. The local controller then determines if it is addressed for itself, or has a global address set. If the message packet has a local destination, the local controller 1012 reads the debug message packet. If the debug message packet contains a command that can be answered by reporting or changing any of the data registers solely within the debug clock domain, such as those illustrated in FIG. 7, then the local controller 1012 immediately performs the requested operation and reports the action by modifying the debug packet, as described above. The local controller 1012 then sends the debug message packet back to the debug master along the backbone debug network 1010.

[0102] If instead the debug message packet contains a command that requires a read or write in the processor domain 1022, the local controller uses a clock crossing channel to access the processor domain registers. In this case, the entire debug network 1010 temporarily (for the time to process the packet) is throttled to operate at the (slowest of the processor or debug clock) speed. Because many of the commands or operations that the debug system controller 610 (FIG. 6) may request a particular processor can be answered by the local controller 1012 without ever having to read or write elements in the processor domain 1022, the debug network 1000 simply runs at full network speed much of the time. In the case where the processor clock in the processor domain 1022 equals the debug clock in the debug domain 1020, the debug network operates at full speed, regardless of whether the particular command specifies action to be performed in the processor domain 1022.

[0103] If the debug message packet received by the local controller 1012 is not addressed locally, the controller 1012 passes the debug message packet to the next traffic controller 1012 on the backbone debug network 1010.

[0104] In contrast to a debug network that is serially coupled through every slave unit in an entire network, a system such as that described with reference to FIG. 10 only (potentially) slows to a local clock rate when debug message packets need to access registers that operate in the processor domain. Otherwise the message packets operate at the full debug clock rate.

[0105] FIG. 11 illustrates a clock crossing circuit 1100 that can be used for the clock crossing networks when a debug message packet passes from the debug clock domain 730 to the processor clock domain 720 and back again. The clock crossing circuit 1100 includes three clock domains: an input clock domain 1160, a clock crossing domain 1180, and an output clock domain 1190. Within each domain, components operate at the clock speed of the domain. With reference to FIG. 10, the input clock domain 1160 is circuitry included in the debug domain 1020, while the output clock domain is circuitry included in the processor clock domain 1022.

[0106] Each of the domains 1160, 1190 may run from a master clock having the same frequency or different frequencies.

[0107] The local clock frequencies may be generated locally from a master clock, as described in the U.S. patent application Ser. No. 11/460,231, filed Jul. 26, 2006, entitled CLOCK GENERATION FOR MULTIPLE CLOCK DOMAINS, the teachings of which are incorporated by reference herein. As taught in the '231 application, the master clock for each domain can be made from a power-of-two divider, which means that the rising edge of any slower clock always aligns with a rising edge of faster clocks. Additionally, each of the domains 1160, 1190 may mask particular clock cycles of its own master clock, using clock enable signals, i_ape and o_cpe to generate its own final frequency.

[0108] In operation, the clock crossing domain 1180 operates at a rate equal to or an integer multiple above the higher of the clock rate of the input clock domain 1160 and the output clock domain 1190. In other words, whichever clock domain has the highest master clock frequency, the input clock domain 1160 or the output clock domain 1190, the clock crossing domain 1180 runs at that clock frequency or an integer multiple above that clock frequency. Although the clock domain 1160 is referred to as an input domain, and the clock domain 1190 is described as an output domain, protocol information in the form of data actually flows in both directions, as illustrated in FIG. 3.

[0109] In the input clock domain 1160, data is stored in flip-flops or registers 1164 and side registers 1162. A selector 1163, such as a multiplexer, controls the origination of the data stored in the register 1164. Data operating as the packet_id is stored in the same manner and will not be differentiated with respect to the description of the clock crossing circuit 1100. A similar configuration also stores an input valid signal, i_valid, in either register 1168 or side register 1166, controlled by a selector 1167. Output of an i_accept signal, which indicates that a successive stage is able to accept data, controls the selectors 1163 and 1167. Additionally, an output of the side register 1166, which indicates whether the data stored in the side registers 1162 is valid, is combined with an output of a register 1170 in a logic gate 1174. Such a configuration allows the data in the side registers 1162 to be updated when the data is invalid, regardless of a state of an output from a register 1170. A logic gate 1172 operates in the same way to allow data in the main registers 1164 and 1168 to be updated as well, based on a state of the output of the logic gate 1172.

[0110] The output clock domain 1190 includes only a single additional gate when compared to a non clock-crossing system. A logic gate 1192 combines an accept signal with a clock pulse enable signal for the output clock domain, o_cpe. In operation, the o_cpe signal is combined with the master clock signal of the output clock domain 1190 to generate the actual clocking signal for the output clock domain 1190. The output of the logic gate 1192 is sent to the clock crossing domain 1180. The logic gate 1192 ensures that only one accept signal is ever generated within one tick of the master clock signal that is used to drive the output clock domain 1190. This avoids multiple accept signals in a single output clock tick.

[0111] The clock crossing domain 1180 includes circuitry that ensures that data passes correctly from the input clock domain 1160 to the output clock domain 1190, no matter what clock speed the domains 1160, 1180 are operating, and no matter how many of the master clock signals are masked to generate the domains' final operating frequency. In this context, correctly passing data means that only one set of data passes from the input domain 1160 to the output domain 1190 for each data transfer cycle.

[0112] In a system where different domains may have different clock rates, a data transfer cycle is measured by the slowest master clock. Thus, a data transfer cycle means that only one set of data will pass from the input clock domain 1160 to the output clock domain 1190 per single cycle of the slowest clock, assuming that the protocol signals authorize this data transfer.

[0113] The circuitry in the clock crossing domain 1180 allows the data in the register 1181 to be set only once per data transfer cycle, and then prevents further data transfers in that cycle by negating the o_valid (forward protocol) signal. In particular, when the o_valid signal is negated, data transfer halts, as described above. The data in the register 1181 cannot be set again until after the rising edges of both of the slow and fast domains next occur at the same time. Note that the circuitry in the clock crossing domain 1180 operates correctly no matter which of the clock domains 1160 or 1190 is the fastest domain, and no matter which of the domains has the highest master clock frequency. When the clock domains 1160 and 1190 are clocked at the same frequency, the clock crossing domain 1180 has almost no affect on the clock crossing circuit 1100. In particular, if both clocks of the input clock domain 1160 and output clock domain 1190 have the same frequency (the synchronous case), o_cpe=i_cpe=1, the logic gates 1184 and 1192 are always enabled, and therefore the clock rate of such a synchronous system would perform at full rate, as if the circuitry in the clock crossing domain 1180 didn't exist, other than a minimal logic gate delay.

[0114] From the foregoing it will be appreciated that, although specific embodiments of the invention have been described herein for purposes of illustration, vanous modifications may be made without deviating from the spirit and scope of the invention.

[0115] More specifically, the debug network could be implemented in a variety of ways. For example, a wider network could access data more quickly from the processors and would also allow instructions or data to be loaded with less delay. Multiple debug networks could be present on the platform. Instead of a ring, the network could be implemented by directly connecting channels to a central hub. Different data could be stored by the slave and requested by the debug master. A different command protocol could be used by the debug network to control the slaves and host processors. Accordingly, the invention is not limited except as by the appended claims.