Combined waveform and data entry apparatus and method for facilitating fast behavorial verification of digital hardware designs

[0002] 1. Field of the Invention

[0003] The invention relates to an electrical circuit timing and verification system, and particularly to a computer-based system that automates and facilitates the entry, modification, analysis and generation of test benches in multiple HDL digital hardware designs via a unique wavetable spreadsheet interface which simultaneously displays data as both waveforms and numerical cell values.

[0004] 2. Description of Related Art

[0005] Hardware Description Languages (HDLs) are language specifications for precisely describing the behavior and programming the function of electrical circuits. Test benches are HDL descriptions specialized in describing and verifying the behavior of electrical circuits.

[0006] Events and data in electrical circuits are routinely described by signals (one bit wide), and vectors (two or more bits wide). The manner in which signals and vectors change as time flows are described by displaying signal snapshots sequentially. The resulting object is called a waveform.

[0007] If signal waveforms are not properly synchronized, the electrical circuit may not behave as intended, and consequently generate unwanted operational errors.

[0008] When a circuit is built, an engineer will use timing verification techniques to identify such errors such that they can be eliminated.

[0009] One popular timing verification technique is the creation of timing waveforms to describe the logical relationships, values, and timing constraints among the signals in a given electrical circuit.

[0010] A technique for testing the behavior of electrical circuits under arbitrary and specific input data streams are also needed to detect data sensitive errors. Spreadsheet-based tables of signal data streams stacked on top of each other are commonly used to spot data-sensitive errors. However, because table-based descriptions of data lacks visual content, it is inherently ill suited to convey the logical relationship among the same data. These tables are usually generated manually via programming scripts of varying complexity.

[0011] HDL simulators are typically used to test and visualize all the various logical and physical conditions that will influence the behavior and proper function of an electrical circuit.

[0012] A problem with simulators is that they cannot be used until the behavioral description of the design has been properly and manually encoded as a script or as an HDL test bench description. This in itself is an error-prone and tedious process.

[0013] Likewise, HDL test bench descriptions of test pattern data are, by themselves, static, thereby requiring considerable effort to modify in response to changes in the underlying electrical circuit specification.

[0014] Another problem with traditional simulators is that, once a problem is encountered, the test pattern must be modified manually in order to keep it up to date.

[0015] Thus, it is highly desirable to create a “seamless” two-way link between the HDL description of the electrical circuit, and the test pattern used to verify it.

[0016] Furthermore, in a truly seamless test environment, changes in the specification of the circuit interface to the outside world would have to be automatically reflected in the test pattern itself. Were this not the case, any HDL test bench description generated from the test pattern would no longer correctly reflect the user's intent.

[0017] Because HDL test benches are the standard format used by engineers to test designs with HDL simulators, there is a need for automating their entry, modification, and generation, resulting in HDL descriptions that are both quantitatively and qualitatively correct.

[0018] The present invention also addresses the problem which arises when an HDL design (VHDL or VERILOG®, or other HDLs) is synthesized to a HDL netlist and the resulting netlist may not be compatible with test benches created for the pre-synthesis HDL source. The most common reasons for this are:

[0019] 1) Unused inputs are optimized out during synthesis, and therefore not present in the port list;

[0020] 2) A signal is defined as bi-directional (inout, buffer) in the HDL source, but is used only as an output. Synthesis correctly recognizes the signal is uni-directional and changes the port mode from bi-directional to uni-directional;

[0021] 3) Complex data types are remapped to synthesized types. An example is in VHDL the INTEGER data type is remapped to std_logic_vector; or

[0022] 4) Parameters (VERILOG®) and Generics (VHDL) are used pre-synthesis to set up the design. Once synthesized, these are dropped from the port definition. Pre-synthesis test benches must instantiate the design using the parameter for it to function as intended. Post-synthesis designs have been flattened, and no parameters remain in the ports; or

[0023] 5) Partially used vectors (those with some unused bits) are changed in width; or

[0024] 6) The order of signals in the port is changed, thereby causing test benches which rely on port order for instantiation to be incompatible.

[0025] The net result is that the test engineer needs to modify all test benches to accommodate to the changes. The designer needs to update all port definitions, component instantiations, signal assignments, and output assertions. Each time changes are made to the pre-synthesis design (then synthesized) the test benches need to be re-updated to maintain consistency.

[0026] For example, pre-synthesis VHDL definition for a counter might look like the following:

[0027] After synthesis, the VHDL netlist ports look like this:

[0028] Notice the last item (COUNT) in the pre-synthesis definition is:

[0029] COUNT: inout integer range 0 to 7:=0

[0030] In this case the counter is of mode “inout” and is of type “integer”.

[0031] After synthesis it becomes:

[0032] COUNT: out STD_LOGIC_VECTOR (2 downto 0)

[0033] Notice the mode is now “out” and the type is “std_logic_vector”.

[0034] The changes made during synthesis render the pre-synthesis (behavioral) test benches unusable without modification. Normally the test benches must be updated by hand. All ports must be updated to match the synthesized netlist. Generics (and parameters) must be eliminated. Unused signals must be stripped from the port and the test bench. The test bench code must then be revised, changing all input assignments, check procedures, file IO and other portions of the test bench which are affected by signal definition changes and port changes.

[0035] During simulation and synthesis of behavioral designs, attributes are passed to the unit under test via generics or parameters. These attributes can be changed in the behavioral description. The last value assumed by the generics must be used during the remap process to ensure continuity between behavioral and post-synthesis testing.

[0036] One solution to this problem is to restrict the HDL constructs used in a design. For example, allowing only std_logic_vectors in place of integers might eliminate data type remapping. Port mode changes can be resolved (manually) by back annotating the synthesized port modes into the original behavioral HDL source. While these solutions reduce the occurrences of port map problems, they do not eliminate them.

[0037] The present invention addresses this problem by including an Automatic Post Synthesis Test Bench Remap feature that automatically re-maps signals based upon their name, mode and data type.

[0038] A computer-based system, which automates and facilitates the entry, modification, analysis and generation of HDL test benches from standard HDL description code.

[0039] The present invention utilizes a unique combined waveform- and table data entry system to facilitate fast behavioral verification of digital hardware circuit designs. A cell based, tabular data entry system consists of rows of signal names and columns of cell values containing time-varying data, which are displayed as waveforms.

[0040] The present invention allows the user to enter data in a way similar to data entry in standard spreadsheets, as well as editing values by the use of a click event. The data values are represented by both waveforms and cell values, which are shown using dynamic text compression to reduce the viewing space required to see the values. Redundant values are hidden, and constant values are maintained until another value is assigned.

[0041] Each cell is partitioned into two distinct phases, input and output. Data entered for a particular signal is automatically entered into the correct portion of the column, automatically setting up the correct timing. Bi-directional signals are allowed two entries per column, one input entry and one output entry. To clarify each column's time relevance, a color-coding scheme is used. Light blue represents input cells while yellow is used to represent output cells. Gray represents disabled times. An attempt to toggle bits or invoke the invention's cell editor in a disabled time region automatically makes the change in the preceding active time cell, guaranteeing correct entry 100% of the time.

[0042] The system and method of the present invention are for use with arbitrary HDL source code descriptions of electrical circuits and with external HDL simulators. The system comprises: (1) an HDL input editor, for dynamically retrieving, viewing and modifying arbitrary HDL descriptions of electrical circuits; (2) a scanning interface, which retrieves specific circuit sub-components to test (unit under test, or UUT) from arbitrary HDL descriptions; (3) a time scaling unit, which specifies basic timing parameters that must be met during normal circuit operation; (4) a wavetable editor, for dynamically viewing, creating and modifying test patterns; (5) a wavetable-to-test bench translation unit, which generates HDL behavioral descriptions (test benches) of the test pattern in the wavetable unit; (6) a back annotation unit which updates waveforms with simulation results, or highlights simulation errors; and (7) a remap unit which updates the waveforms upon source HDL port changes or reassociation with post-synthesis source HDL.

[0043] The Wavetable editor seamlessly blends waveform signals, pattern data and the interface specified by the underlying HDL description. Waveform signals can be altered at specific points in time simply by clicking on their background cells, or through regions of arbitrary size through a pattern generator.

[0044] The HDL Editor and the wavetable Editor contain two-way links, which allows them to maintain consistency and coherency between the test pattern under development and the input/output interface of the UUT.

[0045] Similarly, the time scaling unit has a one-way link to the wavetable editor, which allows it to modify the entire test pattern under development so that it conforms to changing requirements for the electrical circuit being tested.

[0046] The wavetable-to-test bench translation unit automatically generates HDL test bench specifications fully compatible with most HDL compliant simulators, thereby seamlessly linking arbitrary HDL descriptions of electrical circuits with the simulators needed to test them.

[0047] Finally, the back-annotation unit, which takes simulation results and updates the waveforms with the resultant values or highlights mismatches between actual and expected behavior.

[0048] The present invention also includes a feature titled, “Change HDL source”. When a waveform description has been created for a HDL (pre-synthesis) design, the user can simply select the post-synthesis HDL netlist as the new source. The remap unit of the present invention automatically re-maps signals based on their name, mode, and data type. Unused signals are automatically dropped. The waveform can then be re-exported for post-synthesis simulation.

[0049] In the examples given above, the signal COUNT is automatically remapped with no user intervention. In the case where there may be possible conflicts, the present invention will ask the user for information such as which signal to re-map to, and how to resolve the new data type. If post-synthesis uses new data types not currently recognized, the remap unit of the present invention will automatically ask for information which will allow it to use the new types and remap the older type correctly.

[0050] To summarize, the present invention is a computer system for constructing HDL test benches based upon HDL descriptions of electrical circuits, comprising a means for generating and modifying HDL data descriptions; a computer, including a monitor and memory storage means containing instructions for displaying a spreadsheet comprised of timing waveforms superimposed on an array of cells containing time-varying signal data patterns, wherein the waveforms and the data patterns correspond to the HDL descriptions and are simultaneously displayed on the monitor; means for modifying the waveforms and the signal data patterns; and means for automatically translating the waveforms into a complete HDL test bench file, the test bench file being fully compatible with industry-standard HDL simulators.

[0051] In accordance with these and other objects that will become apparent hereinafter, the instant invention will now be described with particular reference to the accompanying drawings.

[0052] FIG. 1 shows a typical computer system utilizing the present invention.

[0053] FIG. 2 shows a block diagram depicting the overall process flow of the test bench generating system of the present invention.

[0054] FIG. 3 illustrates a typical spreadsheet displayed by the present invention

[0055] FIG. 4 shows a flowchart of the test bench generation process used by the present invention.

[0056] FIG. 5 shows a flowchart of the waveform editing process of the present invention.

[0057] FIG. 6 shows a block diagram of the Pattern Wizard cell editing dialog box of the present invention.

[0058] FIG. 7 shows the timing dialog box to allow the user to select timing parameters for imported designs.

[0059] FIG. 8 illustrates, in a block diagram, the process to reassociate post-synthesis HDL netlist results with the waveform description as utilized in the present invention.

[0060] FIG. 9 depicts, in block diagram form, the remap process of the present invention.

[0061] In the following description of the preferred embodiment, reference is made to the accompanying drawings that form a part thereof, and in which is shown by way of a illustration a specific embodiment in which the invention may be practiced. It is to be understood that other embodiments may be utilized and structural changes may be made without departing from the scope of the present invention.

[0062] Referring first to FIG. 1, a combined waveform and data entry system for facilitating fast behavioral verification of digital hardware designs is designated generally by the number 10 and comprises a personal computer (PC) 100 including a monitor 110, a disk storage device 120, input devices such as a keyboard 130 and a mouse 140 and printer peripherals 150. The preferred embodiment of the present invention runs under Microsoft Windows operating systems.

[0063] FIG. 2 shows the process flow used for developing test benches for arbitrary VHDL and VERILOG® designs utilizing the present invention.

[0064] The Text Editor 11 creates the HDL source code via step 12, which is subsequently imported 13. The system 10 revises the waveform 14 and a test bench exported 15 for simulation 16. After simulation, errors are detected 17 and the source code is revised via step 18. The source code is once again imported and the waveform revised 14 until the Text Editor edits the source code to correspond to the revised waveform.

[0065] The VHDL and VERILOG® test benches produced by the test bench generation system of the present invention 10 may be simulated using simulators from VIEWLOGIC®, SYNOPSYS®, MODEL TECHNOLOGY, and others. The resulting test bench is timing constrained, and can be used for post synthesis timing verification.

[0066] HDL sources created external to the invention can be read into the Text Editor, or new designs can be created within the Text Editor. Once an HDL source is read into the Text Editor, it can be imported. The waveforms are then revised and the test bench created and exported. After simulation, errors are detected, and the test bench and the HDL source code are revised and re-imported, automatically updating the waveform attributes with updated port attributes including width, direction, data type and name, to create an updated waveform which shows the simulation output or mismatches between actual and expected behavior. The test bench is automatically re-exported to comply with the changes. The need for manual test modification for each design revision has been eliminated.

[0067] Referring to FIG. 3, a window 21 showing a typical spreadsheet utilizing the present invention can be seen. The waveform window 22 provides waveforms which are displayed and edited. A color scheme is used wherein waveform window 22 is gray and the waveforms are colored according to IO mode, whether it be input, output or bi-directional. The color scheme ensures that waveforms are clearly recognizable and easily identified when selected.

[0068] Signal names 23 are represented in each row. The port direction 24 is also indicated to the right of each signal name. Signals are simultaneously presented in both waveform 25 and as numerical cell values 26.

[0069] The HDL being tested is indicated at 27 and the source code is shown below it in the source editor window 28. Source editor window 28 allows the source file associated with the test bench to be edited and viewed as test bench development proceeds. Source editor window 28 is automatically positioned beneath waveform window 22 whenever a design is imported or a waveform file is opened.

[0070] Text strings 29 are located in source editor window 28, facilitating design revision. Process bar 30 allows the user to select one of several editing, display, or statistical options.

[0071] When an Import command is executed via the Import HDL button 31, the user is able to quickly locate the target file to be tested. Files with .vhd, vhdl. And .vhf extensions as VHDL, and files with extensions .v and .vf as VERILOG® are automatically recognized. The user is queried as to which unit is to be tested.

[0072] Once the user indicates which unit is to be tested, the unit is analyzed for port problems such as syntax and semantic errors, and inconsistencies. The problems are detected, highlighted and a solution is suggested.

[0073] The HDL is further analyzed to determine if the design is clocked or combinatorial. The user is then prompted to select a default timing to be used in the test bench. The waveform is automatically initialized, the clock pattern is automatically set, waveforms are sorted by type, and inputs are assigned initial values.

[0074] Source editor window 28 is opened, and automatically scrolled to the unit under test. Upon completion of the waveform specification, the design may be exported to a VHDL or VERILOG® test bench. Export Testbench button 32 produces a timing constrained test bench which can be ultimately simulated on a third party VHDL or VERILOG® simulator. Executing Export Testbench button 32 causes the present invention to write a VERILOG® or VHDL testbench. The exported test bench is a complete test bench which includes all of the stimuli and assertions in the parent waveform.

[0075] Prior to exporting, all inputs are checked to see if they are assigned. If not, a warning is issued and prompts the user to initialize the inputs to avoid simulation problems, to allow the inputs to be initialized automatically.

[0076] Open Waveform button 33 allows waveforms to be read into the system for editing. Title bar 34 indicates the waveform file name, and window 21 shows the waveform. When the waveform is opened, the corresponding HDL source file is also opened and shown in editor window 28.

[0077] The user may save a current waveform by executing the Save Waveform command 35. This command writes the waveform diagram to the current file name as shown in title bar 34. The Edit HDL Source button 36 invokes the source editor and allows the source code to be edited.

[0078] The invention also automatically shows testing coverage, allowing the user to quickly identify unused portions of the design and to improve design reliability. The Coverage Statistics button 38 provides a summary of how much of the design has been exercised at any point in time. As a waveform is developed, coverage of each signal is shown via small bar graphs to the left of each signal's name. The coverage bar changes from red (untested) to green (tested). Moving the cursor over a bar causes a popup window to display the signal's coverage.

[0079] Values may be represented in hex, decimal, or binary form, via buttons 39 and 40 on the tool bar (the waveform shown in FIG. 3 is currently being displayed in decimal, therefore, no decimal button appears). If the radix is changed, all further usage of numbers assumes the new radix as the default.

[0080] Because larger waveforms cannot be shown on the screen at one time, the present invention includes a Zoom Out button 41 to allow viewing and editing of large test cases. Zoom In 42 provides further magnification.

[0081] Referring to FIG. 4, the overall method of the test bench generation system of the present invention is shown in flowchart form. When a new electrical circuit design is to be tested 44, its corresponding HDL code is first imported 45 and the timing constraints and parameters are set via step 46. A set of waveforms is created 47, opened 48 and edited 49. The user then has the option of saving the waveform 50 and/or exporting the test bench 51. After simulation, the user can back re-import the design 52 into the wavetable with expected results after editing the HDL source code 53.

[0082] Referring to FIG. 5, waveforms can be edited by invoking the waveform editor 54 of the present invention. If the user left clicks on the bit waveform itself 58, further clicks will toggle the cells from “0” to “1” to a blank cell 55. Future cell values can be set in this fashion 56 and the entire screen can be refreshed 57 to accommodate the changes.

[0083] Alternatively, by clicking on the vector 59 or by double clicking 60, the cell editor can be opened and a value can be manually entered, steps 61-64 or the Pattern Wizard invoked via steps 65-67.

[0084] Data values can be entered using a spreadsheet approach where cells are assigned specific values. The user can simply double click a signal at the specific location where it should be changed. The value editor appears and the value can be entered. After pressing the [ENTER] key, the cursor is automatically placed at the next time frame where the next cell can be edited in the same fashion. Windows standard graphics operators such as cut, copy, and paste are fully supported to facilitate waveform manipulation.

[0085] Using the Pattern Wizard (FIG. 6) is an alternate method of waveform editing. Here, bit signals may be toggled, pulsed, or randomly assigned via the Choose Pattern box 68. Vectors can be counted, shifted or randomly assigned.

[0086] A user first can choose a pattern 68, and read the description of the specific pattern in the Description window 69. The pattern can be repeated for a given number of cycles 71, and a customized pattern can be created by entering the initial 70 and terminal values 72, and the amount of incrementation 73.

[0087] The present invention allows the user to specify timing constraints that must be met during post-synthesis verification via the Rescale Timing button 37 (FIG. 3). Timing parameters can be changed at any time, allowing test cases to be updated for faster devices.

[0088] When a design is imported, and the timing-editing feature is invoked, the user is automatically asked to choose a timing parameter (FIG. 7). The imported design is parsed to determine if it is clocked or combinatorial. The maximum output delay and minimum input setup time can be specified. The user simply selects the time units to be used during test benching from the Time scale list box. The duty cycle of the clock is specified by setting the Clock high time and Clock low time parameters.

[0089] The present invention outputs timing constrained VHDL/VERILOG® test benches which can be simulated on a third party simulator. By choosing a specific programmable device, the output can be used to see if the timing constraints are met after synthesis.

[0090] FIGS. 8 and 9 illustrate the portion of the invention which addresses the problem of post-synthesis netlists which may be incompatible with test benches created for pre-synthesis HDL source code.

[0091] An HDL design is imported and a waveform description of the stimulus and expected response are drawn. The waveform may be exported as a complete, timing constrained test bench. Once the design is synthesized, a post synthesis HDL netlist results. The netlist is then re-associated with the waveform description, the waveform is updated, and the post-synthesis compatible test bench is exported.

[0092] During remap, port order of the post synthesis netlist is maintained. Signals with identical names and attributes in the behavioral and netlist HDL are stored in a list. Next, signals with identical names but different attributes are padded, scaled, or re-assigned and moved to the new list. If signals remain in the original and netlist lists, then the signals are remapped where applicable. Remaining signals in the original list are dropped, and new signals in the netlist list are saved.

[0093] During the remap process, the user is asked for each signal in question, and which old signal the new signal should map to. This occurs when signal names are changed by synthesis, or when signals are added during synthesis. When signal types are remapped, the bounding ranges are used to ensure the compatibility of values. When vectors are scaled to smaller vectors, the user is asked which bits to drop. Bi-directional signals converted to unidirectional have the unused components dropped.

[0094] Once remapping is complete, the waveform can be exported as a timing-constrained test bench, which simulates with the post-synthesis netlist. The testbench simulates on any standard HDL simulator, and the results can be back annotated to the waveform for comparison, highlighting of erroneous results, or updating of the waveform for future export as the expected values.

[0095] In summary, a computer system has been described that blends entry and display of timing requirements in order to automate the entry, modification, analysis, and generation of test benches from HDL descriptions.

[0096] The instant invention has been shown and described herein in what is considered to be the most practical and preferred embodiment. It is recognized, however, that departures may be made therefrom within the scope of the invention and that obvious modifications will occur to a person skilled in the art.