SYS:PUBLIC.PROBE;PROBES.TEXT January 30, 1987 COMPLETENESS PROBE UTILITY by David Gray Original version November 19, 1984 Updated January 30, 1987 Pathnames updated August 15, 1988 1 INTRODUCTION The Completeness Probe utility is a tool which aids software testing by telling you which parts of your code were and were not executed while running your test cases. This then enables you to extend the test programs to cover the additional cases and to be sure that everything has been exercised. It can also reveal code that can never be reached. The utility can be used either as a set of functions which can be called from a Lisp Listener, or through the use of a UCL window interface. 2 PROBE FUNCTIONS To use the probe utility functions, first do (LOAD "SYS:PUBLIC.PROBE;PROBES.XLD") to load the utility. Then say (COMPILER:PROBING-ON) to prepare the compiler for inserting probes into your code. (This step can be un-done by (COMPILER:PROBING-OFF).) Then use Compile Buffer, Compile Region, or whatever, to compile the code that you want to test. Anything that you compile will have hooks inserted to keep track of which paths are executed. Then run your tests. You can then use (PROBE-REPORT) Texas Instruments - 1 - Explorer SYS:PUBLIC.PROBE;PROBES.TEXT January 30, 1987 to get a summary listing which has one line for each probed function showing how many times it was called, how many single- entry/single-exit blocks of code it contains, and how many of those blocks were actually executed. Calling PROBE-REPORT with specific function names as arguments or with a single argument of T will display the Lisp source code for the function with the portions that were executed in lower case and the portions that were not executed in upper case. The source code display uses the pretty-printer, so what you see will differ from your source file text in that comments will be gone and the line breaks and indentation are likely to be a little different. The source display is suppressed if either all or none of the paths were executed. Additional details are given in the appendix of this report. 3 PROBER WINDOW To use the window interface, first load the PROBES file as shown above, and then: (LOAD "SYS:PUBLIC.PROBE;PROBER.XLD") Then execute (USER:PROBER) to activate the window. You will then see a menu of items that you can click on to invoke the various functions, including a "Help" display. Besides the convenience of using mouse clicks, the window interface has the advantage that function definitions are displayed in a scrolling window with the unexecuted portions in a bold face font that is much easier to distinguish than the lower- case/upper-case distinction in the simple type-out version. The usual commands CTRL-V, META-V, META-<, META->, and the scroll bar can be used to move around in the display. Examinination of function definitions is typically done by selecting a summary report and then clicking left on a function in the list that you want to look at. Pressing CTRL-META-L, MOUSE-M, or RUBOUT will return you to the summary listing. Texas Instruments - 2 - Explorer SYS:PUBLIC.PROBE;PROBES.TEXT January 30, 1987 4 USER NOTES A few limitations to be aware of: * If a special form is encountered which the probing routine does not understand, the form will be counted as one code block and will be displayed with *** in place of the arguments. Let me know if you encounter this so that I can add handling for any additional special forms that are needed. * The letter case or font is not significant for items in the source text which are not executable, such as DECLARE forms and variable names in lambda lists. These are displayed in the same case or font as the last executable form. APPENDIX A PROBES FUNCTIONS (PROBE-FUNCTION function-spec ...) Inserts probes into the function(s) named and recompiles. The function definition must have been evaluated or compiled in an editor buffer so that the interpreted version of the definition is available. (PROBING-ON) Causes probes to be inserted into everything compiled afterwards until (PROBING-OFF) or (LOGOUT) or (UNADVISE). (PROBING-OFF) Undoes (PROBING-ON), returning the compiler to its normal state. (PROBE-REPORT) For all of the probed functions, lists the function name, number of times called, number of basic blocks of code, and how many of them were executed. (PROBE-REPORT-FREQUENCY) Like (PROBE-REPORT) except the functions are listed in order of the number of times called instead of alphabetical order. Texas Instruments - 3 - Explorer SYS:PUBLIC.PROBE;PROBES.TEXT January 30, 1987 (PROBE-REPORT T) In addition to the above information, for each function for which some but not all of the basic blocks were executed, displays the function definition with executed portions in lower case and unused portions in upper case. [Unfortunately, this does not yet work for functions which were probed in a compile to file.] (PROBE-REPORT function-spec ...) Give information for indicated function(s) only. (PROBE-REPORT 0) List the names of functions which have an execution count of zero; that is, were never called. (CLEAR-PROBE-DATA) Re-initializes the probe data. The probes remain in the functions but all other information is forgotten. Any probed functions will be recorded with a call count of 1 the next time they are called. (FASD-PROBE-DATA "pathname") Writes the probe data to an XLD file. The file can be loaded later to restore the data. Texas Instruments - 4 - Explorer