HINT: JOHN GUSTAFSON'S LATEST MEASURE OF PERFORMANCE Oct. 21 In the News by Norris Parker Smith, Editor at Large HPCwire ============================================================================= A performance measure for high-performance computing known as HINT that evaluates accomplishment in terms of QUIPS may, at first hearing, sound like a second-year graduate student's mixture of boredom and fantasy. It is in fact a serious proposal from a serious source: John Gustafson of the Department of Energy-supported Ames Laboratory at Iowa State University who caused a stir in the performance-evaluation community a few years ago with SLALOM. In a paper co-authored by Quinn O. Snell, also of the Ames Laboratory, Gustafson argues that familiar performance measures like SPEC and almost-forgotten ones like the PERFECT benchmark neglect a crucial consideration: the quality of work done. This leads to QUIPS, the HINT measure of performance, which stands for Quality Improvement Per Second. The paper asserts that this concept is "based rigorously on progress toward solving a problem." So what does HINT mean? Hierarchical INTegration. The authors say that HINT "adjusts to the precision available and has unlimited scalability ... there is no mathematical upper limit to the quality that can be calculated, only limits imposed by the particular computer hardware used (precision, memory, and speed)." As has been argued frequently in this space, memory management, including cache arrangements, and I/O are crucial determinants of actual utility. HINT indicates the effect of the data-juggling hierarchy, reflecting the influence of primary caches within the microprocessor, larger secondary caches adjacent to the processor, and the machine's last-ditch efforts to summon data from disk. The authors have experimented with HINT on various systems available at the Ames Laboratory. Results for a Sun 4/370, Silicon Graphics' original Indigo, an Indy without secondary cache, an Indy with the cache, a DEC 5000/240, and an Indigo2 provide a fascinating glimpse into real capabilities. In terms of MQUIPS, the advantages of greater raw power -- very marked for short run times like 10 milliseconds -- decline as one microsecond is reached and tend to converge as one second of run time is reached. Comparisons of MIMD systems -- the nCUBE 2 and Intel Paragon -- show a delay as the systems get up to speed and then consistent scalability, although smaller systems appear to run out of steam between one and ten seconds. Gustafson and Snell summarize HINT in these words: [It] "is designed to last. It allows fair comparisons over extreme variations in computer architecture, absolute performance, storage capacity and precision. It improves on SLALOM in being linear ... being very low cost to convert to different architectures, and unifying the precision and memory size into the performance." HINT can be downloaded over the Internet via anonymous FTP maintained by the Scalable Computing Laboratory at Ames. A Mosaic home page is being developed. The ftp server is: . The HINT directory is located within /pub. A paper mailed by the authors to about 350 addresses (including this reporter) may be found in the doc subdirectory. Queries and comments should be addressed to . ***************************************************************************** H P C w i r e S P O N S O R S Product specifications and company information in this section are available to both subscribers and non-subscribers. *900) Ampex 915) Genias Software 916) MasPar Computer 912) Avalon Computer *930) HNSX Supercomputers *905) Maximum Strategy 921) Cray Research Inc. 902) IBM Corp. 906) nCUBE 907) Digital Equipment 904) Intel SSD *932) Portland Group 909) Fujitsu America 935) Silicon Graphics *Updated information within last 30 days ***************************************************************************** Copyright 1994 HPCwire. To receive the weekly HPCwire at no charge, send e-mail without text to "trial@hpcwire.ans.net".