|Understanding HINT Graph|
| || |
|John L. Gustafson||Rajat Todi|
|Ames Laboratory||Ames Laboratory|
HINT (Hierarchical INTegration) is a scientific benchmark which is used to measure "anything" that computes. It can benchmark a personal computer or a supercomputer. It can even benchmark a human. It is a universal benchmark which fixes neither time nor workload to measure the performance of a computer system. Unlike other benchmarks, HINT produces a spectrum of performance results with respect to a scalable workload. It also produces a single performance number: NetQUIPS.
HINT uses interval subdivision to find upper and lower bounds of the function (1-x)/(1+x) using only the monotone decreasing property of the function. Quality is defined as the reciprocal of the difference between the upper and lower bounds. At each time step the Quality improvement is calculated with respect to the time spent for the calculation yielding another metric QUIPS which stands for Quality improvement per second. NetQUIPS summarizes the QUIPS over time. For the detail of the metrics, refer to the technical document written by John Gustafson and Quinn Snell.
HINT generates five sets of data in the output file. They are time, QUIPS, Quality, subintervals, and memory used in bytes. The first two data, Time and QUIPS, are used to produce the QUIPS graph. This graph can be thought of as a performance "speed" graph of the computer. The third and fourth data sets are Quality and subintervals. These columns can be used to check for loss of quality due to insufficient precision and poor choice of which rectangle to split. The fifth data, memory used in bytes, is useful in determining the memory regimes, i.e. cache and memory sizes.
HINT results can be used to interpret and explain a variety of aspects of computer performance. Recent study shows that HINT has a potential to express a superset of information given by any particular fixed-size benchmark. The following analyses use a range of actual HINT results from the steadily growing data base. The systems that generated the data are not identified because the point here is to understand the interpretation of the HINT graphs, not to critique specific systems. Each graph is accompanied by an explanation of how to interpret the data. Unless explicitly mentioned, a logarithmic time scale for the x-axis and a linear QUIPS scale for the y-axis is used.
Classical memory-regime revealing graph
Varying clock speed
Varying main memory
Cache-dependent and cache-independent systems
Dedicated machine versus machine with interrupts
Scalable parallel computers
Unscalable parallel computers
Serial versus workstation clusters
Serial versus vector computer
Region of Computation
Superset of other benchmarks
Identical Machine - Different Results