Understanding HINT Graph
John L. GustafsonRajat Todi
Ames Laboratory Ames Laboratory

HINT (Hierarchical INTegration) is a scientific benchmark which is used to measure "anything" that computes. It can benchmark a personal computer or a supercomputer. It can even benchmark a human. It is a universal benchmark which fixes neither time nor workload to measure the performance of a computer system. Unlike other benchmarks, HINT produces a spectrum of performance results with respect to a scalable workload. It also produces a single performance number: NetQUIPS.

HINT uses interval subdivision to find upper and lower bounds of the function (1-x)/(1+x) using only the monotone decreasing property of the function. Quality is defined as the reciprocal of the difference between the upper and lower bounds. At each time step the Quality improvement is calculated with respect to the time spent for the calculation yielding another metric QUIPS which stands for Quality improvement per second. NetQUIPS summarizes the QUIPS over time. For the detail of the metrics, refer to the technical document written by John Gustafson and Quinn Snell.

HINT generates five sets of data in the output file. They are time, QUIPS, Quality, subintervals, and memory used in bytes. The first two data, Time and QUIPS, are used to produce the QUIPS graph. This graph can be thought of as a performance "speed" graph of the computer. The third and fourth data sets are Quality and subintervals. These columns can be used to check for loss of quality due to insufficient precision and poor choice of which rectangle to split. The fifth data, memory used in bytes, is useful in determining the memory regimes, i.e. cache and memory sizes.

HINT results can be used to interpret and explain a variety of aspects of computer performance. Recent study shows that HINT has a potential to express a superset of information given by any particular fixed-size benchmark. The following analyses use a range of actual HINT results from the steadily growing data base. The systems that generated the data are not identified because the point here is to understand the interpretation of the HINT graphs, not to critique specific systems. Each graph is accompanied by an explanation of how to interpret the data. Unless explicitly mentioned, a logarithmic time scale for the x-axis and a linear QUIPS scale for the y-axis is used.

Classical memory-regime revealing graph

Varying precision

Varying clock speed

Varying main memory

Cache-dependent and cache-independent systems

Dedicated machine versus machine with interrupts

Scalable parallel computers

Unscalable parallel computers

Special-purpose computer

Serial versus workstation clusters

Serial versus vector computer

Region of Computation

Superset of other benchmarks

Identical Machine - Different Results

Please mail your suggestions or feedback to hint@scl.ameslab.gov