Metrics of Computer Performance

David J. Lilja, Ph.D., P.E.

Course Overview

Before you can begin to determine anything about the performance of a computer system, you must decide what aspects of the system are interesting and useful to measure. In this course, you will learn about the common metrics that are used to quantify a system's performance, the characteristics of a good metric, the meaning of speedup and relative change, and how to measure the fundamental metric of performance -- time.

This course includes a multiple-choice quiz at the end, which is designed to enhance the understanding of the course materials.

Learning Objective

After completing this 2-hour course, you will be able to:

• Interpret the different types of performance metrics;
• Understand the characteristics of a good performance metric;
• Compute and interpret speedup and relative change; and
• Measure the time required to execute any portion of a program.

The reading assignment for this course is Chapter 2 of Measuring Computer Performance: A Practitioner's Guide, David J. Lilja, Cambridge University Press, 2000.

If you don't have this book, you can purchase Chapter 1 in PDF format online at eBooks.com for a modest cost. The price for this course listed on this website does not include the cost of purchasing the chapter through eBooks.com. However, the price has been reduced to compensate for the cost of purchasing the chapter required. If you plan to take all 6 courses (E132 to E137) based on this book, you may consider to purchase a hard copy of the book or the entire book in PDF format online through eBooks.com.

Key Terms

• Performance metric
• Means-based metric
• Ends-based metric
• Speedup
• Relative change
• Interval timer

### Study Notes

We begin our study of computer systems performance analysis by examining those things that are useful to measure -- that is, the metrics of performance. There are essentially three types of characteristics of a computer system that we can measure directly:

1. A count of how many times an event occurs.
2. The duration of some time interval.
3.
The size of some parameter.

From these different types of quantities, we can derive the other types of metrics in which we might be interested. For instance, a rate metric, which is simply the number of events divided by the time interval in which the events occur, is useful for normalizing event counts to a common time basis, such as operations executed per second.

Usually, we are interested in how quickly a given program is executed. As a result, the fundamental performance metric of any system is time. We can compare times directly, or we can use them to derive appropriate rates. But without a precise and accurate measure of time, it is impossible to analyze or compare any system performance characteristics. The basic technique for measuring time in a computer system is analogous to using a stopwatch to measure the time required to perform some event. Some examples of common interval timers are provided here.

While time is usually considered the ultimate measure of performance, there are many other performance metrics that are in common use. Not all of these metrics are good, however, in that they sometimes can lead to erroneous conclusions. After completing this course, you should understand what makes a good performance metric. Furthermore, you should have developed some sense for why it is so difficult to develop good performance metrics, and why, therefore, it is so difficult to accurately and rigorously compare the performance of computer systems.

Quiz

Once you finish studying the above course content, you need to take a quiz to obtain the PDH credits.

DISCLAIMER: The materials contained in the online course are not intended as a representation or warranty on the part of PDH Center or any other person/organization named herein. The materials are for general information only. They are not a substitute for competent professional advice. Application of this information to a specific project should be reviewed by a registered architect and/or professional engineer/surveyor. Anyone making use of the information set forth herein does so at their own risk and assumes any and all resulting liability arising therefrom.