Thinking Clearly about Performance

Improving the performance of complex software is difficult, but understanding some fundamental principles can make it easier.

Source: Thinking Clearly about Performance via ACM Queue

Reasons for bad performance in distributed systems

Another great figure from DynaTrace, this time by Alois Reitbauer. It summarizes the possible reasons for bad performance in distributed applications.

Reasons for bad performance

The following figure by Andreas Grabner from DynaTrace is an excellent summary of the possible reasons for bad performance in computer programs.

Steps for a performance evaluation study

1. State the goals of the study and define the system boundaries
2. List system services and possible outcomes
3. Select performance metrics
4. List system and workload parameters
5. Select factors and their values
6. Select evaluation techniques
7. Select the workload
8. Design the experiments
9. Analyze and interpret the data
10. Present the results. Start over, if necessary.

Read more about this in the book The Art of Computer Systems Performance by Raj Jain

Common mistakes in performance evaluation

1. No goals
2. Biased goals
3. Unsystematic approach
4. Analysis without understanding the problem
5. Incorrect performance metrics
6. Unrepresentative workload
7. Wrong evaluation technique (measurement / simulation / analytical modeling)
8. Overlooking important parameters
9. Ignoring significant factors
10. Inappropriate experimental design
11. Inappropriate level of detail
12. No analysis (only measurements)
13. Erroneous analysis
14. No sensitivity analysis (to find the relative importance of various parameters)
15. Ignoring errors in input
16. Improper treatment of outliers
17. Assuming no change in the future
18. Ignoring variability
19. Too complex analysis (start simple!)
20. Improper presentation of results
21. Ignoring social aspects
22. Omitting assumptions and limitations

Read more about this in the book The Art of Computer Systems Performance by Raj Jain

Computer performance definitions

Cost-performance trade-off

Select:
- the right measures of performance
- the right measurement environment
- the right techniques

Metrics - criteria used to evaluate the performance of the system (examples: response time, transactions per second)

Workloads - requests made by the users of the system

Factors - system parameters that are varied in the study. Other parameters should fixed or typical values.

2 tools are required to measure the performance of a computer system:
- load generator - to load the system;
- monitor - to measure the results.

Read more about this in the book The Art of Computer Systems Performance by Raj Jain