Steps for a performance evaluation study

1. State the goals of the study and define the system boundaries
2. List system services and possible outcomes
3. Select performance metrics
4. List system and workload parameters
5. Select factors and their values
6. Select evaluation techniques
7. Select the workload
8. Design the experiments
9. Analyze and interpret the data
10. Present the results. Start over, if necessary.

Read more about this in the book The Art of Computer Systems Performance by Raj Jain

Common mistakes in performance evaluation

1. No goals
2. Biased goals
3. Unsystematic approach
4. Analysis without understanding the problem
5. Incorrect performance metrics
6. Unrepresentative workload
7. Wrong evaluation technique (measurement / simulation / analytical modeling)
8. Overlooking important parameters
9. Ignoring significant factors
10. Inappropriate experimental design
11. Inappropriate level of detail
12. No analysis (only measurements)
13. Erroneous analysis
14. No sensitivity analysis (to find the relative importance of various parameters)
15. Ignoring errors in input
16. Improper treatment of outliers
17. Assuming no change in the future
18. Ignoring variability
19. Too complex analysis (start simple!)
20. Improper presentation of results
21. Ignoring social aspects
22. Omitting assumptions and limitations

Read more about this in the book The Art of Computer Systems Performance by Raj Jain

Computer performance definitions

Cost-performance trade-off

Select:
- the right measures of performance
- the right measurement environment
- the right techniques

Metrics - criteria used to evaluate the performance of the system (examples: response time, transactions per second)

Workloads - requests made by the users of the system

Factors - system parameters that are varied in the study. Other parameters should fixed or typical values.

2 tools are required to measure the performance of a computer system:
- load generator - to load the system;
- monitor - to measure the results.

Read more about this in the book The Art of Computer Systems Performance by Raj Jain