2009-12-10

Common mistakes in performance evaluation

1. No goals
2. Biased goals
3. Unsystematic approach
4. Analysis without understanding the problem
5. Incorrect performance metrics
6. Unrepresentative workload
7. Wrong evaluation technique (measurement / simulation / analytical modeling)
8. Overlooking important parameters
9. Ignoring significant factors
10. Inappropriate experimental design
11. Inappropriate level of detail
12. No analysis (only measurements)
13. Erroneous analysis
14. No sensitivity analysis (to find the relative importance of various parameters)
15. Ignoring errors in input
16. Improper treatment of outliers
17. Assuming no change in the future
18. Ignoring variability
19. Too complex analysis (start simple!)
20. Improper presentation of results
21. Ignoring social aspects
22. Omitting assumptions and limitations

Read more about this in the book The Art of Computer Systems Performance by Raj Jain

No comments: