The Fallacy of Premature Optimization
Every programmer with a few years’ experience or education has heard the phrase “premature optimization is the root of all evil.” […] Unfortunately, as with many ideas that grow to legendary status, the original meaning of this statement has been all but lost”
I would not agree with all the proposed solutions to the problem, but I do agree with most of the observations.
One of the most frustrating things I found when working on performance at Apple was that, almost always, user-level performance issues were caused by lots of small inefficiencies spread throughout the software adding up to noticeable chunks of wall-clock time. Issues like these were very hard to “post-optimize”: instead of one major algorithm to change, fixing the issue would require small, but still time-consuming, changes in many independent places.