This week I set out to add performance testing to a project I’ve been working on. Why performance test? The main benefit is that we can hone in on when one of the changes we’ve made to a project has affected performance dramatically(such as memory usage, run-time, etc.). It gives us the ability to review performance historically and subsequently identify areas of improvement.
The catch with performance testing is that run-time can vary between runs, making it tricky to test. There are a couple ways to tackle this problem:
The first of which is to run each performance test multiple times and average the results; then compare it to the previous run. This is the most accurate way to go about it, but not necessarily the most cost effective way to spend your time. For instance, if your performance tests take 30 minutes to complete and you need 10 runs to get a good average, that’s a 5 hour test.
Alternatively, we can compare the results of a single test to previous runs and simply identify whether or not the test falls within a desired percent of our distribution. This doesn’t have quite the accuracy of the first approach, but if you don’t expect extreme performance changes it can be a viable option. The biggest downside to this approach is that you won’t necessarily detect deviations immediately.
Right now we don’t have the time to run each test multiple times, so we will be implementing our tests using the second method. For testing the run-time I created a simple function that takes a lambda expression as an argument, so usage would be something like this:
Every so many months I look up “X number of things every programmer should know” and do my best to incorporate one of those principles into my coding. There are some standard concepts: modularity, short methods, avoid “spaghetti code, ” understand what your compiler is actually doing, the four years and tens of thousands of dollars you spent on an education is worth nothing. Nothing.
My most recent gem is here. I actually found this article sometime ago and tl;dr-ed it, throwing it into the “Read This” folder. I’m not proud of shelving an article due to its length, nor that I ended the last sentence with a preposition, but that’s just how things went.
In any case, after rediscovering it, two points caught my eye. First, #5: Beauty is in Simplicity. (I’ll save you the trouble of changing tabs.) This section is about writing simple code that‘s also easy to read. Most of my short code is pretty much impossible to understand (kind of like this). Maybe it’s because all my variable names are in Bantu, but still it would be worth learning how to write simple code that another person could understand. New principle identified! That was until I saw #10: Consider the Hardware. According to this, lots of small functions aren’t necessarily the best for machine efficiency. Oh, good. Do I go with small methods for modularity and readability, or giant methods for more efficient execution?