Reliability of benchmarking scripts
I was doing some benchmarking when I noticed some weird numbers being reported. I decided to investigate by running the script three times. The reported download speeds of the 100MB test file from Softlayer SG to my server are:
When I downloaded the same file using curl three consecutive times, the speeds are:
Finally, I tried the same file three consecutive times using wget:
There seems to be a clear consistent pattern that the script is under-reporting the bandwidth in the range of 40 - 80% relative to plain curl or wget. I have even more wild swings in other instances. This is making me curious as to what is causing it. Nench uses the curl command in its script, but it doesn't seem to be curl's fault because when I executive the curl command manually, the speeds are consistent with wget.
I think the download speeds of benchmark scripts need to be taken with quite a big grain of salt. I am not sure why, but manual downloads seem to be a much better reflection of downloading speed.
Now I have to think about how to tweak my benchmarking procedure.