S h o r t S t o r i e s

// Tales from software development

Archive for the ‘General’ Category

Calculating typical network data rate

leave a comment »

I have a Windows service that monitors internet connectivity by periodically fetching a web page from a well known web site.

As I’ve recently been having problems with the speed of my internet connection I thought it would be useful to also download a reference file a few times each day and time how long it takes in order to monitor download speeds.

The service is configured to run the download speed test several times, at half hour intervals, in the early hours of the morning.

So, I now have a set of download speed samples for each day but how should these speeds be reported ? The minimum, maximum, or average ?

The problem with the minimum is that it may be spurious. Most internet connections suffer reduced download speed from time to time and it’s also possible that a low download speed may reflect bandwidth limitations in the local network or higher than usual CPU activity on the server where the test is running.

To some extent this applies to all values less than the maximum sample value. The download speed samples should, ideally, all be a single consistent value rather than a range of values fitting, for example, a normal distribution curve. But in the real world, there will be a range of values and I’d like to reject any samples that are perhaps not an accurate reflection of typical download speeds.

There are some statistical methods for rejecting outlier samples such as Chauvenet’s CriterionGrubb’s Test and Peirce’s Test but they are all questionable techniques, especially when applied without any consideration of the nature of the data and its distribution.

Fortunately, I didn’t need a rigorously correct method, I just needed a means to reject low value samples that were not an accurate reflection of download speed and would skew any attempt to determine a typical download speed.

I decided that Chauvenet’s method would be quick and simple to implement but with a small twist. This method relies on calculating the standard deviation for the sample set and then rejecting any samples that are more than one standard deviation from the mean. However, while this sample set may contain spurious low values, the high values are accurate because it’s not possible to have values beyond the actual maximum download speed that the connection is capable of. So, rather than reject all samples more than one standard deviation from the mean, the method I implemented only rejects samples that are below the mean and more than one standard deviation from the mean.

The mean of the remaining samples is calculated and presented as the ‘typical download speed’.

An example set of samples looks like this (download speed in bits per second):

6391320
6052659
6264538
5965232
5500726
5723570
5854644
5867441
5822894
4742675

A graph of these values helps show that while most of the values are in a fairly narrow range, the first and last samples are, arguably, outliers:

download_speed

Plotting the absolute standard deviation shows that the first and last values, with standard deviation greater than 1, are definitely candidates for outlier values and that sample three is right on the threshold:

download_speed_deviation

The first value, 6391320, is 1.25 standard deviations greater than the mean of 5818570 while the last value, 4742675, is 2.35 standard deviations below the mean. Both these samples would be rejected when using Chauvenet’s Criterion but the modified version I’ve implemented only rejects the lower value sample.

An average of all the samples gives a value of 5818570. Applying Chauvent’s Criterion and rejecting the first and last samples results in an average of the remaining samples of 5881463, while using the modified version and rejecting only the last sample gives a sample average of 5938114.

While this modified version of Chauvent’s Criterion skews the result towards the higher value samples, I believe that this is correct for this connection speed data.

Written by Sea Monkey

July 16, 2014 at 6:00 pm

Posted in Environments, General

Tagged with ,

How to lose customers fast. Really fast…

leave a comment »

Auction Sentry is an ebay bid management tool that I’ve been using for the past eight years. The annual subscription is $10 and although I don’t use it much, it’s been very useful when I have needed to use it. Over the years the product has had its problems, usually caused by ebay making changes, and the developers have sometimes lagged behind in resolving issues. However, I felt loyalty to the company as I’d used its product for years and the support team were helpful and responsive and bugs were usually addressed and resolved quickly.

Last week I went to use the software only to find that it had expired. I was a little surprised as I hadn’t received a subscription renewal reminder email. Even more puzzling, when I tried to renew my $10 annual subscription I found that the only option offered was a monthly renewal for $4.95. Thinking that this was some sort of trial offer, I repeatedly tried to renew my annual subscription but each time only the monthly renewal option was offered.

A quick Google search showed that I am not alone and that many other users have encountered the same problem. It appears that the company has decided to increase its subscription fees by 500% with no concession to its existing subscribers. This seems so ridiculous that I raised it with the company’s support team.

No annual renewal offered

Category:  Renewals

Case Status:  Closed

Last Updated: 6/30/2014 9:01:05 AM

Category:
Renewals
Case Topic:  No annual renewal offered
xxxx@xxxxx.com 

Customer

Posted Thursday, 6/26/2014 3:22 AM

I've been an Auction Sentry user since 2006 and have purchased either annual or 3 year subscriptions renewals when required to continue to use it. I only use Auction Sentry occasionally and when I just tried to use it I received a message indicating that it had expired and that I need to renew my subscription. However, the only option that I'm being offered is for a single month at $4.95. I've received no emails indicating that my subscription was due to expire or that it had expired and I don't understand why I'm not being offered a one year or three year renewal option ?

Best regards.

Craig

Support

Posted Thursday, 6/26/2014 3:19 PM

Hello,

I am sorry to hear that you have encountered difficulty with a renewal.

Unfortunately, we no longer offer subscriptions for durations longer than one month, so the currently monthly subscription you see is the only one we have remaining.

Apologies for any inconvenience this may cause.

Please let me know if you have other questions or encounter further difficulty.

Sincerely,
Auction Sentry Support

xxxx@xxxxx.com 

Customer

Posted Friday, 6/27/2014 12:14 AM

After 8 years as a loyal customer, I'm very disappointed to be treated this way. Clearly your company has no regard for its long term customer base and so I will be joining the many who have deleted your software from their computers and found an alternative product from a company that cares about its customers and doesn't treat them with such contempt. I'd like to be able to wish your company every success but the truth is that I think it deserves to go out of business as a result of losing customers because of the way it is treating them.

Best regards.

Todd K.

Support

Posted Friday, 6/27/2014 11:56 AM

Hello

Thanks for following up on this matter. I am sorry to hear of your disappointment. I will pass your comments on to our management team but, my understanding is that our fee structure is permanent and will not be changing.

Sincerely,

Todd K.

xxxx@xxxxx.com 

Customer

Posted Monday, 6/30/2014 1:33 AM

Thank you for your follow up. I do strongly feel that Auction Sentry has got this badly wrong and is treating its existing customers with contempt. It's a basic rule of business that you retain existing customers because it's very costly to acquire new ones. I suspect that most existing customers will be as annoyed and disappointed as I am with the new subscription pricing and will take the same action – find an alternative product. For what it's worth, I have uninstalled Auction Sentry and started using Xellsoft's HarvEx. It may be a less polished product but it offers similar functionality and is more than sufficient for my needs at a price that was what I was previously paying for Auction Sentry.
Todd K.

Support

Posted Monday, 6/30/2014 8:59 AM

Hello again,

Thank you for your additional comments.

Sincerely,

Todd K.

xxxx@xxxxx.com 

Customer

Posted Monday, 6/30/2014 9:01 AM

Auction Sentry has lost another customer.
Status:
Closed 

 

Written by Sea Monkey

July 1, 2014 at 7:00 pm

Posted in Comment, General

Tagged with ,

What does Adobe know about web programming ?

leave a comment »

What does Adobe know about web programming ? Perhaps not as much as it should…

Adobe wants its customers to purchase subscriptions for its cloud based products so you’d think the company would employ the best talent for its web programming and yet it seems that these guys can’t even correctly determine what browser is accessing their web site. Having just purchased a an upgrade to Lightroom 5 from Adobe’s web site using Internet Explorer 10, I was presented with an error message telling me that my web browser, ‘Internet Explorer 4, or earlier’ was not supported:

adobe_ie_10

Written by Sea Monkey

June 14, 2013 at 7:00 pm

Posted in Comment, General

Tagged with ,

ffmpeg.exe – maybe incorrect parameters such as bit_rate, rate, width or height

leave a comment »

After successfully ripping the audio from some video files using ffmpeg.exe I suddenly started getting this error message:

Error while opening encoder for output stream #0.0 – maybe incorrect parameters such as bit_rate, rate, width or height

Unfortunately, this is a generic error message that gives little indication of exactly what the problem is. I googled the error but couldn’t find any suggestions that helped. After trying a few things I realised what the problem was – the video files I’d successfully converted had a stereo audio track but the video files it was failing on had a 5.1 audio track as well as stereo. By default, ffmpeg was selecting the multi-channel audio track which, obviously, couldn’t be output to a stereo mp3 file.

The solution is to add the following command line parameter to indicate that ffmpeg should create a two channel output:

-ac 2

So, the complete command is:

ffmpeg.exe -i “video-file.m4v” -vn -acodec libmp3lame -ac 2 -ab 256k “audio-file.mp3”

Written by Sea Monkey

March 24, 2013 at 8:00 pm

Posted in General

Tagged with

Abraham Wald

leave a comment »

Dr Aoife McLysaght, Associate Professor of Genetics, at Trinity College, Dublin was interviewed by Eddie Mair on the Radio 4’s PM programme on Thursday 6th December 2012 about her work on Evolutionary Genetics.

Dr McLysaght used the work of Abraham Wald as an example to illustrate how a seeming lack of first hand evidence can sometimes be used to make useful deductions.

“There’s a core priciple of evolutionary genetics in terms of what the comparisons can tell you and this World War 2 analogy explains it. So, there was this statistician, Abraham Wald, who in World War 2 was asked to come in and help the air force how to decide to use their limited resources, in terms of metal, to reinforce the planes so that they could reinforce the vulnerable parts of the planes. And what he set about doing was looking for the parts of the planes returning from the battlefield that didn’t have any bullet holes in them. This seemed counterintuitive, why was he looking at the parts of the planes that didn’t have bullet holes in them ? This is because any planes that were hit in that region didn’t return from the battlefield because they were shot down and you have the same kind of logic in evolutionary analysis because when I’m doing these gene comparisons, comparing human DNA to chimp and gorilla and out to fish, when we look for regions of DNA that haven’t changed, it’s not because it never happened, it’s not because the bullet never hit, it’s because they didn’t return from battle in terms of natural selection so these regions of DNA that we don’t observe changes in, are regions where when the change happens it’s really devastating in terms of what the normal, correct, function of that gene could be.”

Wald’s report “A Method Of Estimating Plane Vulnerability Based On Damage Of Survivors” is available at https://www.cna.org/sites/default/files/research/0204320000.pdf.

Written by Sea Monkey

December 11, 2012 at 8:00 pm

Posted in General

Tagged with

Removing the Internet Explorer 8 search box

leave a comment »

After ten minutes of trying everything I could think of to remove the search box shown to the right of the address bar in Internet Explorer 8 (and 7) I googled for a solution. It turns out that the search box can only be hidden by adding a registry setting – unbelievable!

You need to add a DWORD setting called NoSearchBox with a value of 1 under the following key:

HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Internet Explorer\Infodelivery\Restrictions

More info here:

http://www.vistax64.com/tutorials/68167-internet-explorer-search-box-add-remove.html

Written by Sea Monkey

October 11, 2011 at 8:00 pm

Posted in General

Tagged with

Fail quickly, fail small…

leave a comment »

Tim Harwood, author of Adapt, was interviewed on Start The Week this morning and came up with the pithy advice: “Fail quickly, fail small”.

The alternative, as demonstrated by the banks a couple of years ago, is to fail late and fail big.

Written by Sea Monkey

June 20, 2011 at 8:00 pm

Posted in General

Tagged with