Ravings Of A Strange Mind

I've written on this subject in my blog before, but it still staggers me that supposedly experienced software authors still make this most fundamental of mistakes.

Let's just imagine you are a software author, and somebody has come to you with a problem that they think is in your code. And let's say that the user is not one of the clueless "why is my cursor stuck" kind - they are a fellow software developer who is highly motivated, who wants the software to work, and has done all of the kinds of things you would do if you were looking for this kind of problem. Things like isolating the bug to a specific set of failure modes, making sure it is reproducable, looking at alternate environments to see if that affects the failure mode - the usual stuff.

Given this information, you look at the report and think "Hey, that is impossible. There is no way the code can do that, and I should know because I wrote it". You might even try and reproduce the problem yourself and of course, it "works for you".

This means you have to make a choice on how to proceed. Do you:

a) Assume the person has got it wrong and there is no problem.

b) Assume that there is a problem but you can't see it for some reason.

Of these two choices, a) seems like most attractive because it is the one that involves the least work for you. But really, how realistic is this? For this to be the correct course of action, the poor sod reporting the bug would have to be so deluded that they have managed to concoct a whole story with no basis in reality simply for your benefit. Are you really so self-centred as to believe that people have nothing better to do than create works of fiction in order to bother you? Isn't there even the smallest chance that perhaps, just perhaps, there is a bug in your code that under some circumstances can exhibit the behaviour being seen? Or are you that perfect that this is simply not possible...

So really, the only sensible choice is b) - they actually have found a problem which for some reason does not occur on your system. So, you are to have to try and find it, or at least, provide them with some more information so they can try and isolate the problem further. Differential diagnosis can be very helpful in this kinds of situations.

The reverse often happens as well: you are using someone else's code and you can't even get it to work. Supposedly the code works, but damn it, you can't even get off first base. In this situation, you once again have two choices:

a) Decide that the code is fundamentally flawed if you can't get it to work, so it must be truly broken. Report this as a bug and do something else until it is fixed.

b) Realise that other people have used the code, and so you have failed to understand something that perhaps is not obvious.

You'd be suprised how many people chose a)...

Here is a real-world example: I was asked recently to evaluate a video codec from a vendor who had created a highly optimised codec for a particular hardware platform. I wrote a benchmark program, and integrated the codec as per the documentation provided by the vendor. I could push frames into the encoder, which created a nice bitstream that was supposedly compliant with the codec specification. The bitstream had the right bitrate, but I could not decode the bitstream using a reference implementation of the same codec. However, the decoder from the same vendor recreated the source images just fine.

It was tempting to just decide that the code was just crap because, although it was self-consistent, it was demonstrably non-compliant to the specification. But I also knew that the vendor has delivered this code to other companies, and that it had been used before. So even though it was not compliant, it must be possible to make it work. I just had to find out how.

It took me two days, but I eventually discovered that if I reversed each byte in the bitstream end for end, and then reversed three bits in the first byte that specified the codec mode, then the bitstream magically became compliant and worked just fine. This was not documented anywhere, and the vendor was unable (or unwilling) to explain why this was needed. By assuming that the vendor did actually have somewhat of a clue, I was able to find a way to make it work. Of course it would have been nice if the code had done the right thing, or even if the documentation had been correct, but at least I did not embarrass myself by claiming the codec was garbage and then having the vendor prove me wrong.

So please, the next time a user report a bug that seems impossible, then remember what Sherlock Holmes said:

"How often have I said to you that when you have eliminated the impossible, whatever remains, however improbable, must be the truth?"