It doesn’t make a difference…

I thoroughly enjoyed Martin Widlake’s post about a “simple” problem – and how you should never take anything for granted when trying to solve a performance problem.

As I read through Martin’s post, one thing kept jumping out at me.  This was a large system, with lots of controls and a full test system to boot, and yet I wondered why and how this structural difference came to be between the test and production system.

Not only was the column order different, but so was the object name.

And so, why?

Martin’s comments seemed to echo responses I’ve heard in the past when technical managers were confronted with questions about differences between test and production:

“Ignore the difference in name, that was an artifact of the test environment creation, the key thing is the primary key has a different column order. The DBAs had implemented the table wrong {I’m not blaming them, sometimes stuff just happens OK?}.”

To me, this gets at a philosophical or policy decision – why would you allow your production and test environments have differences?  Do you say that the differences “won’t make a difference”?

Clearly in this case, one of these differences invalidated testing and caused performance problems in production.

Which differences are important?  Which are avoidable?  If you’re going to allow differences how will you account for them?  How will you account for their impact?

Is there a way to prevent avoidable differences?  How creative can you be in removing differences?  How hard is it to eliminate differences after you discover them?

The reason I obsess about these things is that I like assuming that test and production are the same – that any differences are clearly ignorable OR unavoidable.

I like the fact that when I say that I’ve tested something in the test environment that it will work the same way in production.  If not, then I start to question to validity of my tests – and I don’t like to explain why I’m wasting time running tests that have limited applicability or come out with false and misleading conclusions…

I have ideas on how to minimize these differences and reduce administrative overhead along the way – and I’ll be presenting many of these ideas at ODTUG during the Sunday Symposium.

I’d be happy to share some of those ideas here as well as hear about things that have worked for others…