Mean Mr. Modeler

With apologies to the Beatles.  I read a post by Tom Kyte the other day about a data model that had passwords as clear-text, unique keys on a table and even used them as foreign keys!  After reading it, I was reminded of a conversation I had with my wife, Wendy, about how she’s seeing a revival of interest in enterprise data modeling due to the SOA / BPEL fad surge ;-).

Despite some people’s opinion to the contrary, I’m less of a purist than I sound.  I love data modeling, but I recognize that it can be an endless quest for non-achievable perfection.  I once worked on a project to fully model all of the financial reporting attributes for a Fortune 500 company — the project took a full year to model, at which point the model was obsolete :-).

After that debacle, I’ve come to realize that data modeling (and the eventual goal of a data model, which is its representation in the physical implementation world) has a heavy dose of artistry along with the science of analysis and normalization.  And I’m not sure how to teach the art — so much of how to model something includes the requirement to understand how the data will be accessed and processed.

A common problem I encounter is a data “model” produced by programmers who have no experience in the art.  These are usually physical models which mimic flat files or maybe, just maybe an advanced structure like a linked-list or tree.  Most of the time they suffer greatly from poor attribute definition, lack of any keys, and no appreciation for scalability or schema evolution.

In the end, those last 2 items appear to be the biggest problems.  I usually handle them with 3 questions every time I’m presented with a physical model:

  1. How will this handle concurrency?
  2. What will it look like with 1 million entries?  10 million?  1 billion?
  3. How hard will it be to add attributes?  Add dimensions (versions, history, etc.)?

Leave a Reply

Posting code can be a pain. To make sure your code doesn't get eaten, you may want to pre-format it first by using HTML Encoder