PAVING THE WAY TO DATA WAREHOUSE PROJECT SUCCESS: PART I
by Doug Laney

Data warehouse projects pose a truly unique set of analysis, design, technology and management challenges which are unlike traditional development projects. More than a set of technologies, successful data warehouse implementations are the result of an effective project approach. Confidently and effectively navigating your way through a data warehousing effort calls for a new kind of road map. This article addresses how traditional approaches can curb a data warehouse project while a data warehouse-specific road map will place a project in the fast-lane to success.

Architecture for the Ages

There are age-old data architectures, and architectures for old-aged data. In data warehousing we desire the latter. Even after reading the books and attending the seminars, without the experience it's hard to know exactly what to do. It's all too easy to fall back on what we've done before -- for the last 30 years. It's difficult to know how to, even if we do know we're supposed to:

• create information models, not data models
• design for analytical efficiency, not operational efficiency
• introduce managed redundancy and denormalization, not shun them
• tune the database for understanding and accessibility, not for update performance
• embrace historical data, not overwrite it
• be concerned more for temporal integrity than referential integrity
• design to manage data content AND data context (meta data)
• architect data delivery structures for multiple user communities, not a single one.

The list of architectural differences goes on. By the time these architectural differences are noted (if they're noted at all), it's often too late for project managers to search for guidance. Probing traditional SDLC, business process re-engineering (BPR), or rapid application design (RAD) methodologies for answers is an exercise in futility. Regardless how many boxes or arrows or levels they may have, they're still geared toward developing operational systems which run the business, rather than informational systems to analyze and report the business.

Not only are the unique architectural considerations for data warehousing absent among common methodologies, but even worse: anything that looks remotely useful in one may in fact be the polar opposite of what an experienced data warehouse architect would advocate. Common methodologies and customary techniques, not technology, are largely to blame for data warehouse projects that are pulled over on the shoulder or have resulted in "fatalities".

The Data Warehouse Road Map

"Alright," you say, "we've put together an all-star data warehousing design team to hammer out these architectural issues -- let's go!" Go where? Go when? With whom? How? This is like the basketball Dream Team without a coach.

Successful data warehouse projects have process as a key consideration.. Again, this is not a traditional development process, but one that accommodates the radical differences in expectations, involvement and activities between data warehouse endeavors and operational business system projects. Any successful data warehouse project needs a methodology, or road map, with three key features: iterative development, parallel scheduling, and data warehouse-specific work items.

Iterative Development

There is a practical reason and a political reason why successful data warehouse projects should be managed iteratively. First, data warehouses, by definition, bring together data from several operational (source) systems. Projects that attempt to bite off too many sources at one time are more likely to end up chasing their tails as one or more of the source systems undergoes maintenance or replacement.

Second, most data warehouses are built to allow the discovery of incredible new sources of business opportunity or glaring business inefficiencies. This creates expectations that are sky-high and introduces windows of opportunity for solving particular analytical problems. In addition, most data warehouses are built to allow the visibility and analysis of cross-functional data. All of this leads to wide-spread political frenzy, and brings about the need for exceptional levels of commitment and involvement.

A life cycle approach to developing data warehouses is contrary to the obvious differences in project pragmatics and politics. To effectively contain the politics and retain the commitment needed to deliver a successful data warehouse, it is imperative to manage scope by delivering iterative successes. Each data warehouse project iteration should have a limited scope focused on delivering a single analytical solution in a three-to-four month rigid "timebox". To this end, your data warehouse road map should illustrate the project on-ramps and off-ramps allowing you to reach many destinations along the way.
---
Part II of this series will appear in next week's edition of D S * .