Analysts Warn End-Users to be Aware of Possible Shortcomings in Data Mart Tools
ACTION ITEMS

ComputerWorld has reported that data mart tool vendors, in their zeal to market products for gathering, cleansing, normalizing, mining and analyzing data, have frequently sold information systems organizations a bill of goods. "A lot of untruths are being propagated in the market, and part of it is vendor-driven by the makers of lower-end tools that can't scale. I think they will do anything to undermine (a top-down) approach," said Janelle Hill, research director of strategic data management at Gartner Group, Inc. in Stamford, Conn. "The whole marketing appeal of faster, smaller, cheaper' falls on very receptive ears."

ComputorWorld noted that Hill estimated when data mart decisions are made tactically, IS ends up with algorithms and data models that are obsolete within 18 months. As business requirements change and end users need to look at data in new ways, the business rules and queries can't be easily altered. So even if a data mart yields some short-term strategic value, "IS has spent a lot of money on something that it will have to retire, and it will probably not see a real return on the investment," Hill said.

Gartner Group recommends a top-down approach, which means starting with a data warehouse and later adding data marts and data mining tools. Yet a bottom-up implementation, which starts with a data mart, has found receptive users because of a misconception that they either must model all the data in the enterprise at once or build departmental marts, Hill explained.

But users can, and should, consider building the data warehouse on a subject-by-subject basis instead, Hill said. Customers, sales and budgets are good places to start. There is evidence that more companies are adopting the top-down approach.

Data warehouses that are larger than 1T-byte will make up 17% of all warehouses this year, from 7% last year, indicating that much more data is being stored centrally and feeding into downstream data marts, said John Ladley, a senior program director at Meta Group, Inc. in Burlingame, Calif. The figures are from a Meta Group poll of attendees at last month's Data Warehouse World conference.

"I don't hear a lot of vendor bashing, but I am hearing regret and hindsight regarding developing a bunch of separate data marts," Ladley said. "I am also hearing a lot of IS organizations trying to convince the business side that these (data marts) need a much higher level of integration. "

In the past five years, Xerox Corp. has built a multitier architecture of 45 data warehouses and marts, including one data mart that supports profit-and-loss decisions by business units. On that project, the faster, smaller, cheaper' message initially was appealing, particularly because the vendor had promised scalability. But results weren't optimal.

Carl Cichetti, a manager at Xerox Information Management in Rochester, N.Y., declined to name the vendor or the product but said there were a lot of problems. "The technology was more difficult to deal with than we had expected, didn't live up to the promises made and was high-maintenance. When we tried to have a lot of business rules and complexity, it really pushed the limits of the technology," he said.

Xerox has since adopted a process to find tools that are scalable and that fit into Xerox's multitier architecture. "Business needs can change quickly, so you need the framework to manage business and technology changes. And to do that, you really need an infrastructure," Cichetti said.

Brooklyn Union Gas, a Brooklyn, N.Y.-based utility, is preparing to roll out a warehouse that will better segment potential customers. The utility uses Sybase, Inc.'s IQ as the database engine and SAS Institute, Inc. tools for data gathering, cleansing and modeling. Gloria Castro, manager of information product solutions, says the utility will use its warehouse, which it is building with Price Waterhouse LLP, to combine internal and external demographic information on customers. It aims to get a picture of who, for example, rents vs. owns their home and how profitable they might be over time. Analysts can then assess who would buy certain products, such as maintenance contracts.

"Before, we just did a spray-and-pray approach -- we'd market our whole customer base with the same promo, cross our fingers and wait for a 1% to 2% response rate," Castro said. "This way we can target our marketing and hope to increase the response rate."

Ultimately, Hill said, "Data marts have run a high risk of jeopardizing the future of data warehouses, but I think that we are far enough into the technology now to demonstrate that this approach is wrong, and we will experience a resurgence of interest in data warehousing."

Yet, not all experts say you need to build a data warehouse first. Data marts in themselves can be an effective strategy as long as there is a road map that indicates what different departments have developed and offers a means to access the data in a data mart, according to London consultancy Ovum Ltd. That kind of setup is known as federated data warehousing, where common metadata -- an accounting of what is where -- is available to people throughout a company.

New technology is designed to enable the federated model. Some data mart tool kits from vendors such as D2K, Inc., Informatica Corp. and Sagent Technology, Inc. offer good support for federated metadata, said David Wells, a principal consultant at Ovum. Also, Microsoft Corp.'s SQL Server 7 database, due by year's end, has built-in data mart building tools and metadata capabilities.