REPORT ON THE DCI DATA WAREHOUSE WORLD CONFERENCE,
NEW YORK, JULY 28-30
by Aaron Zornes, META Group
The META Group/DCI 1998 Data Warehouse World conference in New York City drew 6,500+ attendees:
The audience reflected the diverse nature of applications targeted for data warehouse-style solutions. 35% percent of attendees were executive or business management and 45%+ IT professionals (consistent with DW Worlds held 2H97-1Q98); 62% are in corporate IT groups; 38% in lines-of-business IT group. 200+ attendees completed the on-line survey conducted by Market Perspectives Inc. on behalf of META Group and First Albany Corp. Key issues/trends addressed by the conference include data obtained -- the surveyed DW projects at this conference overlayed against 3,300 data points from prior 2 years of conference survey data:
Interim results from META Group's "Data Warehouse Corporate Scorecard" study was released which dispels the myth of 70% failure rate for DW projects.
The preliminary study results found 75% of surveyed companies report DW projects obtain some degree of success in that 30% "meet or exceed expectations", 50% "meet expectations to some degree" (but may require additional work), and 20% are considered "abject failures." Of approximately 20% of DW projects that are halted as "abject failures" almost all are restarted due to the critical or strategic nature of the business value they provide.
According to META Group, reports of DW failures are based primarily on analysis of large, multi-subject area projects with massive scope: inherently risky projects requiring strict adherence to an architected process. Virtually any large IT project that is embarked upon with multiple lines of business, wide scope, and new technology won't meet user expectations and could be considered failures. The 30% indicating that projects "meet or exceed expectations" are not just the survivors, but the leaders in their respective industries.
Additionally, Bill Inmon, Claudia Imhoff and other leading thinkers provided similar experiential data which refuted the recent spate of negative DW press by Gartner Group, Price-Waterhouse and Forbes.
Overall the market for data warehouse (DW) software, hardware and services will continue to grow at 40% annual compound rate through 1998: from US$2.8B in 1996 to US$8B in 1998 (First Albany Corp./META Group estimates).
From an architectural standpoint, we are seeing continued rapid growth in data marts: growing from 51% of all DW architectures to 61% during 1998/99 (vs. 24% of all DWs during 1996). Operational data store is holding steady at 23-24% while centralized/enterprise DW is declining from 43% of current to 19% planned. Moreover, personal/mobile OLAP cubes will grow from 8% to over 21% of DWs planned.
The number of organizations building 1TB+ DWs will increase from 7% to 17% during 1998/99.
Year 2000 (Y2K) Impact: Although, these percentages previously held steady now for the past 12 months, we are now seeing confirmation that impact is occurring and is accelerating (17% additional in past 3-6 months have seen impact of some sort; with additional 10% seeing staff reductions and additional 8% seeing capital budget reductions). We still believe there will a major fire drill during 1998/99 which will drain significant COBOL and database talent off all IT projects to deal with the pending Y2K triage efforts.
Prevalent Themes/Announcements: "Data Marts," "Customer Centricity," "Knowledge Management," "Web-Enablement" were endemic themes at this conference. Announcements made at this conference included:
There were ongoing guru/consultant skirmishes over the religious issue of architecture: Bill Inmon promoted his relatively new concept of exploration DWs and "near-line" DWs. META Group's John Singer promoted the pragmatic "information logistics" view: i.e., back off to 50,000 feet and ignore formulaic solutions; while making a conscious effort to map appropriate mix and match of components to your business.
Data mart applications for application packages: Numerous vendors and SIs were promoting pre-packaged data mart applications for SAP R/3, PeopleSoft, et al; notable were Information Builders and Informatica.
There was a significant increased emphasis on becoming more "real-time" in data fresh rates for DWs (centralized/enterprise DW vs. line of business data mart)
Vendor announcements at this conference included:
Oracle Financial Analyzer 6.2 and Sales Analyzer 6.2
Broadbase 1.3
Enterprise Performance Management (EPM) solutions to measure, analyze,
and optimize business performance and effectiveness
Enhanced Web-based delivery to easily distribute information and analysis
Seamless integration to Customer Relationship Management (CRM): Clairfy and Vantive: via packaged interfaces or "adapters"
Broadbase CRMperform: Analytic application to optimize customer relationship management (CRM)
Carleton's Enterprise/Integrator: now includes i.d.centric's name/address processing technology from Firstlogic
Decisionism 2.1 Aclue generates Express and Essbase cubes and includes data mart prototyping facilities
Hummingbird Communications: Renamed GQL query and reporting tool to BI/Query and added BI/Broker and BI/Web tools to the suite; BI/Web provides thin client web-based query and reporting vs. BI/Query's enterprise reporting features; BI/Broker provides a repository for reports, data, attached files plus distribution, security and scheduling services
i.d. centric: MERGE/PURGE parallel option IBM: relicensed Exchange Applications Informatica and Trillium Software: joined to integrate data cleansing at API and metadata level
NCR: Exchange Applications partnership
Red Brick: Paragren partnership
SAS Warehouse Administrator 1.3: a new administration module to its DW
software for single point of control for multi-dimensional and relational
data
Center for DW Excellence: Concurrent with DW World, DCI hosted for the third time its "Center for DW Excellence" with chairperson Claudia Imhoff. This parallel conference addressed the more advanced technical issues via 2-4 hour workshops and was well attended by approximately 250 attendees.
Future: Trends outlined for 1998/99 include a new emphasis on DW lifecycle maintenance and garbage collection such as identification and archival of little accessed elements. Also projected is an increased convergence of DW, data mining, database marketing, and knowledge management: i.e., data mining is an overlay of the DW infrastructure and database marketing.
Bottom Line: For production DW, IT must budget the appropriate hardware
and software dollars to support 150 GB+ data stores, with UNIX and NT being
the predominant targets. Decision support workbenches should include both
multi-dimensional-oriented tools (OLAP) for complex dimensional analysis as
well as traditional tools with standard SQL access (managed query
environments). IT organizations must also begin planning for both
Web-enablement and data mining technology initiatives for deployment 6-12
months out. Vendors are still scrapping to produce better metadata
integration, web-enablement while data mining is backing off from discrete
technology to more of an embedded capability. Hot demand continues for
application package-specific data marts. User DW emphasis is being put on
business solutions such as customer service, database marketing, supply
chain logistics, and quality management. IT focus continues to shift
towards data marts over centralized DWs with weekly/daily refresh more
common than monthly. While DW investments are not (currently) at risk due
to Y2K, savvy IT managers should rightfully position DW initiatives as
complementary to Y2K initiatives to avoid budget and staffing reductions.
---
For more information, see http://www.metagroup.com