DATA WAREHOUSING SYSTEM ADMINISTRATION, A BUSINESS-CRITICAL SYSTEM PART I
by Nigel Chang

Introduction

According to a report from International Data Corporation (IDC), 95 per cent of Fortune 500 companies had built their own data warehouse by 1995. There is no doubt that many MIS professionals are now familiar with the design, implementation, and maintenance of data warehouses. However, many MIS professionals still hesitate to call data warehousing a mission-critical system. While it is common sense that online transaction processing (OLTP), the business operating system (BOS), for example, is definitely a mission-critical system, it is not so clear that the data warehouse is also qualified as a business-critical system.

With multiple years of data warehousing experience, I can explain the reason I acknowledge it as a mission-critical system. In a recent MetaEdge white paper, we called OLTP type BOS a system that "runs the business" and the data warehousing type of warehouse operating system (WHOS) a system that "guides the business". An outstanding next century enterprise must be equipped with both the WHOS and the BOS.

An OLTP system typically supports many users and small queries, while a data warehouse system usually supports few decision makers and heavy queries. Can you ask your decision-makers to tolerate system downtime? Do you have enough system resources to catch up on killed heavy queries? OLTP operating systems are the data source of data warehouse operating systems. Warehouse failure may result in system resource overrun on your OLTP. I simply have no reason to conclude that a data warehouse is not a mission-critical system.

The data warehouse is not only a critical system in business, but it is also a difficult system to maintain. It takes more than UNIX systems administration knowledge and experience to make data warehouse systems administration an easy job.

Commonly, enterprise data warehouses are loaded with several hundred gigabytes of data or even multiterabytes of data. This results in your systems administrator being forced to keep up with data warehouse system resources, such as the CPU, memory, network, and the tape that is able to serve the data warehouse with a huge data volume and continually running jobs. Suddenly, your small town sheriff is in charge of NYPD. It is possible that he will do just fine, if he is well prepared.

This article emphasizes the critical administrative tasks that make a data warehouse successful. Minor tasks, such as printing, user accounting, terminal installation, mail, uucp, nfs, and news, are not points of interest in the data warehouse environment.

UNIX Expertise

It is often the case that MIS departments try to transform their own mainframe administrators into open systems administrators. While this may take care of some personnel issues, the greatly needed UNIX expertise is not addressed. Yes, these administrators are trained either before or during the data warehouse development phase. However, the experience wasn't built before the production date. Many data warehouse projects have suffered from this lack of experience from the very start and were never able to get stabilized, even years after the production date.

As we often see, this type of systems administrator usually runs the UNIX system like a mainframe system and overlooks the major differences between the two. Understandably, these systems administrators are not always as energetic as new graduates and often have a lengthier learning curve. Resolving personnel issues through transforming mainframe administrators into open systems administrators rather than injecting the project with the fresh energy that many new graduates possess certainly has its downside.

Data warehouse management teams are commonly forced to face this problem. We do not intend to discourage such internal transfers and make ourselves an MIS enemy. Rather, we want to warn data warehouse management teams of the problem, so that they can develop a solution for it at an earlier stage of the project. One solution would be to hire UNIX top guns externally; another solution would be to train the internal transfers much earlier. It doesn't look healthy when both your applications and DBA teams are far more experienced than your systems administration team. A management team also has to realize that mainframe experience does not automatically translate into UNIX experience.
---
Part II of this article will appear in the next edition of D S * .
---
For more information, see http://www.metaedge.com