WHAT IS THE PURPOSE OF DATA MINING?
by John K. Thompson, Senior Director of Marketing, Magnify

I don't amaze easily. Penn and Teller amuse me, and I can appreciate the run-up in Apple stock. But am I amazed by either? No. What does amaze me, however, is how many people simply don't get what should be the clearly obvious premise behind data mining.

Data mining enables people to discover unanticipated information on an unaided basis. This is information they can act on to better understand, selectively market to and retain their best customers, or sharply cut consumer fraud.

But I've seen too many people engaged in descriptive analyses of the information contained in their companies' databases, and call it data mining.

I've seen too many cases where others decide they can reach into the gigabytes and terabytes of data they have stored, core out a sample, and assume they are getting actionable and accurate results. Neither case, however, can really be called data mining, at least not as it's evolving in the marketplace. Data digging, maybe, but definitely not data mining.

Companies which sample their databases, and assume they're getting a precise picture of customer preferences and patterns, run the risk of missing the very patterns they're trying to find. Even worse, if they do find the patterns, they're only to find a portion of them. And that can lead to misinterpretation of the data, which leads to business decisions they're likely to ultimately regret.

They'll have misunderstood the data and misunderstood the people represented by them.

I strongly believe that true data mining means finding and implementing the powerful tools to search your databases, in their entirety, for the very patterns you didn't know where there -- the patterns you didn't even think of asking about.

Don't get me wrong. ROLAP and OLAP have their place in decision support. But they are only precursors to where data mining can take companies. We owe it to ourselves and the companies we work for to recognize the difference. The real power of true data mining has the capacity to amaze the most skeptical executive. The number of people who perform something with far less capacity, and who mistakenly call it data mining, amazes me.