DATA MINING & PREDICTIVE SERVICES: BREAKING UP THE MONOLITH
by John K. Thompson, Vice President - Marketing, Magnify Inc.

Up to this point in the evolution of the data-mining marketplace, there has been only one method of implementing predictive models. That is not to say that there is only one type of system, or that there is a lack of system choices to utilize, but in fact, the vast majority of systems are implemented in the same manner. Enterprise in scale or desktop in operation, data mining systems work in similar fashion.

For the most part, data mining systems are implemented in a specific location, on a specific platform. The locations and platforms have changed over the last few years, and system options have expanded as well, but the fact remains that once the data mining system is implemented on a computer system, in a specific location, all of the data mining model development, testing, tuning, predictive modeling and scoring occurs at that location, on that machine. To express the point in another way, the data sources, the data mining system and the models that are produced are inextricably intertwined. This, of course, has been acceptable because, well, all data mining systems performed this way.

As innovations have come into the market and new ideas have been tried and tested, the concept of building predictive models that can operate outside their native development environment has been advanced. The divergence can be thought of as two different systems originating from a common legacy. Those two systems are being referred to as traditional data mining and predictive services.

Data Mining is best performed on high performance platforms that provide access to significant amounts of data. They are typically used by professionals that are adept in advanced analytical methodologies, with the end goal being the development of highly accurate predictive models.

Predictive Services systems, on the other hand, are of most value to professionals who are close to the execution of the transactions that drive businesses (i.e. customer service representatives), and to the professionals who set strategy and policy for businesses (business analysts and Line of Business executives). Predictive Services systems need to be lightweight and able to integrate into existing systems and process streams.

Consider this -- If the knowledge derived and built into a predictive model during model development (traditional data mining) can be encapsulated in a form that preserves the input data mappings and output integrity, and thus, becomes portable, then the insights gained are no longer intimately tied to the system and platform on which it was implemented. Such insights can now be transported and implemented in another environment (predictive services) and can be optimized for processing speed in an operational setting.

Predictive services appear to be the next wave in data mining, because they enable more people to make use of derived knowledge.

In my next article, I will expand upon the background and value of functional decomposition, a phenomenon which is occurring rapidly throughout the data mining discipline.

---

John Thompson, Vice President -- Marketing, Magnify, Inc. You can reach me at jkt@magnify.com