GOLDEN MEANS: TECHNOLOGY AND THE STATUS QUO
by Inderpal Bhandari, executive editor at large

In October 1997 my company, Virtual Gold, Inc., and I hosted a conference entitled "The Evolution of Data Mining: Technical Strategies to Beat your Competition by Year 2000". It was held at the Millenium Hilton in New York, a location aptly named for a Y2K event.

It was not only a good place but also a good time for such a conference. Most of us who had initiated projects that helped jump-start the field of data mining had seen our work come full circle. For instance, I had waited four long years to hear a coach from the National Basketball Association give whole-hearted, unqualified credit to data mining for a victory over a tough opponent.

That milestone was achieved in May 1997 (details can be found in my column of March 26). Sure, there would be more such credits from coaches that would follow, undoubtedly. But the fundamental issue of creating a data mining solution that was simple enough analytically for non-technical domain experts such as NBA coaches to use but yet sophisticated enough to find patterns that could truly surprise those experts, had been addressed.

The time to explore fresh ideas was clearly at hand. To present the evolution of data mining into the 21st century, we invited speakers who were creating technologies that were likely to play a role in giving shape to the evolution of data mining over the next three years.

In the coming weeks, I will detail those technologies and approaches in this column. I will also invite some of the speakers to guest write the column. Presently, I summarize the key messages from the conference below.

The success of the early data mining solutions has led to the realization that commercial efforts which package data mining technology as part of an over-all solution to a problem in a specific application area are more likely to succeed than those which offer data mining in the form of a generic platform. While solution-centric approaches will continue to be the name of the game in the near future, the messages from the conference suggest how approaches that are more system-centric could become powerful factors by year 2000.

1. Data mining for the masses. In theory, such an approach would package technology and methodology into a simplified interface that would allow non-technical users to interact meaningfully with data, thereby eliminating the present need to hide the complexity of the data mining process under the covers of a solution.
2. Data mining middle ware. Emerging areas such as electronic commerce, knowledge management, intelligent agents, and network security are going to become increasingly critical for businesses. Data mining technology can enhance the effectiveness of these areas. This suggests the development of modules representing those enhancements, to be deployed between the system and application layers of an implementation.
3. Data mining of massive data sets. With time, data warehouses will accumulate more and more data. Businesses will require the ability to analyze very large data sets. That, in turn, suggests the use of system-oriented methods such as parallel computing.

To quote Milton Berle, a committee is a group that keeps the minutes and loses hours. Given that I will be covering the above minutes of the conference in more detail in later columns, perhaps it is best to end here, lest the Miltons among us rise up in protest.

One final point. Businesses that foresee the evolution of data mining can gain a decisive competitive edge over those who must await the benefit of hindsight. Approaches such as those described above represent the cutting edge of the technology. They can change the status quo; hence, their importance.

Interested in foreseeing the evolution of data mining? Contact us at http://www.virtualgold.com