GOLDEN MEANS: DATA MINING AND THE FIRST FOUR HUNDRED PEOPLE IN THE BOSTON PHONE BOOK
by Inderpal Bhandari, executive editor at large

I remember coming across a Meta Group study that reported that the percentage of data warehouses with 500+ users was increasing rapidly. Juxtaposed with that report was an article about a major bank that was investing 38 million dollars in its data warehouse by setting up a 50-person staff for analysis. The implication: A large number of employees use data. However, only a small fraction of employees actually analyze data.

I asked a senior executive of a major petroleum company what he thought about that trend. Specifically, I wondered whether he was concerned about the ROI of data warehousing, given that only a small fraction of the people were really learning from data. He was not. "The way I see it, data analysis will not appeal to the vast majority of employees. Why should they add to their task? Yes, there will be a small minority that will go the extra mile but these are the guys that will find a way to do this stuff no matter what. The rest of us will benefit by using the results."

The CIO of a major bank did not want the bulk of the employees doing analysis either. "This stuff is not for everyone, nor should it be done by everyone. You have to understand analysis to do it right. I would rather leave it to our trained statisticians. The rest of us should use the results."

And yet my experience leads me to believe that there is something amiss with this picture. There have been two data mining projects that I have worked on that fundamentally changed the way things were done for the better. Both projects involved the frontline employees directly in analysis. For example, in the IBM Advanced Scout data mining for the NBA project, the coach mined game data. In the other project, software developers mined data of bugs to uncover hidden problems that they had missed. Both projects fundamentally changed the core products of those businesses.

The other data mining projects that I have participated in, while successful, were limited in scope to improving a particular aspect of the business but did not impact the core products of a business. In these projects, the frontline employees were users of data mining results but not directly engaged in analyzing the data. For example, the operators in many customer call centers today are alerted to cross-selling opportunities. The operators are the frontline of the call center. They do not analyze data and discover the rules that define cross-selling opportunities. They are merely users of those rules.

I am not sure whether empowering frontline employees with analytical capabilities was the key to producing fundamental impact. It could certainly have been the nature of the data that was being mined in those projects (as opposed to who was doing the mining). Be that as it may, I can tell you that there are definite advantages to be had if you can figure out a way to get your frontline employees involved in learning from data.

First, your employees will develop a mission of self-improvement. They will come to work to learn, not merely to perform their duties. Second, they will be more amenable to think out of the box. Of course, that encourages a certain amount of risk taking which has to be managed appropriately. But it will energize your workforce. Third, you will teach employees to cope with large amounts of data. In today's world, nobody in an organization, from the security guard to the CEO, can escape being inundated with data. This situation is only going to become more and more data intensive with time. You will prepare your workforce for that situation. And, finally, there is William F. Buckley, Jr., who said, "I'd rather entrust the government of the United States to the first four hundred people listed in the Boston telephone directory than to the faculty of Harvard University."

In summary, we need to figure out ways to get mainstream employees more involved in learning from data. It can unleash a chain of positive events that can fundamentally change your business for the better.
---
Inderpal Bhandari can be reached via http://www.virtualgold.com