PUTTING DATA MINING TO WORK: THE SEQUEL, PART II
by Michael J.A. Berry

What the Marketing Data Revealed

The marketing data, while considerably more mining-ready than the call detail data, also required some preparation. Fields with unique values such as account and mobile number were dropped. New fields representing various historical ratios, length of tenure, and aggregate counts of promotions and features were added.

Once the data was ready, it was fed into a data mining product, Information Harvester, which generates models which are collections of rules based on combinations of the input variables. Information Harvester also generates information on which individual variables contribute most to a model.

Although the details are confidential, the models did a very good job both at predicting voluntary cancellations and at suggesting which factors were contributing to customer dissatisfaction. Some of these factors begged further investigation. For instance, the town that a subscriber lived in turned out to have a large impact on his or her loyalty. What does that reveal about the sales channels, demographics or service conditions in those towns?

The data mining effort involved segmenting the population in various ways. We achieved the best results when the population was segmented by length of tenure and separate models built for new customers, medium term customers and long-term customers. The most accurate predictions were for customers with shorter tenure.

What Actions Were Taken?

Predictive modeling is useless if it does not lead to action. In this case the desired action was to generate a list of customers at risk and perform some intervention in order to retain them. Here the project ran into a little trouble. For one thing, the customer contact information contained in the marketing data turned out to be surprisingly inaccurate. About 25% of the phone numbers were wrong or missing. An even bigger problem was that although the project included a budget for outbound telemarketing calls to the people identified as likely to cancel, there was neither budget nor authorization to make any particular offers to the people we called. Furthermore, we lacked the ability to switch a dissatisfied customer with whom we had established contact directly over to the customer service group at the cellular phone company.

In the end, the outbound telemarketing company with which we worked called people on the list, asked them a series of questions and offered to refer and problems reported to customer service.

Measuring the Results

Of course, one cannot measure the results of any retention effort right away. If you make the effort one week and check the next, very few of the targeted population will have defected, but then very few of the control group will have left either. Two months after the outbound calls were performed, results are encouraging. Even though no actual interventions were performed during the calls, the targeted population is showing better retention than the control group. This may be because any contact at all increases customer loyalty or it may be because problems that surfaced during the interviews were referred to customer service and resolved. The data now being collected will make it possible to build a response model in order to predict which customers respond well to being called.

The Need for More Data Mining

The first round of data mining was a success. We were able to build models that defined segments of the customer population with the greatest likelihood of canceling their subscriptions. To be really useful, this data must be combined with information on which of these potential churners are most likely to respond to which offers.

Back to the mines!