IF YOU ASK THE RIGHT QUESTIONS, YOU CAN GET THE RIGHT ANSWERS.
by Ed Colet, Virtual Gold Inc.

It may seem trite to note that when one asks a question, one receives an answer. However, the way a question is asked can strongly influence the answer, and the nature of this relationship can be quite interesting. In inter-personal communications people at times may not be aware of this effect, while at other times, it is explicitly acknowledged. But this effect is also present when technology (rather than another person) is handling the questions and answers.

Research scientists studying eyewitness testimony have demonstrated that people may not be aware of the influence that a question can have on an answer. An example of one study uses the following paradigm. Participants were asked to view videotape of an event, and then answer questions about it afterwards. A portion of the videotape shows a car driving away during the scene. One of the questions presented to the participants asks them to estimate the speed with which the car drove away. It was shown that the way the question was framed influenced the participant's judgments. Speed judgments were higher when asked "How fast do you think the car was going after it sped away from the curb?" than if the subjects were asked "How fast do you think the car was going after it pulled away from the curb?"

It was also interesting to note that the participants denied that they felt their judgments were influenced by the question at all. Lawyers and politicians have been well aware of the fact that the form of a question can influence the answer. If questions are being asked in a way that disadvantages the client an astute lawyer may object on the grounds that the question is "leading the witness", or on some other legal basis.

Politicians have also been well aware that questions influence answers.

We have often seen that regardless of the tough questions put forth by an aggressive reporter, an experienced politician may always provide the same answer. In this case, the politician is absolutely and explicitly refusing to let a question influence an obviously prepared answer.

When it comes to questions and answers handled via technology rather than via person-to-person, the notion that answers can also be influenced by the nature of the question (query) seems to have been overlooked. Perhaps we think that the automated nature of database querying leaves no room for biases and subtle influences to operate upon the retrieved answers. This view can be dangerously misleading.

To illustrate, one can issue a database query (the question), and the analytical software will retrieve a result from the database(s). The end-user/analyst may think the right question was asked, and the resulting answer based on the query may be "technically" true, and conclusions are drawn. However analyzing data via this type of interactive query and retrieval process can be short sighted. For example, in Virtual Gold's SoccerMiner (an application to analyze soccer games, built using the patent-pending VirtualMiner(framework), one can issue a query about various aspects of players performances and look at the percentage of scoring opportunities created vs. not created. The data from one of Brazil's games reports that when the Brazilian goalkeeper is involved in a possession, a scoring opportunity is created 26% of the time (quite high for a goalkeeper!). At this point, one might conclude that Brazil's offensive prowess extends through to their goalkeeper as well. But Virtual Gold's patent-pending technology included in SoccerMiner reveals that if other questions (involving other players, and considering score differential) were considered, then a more meaningful and valid result would be discovered and one would not have been mislead. In other words, one needs to know how to ask the right question. The difficult part is knowing what the other relevant variables are - and this is not easy. But rather than leaving the user analyst to struggle with ways to formulate the right question, Virtual Gold's patent-pending technology in SoccerMiner is used to identify the relevant question.

This approach of integrating technology that helps one identify the right questions with database querying results in a powerful and accurate way of exploring and understanding one's data, with minimal risk of drawing erroneous conclusions. Ultimately, being able to ask the right questions can provide you with the right answers.

---

Ed Colet is the Acting Director of Research at Virtual Gold Inc., responsible for developing analytical methods for data mining and for investigating human factors and usability issues of business intelligence systems. At present, he is in the final stage of completing a doctoral dissertation in the Cognition and Perception program at New York University's Department of Psychology. Ed has also worked for IBM Research at the T.J. Watson Research Center. At IBM, Ed was a member of the group that developed Advanced Scout, the data mining application for NBA teams. His research interests focus on statistical methods and human factors.

For more information, see http://www.virtualgold.com.