“No great marketing decisions
have ever been made on quantitative data.”
John Scully
Former PepsiCo president, former Apple CEO
The Tree of Knowledge
Marketers commonly use decision trees to assess features and benefits to determine what is most important to consumers. Quantitative results can be obtained by asking respondents a sequence of very specific questions that branch out using if/then methodology.
Unreasoned Response
In a focus group years ago, an outspoken man was asserting himself by speaking out of turn, disparaging the process, and scoffing at the premise that brand had any bearing on his buying decision, ultimately proclaiming, “I’m just here for the money.”
“Control” Group
Experienced focus group moderators realize if unaddressed, dominant individuals can establish control, affect the group and ultimately hinder true and useful input. The deft moderator began to ask a series of if/then comparative questions that challenged the man to reconsider his inherent assumptions. In essence, the moderator drove him through a decision-making process to help him formulate reasoned positions.
Once back on topic the naysayer became the moderator’s most vigilant and attentive advocate – offering considered and definitive feedback. The rest of the group followed suit.
“The only relevant test of the
validity of a hypothesis is comparison
of prediction with experience.”
Milton Friedman
Nobel Prize-winning economist
Overrated Ratings
Similar principles apply to common online qualitative tools such as the five-star, numerical value, or Likert scales used to value or measure a respondent’s level of agreement with a given statement. Although quick and simple for respondents to complete, unlike decision trees, these methods ask subjects to value an attribute or preference without any measure of comparison, which lacks objectivity and is prone to positive or negative bias when respondents rank nearly everything of high (or low) importance.
Minimize Error
In What Do Customers Really Want on the Harvard Business Review site, Eric Almquist and Jason Lee explore Maximum Difference scaling. An extension of the Method of Paired Comparisons where subjects select a preference from two choices, MaxDiff asks respondents to identify their highest and lowest preference from a subset of attributes or statements. Multiple subsets are tested as part of a series. Almquist, a partner at Bain & Company, talks through one MaxDiff study on the relative importance of restaurant attributes in this presentation.
Asking respondents to rate selections is helpful and informative, but requiring them to decide between selections forces them to weigh answers. It inspires considered input, and generates more defined, useful and valuable feedback while eliminating undecided responses and mitigating positive and negative bias.
Maximize Outcome
Qualitative research adds relevance and validity to quantitative findings. In brand marketing research, consider your premise and process carefully from the outset to limit risk and maximize return. Remember, research often drives strategy, strategy drives spending, and spending drives outcomes – both good and bad.
Let well-considered decision trees help you branch out in the right direction.
image: pkeyn
