Challenges with ranking responses from focus group discussions
In qualitative research, responses and viewpoints are identified and reported regardless of how often they were mentioned and irrespective of the point in the discussion or interview process they were mentioned. In other words, both spontaneous and probed responses are important. Although there is scientific merit in attaching certain interpretations to responses that are given first or spontaneously, it ought to be borne in mind that some important points or insights often only emerge after encouragement and probing by the researcher.
Another important point here is that whereas determining “order of mention” is quite feasible in unstructured qualitative interviews with just one respondent (i.e. one-to-one depth interviews), it would be a methodological flaw to try and do so in focus group discussions. First, and by instruction, focus group participants speak one at a time. Therefore “first mention” in a group discussion is really only “first mention” by the participant who happened to speak first. It does not follow that a different participant speaking first would have said the same things and/or in the same order. Secondly, participants in focus groups tend not to repeat points already covered by others. It can therefore be misleading trying to infer the importance of certain ideas or viewpoints based just on the number of times something was said in a focus group (e.g. through a basic word count). Indeed, once a point has been made and noted, moderators tend to probe for additional, or even opposing viewpoints, and not for confirmation of points already made.
In addition to the above within-group challenges, researchers often conduct the analysis across several focus groups and there is usually no uniformity across the groups in relation to what is mentioned first or what is mentioned the most. The analysis therefore focuses on identifying ideas that came up within and across the groups or ideas that came up in some groups but not in others, but without attaching too much importance on identifying the order and frequency of the opinions. That is the function of quantitative surveys.
There are several dynamics at play in focus groups that have biasing effects too. One such bias is dominant characters who tend to speak first and/or the most. There is thus a danger of ending up taking their individual opinions and projecting them as overall group opinions. That is why, where some form of ranking or quantification is required, some structured response forms are handed out for self-completion by the individual participants, or the facilitator asks for a show of hands and performs a count.
Ultimately, the real value of qualitative research lies in identifying and listing all the relevant factors in relation to the phenomenon being investigated. The importance and/or prevalence of those factors within the target audience can then be measured and reported more definitively through a follow-up quantitative survey.