The collection of data through surveys is a critical component of most market and social research projects. This is because even where secondary or archival data is available, it often needs to be analysed and triangulated against some primary data collected from a representative sample. As research consultants we therefore create and administer surveys, mostly online, on a regular basis.
Each time we create an online survey, a key consideration is whether we will get a good response rate. Given the traditionally low response rates to online surveys in general, you always hope that everyone who starts the survey finishes it. Survey dropouts do not only leave you with an insufficient sample but with potentially biased results too as those who dropout may be unique members of your target population.
There are several factors that influence the survey completion rate, primary of which is survey length. Apart from surveys that are too lengthy, other factors that have been shown to drive non-completion is having too many open-ended questions and making answering the questions mandatory.
We recently scripted a questionnaire that was pre-designed by a client into an online survey. In the process, we noticed that, apart from a couple of standard and easy to answer demographic questions, the substantive section of the survey started off with an open-ended question which was also mandatory to answer. Having discussed the potential negative effect of this question on survey completion, we decided to proceed without any amendments while we monitored the response trends. We subsequently noticed that a mere 53% of those who were starting the survey were completing it fully. Moreover, almost all the dropouts were occurring at the point where respondents had to answer the mandatory open-ended question referred to above. We also noticed that although there were many more survey questions after compared to before this specific question, those who went past this question tended to complete the remainder of the survey. As such, survey length did not appear to be the main impediment in this case. We thus duly removed the mandatory answer instruction from the question and although the completion rate was still not ideal, it did improve significantly by 10 percentage points to 63%.
As the popular Survey Monkey online survey platform advises, surveys are a form of conversation and should, among other things, start with a “friendly, easy to answer question”. As we have also highlighted in some previous articles, one of the key questionnaire design principles is to minimise the burden on respondents. Our observation from this survey lends further support to this proposition.
