With the growth of internet research and its promise to deliver speed, it is often debated among market researchers if time of response and reduced field times result in biased data. To understand these issues and develop sampling best practices Zoomerang’s parent company, MarketTools, conducted a series of internal tests.
The results determined that differences do occur between day and evening respondents, as well as those who respond early versus late during the field period. However, the differences are not sharp enough to impact overall business decisions. When fielding internet-based research, it is still important to consider field length and how it may impact results and responses from various demographic groups.
With that in mind, we are making the following recommendations:
For research programs that require quick field periods of two days or less, specific attention should be given to quota management to allow for a balance among region, employment age and gender in order for fair representation for all cells that are pertinent to the research program
When fielding surveys among a younger population, we recommend an extended field in order to maximize response rates and to encourage balance among region, income and employment within each age group
When designing a test for absolute comparisons (i.e. A&U tracker), it would be wise to leave the survey in the field for longer periods of time to ensure adequate demographic representation occurs that balances out demographic biases occurring with early versus late responders. Our recommendation is to allow for at least three field days
When designing experiments such as a multi-cell concept tests where absolute comparisons are required, effort should be made to ensure early versus late responders are balanced by directing each new respondent to a rotating cell as they enter the survey location. The same consideration for concept rotation should be used for day versus evening respondents
Finally, when conducting programs of research where concept or product scores are compared in a database, standard field times and sample specs should be as closely matched as possible across projects
View the full MarketTools report to further explore how quick delivery of data and time of response may impact survey results and ultimately the business decisions that are made from them.