Creating great online surveys

Creating online surveys can be a great way to create some quick thought capital for your business. But if you're going to create a survey, it takes a little more preparation than just writing some questions. Here are some thoughts on how you should

1. Clearly define the purpose of the survey.
Fuzzy goals lead to fuzzy results and the last thing you want to end up with is a set of results that provide no value. So spend time up front to identifying:

(a) What is the goal of this survey? What do you hope to accomplish with this survey?

(b) How will you use the data you are collecting? What decisions do you hope to impact with the results of this survey? (This will later help you identify what data you need to collect in order to make these decisions)

It sounds obvious, but we have seen plenty of surveys where a few minutes of planning could have made the difference between receiving quality responses or uninterpretable data.

Consider the case of the accountancy firm that wanted to improve their website and find out was most important to its clients. Their survey asked ‘How can we improve our website?’ The resulting answers ranged from ‘Make it cleaner' to ‘Add an update button on the recruiting page’. While interesting information this data is not really helpful for the partner who wanted to take an itemised list for the development team with client input driving prioritisation.

Spending time identifying the objective might have helped the them determine:

1) are we trying to understand our clients' perception of our website in order to identify areas of improvement e.g. hard to use, time consuming, unreliable; or

2) are we trying to understand the value of specific enhancements. They would have been better off asking customers to please rank from 1 – 5 the importance of adding X new functionality.

Upfront planning helps ensure that the survey asks the right questions to meet the objective and generate useful data.


2. Keep the survey short and focused
Short and focused helps with both quality and quantity of response. It is generally better to focus on a single objective than try to create a master survey that covers multiple objectives. Shorter surveys generally have higher response rates among survey takers. It’s human nature to want things to be quick and easy – once a survey taker loses interest they simply abandon the task – leaving you to determine how to interpret that partial data set (or whether to use it all).

So make sure each of your questions is focused on helping to meet your stated objective. Don’t toss in ‘nice to have’ questions that don’t directly provide data to help you meet your objectives.

To be certain that the survey is short; time a few people taking the survey. Gallup research has shown that online surveys should take 5 minutes or less to complete. 6 – 10 minutes is acceptable but there are significant abandonment rates after 11 minutes.


3. Keep the questions simple
Make sure your questions get to the point and avoid the use of jargon. We have often received surveys with questions along the lines of. “When was the last time you used our RGS?” (What’s RGS?)

Don’t assume that your survey takers are as comfortable with your acronyms as you are. Try to make your questions as specific and direct as possible. Compare: "What has your experience been working with our litigation team?" to "how satisfied are you with the response time of our litigation partners?"


4. Use closed ended questions whenever possible
Closed ended questions can take the form of yes/no, multiple choice or rating scale and by giving survey-takers specific choices. Open-ended question allow people to answer a question in their own words and are great supplemental questions, providing useful qualitative insights. However, for collating and analysis purposes, close-ended questions are the better choice.


5. Keep rating scale questions consistent through the survey
Rating scales are a great way to measure and compare sets of variables. If you elect to use rating scales (e.g. from 1 – 5) keep it consistent throughout the survey. Use the same number of points on the scale and make sure meanings of high and low stay consistent throughout the survey. Also, use an odd number in your rating scale to make data analysis easier. Switching your rating scales around will confuse survey takers, which will lead to untrustworthy responses.


6. Logical ordering
Make sure your survey flows in a logical order. So begin with a brief introduction that motivates survey takers to complete the survey (e.g. “Help us improve our service to you. Please answer the following short survey.”). Then start with broader-based questions and move to those narrower in scope.

It is usually better to collect demographic data and ask any sensitive questions at the end (unless you are using this information to screen out survey participants). And if you are asking for contact information, place that information last.


7. Pre-test your survey
Make sure you pre-test your survey with a few members of your target audience and/or co-workers to find glitches and unexpected question interpretations.


8. Consider your audience when sending survey invitations
Recent statistics show the highest open and click rates take place on Monday, Friday and Sunday. In addition, our research shows that the quality of survey responses does not vary from weekday to weekend. That being said, it is most important to consider your audience. For instance, for employee surveys, you should send during the business week and at a time that is suitable for your business. i.e. if you are a sales driven business avoid sending to employees at month end when they are trying to close business.


9. Consider sending several reminders
While not appropriate for all surveys, sending out reminders to those who haven’t previously responded can often provide a significant boost in response rates.


10. Consider offering an incentive
Depending upon the type of survey and survey audience, offering an incentive is usually very effective in improving response rates. People like the idea of getting something for their time and research shows that (appropriately-sized) incentives can typically boost response rates by about 50%.


So, with these thoughts in mind, speak to your marketing & web teams about how you might best use an online survey to develop some thought capital and build your own reputation.


the Heresiarch
+ + +