Test your survey

Never just start a survey without testing it first. Especially with online surveys you have limited opportunity for providing context and eliminate confusion.

Who to test with

Dry-run your questionnaire with a few people before you finalise it. Don't test your survey with random people. Use testers who are similar to your final group of participants.

You don't need a large number of testers; from three people on you'll begin to what major issues your survey has, if any.

What to test

The introduction

Check whether people understand the context of the survey. You want them to have the same mental construct as you. If your questions are about the different aspects of ordering movie tickets online, check whether they are seeing your questions in that hierarchical context. You don't want to be framing your questions in another context (like the movie theater programming, its location, ...). Define the context better if people see your questions in a broader (e.g. movie going in general) or stricter (e.g. only the online payment aspect) construct.

Not everyone immediately understands the format of a Kano survey. Test your explanation of the format. Check whether people understand the meaning of the different answers and that the answers are not on a scale. When you've found a good way of explaining how a Kano survey works, use that as an introduction to your final (online) survey.

The questions

Ask your testers to think aloud while completing your survey. Ask them to repeat the questions in their own words. You want to make sure they understand what you're asking, and that they understand the questions the way you intended.

You'll also get a feel of whether your survey is too long. You shouldn't ask more than 20 questions per survey, but even that could be too much, depending on the context of the survey or the complexity of the questions.

If you're doing an online survey, consider adding “Other” to the possible answers. Then follow-up with testers who did not complete all questions to find out why.