In person or online?

Short answer: in person.

Despite its quantitative angle, you will get the most out of a Kano study if you approach it as a qualitative study. The benefits of talking to your survey participants with the Kano survey as an underlying structure far outweigh the disadvantages.

If you cannot talk with survey participants, a remote survey (online or other) is the next best thing. But even then, do a live test of your survey first.

Eliminating confusion

Our experience has shown that standardized, oral interviews are the most suitable method for Kano surveys. A standardized questionnaire reduces the influence of the interviewer, the return rate is very high and, in case of comprehension difficulties, the interviewer can explain. Usually the questionnaire must be explained because of its new and unfamiliar nature. (Matzler & Hinterhuber 1998)

The survey format, its questions and its possible answers are all potential sources of confusion. You immediately feel if a participant is confused when you are in front of or on the line with them.

Participant confusion is the number one reason for a Kano survey's unreliability. You cannot feel, let alone eliminate, a participant's confusion if you are not talking with them.

Finding out why

When you're talking to a survey participant, you can immediately get a sense of why they are answering they way they do. You'll get a glimpse of the Job they are trying to get Done and whether they think the feature will help them achieve it.

Understanding the underlying reasons makes the analysis of your survey a lot more valuable. You won't have to guess why a feature is predominantly in the Attract category for example.

With a remote survey, a lot of that qualitative information is lost. All you'll know is what category a feature is in, but you'll still make assumptions as to why that is the case.

Discovering real segments

Understanding why people answer what they do also allows you to detect the real segments in your customer base.

Matzler et al. found for example that the significance of the edge grip of skis they were surveying customers about was dependent on the skill level of the customer.

Had they done their surveying remotely, they might not have thought about asking about the skiers skill levels. They would not have known that skill level was such a determining factor in how skiers perceived the quality of skis. Knowing this has major ramifications for product development, but also on marketing, product positioning and pricing.

I’ve done surveys where digital savviness was a major factor in how customers perceived a feature. Again, that this was relevant only became clear while I was doing the survey. I would not have thought about this up front. (Also: good luck getting honest answers when asking in an online form about the customer's online saviness). The product we were developing would have contained features less digital savvy users would have been nervous about had we done only remote surveying.

You don’t always know what factors will determine someone’s attitude to product features. Live interviews will help you uncover that information while remote surveys won't.

Bonding with participants

Don't underestimate the value of a live interview with a (potential) customer. Customers are happy to talk about their needs and relationship with you. I've yet to encounter a customer who's not happy to do a follow-up interview after the first one too.

Customer relations are deepened through actions such as these. An advantage of using the structure of a Kano survey for interviews is that there is a clear purpose and direction to the interview. Customers appreciate this: it shows that you have put time in preparing your talk with them and that makes them feel valued.

Cost per survey

The cost per survey is higher for a live interview than a remote survey. Yet the value per survey is higher by orders of magnitude. You also don't need to interview large numbers of people for a Kano survey to be reliable.

Unfortunately, although the validity of your findings is not proportionate to the number of participants, higher numbers seem more convincing. Reporting to the your boss that you are making a decision based on interviews with 20 people is more difficult than telling him you're basing your decisions on a 1.000 people survey (however flawed that survey may be).

If you're in this situation, a mix between live and online surveys is ideal. Use the live interviews to finetune your online survey and understand the reason why features get categorized the way they are. Then use the online survey to support your findings.

So are remote surveys bad?

Not at all. They are the next best thing to live surveys. The value you get from remote surveys is a lot higher than not doing a survey at all.

If you have prepared your survey well, its results will be a solid base for making decisions. As a rule of thumb, the less abstract and the more concrete the topic of your survey, the more suited it is for online administration.

Do try to reach out to some participants before and after the survey; it'll be worth it.