People use a product for many different reasons. Sometimes people will pick up a product once and never return. Trying to understand why people do what they do and react is a difficult challenge. User interviews are a common and trusted approach to gain user insights, but there are a few downfalls that I think require discussing.
People tend to bend the truth and tell you what you want to hear
User interviews inherently introduce bias; where social norms impact how people respond. People subconsciously want to be pleasant and make you happy, shading their answers to surveys, focus groups and interviews. People may answer your questions with what you want to hear, not because they want to lie to you, but because this is what a lifetime of social interaction has trained them to do. This type of response could lead to failure because people may not take the action they said they would, they just didn’t want to hurt your feelings.
Users' accounts of what they remember doing can be very different from what they actually did
It is also very difficult to quantify what happens while your product is being used. During interviews, you are relying on users to provide feedback on their experience. The problem is that what they think happened, and what actually happened can be two completely different things. This reality can be extremely dangerous as the feedback may lead you to change an aspect of your product that isn’t the cause of concern. #### People tend to rationalize their behaviour
People tend to rationalize their behaviour and come up with reasons or excuses for their actions. This feedback could give you an assortment of problems, and how do you know which are true? For example, you could be noticing that you are getting a lot of people interested in your product but you do not see a correspondingly high sign up rate. When you conduct a user interview and ask why they didn’t sign up, you could get a variety of answers, such as:
- I didn’t see a button on the page, maybe if you put it in a different location I would have
- I didn’t know what it did
- Maybe if the colour was different
- The button was too small
- The process was too difficult
- I didn’t know what you were asking me to do
How do you increase sign ups if you get a variety of reasons to why someone didn’t complete the action? In this scenario, you have no solid data to show the true cause.
How do you get a wide range of users to participate and provide feedback?
One of the most challenging parts of conducting user interviews is getting a broad spectrum of users willing to give you feedback. You want to know what you are doing well and also what needs improvement. You want to understand why people love your product but also why people do not. With all the avenues trying to capture a person’s time, it can be hard to get users to provide feedback. In order to increase the amount of participation, feedback must be easy to provide.
Gain insight from all users by A/B Testing
User interviews still defiantly have a place in researching your market and trying to grow your business. Understanding the dangers of user interviews allows you to see the benefits of A/B Testing. A/B Testing allows you to gain insight and track user interactions as they are happening. You will be able to see at what point a user turned away from your product and what actions they took up to that point. You can then create experiments to modify your product and constantly create a better user experience.
Let's revisit the problem of having good traffic to your product with low user sign ups. Based on the user feedback, you were given reasons as to why a user thinks they didn’t sign up, but no concrete evidence on the true reason. With A/B testing you can test different variations of the sign up process across your entire user base and determine which variation is the most successful. Instead of relying on the user to provide you with feedback you are instantly receiving feedback from all your users as they use your product. The data will then show which sign up variation resulted in the most signups.
In order to truly understand your user base and have a better chance at answering the big question of what does the user want, you need to look further. A/B Testing is a proven way to make that happen and the best app developers use this tool to constantly iterate and test their way to success.