Introducing A/B testing to your company may sound scary, but once you shift your culture to one that is data driven you won’t know how you lived without A/B testing.
For most people deciding what needs to change in a mobile application is rarely driven by real data. You have your design team that creates beautiful screen layouts and developers who know how to make these designs a reality. Your typical new project launch meeting involves introducing a new design and making a plan to execute it. You talk about the project schedule, resources, and how this design is going to grow the business. The problem with this mindset is that most have no idea how these changes will affect their business, and you rarely follow up after implementation to confirm. You may think your design is a home run, but your users might not and there is very little data to back it up. If this sounds, like your company, then let's start making some changes.
Create a culture driven by data
In order to start developing an A/B testing culture, you need to introduce key metrics. Your key metric should be based on the factors that influence the performance of your business. Your metrics should indicate whether or not your business is growing and how much money you are spending to grow it. Once you have established what your key metrics are, you need to instill a culture that is driven by these metrics. Everyone in your organization should know what these metrics are, and everyone's individual goals should be influenced by the key metrics.
Make people accountable to the metrics
When introducing key metrics into your company you need to ensure that everyone has a common goal, and you need to make everyone accountable to the metrics. Everything a team implements should have a goal to improve one of your key metrics. If you are implementing changes that are not driven by your metrics, you are probably wasting a lot of time and money with very little return.
Change your design process
When you start looking at your design process and making design changes, you need to start asking different questions.
- What is the problem?
- What are we trying to achieve?
- How will this affect our metrics?
- How are we going to measure the success?
- Could there be more than one solution?
When you start asking these questions you are making your team accountable to the metrics and pushing them to test multiple solutions. Your team will quickly realize that only your users know what the solution is, and sometimes the solution that seems the worst on paper, is the best performer.
Test every theory
Once you start asking the right questions, your team will be driven by your data and a/b testing will come naturally. Your team will now depend on running A/B tests to get the data to confirm a hypothesis. This data will start dictating your design, and only the right changes will be implemented.
In order to get your a/b testing culture to stick you need to follow up on your completed each project. Don’t just push the winning variation out to your users and move onto the next thing. If a team implemented a change that increased your growth by x%, make that known. By making people aware of how the changes, they make affect your metrics everyone will start pushing to go beyond. Even better everything will become an A/B test because your team will want to know how even a small change can have an impact on your business.
Let your team explore and test
This is probably one of the most crucial parts to forming a strong culture. Give your team the power and freedom to drive changes. Everyone in your organization should be able to suggest a problem and be given the opportunity to test their hypotheses Once your team buys into the fact that they have the power to change your product, you will have engaged employees, focused on growing your business.
Instead of Alex in Marketing getting mad that a designer isn’t listening to a design idea he has to improve your checkout flow, let him develop an A/B test see the impact. By not giving people power to make data-driven changes, you are just losing out. You are potentially losing an opportunity to grow your business, and you are also going to start having less engaged employees unwilling to bring up new ideas.
The last key factor in getting an A/B testing culture to stick is to keep iterating. No design is perfect, and everything can change. If you stop testing new ideas and stop making changes, your product will become out of date and your users could start leaving for newer and better things. You need to iterate constantly in order to continue to grow. Even the biggest and best are constantly iterating and improving their user experience, because nothing is ever perfect.