UX is at the intersection of product and design. If UI designers are fixated on the look and feel of an app, while product managers are focused on its functionality and usability, then UX designers fall somewhere in between.
A/B testing plays as big a role in UX design as it does in marketing. It’s difficult to make good design decisions purely based on gut feel, so many designers find themselves iterating based on real user behavior.
The problem is, there are countless factors that go into app design. And you can’t just A/B test everything — testing is only useful if you have a hypothesis first.
Here are a few elements that we recommend every designer to A/B test. Start with these, and you’ll be well on your way to optimizing your app’s entire user experience.
1. A/B Test Iconography
Iconography is one of an app’s most basic design elements. Frequent users won’t think twice about it, but your icons are one of the first things that new users see.
As with much of UX design, the key to iconography is clarity. A new user should be able to figure out what an icon does before they tap it. Because iconography is something that you get used to (just think of the OS icons that you see every day at work), it’s difficult for a designer to evaluate the clarity of their own icons.
A/B testing the app on new users can help you see how many people click on an icon and immediately click back, implying that they didn’t intuitively understand what screen the icon would lead to.
2. A/B Test Button Placement & Color
This is one of the classic design elements to A/B test, and for good reason. UI navigation happens faster than people can read, so text is often subservient to color and placement.
For example, in a “yes or no” prompt, the “yes” button is usually a brighter color. If you’re accepting a license or setting up a new account, you’re probably more inclined to click “yes” than “no”, so UX designers make the button easier to find.
Button placement is harder when there are more elements on the screen. Even if there are several options available to the user, it helps if the UX designer can identify the primary action(s) and make sure those are easy to find. Camera apps have a lot of functionality hidden behind the settings button, yet it’s still easy to find the shutter button at a glance.
If there are multiple actions on the screen that are equally important (e.g. a list of bookmarks in a browser), you can represent this by making them symmetrical. In doing so, it should be clear that there’s no primary action on the screen — and A/B testing will help you make sure.
3. A/B Test Color Choice
Color is an important choice for UX designers. On one hand, it’s important to stick to a specific palette to establish brand consistency. But on the other hand, color plays an informational role in helping users navigate the app. Balancing the two goals is one of the major challenges that UX designers face.
Rigorous A/B testing is the best way to determine whether users are getting lost in your app, and whether color can help with that.
If the colors of your app are too uniform and people are having trouble finding the primary action on each screen, you can try to subtly change the shade of different elements. If that doesn’t work, you can then change the color entirely, and A/B test to confirm if the color change is necessary to improve navigation.
Don’t be afraid of flat test results at this stage. Discovering that color tweaks don’t aid navigation is just as valuable as discovering that they do.
4. A/B Test Typography & Legibility
UX designers aren’t copywriters, but text is still part of the designer’s job. The visual properties of text have a huge impact on its readability, especially when the user is only skimming. Simple, clear copy can be obfuscated by poor design.
Size and color are two important factors to consider. Changing font size by a single point could make a difference to readability. At the same time, you can make text surprisingly small without sacrificing legibility, making room for more content on the screen or simply a more spacious layout.
Black-on-white might be the default color combination, but there are plenty of aesthetic reasons to go for something else. With enough tweaking, you may be able to find a text and overlay combination that can legibly sit on top of a complex background, enabling more variety in your app’s design.
5. A/B Test Screen-to-Screen Navigation Flows
Even though navigation flows aren’t visual, they’re an aspect of the app experience that should be consciously designed — and tested.
It’s easy to get lured into evaluating each screen in a vacuum, without paying attention to the overall flow. Yet there aren’t many apps that live on a single screen. If users have to navigate too many screens to access common functionality, or if it isn’t obvious how to get to a particular screen, you could be introducing friction without knowing it.
A/B testing app navigation means finding different ways of organizing your hierarchy of screens until you land upon the one that makes intuitive sense. As always, it’s difficult to rely on your own intuition, as you’re an expert at navigating your own app.
With enough data, you can develop a feel for which screens are the most commonly accessed, or which ones users visit after accessing another screen. Once you develop a hypothesis, you can A/B test alternate hierarchies to see if users are able to get where they’re going more quickly.
Navigation testing is tricky because there are more factors at play and it may require reorganizing your menus from the ground up. Still, it’s an important factor to get right, and it’s very difficult to get it right without data.
Iteration in UX Design
We’ve listed the design elements that you can’t afford not to A/B test, but there are many more that designers would benefit from testing. Furthermore, it’s important to understand how different segments of your audience respond to each design, rather than A/B testing on everyone and making blanket judgments.
The key takeaway here is the mentality of iterative design. A/B testing allows designers to quantitatively see the app through the eyes of the user. When combined with the designer’s own qualitative understanding of the user, the most effective designs are suddenly within reach.