We often get asked as to how we make decisions on what products to show within Aura: What inputs are used? How are they processed? What signals matter the most?
The answers to these questions are complex and written in a language that is largely incomprehensible unless you are familiar with python, scikit-learn and google cloud and think in terms of information retrieval and machine learning (Note: We are hiring for more data scientists). In this blog post, I’m going to provide some clarity on how—and why—we’re using Machine Learning (ML) in Aura.
There are five important elements of our machine learning within Qubit Aura: structure, measurement frameworks, evidence selection, focus on variety and revenue, and business rules.
1. Structure in design and structure in data
Any machine learning system is only as good as its data. Good, structured data gives good, fast results. Just as Facebook’s Newsfeed uses a specific structure of information to determine most relevant content for each user, Qubit Aura also benefits from “structure”. There are two main aspects to how the structure is created within Aura.
Firstly, the data collected is structured through our QProtocol format (Qubit customers can also read more here). This means that every single interaction is used to create a better, more personalized experience. The strict schema means that our systems know exactly how to process this data and can use it appropriately to improve the results as well as to also quickly implement Aura for new properties.
Secondly, Aura has structure in its design. The user experience is designed to maximise the likelihood of a customer swiping, scrolling and interact with products and content. In this way, Aura is maximising the amount of data available for it to learn from. These in turn drive better suggestions and, as a result, more impact on revenue.
2. Measurement frameworks
To evaluate the performance of a ML system you need a basis for measurement.
The visual impact of the ML used within Aura are evident through the ranking and prioritization of products. Measuring this is not a simple task: there are too many combinations and permutations. With personalized recommendations, every potential interaction results in a potentially different recommendation.
As such, we use a combination of offline measurement and randomized controlled trials within Aura to measure the impact of any changes to our models, and provide the data to iterate them.
Read more on our approach here.
3. Evidence Selection
The best personalization looks like it isn’t there at all. As if by magic, all the things people see are the things they want to see. But that isn’t always a comfort to users, who want to understand “How did you know I’d like that?” “Why are you showing me this?”.
Qubit Aura provides information to the user on why they are seeing an item. This information is referred to as ‘Evidence Selection’—that is, Qubit Aura selects the piece of evidence it has that is most suitable for that user.
As Netflix describe it:
“Another important element in Netflix’ personalization is awareness. We want members to be aware of how we are adapting to their tastes. This not only promotes trust in the system, but encourages members to give feedback that will result in better recommendations. A different way of promoting trust with the personalization component is to provide explanations as to why we decide to recommend a given movie or show. We are not recommending it because it suits our business needs, but because it matches the information we have from you: your explicit taste preferences and ratings, your viewing history, or even your friends’ recommendations.”
For Qubit Aura, providing a reason to the user promotes trust. And it increases engagement and gives users more confidence to click through. In turn, this results in more data for Aura to learn from, so it can make even better suggestions going forward.
4. Focus on Variety and Revenue
A key goal of Qubit Aura is to increase the number of products that customers view, getting them deeper into your product catalog (to understand why this is important, check out our mobile discovery report, which summarizes findings of an analysis of 1.2Bn user journeys).
In theory, this goal could be met by either showing lots of very similar products side by side, or by showing a whole load of very diverse products. In reality, there’s an optimal point between these two extremes. This goal is to provide a great user experience, to enable customers to compare multiple products in the feed so that they are informed of the possibilities of the catalog, but are able to easily find the item most suitable.
Key to the user experience is not just surfacing the most popular products, or the ones the customer is most likely to interact with. To provide broader context and a richer experience, customers should be kept informed of items that are increasing in popularity, and the bestsellers. The balancing act is in ensuring that the consumer is provided with an engaging experience that can increase the diversity of products a customer sees—giving them a broader exposure than they might experience through search or a PLP (product listing page).
5. Coming soon: Ability to merchandise and apply business rules
Machine learning systems are as good as their data, but there’s always an essential place for human instinct, expertise and inspiration.
The Aura team are currently making large investments in giving brands controls that enable them to set out their editorial judgment or to guide the customer to specific areas of the catalog which they believe are about to be on the up.
Coming soon to Aura are features enabling brands to merchandise the items that they would like their customers to see based on brand essentials, exclusives, new designer launches and more. Balancing ML with human creativity will ultimately result in greater product discovery and a better experience for the customer all round!
Watch this space to learn more on our plans.
Read more on Aura here.