Designing an AI Driven Product Roadmap
by Charu Manglani –
Artificial Intelligence (AI) has invaded nearly every sphere of our lives. It is the hidden force of math suggesting the best route on google maps or what to watch on Netflix and Amazon Prime Video. AI is like a wizard that behind the scenes automates our most tedious tasks and makes our interaction with products more personal and engaging.
Does the success of an AI product depend only on the rigorous math running the model? What are the challenges faced with managing a data based products? How are algorithms kept up-to date? More in general, how do you approach AI when building a product?
Daniel Shenfeld of Manganese Solutions simplified the journey of building a product driven AI Roadmap at a recent event held by BPMA. The discussion focused on four key product management fundamentals that apply to building a successful AI based product.
Understand the product problem(s) that AI solves
Product managers must always first understand the problem that they want to solve for users. Similarly, to utilize AI successfully, we need to understand what problem AI can solve and connect the features we’re trying to build to the product goals we want to achieve.
Machine Learning, a component of AI, helps address the following types of problems:
Need for Personalization (e.g. Netflix, Amazon, Uber): AI helps customize the product and optimize information most relevant to the user.
Need for Automation: AI helps automate routine tasks such as Frequently Asked Questions (FAQs) for retrieving information on account balances and due payments that would otherwise consume productive time of human resources.
Map decisions in your product
AI products essentially take over certain decision making human tasks, such as interpreting radiology results or drawing patterns from various healthcare reports to recommend the right treatment.
The first step requires the Product Manager to map decisions on various possibilities that could emerge during an interaction in order to train the model on these scenarios. Once the model is trained on an initial dataset, the algorithm is further refined with more data to identify the correct pattern and produce more accurate results. AI should not be used for products where the current user base is small and hence there’s lack of adequate data to train the model. The more data, the better.
Connect decisions to product objectives
Let’s consider the examples of TripAdvisor (travel recommendations) and Stitch Fix (styling recommendations) to understand how to link decisions to objectives. The end objective for both of these products is to make recommendations to the customer based on certain criteria. However, personalization for Stitch Fix is different than for TripAdvisor.
While for TripAdvisor, a criterion like location is objective and standard, for Stitch Fix, making style suggestions involves much subjectivity. Also, in the case of Stitch Fix, the number of product options are limited to five and they must be distinct choices of clothes to induce the customer to buy more than one item. A customer cannot be expected to purchase similarly styled shirts in different colors. In the case of TripAdvisor instead, knowing that the customer will book only one hotel for a night, allows for recommendations of several but similar hotels of similar range and location. The different degree of complexity at different points of the customer journey, leads to different objectives.
Product Managers must focus on these granular considerations when building a decision tree for the end customer and take into account the broad statistical components that go into an ML model to properly leverage its abilities.
Make room for mistakes
Another important aspect of an AI product strategy is managing uncertainty of the model when making predictions and not solving the product’s stated objectives. For example, there may be variance between the actual dataset and the training dataset which would cause the model’s performance to deviate from the intended outcome. The cost impact of a wrong decision based on faulty prediction could be high.
To mitigate these risks, the Product Manager should adopt the following practices:
Request the data scientist to share the confidence level of the prediction – the higher the better. Gather more data samples if faced with low confidence level.
Keep a human in the loop – recognize the inability of the model to interpret every piece of information, especially if it has not been trained for a certain scenario. The model should therefore be programmed to involve humans for further decisions. For example, Microsoft has built an AI model for mapping data from a table: the user can take a picture of a data table printed on paper and import the image to Excel. The image recognition functionality automatically converts the table image to excel format. However, to avoid the mistake of selecting any text outside of the table, the task of table selection is left to the user and not the AI model.
Build a successful AI apprenticeship: Use this approach to build a virtuous cycle of data to train the model under various scenarios and update it with new information for better decision making.
Start building the model with easier questions – use uplift modeling used in direct email marketing to model the incremental impact of a test on individual response.
AI helps enhance decision making but it cannot fix a process that is natively bad or suboptimal. Unlike building products using traditional software development, which is a deterministic approach; AI helps build products that analyze data, detect logical patterns and make decisions automatically.
The emphasis of an AI strategy should be on identifying the problem and how well AI can be used to automate decisions that can augment human efforts, reduce cost and eventually enhance customer value while delivering ROI.
About the author
Charu Manglani has worked in the banking and asset management industry in strategy & advisory roles. Charu is passionate about learning and applying new ways of building and improvising products. Her current focus is on exploring optimal applications of emerging technologies.