How AI 'prediction products' could improve performance

In increasingly turbulent and dynamic business environments, managers are seeking tools that provide reliable data, generate accurate predictions, and support them in making better decisions.

Existing approaches draw on traditional performance measurement and management tools - such as KPIs, performance targets, and scorecards - as well as big data analytics.

However, recent uses of Artificial Intelligence (AI), especially by online platforms, are creating novel scenarios in relation to the acquisition, analysis and use of data with the aim of predicting human behavior and organisational performance with potentially radically new consequences for individuals and organisations.

So far, researchers and managers have mainly concentrated on how platforms use AI to understand and predict human behavior via machine learning algorithms that interact with users. For example, at Netflix machine learning algorithms are deployed to understand viewers’ preferences and to provide recommendations to engage them with the movies they most prefer.

Here, we consider situations where three parties are involved: a platform (first party) collects data on users (second party) and generates and sells predictions to a buyer (third party). In this context, the platform monetises users’ data not by directly sharing it with the buyer, but rather by 'packaging' the data, using AI, into a 'prediction product', that is, the prediction of users’ behaviors.

Performance measurement and management: measure-to-modify

Traditionally, organisations have adopted a hypothesis-driven approach to the process of data collection, analysis, and communication where purposes such as monitoring, improving and predicting performance, are defined up front.

In practice, strategic objectives are articulated and several tools, such as scorecards, KPIs and targets, are introduced to both control and direct efforts towards the attainment of the strategic objectives. To further motivate employees to achieve them, Professor Micheli's research with Jean-Francois Manzoni, of the International Institute for Management Development (IMD) in Switzerland, found a system of incentives is often implemented in the form of financial and non-financial rewards.

This approach is mainly top-down: strategic objectives are identified first and then mechanisms are put in place to promote their achievement; sometimes the objectives are changed or refined because of previous results.

Importantly, even though the collection, analysis and communication of data are often regarded as 'neutral', this process is 'performative': what is being measured changes because of the very fact of it being measured. Hence, we label this approach 'measure-to-modify'.

For example, if a KPI is introduced to evaluate employee productivity, such as to measure how many patients are seen by a doctor at a hospital, it is likely that employees will try to improve their productivity, whether positively – by shortening patients’ waiting times and improving their pathways – or negatively – by seeing more patients but eventually achieving worse clinical outcomes, because assessments are too brief.

When deploying performance measurement tools, organisations have increasingly attempted to predict, rather than simply monitor, their performance, for instance, by introducing leading, rather than lagging indicators.

In this context, prediction refers to using a KPI to increase the chances of achieving a certain goal, as the KPI could provide insight and greater confidence that a goal would be attained. This confidence is based on the hypothesised causality between attributes and depends on an organisation’s ability to create and refine causal models.

For example, in a production plant, if setup times are reduced then it can be reasonably hypothesised that lead times will reduce too. In this paradigm, predictive ability could be also enhanced by combining measurement with other tools such as scenario planning and forecasts.

Big data analytics: predict-to-modify

Thanks to the increasing availability of data from a variety of sources, a second approach is becoming commonplace. In this case, Nada Sanders, of Northeastern University, found that while performance measurement tools may still be used, the emphasis is more on an organisation’s capacity to leverage data gathered from different sources (such as, internal data, sensors, wearable devices) to predict future performance using algorithms.

For example, we have worked with companies like GE, Siemens, and Rolls-Royce that have invested in predictive maintenance whereby data on the wear and tear of their products is gathered to predict breakdowns or malfunctions.

Similarly, in many companies the allocation of employees across stores or work sites now depends on the demand or workload predicted using past data. While data can also be gathered to predict customer choices; for example, based on which genres and songs a person has listened to, Spotify offers suggestions through personalised playlists.

These are all examples of predictive analytics which consist of examining historical data, detecting relationships, and extrapolating to formulate predictions.

We label this approach 'predict-to-modify' because organisations formulate predictions with the aim of modifying their work processes or offerings - in the examples above, their maintenance schedules, employees’ workloads, and users’ playlists.

While sharing some similarities with the 'measure-to-modify' approach, in these contexts, data may be gathered in a variety of ways and perhaps with different purposes but are then used to identify patterns and correlations, and to formulate predictions.

In this case, there may be no targets, incentives or intentional feedback loops: the performative element is less pronounced, and individuals may be aware that data are being collected, but not why.

At the same time, tools and practices from the two approaches can be used jointly: various authors have emphasised that the combination of big data analytics and performance measurement tools such as KPIs can be a key way to provide prediction, “not just rearview-mirror reviews”, say Michael Schrage, of MIT Sloan School and David Kiron, Executive Editor of MIT Sloan Management Review.

For example, customer loyalty may be hypothesised as a primary driver of sales volumes; however, data may show that investments to increase customer loyalty do not significantly impact sales. Therefore, a more inductive approach might be adopted to identify drivers of sales, for example, by conducting small experiments in different markets.

This could then lead to a reconceptualisation of the causal model – which addresses the question, 'what drives sales?' – that could then result into further hypothesis testing.

AI-based approach: predict-then-modify

Drawing on secondary data and on experience we gained by researching digital platforms, such as Google, Amazon and Meta, we concentrate on a third type of approach, which makes it possible for platforms to generate almost perfect predictions (prediction products) for their business customers, something Galit Shmueli, of National Tsing-Hua University College of Technology Management, and Ali Tafti, of University of Illinois also discovered.

For instance, Google’s 'predictive audiences' tool aimed at e-commerce customers uses machine learning to “predict the future behavior of your users” and provides business customers with lists of users who are most likely to purchase, churn, or generate the most revenue in the near future, such as over the next 28 days.

More generally, in our research we have come across a wide range of predicted user behaviors which can be used for a variety of purposes, including insurance companies using predicted user behaviors to tailor personalised insurance products, advertisers using predicted consumer preferences to offer personalised ads, and political campaign managers predicting voting intentions to target suitable audiences, as unearthed by Shmueli and Tafti.

When offering prediction products, the seller aims to showcase high prediction accuracy and the buyer expects that this is achieved via smart algorithms applied to quality data.

While prediction accuracy can indeed be improved by using larger, more robust datasets and by enhancing the quality of the algorithms, research by Travis Greene, of National Tsing Hua University, David Martens, of the University of Antwerp, and Shmueli found it can also be achieved by combining AI tools with 'persuasive technology', which refers to behaviour modification techniques implemented on online platforms.

In this context, behaviour may refer to clicking on an ad, purchasing an item, and posting information. Behaviour modification techniques, such as nudging, refer to ways of manipulating environments to subtly guide users towards making certain choices.

For instance, nudges are built into the design of recommender systems on e-commerce platforms such as Amazon, so that “personalised product recommendations” and “relevant search” results push users to purchase more or specific products in order to generate profit.

While nudging and similar techniques have been used in business for years, the use of persuasive technology by online platforms is more recent and often implemented via various machine learning algorithms that operate in a data-driven, autonomous, interactive, and sequentially adaptive manner.

This means that the combination of prediction and behavior modification – that we label 'predict-then-modify' – may result in a black box system where the goal is to have perfect alignment between predicted and real values. We describe the system as a 'black box', because how this is attained is not clear to platform users, customers, or even the platform data scientists.

In the words of former Facebook Platform Operations Manager, Sandy Parakilas, “these systems have been built in such a way that they’re hard to control and optimise. I would argue that we humans are now out of control. We’ve built a system that we don’t fully understand”.

What is particularly problematic is that it is in the platforms’ interest to push users’ behaviours towards the algorithmic predictions they generate to showcase its predictive accuracy and ability to influence behaviors.

We regard this as problematic, because pushing outcomes towards predictions can be harmful to manipulated users and at odds even with companies’ intentions and goals. For instance, a platform like Amazon could sell predictions of customer purchases to companies offering products on the platform (such as 'Amazon sellers').

These companies could use the prediction products they purchased to adjust their production, marketing and sales efforts and strategies. However, a dysfunctional effect would be that, for customers predicted to be unlikely to purchase these companies’ products, the platform could achieve high prediction accuracy by actively dissuading these customers from purchasing the product, by promoting the products less to these users or by recommending to them other products or sellers, even if these are less relevant for them.

In all these cases, it is evident how the predict-then-modify approach can have negative consequences not only for the second party (users), but also for the third (buyers).

Comparing this approach to the ones outlined earlier, we can see some similarities, but also substantial differences. For example, in both the measure-to-modify and the AI-driven predict-then-modify cases there is a performative element - we want to modify behaviours and therefore feedback loops are present - there are three key differences.

First, in the AI scenario individuals are not aware that data is being collected, by whom or why. Second, in the predict-then-modify case, the data is typically owned by the seller (platform) whereas in the measure-to-modify one the organisation owns the data. Third, in the world of AI, unexpected consequences are easily exacerbated because of faster feedback loops.

Moreover, in the predict-then-modify approach, the end goal of achieving similarity between prediction and reality is not in the usual sense of improving predictions to fit reality; rather, vice versa: modifying reality to fulfill prediction. In this setting, what is being created is neither a model nor a 'neutral prediction' anymore, as the algorithm’s main role is not to represent reality, but to modify it.

Predictions therefore serve as self-fulfilling prophecies. Unlike the first scenario, where performance indicators are intended to be known, the algorithmic predictions are not revealed to any of the parties but are used by the platforms’ automated persuasive technology to modify behaviours toward those predictions.

This means that the performative element is not clearly owned, and responsibility and accountability are opaque. Indeed, we could say that the algorithm is in control, but the algorithm itself is a black box.

In recent years, we have witnessed a substantial change in the uses of performance data to generate predictions in organisational contexts. In this article, we have discussed the main differences between three approaches and their varied effects on the relationship between prediction and behaviour modification.

The case of prediction products highlights a new capability that has recently emerged due to platformisation: predicting and modifying human behavior via algorithms that interact with users.

This predict-then-modify scenario has significant managerial implications. First of all, since the quality of the service delivered by platforms is mainly expressed as the similarity between predicted and actual behaviours, it is imperative that managers understand how platforms attain such similarity and do not simply trust the basic similarity scores.

While identifying prediction products that use persuasive technology to improve their accuracy is not easy, managers could ask providers for greater transparency as to the algorithms used, and explicitly probe whether and how behaviours are being manipulated. For example, if one of the aims is to maximise user engagement, how is the prediction being generated?

Furthermore, as data ownership and accountability are shifting from being business-owned to platform-owned in various industries, the AI-driven approach discussed here creates new relationships between firms and platforms.

Platforms use a variety of data monetisation strategies that managers should be aware of. Such strategies may offer new opportunities to firms but can also create unintended harms. For example, Uber now monetises its hugely rich data on drivers, riders, and routes to offer business customers targeted advertising to riders.

Fitbit, the provider of wearable technology, physical fitness monitors and activity trackers, uses its data to offer a Health and Wellness Insights Platform that provides anonymised, aggregated data to healthcare providers, insurers, and researchers. eBay monetises data by offering retailers, manufacturers, and industry analysts access to aggregated and anonymised sales data. 

In this article, we have discussed the known examples of prediction products sold by Google and Facebook. Uber, Fitbit, eBay and others could potentially create prediction products as well and sell them to business customers.

For example, Uber might sell predicted risk scores of drivers to insurance companies. Fitbit could sell predicted health risk scores of its wearable users to healthcare providers, and pharmaceutical and insurance companies. eBay could sell predictions of its sellers’ behaviours, including chances of successful future transactions to third parties such as financial services firms.

Managers should assess the potential consequences of platforms creating and selling these products. Also, they should consider up front whether the prediction they are asking for is related to any specific risk. For instance, is there a risk of overly engaging users (especially of young people)? Could addictive or divisive behaviors be promoted? What is the likelihood of putting users or society at risk?

Reflecting on these aspects at the very beginning would help managers frame the focus of the prediction in such a way that, however generated, it would be less likely to lead to unintended consequences at a later stage.

This article was originally published in California Management Review.

Further reading:

Working on the jagged frontier: How companies should use generative AI

Three steps to integrate AI into your organisation

Why pension funds must reduce the AI skills gap

Pietro Micheli is Professor of Business Performance and Innovation and teaches Managing Organisational Performance on the Executive MBA, Global Online MBA, and Accelerator MBA. He also lectures on Design Thinking on the Master's portfolio and Design in Business on the Undergraduate programme.

Galit Shmueli is Chair Professor, Institute of Service Science, at the National Tsing Hua University, Taiwan.

Discover more about Digital Innovation and Entrepreneurship with the Core Insights newsletter.