Prediction Studio

Prediction Studio is the workspace that provides tools to create, monitor and update predictions and the predictive models that drive them.

This video gives you an overview of the features of Prediction Studio, the data scientist workspace that provides tools to create, monitor and update predictions and the predictive models that drive them.

The responsibility of a Customer Decision Hub™ data scientist during the implementation phase of a project is to configure the AI.

This includes configuring the out-of-the-box Customer Decision Hub predictions, adding relevant predictors to the adaptive models that drive these predictions. You can create predictions that are driven by existing client models, for example a predictive model that determines the likelihood that a customer will churn in the near future. You also configure the notifications that will alert you when the AI performance metrics change over time.

Once the project has reached the production phase, the data scientist monitors the AI in response to Prediction Studio notifications or as a regular health check.

The impact of AI on the business objectives, often referred to as 'lift,' is an important KPI. The data scientist analyzes the predictions, models, and predictors with the aim of fixing any issues with prediction performance and providing insights to the governance team.

Prediction Studio offers a convenient suite to accomplish many of these tasks. There are five work areas in the portal. Predictions, to manage your predictions, Models, to manage the models that drive the predictions, Data, to create data sets sourced from a database table, from stream services, or even social media, such as Twitter and YouTube.

Resources, which include taxonomies and the default sentiment lexicon to use in building machine learning models. When enabled, historical data used for the training of adaptive models and monitoring of predictive models is recorded for offline analysis. The Reports work area contains numerous out-of-the-box reports on models and predictors. In the Settings work area, you can manage general Prediction Studio settings, review the model transparency settings, connect to third-party machine learning platforms, and set up the notifications that Prediction Studio sends you.

Out of the box, Customer Decision Hub comes with predictions that optimize 1:1 customer engagement for every channel in both inbound and outbound directions if applicable. For example, in a cross-sell scenario, U+Bank uses its website as a marketing tool. When a customer logs in, the website displays a personalized credit card offer in a banner.

When a customer is eligible for multiple credit cards, the Predict Web Propensity prediction calculates the propensity of receiving a positive response from the customer for each of these cards. Customer Decision Hub decides which credit card to offer based on business rules, interaction context, and propensity.

Self-learning, adaptive models drive these out-of-the-box Customer Decision Hub predictions. The Predict Web Propensity prediction is based on the Web Click Through Rate model configuration. The configuration supports both customer-level and account-level decisions.

Adaptive Decision Manager is the closed loop system that manages the adaptive model creation, deploying, and monitoring process of a large number of models without human intervention.

For the U+Bank cross-sell use case, Adaptive Decision Manager creates an adaptive model for every credit card offer, based on the Web Click Through Rate model configuration, the first time Customer Decision Hub makes a decision that references the offer. In the production phase, Adaptive Decision Manager captures the customer interactions and updates the models every hour, closing the loop. This ensures that U+Bank automatically reacts to changes in customer behavior as the adaptive models self-optimize.

As a consequence of testing the system by logging in to the U+Bank website, Adaptive decision Manager has created four models and captured some responses. The bubble chart plots the success rate versus the model performance.

In this click-through scenario, the success rate is the fraction of customers that click on the web banner among all customers that see the offer. The model performance is calculated as the area under the curve of an ROC-graph and ranges from 50 to 100. The models learn at the treatment level, which is reflected in the naming as the combination of a credit card offer and the web treatment. The new models have the minimum performance of 50, as no significant interaction data is available to them in the implementation phase.

A data scientist adds input fields to the model configuration as potential predictors for the models. It is highly recommended to add many uncorrelated predictors, as the models figure out which ones to use. Pega has defined best-practice data models for real-time decisioning use across multiple industries, including financial services, communications, healthcare, and insurance. After a system architect installs the Customer Profile Designer Accelerator component from Pega Marketplace, you can use the financial services clickstream summary fields as new potential predictors in the adaptive models.

The U+ Bank data scientist team develops predictive models on third-party platforms. The offline scores represent the model scores produced by the predictive models. Input fields that are not directly available in the customer data model can be made accessible to the models by configuring these fields as parameterized predictors. The Interaction History dataset captures the customer responses. Aggregated fields from IH summaries are automatically provided to the models as predictors. An example of such a predictor is the group of the most recently accepted offer in the call center.

You can create custom predictions. For example, a prediction that predicts churn, as U+Bank wants to proactively offer a retention offer instead of a credit card offer to customers that are predicted to leave the bank in the immediate future.

Create a churn model to drive the prediction.

Use Pega machine learning, import predictive models into Prediction Studio in the PMML or H2O.ai format, or connect to a machine learning service, such as Google ML or Amazon SageMaker.

Create a churn prediction and configure your predictive model to drive the prediction. This prediction is a hand-off to an NBA designer, who can then create an engagement policy to offer a retention offer to customers with a high churn risk. This strengthens the separation of concerns between the two roles. You can also use the output of the prediction as real-time predictive model scores in the adaptive models.

In the production phase, a data scientist gets involved based on definitions of things to pay attention to, as opposed to regularly coming to Prediction Studio to perform investigations. You get notifications when metrics on model performance, responses, output, and predictors change as an email digest. For example, you can include a notification in the email digest that alerts you if the lift that a model generates drops by 10 percent over the course of a week.

This demo has concluded. What did it show you?

• The work areas of Prediction Studio.
• What tasks a data scientist performs during the implementation phase and the production phase.
• How Adaptive Decision Manager automatically updates adaptive models to react to changes in customer behavior.
• How to add predictors to an adaptive model.
• How to create custom predictions, driven by predictive models.
• How to set up the notifications that a data scientist receives in the production phase.