Monitoring adaptive models

3 Tasks

10 mins

Pega Customer Decision Hub '25

Pega Customer Decision Hub '24.2 Pega Customer Decision Hub '24.1 Pega Customer Decision Hub '23 Pega Customer Decision Hub '23 Pega Customer Decision Hub 8.8 Pega Customer Decision Hub 8.7 Pega Customer Decision Hub 8.6 Pega Customer Decision Hub 8.5

Visible to: All users Applies to: Pega Customer Decision Hub '25

Beginner

English

Scenario

U+Bank is cross-selling credit cards to customers on the U+Bank web channel.

A Gradient boosting model calculates the propensity that a customer will respond positively to the offer for each combination of action and treatment.

Monitoring the health of the model and its predictors is a regular data scientist task that can be performed in Prediction Studio. Your task in this challenge is to inspect the model and report on which actions and predictors are performing well, and which are not.

Use the following credentials to log in to the exercise system:

Role	User name	Password
Data scientist	DataScientist	rules

Your assignment consists of the following tasks:

Task 1: Inspect the bubble chart

Inspect the bubble chart by answering the following questions:

1. Which actions require attention, and what is the performance of the model for these actions?
2. Which banner is the most successful?

Task 2: Inspect the predictors of the Gradient boosting model

Inspect the predictors in the model by answering the following questions:

1. Which customer predictor has the highest performance across all actions?
2. Which predictors are not used in the model?

Task 3: Inspect a model report

Identify the model with the highest performance, inspect it, and answer the following questions:

1. What is the best predictor for the action with the highest success rate?
2. Does the model successfully identify customers more likely to respond positively to the RewardsPlusCard offer?
3. Is the model learning over time?
   Note: The values you see in your exercise environment will differ slightly from the screen shots below.

 

You must initiate your own Pega instance to complete this Challenge.

Initialization may take up to 5 minutes so please be patient.

• Log In

Challenge Walkthrough

Detailed Tasks

1 Inspect the bubble chart

1. On the exercise system landing page, click Pega Infinity™ to access Customer Decision Hub.
2. Log in as a data scientist:
   1. In the User name field, enter DataScientist.
   2. In the Password field, enter rules.
3. In the navigation pane on the left, click Intelligence > Prediction Studio to open the Prediction Studio portal.
4. Click the Predict Web Propensity prediction tile to open the prediction.
5. On the Models tab, in the Propensity to Click section, click Web Click Through Rate GB Customer to open the model.
6. Expand the Business issue / Group list, and select Grow > Creditcards.
7. On the Monitor > Models tab, inspect the bubble chart.
8. Question - Which actions require attention, and what is the performance of the model for these actions?
   1. Hover over the actions in the chart to see the performance.
      
      Tip: The RewardsCard and the StandardCard actions show a model performance close to random (50), and require attention.

9. Question - Which banner is the most successful?
   1. Hover over the blue bubble.
      
      Tip: The PremierRewardsCard has the highest success rate.
      Note: In this web scenario, the success rate of the action is the click-through rate, the fraction of customers that clicks on the banner.

2 Inspect the predictors of the Gradient boosting model

1. Question - Which customer predictor has the highest performance across all actions?
   1. Click the Monitor > Predictors tab.
      
      Tip: The predictor with the highest importance is the Context predictor pyTreatment. The best-performing customer predictor is AnnualIncome.

2. Question - Which predictors are not used in the model?
   
   Tip: Predictors with an importance of zero do not contribute to the decisioning process.

3 Inspect a model report

1. Question - What is the best predictor for the action with the highest success rate?
   1. Click the Models tab.
   2. Click the Success rate (%) column header twice to sort the list of actions in descending order.
   3. Click Model report in the first row.
      
      Tip: The top predictor for the PremierRewardsCard action is CreditScore.

2. Question – Does the model successfully identify customers more likely to respond positively to the RewardsPlusCard offer?
   1. Click Score distribution to inspect how the model distributes cases across different score ranges.
      
      Tip: In a well-performing model, the propensity trend line should generally trend upward as scores increase.

3. Question – Is the model learning over time?
   1. Click Trend.
      
      Tip: The performance of the model increased from 50 (or random) to around 80.

---

This Challenge is to practice what you learned in the following Module:

• Monitoring adaptive models v9

---

Available in the following mission:

• AI for 1:1 Customer Engagement v6