Monitoring adaptive models

3 タスク

10 分

表示の対象：All users Applies to: Pega Customer Decision Hub '25

初級

英語

シナリオ

U+Bank is cross-selling credit cards to customers on the U+Bank web channel.

A Gradient boosting model calculates the propensity that a customer will respond positively to the offer for each combination of action and treatment.

Monitoring the health of the model and its predictors is a regular data scientist task that can be performed in Prediction Studio. Your task in this challenge is to inspect the model and report on which actions and predictors are performing well, and which are not.

Use the following credentials to log in to the exercise system:

Role	User name	Password
Data scientist	DataScientist	rules

Your assignment consists of the following tasks:

Task 1: Inspect the bubble chart

Inspect the bubble chart by answering the following questions:

Which actions require attention, and what is the performance of the model for these actions?
Which banner is the most successful?

Task 2: Inspect the predictors of the Gradient boosting model

Inspect the predictors in the model by answering the following questions:

Which customer predictor has the highest performance across all actions?
Which predictors are not used in the model?

Task 3: Inspect a model report

Identify the model with the highest performance, inspect it, and answer the following questions:

What is the best predictor for the action with the highest success rate?
Does the model successfully identify customers more likely to respond positively to the RewardsPlusCard offer?
Is the model learning over time?
補足： The values you see in your exercise environment will differ slightly from the screen shots below.

このチャレンジを完了するには、Pegaインスタンスを起動する必要があります。

起動には5分ほどかかることがありますので、しばらくお待ちください。

ログイン

チャレンジウォークスルー

詳細なタスク

1 Inspect the bubble chart

On the exercise system landing page, click Pega Infinity™ to access Customer Decision Hub.
Log in as a data scientist:
1. In the User name field, enter DataScientist.
2. In the Password field, enter rules.
In the navigation pane on the left, click Intelligence > Prediction Studio to open the Prediction Studio portal.
Click the Predict Web Propensity prediction tile to open the prediction.
On the Models tab, in the Propensity to Click section, click Web Click Through Rate GB Customer to open the model.
Expand the Business issue / Group list, and select Grow > Creditcards.
On the Monitor > Models tab, inspect the bubble chart.
Question - Which actions require attention, and what is the performance of the model for these actions?
1. Hover over the actions in the chart to see the performance.
  
  ヒント： The RewardsCard and the StandardCard actions show a model performance close to random (50), and require attention.

Question - Which banner is the most successful?
1. Hover over the blue bubble.
  
  ヒント： The PremierRewardsCard has the highest success rate.
  
  補足： In this web scenario, the success rate of the action is the click-through rate, the fraction of customers that clicks on the banner.

2 Inspect the predictors of the Gradient boosting model

Question - Which customer predictor has the highest performance across all actions?
1. Click the Monitor > Predictors tab.
  
  ヒント： The predictor with the highest importance is the Context predictor pyTreatment. The best-performing customer predictor is AnnualIncome.

Question - Which predictors are not used in the model?

ヒント： Predictors with an importance of zero do not contribute to the decisioning process.

3 Inspect a model report

Question - What is the best predictor for the action with the highest success rate?
1. Click the Models tab.
2. Click the Success rate (%) column header twice to sort the list of actions in descending order.
3. Click Model report in the first row.
  
  ヒント： The top predictor for the PremierRewardsCard action is CreditScore.

Question – Does the model successfully identify customers more likely to respond positively to the RewardsPlusCard offer?
1. Click Score distribution to inspect how the model distributes cases across different score ranges.
  
  ヒント： In a well-performing model, the propensity trend line should generally trend upward as scores increase.