
Monitoring adaptive models
3 Tasks
10 mins
Beginner
Pega Customer Decision Hub 8.7
English
Scenario
The models for the U+Bank implementation of cross-sell on the web of their credit cards have been learning for some time. Your task in this challenge is to inspect the models and report on which predictors are performing well, and which are not.
Use the following credentials to log in to the exercise system:
Role | User name | Password |
---|---|---|
Data scientist | DataScientist | rules |
Caution: This challenge requires specific artifacts. Ensure that you click Initialize (Launch) Pega instance for this challenge to get the correct exercise system.
Your assignment consists of the following tasks:
Task 1: Inspect the adaptive models
Inspect the adaptive models by answering the following questions:
- Which model requires attention, and what is the performance of this model?
- What is the performance of the two best performing models?
- What is the performance of the two worst performing models, and what is their number of responses?
- Which banner is the most successful?
Task 2: Inspect the predictors of the models
Inspect the predictors in the models by answering the following questions:
- Which three predictors have the highest performance across all models?
- Which predictors are not used in any of the models?
Task 3: Inspect a specific adaptive model
Identify the model with the highest number of responses, then inspect it and answer the following questions:
- What are the top three predictors used in the model with the best performance?
- Which predictor, that is not currently relevant to the adaptive model, has the highest performance?
- Which age categories show the highest and lowest interest in this banner?
Challenge Walkthrough
Detailed Tasks
1 Inspect the adaptive models
- On the exercise system landing page, click Pega CRM suite.
- Log in as the Data Scientist with the user name DataScientist and password rules.
- In the navigation pane on the left, click Models.
- Click the Web_Click_Through_Rate tile to open the model configuration.
- Click Refresh data.
- Question - Which model requires attention, and what is the performance of this model?
- Hover over the green model in the chart.
Tip: Note the name of the model and its performance. The StandardCard model with a low performance of 50 requires immediate attention.
- Hover over the green model in the chart.
- Question - What is the performance of the two best performing models?
- Hover over the two models in the middle of the chart.
Tip: The two best performing models are RewardsPlusCard with a performance of 71.7, and PremierRewardsCard with a performance of 84.8.
- Hover over the two models in the middle of the chart.
- Question - What is the performance of the two worst performing models and what is their number of responses?
- Hover over the smallest models in the chart.
Tip: The RewardsCard and StandardCard models have both the worst performances, of 67.2 and 50.0 respectively, and the lowest number of responses at 1516.Note: In this web scenario, the success rate of the model is the click-through rate, the fraction of customers that clicks on the banner.
- Hover over the smallest models in the chart.
- Question - Which banner is the most successful?
- Hover over the model with the highest success rate in the chart:
Tip: The RewardsPlusCard has the highest success rate of 43.22%.
- Hover over the model with the highest success rate in the chart:
2 Inspect the predictors of the models
- Question - Which three predictors have the highest performance across all models?
- Click the Predictors tab.
- Click the Average performance column header twice to sort by the average performance.
Tip: The three best performing predictors are: CreditScore, AnnuaIIncome, and Age. All three predictors are used in all 4 models, because the # Models inactive value is 0.
- Click the Predictors tab.
- Question - Which predictors are not used in any of the models?
- Click the # Models active column header to sort the models in ascending order:
Tip: The predictors with a # Models active value of 0 are currently not used in any of the models.
- Click the # Models active column header to sort the models in ascending order:
3 Inspect a specific adaptive model
- Question - What are the top three predictors used in the model with the best performance?
- Click the Models tab.
- Click the Performance (AUC) column header twice to sort the list of models in descending order.
- Click Model report in the first row.
- Click the Performance (AUC) column header twice to sort the predictors in descending order.
Tip: The top 3 best performing predictors for this model are CreditScore, AnnualIncome, and Age.
- Question - Which predictor, that is not currently relevant for the adaptive model, has the highest performance?
- Scroll to the bottom of the list.
Tip: Customer.HasOptInEmail is the inactive predictor with the highest performance of 51.71.
- Scroll to the bottom of the list.
- Question - Which age categories show the highest and lowest interest in this banner?
- Click the Age predictor.
Tip: The highest interest for this banner is in the last two bins, with customers approximately 49 years old and above. The lowest interest is shown by customers ranging in age from approximately 22 to 25 and 29 to 31 years old.
- Click the Age predictor.
Available in the following mission:
If you are having problems with your training, please review the Pega Academy Support FAQs.
Want to help us improve this content?