Replacing a predictive model
U+ Bank uses Pega Customer Decision Hub™ to personalize the credit card offers that a customer is presented on its website. To customers that are likely to leave the bank soon, the bank makes a proactive retention offer instead of a credit card offer. The bank has recorded historical churn data for its customer base. As a data scientist, you used the historical data to create a churn model using an external machine learning service, and you replace the active model with the new one.
Use the following credentials to log in to the exercise system:
Caution: This challenge requires specific artifacts. Ensure that you click Initialize (Launch) Pega instance for this challenge to get the correct exercise system.
Your assignment consists of the following tasks:
Task 1: Create a validation data set
Use ValidationData to create a data set for model validation.
Place the ChurnH20 model in shadow mode in the Predict churn propensity prediction.
As a data scientist, replace the active model with the candidate model in the prediction.
1 Create a validation data set
- Log in as Data scientist with user name DataScientist and password rules.
- In the navigation pane on the left, click Intelligence > Prediction Studio > Data > Data sets.
- In the upper right, click New.
- Configure the new data set with the following information:
- Name: ValidationData
- Type: File
- Apply to: PegaCRM-Data-Customer
- Development branch: No branch
- Add to ruleset: PegaCRM-Artifacts
- Ruleset version: 01-01-99
- Click Create.
- In the Data source section, select Embedded file.
- Download the ValidationData data set and extract the .csv file.
- Click Upload file.
- Click Choose File.
- Select the ValidationData.csv file.
- In the upper right, click Save.
2 Place the new model in shadow mode
- Download the ChurnH20 model and extract the ChurnH2O.mojo file.
- In the navigation pane on the left, click Predictions.
- Click the Predict churn propensity tile.
- In the Models tab, click the More icon of the Churn model.
- Click Replace model.
- Ensure that Model is selected, and then click Next.
- In the Upload tab, click Choose File.
- Select the ChurnH20.mojo file.
- Click Next.
- In the Dataset field, select ValidationData.
- In the Outcome column field, select Outcome.
- Click Next.
- In the Model name field, enter ChurnH20.
- Click Replace.
Note: Keep monitoring the Status. During the process of replacing the model the status changes to CONFIGURATION IN PROGRESS, VALIDATION IN PROGRESS, and READY FOR REVIEW..
- When the status of the H20 model changes to Ready for review, click H20 (M-1101).
- Inspect the model comparison.
- In the upper right, click Evaluate.
- Confirm that Approve new candidate model and start shadowing (recommended) is selected.
- In the Reason field, add the appropriate information ('The new model outperforms the active model') .
- Click Save.
3 Promote the new model to the active state
- In the Models tab, click the More icon of the ChurnH20 model, and select Promote model.
- Click Promote model to confirm that you want to promote the model.
- Confirm that the ChurnH20 model is now the active model.