Challenging a Predictive Model

1. Download the ValidationData, ChurnBaseline, ChurnImproved files to a local drive on your computer.
2. Extract ValidationData.zip folder to extract the CSV file.

1. On the exercise system landing page, click Pega Infinity™ to log in to Customer Decision Hub.
2. Log in as a Data Scientist:
   1. In the User name field, enter DataScientist.
   2. In the Password field, enter rules.
3. In the navigation pane of Customer Decision Hub, click Intelligence> Prediction Studio.
4. In the header of Prediction Studio, click Prediction Studio > Infinity Studio.
5. In the header of Infinity Studio, click Create > Data Model > Data Set.
   
6. On the Create Data Set page, configure the new Data Set with the following information:
   1. In the Label field, enter ValidationData.
   2. In the Type list, select File.
   3. In the Apply to field, enter or select UBank-CDH-Data-Customer.
      
7. Click Create and open to create the Data Set.
8. On the File tab of the Data Set, in the File location section, select Embedded file.
9. In the File Management section, click Upload file.
   1. In the Upload file dialog box, click Choose File.
   2. Select the ValidationData.csv file, and then click Upload.
10. In the upper-right corner, click Save.
11. In the header of Infinity Studio click Create > Data Model > Property.
    
12. On Create Property page, complete the following details:
13. In the Label field, enter Outcome.
14. In the Apply to field, enter or select UBank-CDH-Data-Customer.
15. Click View additional configuration options.
16. Verify Data type is set to Text by default.
17. Click Create and close.
    

1. In the header of Infinity Studio, click Infinity Studio > Prediction Studio to switch workspaces.
2. On the Predictions page, click New to create a new prediction.
3. In the Create a prediction dialog box, confirm Customer Decision Hub is selected and click Next.
   
4. In the Create a prediction dialog box, complete the following steps.
5. In the Prediction Name field, enter Predict Churn Risk.
6. In the Outcome field, select Churn.
7. In the Object field, select Customer.
   

5. Click Create.

6. On the Prediction: Predict Churn Risk page, navigate to Models tab.

7. In the Churn section, click more icon on Predict Churn Risk row.

8. Click Replace Scorecard and complete the following steps:

1. In the Introduce challenger model dialog box, select Model and then click Next.
2. Confirm that Upload model file is selected.
3. Click Choose File, and upload ChurnBaseline.zip file you downloaded.
4. Click Next.
5. Ensure that Compare models is not selected and click Next.
6. In the Candidate model name field enter ChurnBaseline.
7. Click Add Challenger Model.
   Nota: It will take 1-2 minutes to load, wait until the status of the model shows Challenger (Pending review).

9. On Predict Churn Risk landing page, under Churn section, click Churn Baseline (M-XX).

10. On the Model review: ChurnBaseline page complete the following steps:

1. Click the Mapping tab.
2. Verify that the predictors (RelationshipLengthDays, TotalLiabilities, DebtToIncomeRatio, Gender, Age, CLV, CreditScore, AnnualIncome) are auto mapped.
3. Click Approve.
   
4. In the Approve Challenger model dialog box, in the comment field enter Baseline H2O model with 8 features for churn prediction and click Approve.

1. On the Models tab, in the ChurnBaseline row, click the more icon, and then select Introduce challenger model:
2. In the Introduce challenger model dialog box, complete the following steps:
   
   1. Ensure that Upload model file is the active selection, and then click Choose file.
   2. Select the ChurnImproved.zip file, and then click Next.
   3. Select the Compare models checkbox to run a validation test of the active model against the candidate model.
   4. In the Validation data set list, enter or select ValidationData.
   5. In the Outcome column field, select Outcome, and then click Next.
      
   6. In the Candidate model name field, enter ChurnImproved.
   7. Click Add challenger model.
      Nota: This takes 1-2 minutes to complete. Wait until the model status turns to CHALLENGER (PENDING REVIEW).

3. In the Churn section, when the status of the model changes to CHALLENGER (PENDING REVIEW), click ChurnImproved(M-XXX).
4. Review the model comparison, and then click Approve.
   
5. In the approval dialog, select the Approve and start champion/challenging radio button.
6. In the comment field enter Approve for champion/challenging mode - testing improve mode and click Approve.