Monitoring predictions

Archived

5 Tasks

15 mins

Pega Customer Decision Hub 8.4

Pega Customer Decision Hub 8.8 Pega Customer Decision Hub 8.7 Pega Customer Decision Hub 8.6 Pega Customer Decision Hub 8.5

Visible to: All users

Beginner Pega Customer Decision Hub 8.4 English

Scenario

In the Summer Sales marketing promotion brief, U+Bank has adopted predictive analytics. The sales manager wants you to report on the uplift of the adaptive models that are employed in the one-to-one customer engagement strategy. To measure the uplift, you modify the prediction strategy to take a random action to a small percentage of the customers as a benchmark.

Use the following credentials to log in to the exercise system:

Role	Username	Password
Data Scientist	DataScientist	rules

Your assignment consists of the following tasks:

Task 1: Train the models

Train the MeasureLift adaptive models by running the MakeDecision and CaptureResponses data flows. This warms up the system by capturing the customer responses to all actions.

Task 2: Confirm model creation

Confirm that four models are generated for the credit card actions.

Task 3: Adjust the strategy

Adjust the PredictionStrategy to take a random action for 10% of the customers.

Task 4: Simulate customer interactions

Run the MakeDecision and CaptureResponsesIH data flows to simulate customer interactions and store the data in Interaction History for reporting on the .pyJourney property.

Task 5: Report on uplift

Report on the uplift of the models.

Challenge Walkthrough

Detailed Tasks

1 Train the models

1. Log in as Data Scientist with user name DataScientist and password rules.
2. In the navigation pane on the left, click Data > Data Flows.
3. On the Data Flows landing page, click the MakeDecision data flow.
4. In the upper right, click Actions > Run and wait until the run finishes.
5. Scroll down and check that for each of the 10.000 customers in the source dataset four decisions are made, resulting in 40.000 records in the destination component.
6. Return to Customer Decision Hub.
7. In the navigation pane on the left, click Data > Data Flows to open the Data Flows landing page
8. On the Data Flows landing page, double-click the CaptureResponses data flow.
9. In the upper right, click Actions > Run and wait until the run finishes.
10. Check that 40.000 responses are captured.

2 Confirm model creation

1. In the navigation pane on the left, click Intelligence > Prediction Studio and open the MeasureLift model.
2. Click Refresh reporting data to update the models.
   
3. Confirm that each action has received 10.000 responses.

3 Adjust the strategy

1. Return to Customer Decision Hub.
2. In the navigation pane on the left, click Intelligence > Strategies
3. On the Strategies landing page, click SummerNBAStrategy.
4. Right-click the Prediction Strategy component, and then select Open strategy.
5. Check out the strategy.
6. On the strategy canvas, add a Set property component.
7. Configure the component with the following information:
   1. Name: Random Propensity
   2. Set the .pyPropensity property to equal @Random.random().
   3. Click Submit.
8. On the strategy canvas, copy and paste the Model Prioritization component.
9. Configure the component with the following information:
   1. Name: Random Prioritization.
   2. Set the .pyJourney property to “Random”.
   3. Click Submit.
10. On the strategy canvas, add a Champion challenger component.
11. Configure the component with the following information:
    1. Name: Challenge Model
    2. Select Model Prioritization in 90% of the cases and Random Prioritization in the remaining 10%.
    3. Click Submit.
12. On the canvas, connect the components to reflect that the Model Prioritization component is used for 90% of the actions, and the remaining 10% use the Random Prioritization components. The strategy should look like the following image:
    
13. On the right, open the Test Run pane.
14. Click Settings.
    1. In the Data transform field, enter or select Troy.
    2. For external inputs, select SummerNBAStrategy.
    3. Select the Specify as single component within the strategy check box.
    4. In the Component list, select CreditCards.
    5. Click Save & Run.
15. Under Show component level values for, select Propensity.
    
16. Check that the Random Propensity component overrides the propensity determined by the model and that the Champion Challenger component propagates one of these two propensities (propensity values may vary).
17. Check in the strategy with appropriate check-in comments, and then close the canvas.
18. Check out the SummerNBAStrategy.
19. On the canvas, open the Prioritize component and change the output to Top 1 to simulate customer interactions.
20. In the Test Run pane, click Settings.
21. In the Data transform field, enter or select Troy.
22. Click Save & Run. Notice that a single action is taken for Troy.
23. Check in the strategy with appropriate Check-in comments.

4 Simulate customer interactions

1. In the navigation pane on the left, click Data > Data Flows.
2. Open the MakeDecision data flow.
3. Click Run.
4. Confirm that for each of the 10.000 customers in the source dataset a single action is taken.
5. Repeat steps 1-3 for the CaptureResponsesIH data flow. Confirm that 10.000 responses are captured.

5 Report on uplift

1. Return to Customer Decision Hub.
2. In the header of Customer Decision Hub, click the Report Browser icon to show recent reports.
3. In the Public categories section, click Interaction History.
4. Open the Uplift report. Notice the uplift of the models. In this case, the uplift (the increase in the success rate) is around 65% (percentage may vary due to the randomization factor).
   

---

---

Available in the following mission:

• Data Scientist v1