Detecting unwanted bias in engagement policy conditions
7 Tasks
15 mins
Scenario
U+ Bank is currently cross-selling on the web by showing various credit cards to its customers.
The bank wants to run an ethical bias simulation in Pega Customer Decision Hub™ and identify any unwanted bias in the engagement policy conditions. If any bias is detected, the bank would like to adjust the bias threshold to allow slight bias to comply with their business regulations and requirements.
Use the following credentials to log in to the exercise system:
Role | User name | Password |
---|---|---|
Next-Best-Action Designer | NBADesigner | rules |
Your assignment consists of the following tasks:
Task 1: Prepare data set for simulation run
Run the PrepareSimulationData data flow to prepare the data set for the simulation run.
Note: The SampledCustomers_Inbound is not available in a persisted store. To initialize the customer data, first, run the PrepareSimulationData data flow.
If you already run the PrepareSimulationData in this exercise system as part of a previous challenge, you do not need perform this task.
If you already run the PrepareSimulationData in this exercise system as part of a previous challenge, you do not need perform this task.
Task 2: Create an ethical bias policy
Create an ethical bias policy to include the .Customer.Age and .Customer.Gender properties with the Data-Decision-Request-Customer-CDH class as the context.
Task 3: Create an ethical bias simulation run
Create an ethical bias simulation run to check unwanted bias in the engagement policy conditions.
Task 4: Examine the simulation results
Examine the simulation results to identify on which property bias was detected.
Task 5: Check engagement policy conditions
Check the engagement policy conditions to know which condition is causing the bias.
Task 6: Modify the ethical bias policy
Modify the ethical bias policy to increase the threshold of the Age property.
Task 7: Re-run the ethical bias simulation run
Re-run the ethical bias simulation to verify the changes made to the ethical bias policy.
Challenge Walkthrough
Detailed Tasks
1 Prepare data set for the simulation run
- On the exercise system landing page, click Pega CRM suite to log in to Customer Decision Hub.
- Log in to Customer Decision Hub as a Next-Best-Action Designer:
- In the User name field, enter NBADesigner.
- In the Password field, enter rules.
- In the navigation pane of Customer Decision Hub, click Data > Data Flows to view the list of data flows.
- Search for and then open the PrepareSimulationData data flow to prepare the data set used for simulations. This data is based on a Monte Carlo dataset, which is generated by the system.
Note: The Monte Carlo data set generates a mock data set. As a result, different simulation runs have different results.
- Click Actions > Run to initialize the customer data.
- Click Submit.
- Click Start to process the data, and then wait for the processing to complete.
Note: Notice that the prospect data is populated once the test run is complete.
- Close the data flow window.
2 Create an ethical bias policy
- In the header of Customer Decision Hub, click the Configuration icon, and then select Ethical Bias Policy to configure an ethical bias policy.
- In the Bias fields section, in the Context field, enter or select Data-Decision-Request-Customer-CDH.
- Click Add bias field to include the properties that you want to test.
- In the Add bias field window, in the Field field, enter or select .Customer.Age.
- Click Next.
- Leave the Do the numbers represent categories? set to No.
- In the Add bias field window, click Add bias field.
- Click Add bias field.
- In the Add bias field window, in the Field field, enter or select .Customer.Gender.
- Click Add bias field.
- Click the Bias threshold tab.
- In the Grow section, click Customer.Age.
- In the Customer.Age window, select 0 Gini (No shift).
- In the Grow section, click Customer.Gender.
- In the Customer.Gender window, select 1 Rate ratio (No shift)
- Click Save to save the ethical bias policy.
3 Create an ethical bias simulation run
- In the navigation pane of the Customer Decision Hub, click Simulation Testing to access the simulation tests.
- In the upper-right corner, click Create > Ethical bias to configure the simulation run of the ethical bias policy.
- In the Configure inputs section, next to Strategy, click Configure to select a strategy on which you want to run the simulation.
- In the Strategy dialog box, to the right of the NBA_Grow_Creditcards strategy, click Add.
- Click Apply to add the strategy.
- In the Configure inputs section, next to Audience, click Configure to select an audience on which you want to run the simulation.
- In the Audience dialog box, to the right of the SampledCustomers_Inbound audience, click Add.
- Click Apply to add the audience.
- Click the Edit icon to update the Simulation name field.
- In the Simulation name field, enter EthicalBias_NBAGrow.
- Click Done.
Note: The simulation results are output to the Insights data set.
- Click Submit and Run to run the simulation.
4 Examine the simulation results
- Once the simulation run is complete, click Review reports.
Note that bias is identified. - Click Bias report to learn which property has the bias identified and for which action.
- In the Bias report, click the Bias detected column, and then click Sort > Highest to lowest to sort the report by listing the actions with bias on the top.
Note: The bias is detected on Age.
- Close the report.
5 Check the engagement policy conditions
- In the navigation pane of Customer Decision Hub, click Next-Best-Action > Designer to open Next-Best-Action Designer.
- Click Engagement policy.
- In the Business structure section, click Grow > Credit cards.
- In the Customer actions section, verify that Age is defined as an eligibility condition.
Note: As Age is used as an eligibility condition, which cannot be ignored, the bias threshold for the property needs to be adjusted to allow little bias.
- Close Next-Best-Action Designer.
6 Modify the ethical bias policy
- In the upper right, click the Configuration icon, and then select Ethical Bias Policy.
- In the Grow section, click Customer.Age.
- In the Customer.Age window, select 0.1 Gini.
- Click Save to save the ethical bias policy.
7 Re-run the ethical bias simulation run
- In the navigation pane of Customer Decision Hub, click Simulation Testing to access the simulation tests.
- Click S-##### to open the ethical bias run.
- Click Actions > Restart simulation to re-run the ethical bias simulation.
- Click Submit to confirm the simulation run.
- Once the run is complete, click Review reports.
Note that there is no bias detected now. - Open the Bias report.
- Verify that the Bias detected field is false for all the actions.
This Challenge is to practice what you learned in the following Modules:
Available in the following missions:
If you are having problems with your training, please review the Pega Academy Support FAQs.
Want to help us improve this content?