Skip to main content

Running an audience simulation test

You can improve the performance of your next-best-action strategy by testing the engagement policy configuration against a set of customers. In this way, you can check how many potential actions are filtered out by each component of the policy and discover how a criterion might affect your requirements.

Audience simulation


This demo will show you how to run an audience simulation test. It will also explain how many potential actions are filtered out by each section of the engagement policy and reveal if a particular criterion is too broad or too narrow for your requirements.

U+, a retail bank, has recently implemented a project in which credit card offers are presented to qualified customers when they log in to the web self-service portal. The bank ran a value finder simulation to find under-engaged customers. In that simulation run, a set of customers is identified with no actions after engagement policy conditions are applied.

Value finder results
Value finder opportunities

The bank would now like to run an audience simulation to investigate why these customers have no actions presented to them, and correct the condition that causes the deviation.

This is the Pega Customer Decision Hub™ portal. First, take a look at the set of eligibility criteria that has already been configured by U+.

In Audience simulations, you can create a new simulation run.

Audience simulation menu
Create audience simulation

To create an audience simulation run, select an audience on which you would like to do the simulation. The audience is a list of potential target customers. You can modify the simulation name as required to easily identify the specific runs. Then, choose the scope of the run. That is, you can choose to simulate only the engagement policies to validate the eligibility, applicability and suitability conditions. Or, you can simulate on both engagement policies and arbitration to understand how the conditions work when arbitration across all actions is also considered.

New Auidence simulation window

Once the simulation run is complete, you can view the details of how the audience is filtered at the group level based on the configured engagement policy conditions.

Audience simulation in progress

For each component of the engagement policy, the simulation test shows a numerical or percentage value of the audience that will receive the action based on current criteria. For example, if the result of a criterion that checks if the action is active returns 100%, the component did not filter out any audience members.

The filtering process that happens in this simulation is similar to a funnel filtration for every engagement policy condition type. However, within an engagement policy condition type, each of the criterion is applied separately to the corresponding input audience.

In this case, the total number of customers in the audience is 1000. For this audience, the number of customers who received offers is 182. This simulation run also provides details of the audience filtration that happens with each engagement policy condition. When the eligibility condition is applied, 979 customers qualify for the offers. Within the eligibility criteria, the first condition does not filter any customers, as all qualify. When the second condition is applied, only 979 customers pass through. Thus, the intersection of these two conditions, 979, is the final number of customers who pass through to the eligibility level.

In this case, there are no eligibility conditions defined at the action level. If there are, the final result is the result of the eligibility conditions at the group level, combined with the results at the action level.

The output population of the eligibility level is passed on as the input to the next level.

The input population considered for the applicability condition is therefore 979. When the applicability condition is applied, 344 customers qualify for at least one offer out of the 979 customers. Note, since there are no specific action-level applicability conditions, the result of the individual group-level condition is the same as that of the overall applicability component level, 344.

The output population of the applicability level, 344 customers, is passed on as the input to the suitability level.

When the Suitability group-level condition is applied, 215 customers qualify for the offers out of the 344 customers. If you check the engagement policies at the action level, notice that there are additional suitability conditions. When all of these conditions apply, the total number of customers who qualify for at least one offer is 182.

Audience simulation result

At the bottom, you can also view the number of customers who qualify for the offers. When only engagement policy conditions are considered in the simulation run, the number you see here is the number of customers who qualify for at least one action. Notice that the Premier Rewards card offer is rarely presented. Let's try to find out why this is so.

Open the Premier Rewards card to view the audience simulation filtering at the action level.

Audience simulation result per action

You can choose the same audience simulation to view the action-level filtration details and investigate which engagement policy condition is causing the current outcome.

Running audience simulation at action level
Selecting the audience simulation at action level

These numbers show how the engagement policies that are inherited from the CreditCards group are filtered. At the bottom, you can also view the final number of customers who qualify for the specific offers once the action-level engagement policy is applied.

Notice that at the suitability-level, only eight customers qualify for the Premier Rewards card offer.

Action level audience simulation results

This is because the configured engagement policy criterion is too narrow. The results of this simulation show us that from the 215 customers who passed the eligibility and applicability conditions, all qualify for the group-level condition, but only eight customers qualify for the action-level condition. As the action-level suitability condition is too narrow, a very small number of customers are presented with the offer. The bank then decides to reduce the CreditScore from 825 to 750 to ensure the Premier Rewards card offer is also presented to the customers. 

Now, modify the credit score condition to reflect the correct value. Then, rerun the same audience simulation at the group level. Notice that now, 35 customers are presented with the Premier Rewards card offer.

NBA level action simulation results

Let’s now run an audience simulation with engagement policies and arbitration in scope.

When you select engagement policies and arbitration as the scope, the Include a second-pass simulation without business levers (for Scenario Planner) check box is selected by default. When this option is selected, two simulations run in parallel. One simulation is based on your configuration in the Next Best Action Designer, with levers. The second simulation is based on your configuration in the Next Best Action Designer, without any levers. This simulation is used to determine the opportunity cost that is associated with applying levers. You can then use these two simulations in the Scenario Planner simulation later for analysis. It is a best practice to leave this option selected.

Scenario planner second pass simulation option

Note that the simulation results are different. This is because the simulation includes arbitration, adaptive analytics, treatment and channel processing, and constraints. That is, this simulation result shows the number of customers who will receive an action as their top action. Thus, the numbers at the group level and the action level tally.

That is, the sum of all customers (12+118+22+30) is 182.

This demo has concluded. What did it show you?

- How to configure an audience simulation.

- How to view the simulation filtering details at the group and action level.

Decision funnel explanation simulation


This demo shows you how a decision funnel explanation simulation is auto-created for every audience simulation that is run at the Next-Best-Action Designer level.

Every time an audience simulation is run, a decision funnel explanation simulation is created in the background. Now, navigate to the Simulations landing page to view the auto-created funnel simulation.

Simulation testing option

This is the auto-created funnel simulation. Alternatively, you can also create a simulation run from here and show the simulation results in the Engagement policies > Audience Simulation.

Open a simulation

To view the funnel simulation, open the simulation.

Notice that the NBA_Sales_CreditCards strategy is used to run the simulation. This strategy is used is because you selected Engagement only in the audience simulation, without the arbitration.

Simulation strategy

When a funnel simulation is created, a set of reports is added to the run results.

The simulation reports provide detail about the filtering that occurred.

Decision Funnel report

You can download these reports offline to analyze the outcome.

Decision funnel simulation report view

If you do not want to re-run the simulation in the future, you can wrap up the simulation. Any attached reports remain available for review.

Simulation wrap-up

All the resolved simulations are accessible from the Include resolved simulations section.

Resolved simulations

This demo has concluded. What did it show you?

- How to view the auto-generated decision funnel explanation simulation

This Topic is available in the following Modules:

If you are having problems with your training, please review the Pega Academy Support FAQs.

Did you find this content helpful?

Want to help us improve this content?

We'd prefer it if you saw us at our best.

Pega Academy has detected you are using a browser which may prevent you from experiencing the site as intended. To improve your experience, please update your browser.

Close Deprecation Notice