Aggregated data
Data is a critical component that spans the entire case management lifecycle. It enables case workers and participants to make informed decisions. With advancements in technology and a growing user base, data has become increasingly complex and voluminous. Data is generated from various sources and segregated into different systems, where it is persisted after processing according to business requirements. To effectively gather, analyze, summarize, and present this data, a robust data aggregation mechanism is necessary.
As a Lead System Architect (LSA) for an enterprise Pega implementation, it is critical to the optimal methods for data aggregation within Pega applications. Some of the available options are:
- Aggregation of data in reports
- Aggregation of data by using Data Transforms
- Aggregation of Data Pages
- Aggregation of data by using Data Sets and Data Flows
Aggregation of data in reports
Pega Report Definitions offer a straightforward way to query data and present it to case workers. They can use joins, associations, or sub-reports to aggregate data from different sources or tables and present it in a single sheet. Joins and associations focus on aggregating data from various sources or tables, while sub-reports aggregate data from different reports. If you have multiple reports that meet specific business needs and require a consolidated report, sub-reports are the optimal choice.
For example, consider reports on customer purchase history, customer engagement levels (loyal, high-spender, infrequent shopper), and customer demographics. If you need a report for customer demographic segmentation based on engagement levels for a marketing campaign, you can aggregate data from all these reports into a main report. The existing reports function as sub-reports within the new main report for customer segmentation.
Aggregation of data in data transforms
Pega Data Transforms are designed for straightforward data transformation and manipulation. The Append to and Append and Map to options enable the creation of aggregated data lists. These options are particularly useful when you have data spread across various pages and need to consolidate it into a single page.
Using a combination of main reports and sub-reports is beneficial for aggregating data through queries from different sources. In contrast, the append options within data transforms are effective when the target page already exists. This existing page could be a user page or a data page, making the append options a straightforward solution.
Aggregate Data Pages
Aggregate Data Pages are an option for aggregating data from multiple sources. They are especially useful for data collection with complex requirements. While combinations of reports, sub-reports, or data transforms are suitable for simpler business scenarios, Aggregate Data Pages are excellent in more complex situations. Aggregate Data Pages are particularly beneficial when retrieving data from external sources using connectors or robotic automation. They can also aggregate data from internal sources, for example Activities, Data Transforms, Report Definitions, and Lookups. Aggregate Data Pages are advantageous when you need to consolidate data from multiple sources into a single page. They support both list and page structures, making them versatile for various data aggregation needs.
When designing Aggregate Data Pages, LSAs should consider the following:
- Asynchronous Loading: If the data page is fed by numerous sources and the data volume is large, implement asynchronous loading to improve performance.
- Keyed Page Access: Using keyed page access can help avoid multiple traversals of the sources defined for aggregation, which enhances efficiency.
- Refresh Strategy: Clearly define the refresh strategy for Aggregate Data Pages to prevent unnecessary reloading, which can negatively impact application performance.
Aggregation of data using Data Sets and Data Flows
Data Sets are designed for managing complex data structures efficiently and can help you more effectively organize and manage your data. One type of Data Set is the Summary Data Set, which aggregates various types of data to refine it for use in decision strategies, models, or Data Flows. Summary Data Sets source their data from Stream Data Sets or Data Flows with a stream source and an abstract destination.
Data Flows function like pipelines. They enable you to sequence and combine data from various sources and write the results to a destination. The source and destination points can be abstract or driven by Data Sets and other decision data flows. When you use Data Flows to aggregate data from different sources and move it to a destination, between the source and destination, you can apply various operations. For example, composing, converting, merging, and running other strategy instructions.
Data Flows support the combination of data from two sources into a single page or page list, ensuring that all necessary data is consolidated into one record. To combine data, you identify a matching property between the two sources. The system appends data from the secondary source to the incoming data record as an Embedded Data Page.
Both Data Sets and Data Flows can be used for data aggregation in complex use cases. As an LSA for an enterprise Pega implementation, you should use the most optimized design approach.
Check your knowledge with the following interaction:
This Topic is available in the following Module:
Want to help us improve this content?