Use Case - Setting up data ingestion
2 Tasks
20 mins
Scenario
The U+ Comms data warehouse team created the subscription and account data files, and their respective manifest files. IT built the technical infrastructure to upload the data daily to a repository that Pega Customer Decision Hub™ can access. A token file will initiate the data ingestion process.
Prepare the import data jobs to populate the subscription and account tables in Customer Decision Hub.
The IT development team has provided access to the file repository through File Browser, a third-party application. Use File Browser to browse, review, and create new files in the repository.
Tip: A variation of this scenario exists in Import Data Jobs.
Detailed Tasks
1 Other details
- As the Decisioning Architect, create two different data jobs with Profile Designer:
- Subscription data
Requirement Detail Data Job Name
Import Subscription
Target Profile Data source
Subscription
Repository
filerepo
File path
/IngestionData/Subscription/
Manifest file name
SubscriptionManifest.xml
Data file name
SubscriptionData.csv
CSV file details
First row contains fields (header)
Delimiter character
Comma ( , )
Quotation marks
None
Date field formats
Date time format: MM/dd/yyyy HH:mm
Date format: MM/dd/yyyy
Time format: HH:mm:ssImport using
File Detection
Failure policy
10 records per file.
- Account data
Requirement Detail Data Job Name
Import Account
Target Profile Data source
Account
Repository
filerepo
File path
/IngestionData/Account/
Manifest file name
AccountManifest.xml
Data file name
AccountData.csv
CSV file details
First row contains fields (header)
Delimiter character
Comma ( , )
Quotation marks
None
Date field formats
Date time format: MM/dd/yyyy HH:mm
Date format: MM/dd/yyyy
Time format: HH:mm:ssImport using
File Detection
Failure policy
10 records per file.
Caution: Ensure that the date and time formats are correctly set when you create the data jobs. If an error occurred during the data ingestion, you can reuse the files from the ArchivedData folder by using the File Browser.
- Subscription data
- For this use case, you have access to a third-party application, File Browser. With this application, you can browse, upload, and download files from the file repository. On the exercise landing page, click File Browser to log in with the following credentials and browse the files, then create new token files in the repository:
- Username: pega-filerepo
- Password: pega-filerepo
- To begin data ingestion, create a new .tok file in the IngestionData/Subscription folder. Then, create another .tok file in the IngestionData/Account folder to initiate the data job for Account data, as shown in the following figure:
Tip: To create a new file, click New file in the left navigation pane. After you create the new .tok file, click Save in the upper-right corner.
2 Acceptance criteria
- A new data job is created to ingest subscription data.
- A new data job is created to ingest account data.
- Both data jobs are triggered by a token file.
- Both subscription and account data sources are populated with data.
Available in the following mission:
If you are having problems with your training, please review the Pega Academy Support FAQs.
Want to help us improve this content?