Enhancing entity extraction with machine learning

5 Tasks

25 mins

Pega Customer Decision Hub '24.1

Pega Customer Decision Hub '23 Pega Customer Decision Hub 8.8 Pega Customer Decision Hub 8.7

Visible to: All users

Beginner Pega Customer Decision Hub '24.1 English

Scenario

The U+ Air chatbot channel detects U+ Air ticket numbers using a RUTA-based model. The current entity model recognizes a ticket number in the following format: two letters followed by three digits (for example, WZ266 or AA132). In case of a rescheduled ticket or human error, the application also needs to detect unusual ticket number patterns. Running a RUTA script with machine learning can achieve this business outcome.

As a data scientist, enhance entity extraction with a machine learning model to satisfy the business requirements.

Use the following credentials to log in to the exercise system:

Role	User name	Password
Data Scientist	NLPDataScientist	rules
Application Developer	NLPApplicationDeveloper	rules

Your assignment consists of the following tasks:

Task 1: Test the text extraction in the chatbot for unusual ticket number

As an application developer, test the entity extraction for ticket numbers different than the default pattern (two letters followed by three digits). Test the chatbot for the following message: I want to cancel my ticket number WZ-266.

Task 2: Train the machine learning entity extraction model by record

As a data scientist, add training data manually by record to train the ticket_number entity extraction machine learning model so that it uses machine learning to detect entities.

Task 3: Train the machine learning entity extraction model with a dataset

Import the Airlines_entity_dataset.xlsx dataset as training data to the ticket_number entity extraction model. Train the entity model with all the available data.

Task 4: Test the entity extraction

Test the entity extraction model, and then observe the results.

Task 5: Test the text extraction in the chatbot after building the model

As an application developer, test the entity extraction model, and then observe the results.

 

You must initiate your own Pega instance to complete this Challenge.

Initialization may take up to 5 minutes so please be patient.

• Log In

Challenge Walkthrough

Detailed Tasks

1 Test the text extraction in the chatbot for unusual ticket number

1. On the exercise system landing page, click Launch Pega Infinity™ to log in to App Studio.
2. Log in to App Studio as an application developer:
   1. In the User name field, enter NLPApplicationDeveloper.
   2. In the Password field, enter rules.
3. In the navigation pane of App Studio, click Channels to view the list of current channels.
4. In the Current channel interfaces section, click the icon that represents your existing Airline Digital Messaging channel.
   
5. In the Preview console on the right, in the Type your message here text box, enter I want to cancel my ticket number WZ-266 to test the chatbot.
6. Turn on the Show analysis switch to see the details.
   
7. Click Yes.
8. Confirm that the chatbot detects the cancel ticket topic with high confidence and runs a preconfigured Cancel a ticket case type but does not recognize the ticket number due to an unusual pattern. The chatbot requests the ticket number even though it is provided in the first message.
   
9. In the lower-left corner, click the user icon, and then select Log off to log out of App Studio.

2 Train the machine learning entity extraction model by record

1. Log in to Prediction Studio as a data scientist:
   1. In the User name field, enter NLPDataScientist.
   2. In the Password field, enter rules.
2. On the Predictions landing page, click Airline to open the prediction workspace.
   
3. Click the Entities tab to view the list of entities.
   
4. In the ticket_number row, click the Gear icon to configure the machine learning data.
   
   1. In the ticket_number dialog box, click Add training data to add a new piece of data to the ticket_number entity.
   2. In the text box, enter I want to cancel my reservation for ticket number JK-294.
      
   3. Click Add.
   4. On the right, in the preview pane, select JK-294.
   5. Right-click JK-294, and then select #ticket_number.
      
   6. Click Save.
   7. In the ticket_number row, in the Total training data column, click 0 and confirm that there is 1 reviewed feedback, as shown in the following figure:
      

3 Train the machine learning entity extraction model with a dataset

1. Download the Airlines_entity_dataset.xlsx dataset.
2. Open, and then inspect the downloaded dataset:
   1. Confirm that the entity in the training data is <START:ticket_number> LO-127 <END>.
      
      Note: The ticket number has a single space in front of the two letters, and double space after the three numbers.
   2. Close the Airlines_entity_dataset.xlsx file.

3. In the list of entities, in the ticket_number row, click the Gear icon.
   
   1. In the ticket_number dialog box, click Upload to open the dataset upload dialog box.
      1. Click Choose File, and then select the Airlines_entity_dataset.xlsx file.
         
      2. Click Upload.
   2. In the ticket_number dialog box, confirm that the status of the newly added entities is Reviewed, as shown in the following figure:
      
      Note: There are seven pages of new training data to view. You can inspect and edit every record similarly to manually-added training data.
   3. Click Save.

4. Note the new available reviewed feedback:
   
5. In the upper-right corner of the Airline prediction workspace, click Build to build the model.
6. In the Build models dialog box, select the Airline_entities model:
   
7. Click Build to build the model.
   Note: Building process may take up to few minutes. You will see a green information ribbon at the top of the Prediction window after the process is completed. If the green information ribbon does not appear after few minutes, In the top right of the prediction window click Actions > Refresh.

8. Once the build is completed, in the top of the prediction window, click View build report.
   
   1. In the Model training report window, review the build result.
   2. Click Close to close the Model training report window.

4 Test the entity extraction

1. In the upper-right corner of the Airline prediction workspace, click Test.
   1. In the Test prediction dialog box, in the text box, enter I want to cancel my ticket number WZ-266.
      
   2. Click the Entity tab.
   3. Confirm that the test correctly identifies WZ-266 even though it is not part of the training dataset.
   4. In the Test prediction dialog box, enter I want to cancel my ticket AAL325.
   5. Confirm that the test correctly identifies AAL325 as a ticket number even though it is not part of the training dataset.
   6. Close the Test prediction dialog box.
2. In the upper-right corner of the Airline prediction workspace, click Save.
3. In the lower-left corner, click the user icon, and then select Log off to log out of Prediction Studio.

5 Test the text extraction in the chatbot after building the model

1. Log in to App Studio as an application developer:
   1. In the User name field, enter NLPApplicationDeveloper.
   2. In the Password field, enter rules.
2. In the navigation pane of App Studio, click Channels to view the list of current channels.
3. In the Current channel interfaces section, click the icon that represents your existing Airline Digital Messaging channel.
   
4. In the Preview console on the right, in the Type your message here text box, enter I want to cancel my ticket number WZ-266 to test the chatbot.
5. Turn on the Show analysis switch to show the details.
   
6. Confirm that the chatbot recognizes WZ-266 as a ticket number.
7. In the upper-right corner of the Preview console, click Reset.
8. In the Type your message here text box, enter I want to cancel my ticket number AAL325 to test the chatbot.
   
9. Confirm that the chatbot recognizes AAL325 is recognized as a ticket number.
10. Click Yes to initiate the routing of the case.

---

This Challenge is to practice what you learned in the following Module:

• Using entity extraction with chatbot channel v4

---

Available in the following mission:

• Pega NLP Essentials v4