Voice AI models

The Speech Service component of the Voice AI architecture transcribes audio from the customer conversation in real time. The Speech Service, as shown in the following diagram, uses an AI model to detect and transcribe the audio content:

Pega Voice AI natively supports a set of built-in languages. To extend transcription and analysis to additional languages, customers can connect Voice AI to Amazon Web Services (AWS) Transcribe. This integration enables Voice AI insights for any language that AWS Transcribe classifies as having streaming support.

The following figure shows the Voice AI architecture with Amazon Transcribe:

Transcription Engine Options

Voice AI supports two transcription engines, selectable per interaction using the GetVoiceAISpeechEngine decision table:

The engine selection can be driven dynamically by business logic. Both options can coexist in one implementation, with the decision table routing each call to the appropriate engine.

Language Selection options

The GetVoiceAILanguage decision table determines which language is applied per interaction. If this table returns an empty outcome, Voice AI falls back to the default language set in the CRM – Voice AI configuration set (en_US by default). The language can be resolved from the following sources:

These same five sources also apply to transcription engine selection -- each has a corresponding decision table example for engine routing.

Key Configuration components for using AWS Transcribe

AWS Transcribe Service Limits

By default, AWS has a streaming limit in Voice AI is 25 HTTP/2 streams, supporting approximately 12 concurrent calls. To support Voice AI demand, you need to scale up you AWS environment:

• Number of concurrent HTTP/2 streams = 2× your required concurrent calls.
• Transactions per second (StartStreamTranscription) = 2× the HTTP/2 stream quota.

Quota increases are requested directly in the AWS Service Quotas console.

For more information, see Implementing multi-language support.

The Pega NLP component of the Voice AI architecture analyzes the customer conversation to identify topics and entities, and then pushes suggestions to CSRs in real time in the Interaction Portal. Pega NLP is part of the Pega Server, as shown in the following diagram.

When you create a Voice AI channel, a Text Prediction is generated. The Text Prediction includes models for topic, sentiment, and entity extraction. The following screen shows the models used with a sample Voice AI channel.

The topic models analyze the customer conversation to suggest the appropriate case type to the CSR.

Sentiment models detect the emotion expressed in a conversation as generally positive or negative.

The entity models detect words in the customer conversation that can be associated with case data, such as a name or address.

You can use the Prediction Studio to modify and train models. In most cases, a data scientist maintains the models.

Training data

You can help train a topic model by adding records to the model. Each training record is a combination of a topic and an associated word or phrase relevant to your organization. You classify each record by associating it with the appropriate topic. For example, the following screen shows a training record from a typical customer conversation.

The NLP analysis for the selected record shows the identified entities, such as street address and city. This record is associated with the address change topic

You can review the NLP analysis and mark records as reviewed.

You can monitor a model for accuracy, rebuild a model, and export or import the models across different Pega environments. Each time you rebuild the model, Voice AI calculates a new score, which represents the overall accuracy of the model for all topics.

For more information about configuring and training NLP models, see the Pega Academy mission Pega NLP Essentials.