Use analytics to review your entire assistant at a glance
The Analyze page provides a summary of the interactions between users and your assistant.
Analytics help you to understand the following things:
- What do customers want help with today?
- Is your assistant understanding and addressing customer needs?
- How can you make your assistant better?
To see analytics information, select Analyze in the navigation bar.
You can search and analyze the conversational search requests from the customer in the Analyze page.
Choosing the environment and date range
- Go to Analyze > Conversations.
- Click the filter icon
to display the filter options. - Choose the options for the environment, date range, recognition type and action fields to analyze them in detail.
All charts and cards reflect data based on the environment and the date range you select. When you change the environment or the date range, the charts and cards on the page update to reflect the new date range. You can also use Refresh to ensure the latest data is shown.
Unique users, conversations, and user requests
These three traffic metrics -- unique users, conversations, and user requests -- provide you with data about the volume of user engagements with your assistant.
Unique user is anyone who interacts with your assistant. User ID identifies each user, using a unique label to track the level of service usage. A unique user can have multiple conversations, but a conversation never has more than one unique user.
Conversation is a set of messages consisting of the messages that an individual user sends to your assistant, and the messages your assistant sends back. Conversations can have multiple requests within a single conversation, but a single request doesn't span more than 1 conversation.
Request is a root-level utterance, such as an initial question or request, that signals the start of a specific flow. A user can initiate multiple requests. Requests are meant to represent the core concepts or topics your users are
asking about. A request can have multiple steps within it, for example I want to order a pizza is a request. Delivery/takeout, Small/Medium/Large, Cheese/Pepperoni/Mushrooms/Peppers are all
steps within the request of ordering a pizza.
Action completion, conversational search and recognition
The action completion, and recognition charts provide information about the actions in your assistant. The conversational search shows the confidence scores and other metrics for the overall assistant and individual conversational search requests.
Action completion section
The Action completion section in the analytics page shows how often users are able to successfully get through all the steps of an action.
Your assistant measures when someone reaches the final step of an action. The action completion chart provides an overview of all the actions built, showing how many actions are completed and how many are not.
Action completion applies only when a user's question or request matches an action, and the action starts.
One action can be triggered multiple times. To better understand individual action performance, click Action completion.
Conversational search section
The Conversational search section in the analytics page shows the performance metrics of your assistant's conversational search functions for a selected period. It also shows the detailed data for each individual conversational search query over a specified timeframe. The analytics of custom conversational search calls through custom extensions are not gathered in the conversational search analytics page.
Click Conversational search to open the conversational search chart. The Conversational search chart provides the percentage of response confidence, retrieval confidence, and extractiveness. It also shows the number of
citations per response and the response length in characters. Click the filter icon and choose the type of response that you want to display from the drop-down menu.
For more information on performance metrics, see Conversational search analytics
Recognition section
The Recognition section in analytics shows the measurement of the requests that are being recognized by the assistant and routed into starting an action.
The recognition chart provides you with a view into how many requests are matched to actions. This helps you to understand where you may have content gaps, where you might want to build new actions, or how existing actions aren't matching properly to user requests.
Customer requests are considered unrecognized if:
- The request triggers the No matches action
- The assistant asks a clarifying question and the customer chooses
None of the above
For more details, click Unrecognized requests to review the requests that are not being recognized by the assistant, so you can create new actions that address questions and issues that aren't being answered.
Most frequent, least frequent, and least completed
These most/least cards help you to quickly identify specific actions that might need your attention. From these lists, you can click a specific action to do more analysis.
Most frequent actions shows the actions that have the most recognized requests matched with that action.
Least frequent actions helps you to identify the actions that are possibly poorly trained and are not matching well to actual requests from your users.
Least completed actions shows a list of actions with the lowest completion rate, by percentage. This measurement does not take the traffic into account. This measurement can be another good indication of actions where the flow or content can be improved. You can spend more time investigating these actions.