Enhancing agents with rich responses
Enhance user interactions by designing agents that return structured, visually rich responses instead of relying only on plain text. Rich responses help users understand information faster, reduce ambiguity, and make it easier to take the next step.
In chat experiences, rich responses turn conversations into guided experiences. Rather than asking users to interpret long text or remember instructions, the UI itself communicates what the system knows and what the user can do next.
Rich responses can originate from:
- Tools: Tools can return interactive UI widgets directly in their responses. Python, OpenAPI, and MCP tools embed widget contracts that define forms and other interactive elements. These widgets allow users to enter data, make selections, and trigger follow‑up actions without leaving the conversation.
- AI assistants: Assistants return rich responses that are configured in conversation steps. When an agent routes a user message to an assistant, the assistant responds with the rich element associated with the triggered step.
When to use rich responses
Use rich responses when plain text creates friction for users or increases system complexity. Read about the common scenarios where you can use rich responses to reduce ambiguity, support user understanding, and improve the interaction experience.
Users can easily understand structured information
Users benefit from rich responses when information contains multiple items or attributes that need to be scanned, compared, or interpreted. Common user scenarios include:
- Reviewing lists of results.
- Comparing options.
- Scanning summaries with multiple fields such as status, date, owner.
Users can clearly see what actions are available
Users benefit from rich responses when the system expects them to take a next step. Common user scenarios include:
- Choosing from predefined options.
- Confirming or canceling an action.
- Navigating to details or follow‑up steps.
Users are less likely to make mistakes
Rich responses help users avoid errors when only certain inputs are valid. Common user scenarios include:
- Selecting accounts, environments, or configurations.
- Approving or rejecting changes.
- Choosing dates, statuses, or categories.
Users complete tasks faster with fewer steps
Rich responses improve efficiency when users want to reach an outcome quickly. Common user scenarios include:
- Frequently repeated tasks.
- Time‑sensitive interactions.
- Guided task completion.
Response types and UI widgets
Response types determine how an agent delivers information in the chat. Instead of returning only text, assistants and tools can return formats that best match the task such as a form for data collection or a card for summaries.
For AI assistants, these response types support more natural and efficient conversations by reducing interpretation and follow‑up questions. For tools, interactive forms bridge conversational AI with application logic, enabling structured input and predictable outcomes.
Forms response type can contain UI widgets to define how users interact with the content, such as entering text, selecting dates, or choosing from predefined options. You can configure UI widgets to match workflows, business rules, and user experience requirements.
Supported response types
The following table provides a comprehensive reference of all supported response types, their descriptions, and their sources:
| Response type | Description | Source |
|---|---|---|
| Audio | Plays sound clips, voice instructions, or confirmations to enhance accessibility or engagement. |
|
| Button | Offers clickable actions that trigger workflows, navigation, or responses. |
|
| Card | Presents structured information in a compact format, often with a title, description, and action buttons. |
|
| Carousel | Displays a scrollable set of cards side by side, useful for showcasing multiple options, steps, or items in a compact, swipe-friendly format. |
|
| Connect to agent | Provides an option to escalate the conversation to a live support agent. |
|
| Date | Use a calendar interface to select a date. This widget supports scheduling, filtering, and other time-based inputs. |
|
| Form | A form that include default Submit and Cancel buttons. You can add UI widgets to forms to design it to match your requirements. |
|
| Grid | Arranges multiple items in a visual grid layout for comparison or selection. |
|
| Image | Shows visual content such as diagrams, screenshots, or illustrations to support guidance. |
|
| Inline frame | Embeds external web content directly into the chat, such as forms, dashboards, or interactive tools. Use inline frames when users need to interact with external systems or view dynamic content as part of the conversation. |
|
| Option | Use a predefined list to present selectable choices. When the list contains fewer than four options, they appear as pills to simplify decision-making. |
|
| Text | Displays plain or formatted text messages to convey information or instructions. |
|
| User defined | Inserts custom HTML or reusable content blocks for advanced formatting or branding. This feature is supported only in embedded chat experiences and is not available in platform chat. |
|
| Video | Embeds video tutorials or demonstrations to walk users through tasks or solutions. |
|
Supported UI widgets for forms
Forms with interactive UI widgets that are defined programmatically within the tool's response follow the Model Context Protocol (MCP) specification. These widgets are rendered within forms that include the default Submit and Cancel buttons.
The following table provides a comprehensive reference of all supported UI widgets, and their descriptions:
| UI Widget | Description |
|---|---|
| Checkbox | A toggle control that allows users to select or deselect a boolean option. |
| Combobox | A dropdown selection control that allows users to choose from a predefined list of options. |
| Date Picker | A calendar interface that allows users to select a single date. |
| Date Range | A calendar interface that allows users to select a range of dates with start and end dates. |
| Download a File | A control that provides a downloadable file link for users to retrieve files. |
| Number Input | An input field that is specifically designed for entering numeric values with optional validation. |
| Radio Button | A selection control that allows users to choose one option from a predefined set of mutually exclusive choices. |
| Table | A data grid that displays structured information in rows and columns, supporting features like sorting, searching, and row selection. |
| Text Area | A multi-line text input field for entering longer text content such as descriptions or comments. |
| Textbox | A single-line text input field for entering short text values such as names or identifiers. |
| Upload a File | A file upload control that allows users to select and upload one or more files with configurable file type and size restrictions. |
Creating rich responses
Tools
Tools can return forms with interactive UI widgets that enable users to provide structured input, make selections, and trigger actions directly within the conversation. These widgets are defined programmatically within the tool's response by using widget contracts that follow the Model Context Protocol (MCP) specification. To configure UI widgets in tools, you embed widget contracts directly in the tool's response structure. The widget contract defines the form schema, UI elements, default values, and actions that occur on user interaction.
The following tool types can return interactive UI widgets:
- Python tools
- OpenAPI tools
- MCP tool
These tool types use a unified approach based on the Model Context Protocol (MCP) specification and use JSON Schema to define form fields and React JSON Schema Form (RJSF) to specify UI rendering. Each tool type implements widget contracts by using its native response format while maintaining consistency with the MCP standard.
For detailed implementation guidance on how to configure form responses with UI widgets, see https://developer.watson-orchestrate.ibm.com/tools/create_tool#creating-a-form-tool.
AI assistants
The assistant defines which rich element appears in a response. When you create an action in the assistant, you add a conversation step and specify the rich response that step returns. If a user message triggers an intent that routes to a step configured with a rich element, the chat shows that element.
When you add AI assistants as collaborator agents, the agent routes the user's message to the appropriate assistant as soon as it detects that the intent belongs to that assistant. The assistant then returns the rich response that is associated with the triggered step.
For implementation details on each supported response type for AI assistants, see Response types reference.
Channels that support rich responses
You can use rich responses in the following channels:
- Orchestrate chat on the home page.
- Orchestrate chat embedded on a website.