Adding generative prompts

You can add and edit generative prompts in agentic workflows. A generative prompt uses the capabilities of large language models (LLMs) for tasks that involve natural language understanding such as generation of text or structured content in agentic workflows.

To add a generative prompt to an agentic workflow:

From the menu , go to Build.
Select All tools and click Create tool.
Select Agentic workflow.
Click the Add tools icon .
Select the Create new tab.
Drag Generative prompt to the agentic workflow.

Managing input variables for prompts

Define input variables that you can add in the System prompt and User prompt to map inputs.

To define an input variable, complete the following steps:

In the Generative prompt dialog, click Add.
Select the data type for the input variable.
Specify a name, description, and value. The value is a test value that is used for inputs when you test the generative prompt in the Response panel by clicking Generate response.

You can also edit and delete the existing input variables. The options to edit and delete appear when you hover over an input variable.

Using the chat session context as input

Chat session context provides context from the chat as input to the prompt. When you enable the chat session context, the last five conversation turns from the chat history is provided as input to the prompt. A conversation turn refers to a single exchange between a user and the agent in a chat.

To enable the chat session context, in the Generative prompt dialog, set the Include chat history as input switch to on.

Defining prompts

Prompts are instructions that you provide to the LLM model to get the expected response or behavior.

You can define system prompts and user prompts. The user prompt is task-oriented and specifies what tasks to do. In contrast, the system prompt defines the context such as tone, formatting, and behavior that guide how the response is generated.

To define the prompts, complete the following steps:

In the Generative prompt dialog, select the Define prompts option.
In the System prompt field, provide an explanation about how to respond to conversations. You can specify the tasks to do, tone and intent, and any limitations or restrictions to follow during a conversation. The system prompt helps ensure consistent behavior across all conversations. You can include input variables in the System prompt field to map inputs.

For example, a system prompt that defines the tone and format is "Use a friendly but professional tone. Address the recipient by first name."
In the User prompt field, provide task-specific instructions to the LLM model. You can include input variables in the User prompt field to map inputs.

For example, a task-specific user prompt is "Generate an email that welcomes a new recruit to the team and which is sent to them by their manager. The recruit's first name is {recruit_first_name}. Their manager's full name is {manager_full_name}."

Using variables in the prompt

You can use a flow variable, flow input, or the output of an upstream node.

To add variables to the System prompt and User prompt fields:

Place the cursor where you want to insert a variable, and click the Variable icon .
Select a variable from the list.

Adding examples to improve the generative prompt output

You can specify example prompts and their expected output to improve the output or results of a generative prompt.

To add examples, complete the following steps:

In the Generative prompt dialog, select the Add examples option.
Click Add examples
In the Input prompt field, add a clear and task-focussed sample prompt.
In the Expected output field, define a sample output that you expect as a response.

By default, the Active toggle switch to use the example is set to on. If you prefer not to use the example, set the switch to off.

You can add a maximum of 10 training examples. You can also delete the existing training examples. The option to delete appears when you hover over a training example.

Defining an object as the generative prompt output

You can change the output type of a generative prompt to an object.

Important: A generative prompt response is enhanced by using training examples that you specify. When you change the response of a generative prompt to an object type, all existing training examples are removed.

To change the output type to an object, complete the following steps:

In the Generative prompt dialog, set the Output as Object toggle switch to on.
Click Edit object definition to specify the object definition in the object editor.

To learn more about defining object properties, see Object data type.

Adjusting the LLM settings

To configure the LLM settings, in the Generative prompt dialog, select the Adjust the LLM settings option. You can configure the following LLM settings:

Minimum and maximum tokens in a response.
Stop sequence to stop token generation when a particular sequence is generated
Creative threshold to control how creative or precise the responses are. Move the slider to automatically set the creative threshold. To set manually, select the Manually set the creative threshold settings checkbox.

Selecting a model and testing your generative prompt

You can select a model to use for your generative prompt from the Model list on the Generative prompt dialog. After you select the model, you can test the generative prompt to verify whether the generated response matches your expected outcome.

To test the generative prompt, click Generate preview, and review the generated response to see how it aligns with your expectations.

You can add a generated response as a training example by clicking the Save as training example icon addtraining icon .

Manage error handling

Error handling allows you to customize the error message shown to the user if the generative prompt fails. You can also optionally configure the agentic workflow to retry the call the generative prompt if it fails.

To define error handling, complete the following steps:

In the Generative prompt dialog, click next to Error handling.
In the Error handling window, Modify the Error messsage if necessary.
If you want, you can configure the generative prompt to retry on error. To configure:
1. Enable Retry on error.
The following error codes are supported:
- 408 Request Timeout: The client took too long; retry may succeed.
- 409 Conflict: Often used for version conflicts; retry after resolving conflict.
- 425 Too Early: Request was sent too soon; retry later.
- 429 Too Many Requests: Throttling; retry after waiting for Retry-After header.
- 500 Internal Server Error: Generic server error; may be transient.
- 502 Bad Gateway: Temporary proxy/server issue; retry may succeed.
- 503 Service Unavailable: Server temporarily overloaded or down for maintenance.
- 504 Gateway Timeout: Upstream server didn’t respond in time; retry may succeed.
1. Enter the Maximum number of retries (between 1 and 5).
2. Adjust the Wait time between retries in milliseconds (between 1000 and 60000). The default value is 1000 ms.

Creating agentic workflows

An agentic workflow defines a set of linked activities and controls that are designed to achieve a specific business purpose or goal. Learn more about agentic workflows.