Adding generative prompts

Edit online

You can add and edit generative prompts in agentic workflows. A generative prompt uses the capabilities of large language models (LLMs) for tasks that involve natural language understanding such as generation of text or structured content in agentic workflows.

To add a generative prompt to an agentic workflow:

  1. Open the agentic workflow in the workflow builder.

  2. Do one of the following to add a generative prompt:

    • Click add flow items add new. From Flow nodes, click and drag Generative prompt onto a connector line.

    • Click the connector line between the start and end nodes, then select Add a flow activity > Generative prompt.

You can download an example agentic workflow that is related to the Generative prompt from https://github.com/IBM/ibm-watsonx-orchestrate-adk/tree/main/examples/flow_builder/car_price_calculator. For more information, see Example agentic workflows.

Managing input variables for prompts

Define input variables that you can add in the System prompt and User prompt to map inputs.

To define an input variable, complete the following steps:

  1. In the Generative prompt dialog, click Add.

  2. Select the data type for the input variable.

  3. Specify a name, description, and value. The value is a test value that is used for inputs when you test the generative prompt in the Response panel by clicking Generate response.

You can also edit and delete the existing input variables. The options to edit and delete appear when you hover over an input variable.

Using the chat session context as input

Chat session context provides context from the chat as input to the prompt. When you enable the chat session context, the last five conversation turns from the chat history is provided as input to the prompt. A conversation turn refers to a single exchange between a user and the agent in a chat.

To enable the chat session context, in the Generative prompt dialog, set the Include chat history as input switch to on.

Defining prompts

Prompts are instructions that you provide to the LLM model to get the expected response or behavior.

You can define system prompts and user prompts. The user prompt is task-oriented and specifies what tasks to do. In contrast, the system prompt defines the context such as tone, formatting, and behavior that guide how the response is generated.

To define the prompts, complete the following steps:

  1. In the Generative prompt dialog, select the Define prompts option.

  2. In the System prompt field, provide an explanation about how to respond to conversations. You can specify the tasks to do, tone and intent, and any limitations or restrictions to follow during a conversation. The system prompt helps ensure consistent behavior across all conversations. You can include input variables in the System prompt field to map inputs.

For example, a system prompt that defines the tone and format is "Use a friendly but professional tone. Address the recipient by first name."

  1. In the User prompt field, provide task-specific instructions to the LLM model. You can include input variables in the User prompt field to map inputs.

For example, a task-specific user prompt is "Generate an email that welcomes a new recruit to the team and which is sent to them by their manager. The recruit's first name is {recruit_first_name}. Their manager's full name is {manager_full_name}."

Note:

Generative prompt fields and agent node prompt fields treat text within curly braces as inline variable references. To display curly braces as plain text instead of variable references, escape them using double curly braces {{}}. For example, {{customer_id}}in the prompt is displayed as {customer_id} in the output. In this case, the text within the double braces is treated as plain text rather than a variable.

Using variables in the prompt

You can use a flow variable, flow input, or the output of an upstream node.

To add variables to the System prompt and User prompt fields:

  1. Place the cursor where you want to insert a variable, and click the Variable icon variable icon.

  2. Select a variable from the list.

Adding examples to improve the generative prompt output

You can specify example prompts and their expected output to improve the output or results of a generative prompt.

To add examples, complete the following steps:

  1. In the Generative prompt dialog, select the Add examples option.

  2. Click Add examples

  3. In the Input prompt field, add a clear and task-focussed sample prompt.

  4. In the Expected output field, define a sample output that you expect as a response.

By default, the Active toggle switch to use the example is set to on. If you prefer not to use the example, set the switch to off.

You can add a maximum of 10 training examples. You can also delete the existing training examples. The option to delete appears when you hover over a training example.

Defining an object as the generative prompt output

You can change the output type of a generative prompt to an object.

Note:

A generative prompt response is enhanced by using training examples that you specify. When you change the response of a generative prompt to an object type, all existing training examples are removed.

To change the output type to an object, complete the following steps:

  1. In the Generative prompt dialog, set the Output as Object toggle switch to on.

  2. Enter a Name and Description.

  3. Click plus next to the object name in the object tree or click Add + in the Object properties section.

  4. Select a property from the list.

  5. Enter a Name and Description.

  6. Optional: Select the List of {data type} checkbox to store a list of values in the object property.

  7. Optional: Turn on the Set default values toggle switch to set the properties with a default value.

  8. Enter the default values and press enter.

  9. Toggle between Simple view and JSON view to update the object properties. By default, simple view is selected. For an introduction video of the Simple view, see Low-code object editor in agentic workflows.

  10. Click Save.

  11. Click Edit object to update the created object.

    • The Set default value option is not available for object variable.

To learn more about defining object properties, see Object data type.

Adjusting the LLM settings

To configure the LLM settings, in the Generative prompt dialog, select the Adjust the LLM settings option. You can configure the following LLM settings:

  • Minimum and maximum tokens in a response.

  • Stop sequence to stop token generation when a particular sequence is generated

  • Creative threshold to control how creative or precise the responses are. Move the slider to automatically set the creative threshold. To set manually, select the Manually set the creative threshold settings checkbox.

Selecting a model and testing your generative prompt

You can select a model to use for your generative prompt from the Model list on the Generative prompt dialog. After you select the model, you can test the generative prompt to verify whether the generated response matches your expected outcome.

To test the generative prompt, click Generate preview, and review the generated response to see how it aligns with your expectations.

You can add a generated response as a training example by clicking the Save as training example icon addtraining icon.

Manage error handling

Error handling allows you to customize the error message that shown to the user if the generative prompt fails. You can also optionally configure the agentic workflow to retry the call the generative prompt if it fails.

To define error handling, complete the following steps:

  1. In the Generative prompt dialog, click edit next to Error handling.

  2. In the Error handling window, Modify the Error message if necessary.

  3. If you want, you can configure the generative prompt to retry on error. To configure:

    1. Enable Retry on error.

    The following error codes are supported:

    • 408 Request Timeout: The client took too long; retry may succeed.

    • 409 Conflict: Often used for version conflicts; retry after resolving conflict.

    • 425 Too Early: Request was sent too soon; retry later.

    • 429 Too Many Requests: Throttling; retry after waiting for Retry-After header.

    • 500 Internal Server Error: Generic server error; may be transient.

    • 502 Bad Gateway: Temporary proxy/server issue; retry may succeed.

    • 503 Service Unavailable: Server temporarily overloaded or down for maintenance.

    • 504 Gateway Timeout: Upstream server didn’t respond in time; retry may succeed.

  4. Enter the Maximum number of retries (between 1 and 5).

  5. Adjust the Wait time between retries in milliseconds (between 1000 and 60000). The default value is 1000 ms.

An agentic workflow defines a set of linked activities and controls that are designed to achieve a specific business purpose or goal. Learn more about agentic workflows.