You can author a generative AI task to implement generative AI logic in your
service flow. Generative AI tasks enable you to send a prompt to a large language model (LLM) and to
receive the generated output.
Before you begin
To use the generative AI tooling in the service flow editor, you must first configure the server
with a connection to the watsonx.ai provider. If you navigate to the Gen AI
tab in the service flow editor and the server is not configured, a page displays an error message
and a link to the configuration documentation.
To know how to configure generative AI tasks, see Enabling generative AI in Business Automation Workflow on containers.
After the server is configured, a new AI entry (
) for the generative AI task is
added to the Activity section of the Service Flow editor palette.
Procedure
-
Open the process application in the designer, and open an existing service flow, or create a new one from
the navigation tree on the left side.
- In the Diagram view of the service flow, click the arrow besides
the Activity icon in the component palette to open the sub-palette, and drag
the AI icon () onto the diagram canvas.
- To configure the generative AI task, click that task in the diagram, go to the
Implementation tab, and click Open Gen AI, or click
the Gen AI tab at the top of the editor, and click the task name in the
left-hand panel.
An editor opens in the Gen AI
tab.
- Select the large language model (LLM) that you want to use for your
prompt.
Each entry includes a short description of the LLM.
The
selected LLM gives you access to the watsonx.ai project associated with the server configuration
that you set up in the
Before you begin section. For more
information about the available LLMs, see the
Foundation model library.
- Create a prompt that will be sent to the LLM for processing.
The prompt
includes two parts:
Context and
Input. The context
consists of instructions and background information, while the input is the data provided to the
context. Note that the input may not be required for every context. For examples on how to create a
prompt, see
Sample prompts, and the references listed in
What steps are required to implement a generative AI task?.
- In the Context field, enter an imperative
statement.
- Optional: In the Input field, enter the text that
you want the model to respond to.
- Optional: Add variables to make the context and input more reusable.
- Go to the service flow Variables tab, and for each input and output
variable required in the service flow, click the + sign next to the
Variables section. In the Name field, replace the
default name with a more descriptive name. Click Select next to
Variable type, and choose String System Data from the
list. Optionally, you can set a default value for the variable, by selecting Has
default, and entering a value in the corresponding field in the Default
Value section.
- From the editor in the Gen AI tab tab, click Add Variable
+ in the Variables section.
- Select variables to add to the Gen AI task from the drop-down menu, and click
Add. Only variables of type String and which are not a list are supported and
are available from the drop-down menu.
The variables are now added to the generative AI task, and
are listed under the
Variables section. Those variables can be later removed
by clicking the trash can icon.
Warning: If a variable is deleted from the
Variables tab, it will no longer be available for use in the prompt input and
context, and must be deleted from there as well.
- To see what different types of output are generated for the LLM with
different input values, enter a test value for the variable in the Value
field to be used in your prompt. This value will not be used at run time.
Note: If a default value
was added for the variable in the service flow, it will automatically display in the
Value field when the variable is added to the Gen AI task.
- Add variables from the service flow into the context or the input by clicking the
+ sign at the bottom left corner of the areas, or by pressing
ctrl+space. The variable name then displays in the related field, enclosed in
double curly brackets, for example:
{{topic}}
- Optional: To limit the length of the generated output, from the
Parameters pane, specify the minimum and maximum number of tokens that the
LLM can generate.
Tip: A token is a sequence of characters that holds semantic meaning for a model. Before
LLM processing, the words in your prompt text are converted into tokens. Similarly, the raw output
from the model is also in the form of tokens. These output tokens are then converted back into words
to be displayed as the final result.
- The minimum and maximum default values are set to 1 and 50.
- The minimum value cannot be 0.
- Each LLM has a maximum limit on the total number of tokens, which includes both the tokens in
the input prompt and the tokens in the generated output. You can view those limits in the LLM
descriptions in the Large language model drop-down menu. For more information
about tokens, see Tokens and tokenization.
The cost of calling the LLM depends on the number of tokens it consumes and produces.
To ensure that the response is relevant without being overly verbose, it is recommended to limit the
result. For example, if a "yes" or "no" response is expected, you can set the maximum number of
tokens to 1.
- Optional: To improve the precision and relevance of the generative AI task,
add training examples and specify one or more pairs of sample input and corresponding output by
clicking New example in the Training examples section,
and typing in the input and expected output.
Providing the LLM with appropriate training
examples will result in better-matched outputs but make sure not to exceed the maximum token input
limit set by the Large Language Model and the overall maximum token limit for both input and
generated output tokens.
- Click Generate.
Tip: Clicking RAW beside the Generate
button displays information that is sent to the LLM.
- Review the result in the Generated Output pane by clicking the
Raw prompt icon to see the raw prompts.
In the
View raw
prompt window, you can see the context, the input, and training examples that are used
to obtain the generated output:
- To further improve your prompt results, you can edit the input or expected output, and provide
more examples.
- When you are satisfied with the generated output, you can click Save as example to save
the prompt input and generated output as a training example, which will then be listed in the
Training examples section.
- Map the generated output to the variables in your service flow by going to the
Data mapping section in the properties pane of the
Diagram tab.
The output of the task is mapped to the variables in
the service flow so that the output can be used by other downstream tasks or calling processes or
service tasks. There is no input mapping to fill in, as it is implicitly done when adding variables
in the
Gen AI tab. Four outputs from the task are available to map:
- Generated text: The LLM output.
- Generated token count: The number of tokens that are generated by the LLM
for the Generated text.
- Input token count: The number of tokens that are consumed by the LLM for
the given input.
- Stop reason: The reason why the call stopped. It can be:
- not_finished: Possibly more tokens to be streamed.
- max_tokens: Maximum requested tokens reached.
- eos_token: End of sequence token encountered.
- canceled: Request canceled by the client.
- time_limit: Time limit reached.
- stop_sequence: Stop sequence encountered.
- token_limit: Token limit reached.
- error: Error encountered.
Results
The generative AI prompt you configured for a generative AI task in a service flow can now be
called from within your process application.