Choosing a foundation model to tune

Find the right foundation model to customize for your task.

You can prompt tune the following models from the Tuning Studio in watsonx.ai:

• flan-t5-xl-3b
• granite-13b-instruct-v2
• llama-2-13b-chat

These foundation models must be installed by your system administrator.

To help you choose the best model, follow these steps:

1. Consider whether any measures were taken to curate the data that was used to train the foundation model to improve the quality of the foundation model output.

2. Review other general considerations for choosing a model.

   For more information, see Choosing a foundation model.

3. Experiment with the models in the Prompt Lab.

   Use the largest version (meaning the version with the most parameters) of the model in the same model family for testing purposes. By testing with a larger, more powerful model you can establish the best prompt pattern for getting the output you want. Then, you can tune a smaller version of the same model type to save compute resources. A prompt-tuned version of a smaller model can generate similar, if not better results and requires fewer resources to inference.

   Craft and try prompts until you find the input pattern that generates the best results from the large foundation model.

   For more information, see Prompt Lab.

The following table shows the foundation models to experiment with before you choose a foundation model to tune.

Parent topic: Tuning Studio