Tuning a foundation model programmatically (Python)

You can tune foundation models in IBM watsonx.ai programmatically by using the Python library.

You can tune a foundation model in the following ways:

When you tune a foundation model, you run an experiment that uses training data provided by you. The experiment is a machine learning process that shows the foundation model the outputs you expect the model to return for your model inputs. The tuning process is complex and involves a data asset, a training asset, and a deployment asset.

The python library has methods and helper classes for tuning foundation models. Use functions that are available in the watsonx.ai Python library from notebooks in watsonx.ai to tune foundation models.

Getting a list of models that can be tuned

A subset of the foundation models that can be inferenced in watsonx.ai can also be tuned. The Python library has helper classes that you can use to get a list of foundation models that are tunable.

Sample notebook for prompt tuning

The Use watsonx.ai to tune Meta llama-2-13b-chat model with Consumer Financial Protection Bureau documents sample Python notebook contains code for prompt-tuning foundation models in watsonx.ai.

The sample notebook helps you with the two main phases of tuning:

The sample notebook is designed to prompt-tune the llama-2-13b-chat model. But you can use it for tuning the flan-t5-xl-3b foundation model also. To do so, replace references to base_model='meta-llama/llama-2-13b-chat' with base_model='google/flan-t5-xl'.

If you change the foundation model, you must replace the training data also. Replace the file path in the Data loading section of the notebook.

url = "{path to your file}"
if not os.path.isfile(filename): 
    wget.download(url)

Sample notebook for fine tuning

Step through the provided sample notebook to learn more about how to use the Python library to run a fine-tuning experiment, and then deploy your tuned foundation model programmatically.

See Use watsonx, and meta-llama/Meta-Llama-3-8B to Fine Tune with online banking queries.

Sample notebook for LoRA fine tuning

Step through the provided sample notebook to learn more about how to use the Python library to run a LoRA fine-tuning experiment, and then deploy your tuned foundation model programmatically.

See Use watsonx, and meta-llama/llama-3-1-8b to Fine Tune with LoRA on online banking queries.

Using the sample notebook to optimize tuning parameter values

The sample notebook has code that optimizes the learning_rate parameter value. The sample notebook systematically changes the learning rate value and reruns the experiment 10 times, so the loss can be compared across the 10 runs. The sample notebook calculates the optimal learning rate value for you.

The sample notebook generates 10 separate experiments; it does not run the same experiment 10 times.

The parameter to optimize is defined in the Search space and optimization section of the notebook.

You can edit or add to the sample notebook to run automated code to optimize the following parameters in addition to learning rate:

  • accumulate_steps
  • batch_size
  • num_epochs

To check for the optimal values for many parameters at once, you can change the sample notebook to use code like this, for example:

SPACE = [
    skopt.space.Real(0.001, 0.09, name='learning_rate', prior='log-uniform'),
    skopt.space.Integer(1, 50, name='num_epochs', prior='uniform'),
    skopt.space.Integer(1, 16, name='batch_size', prior='uniform')
]

Optimizing many parameters at once can save time because the parameters work together. Their values affect one another and the right balance of values among them leads to the best results.

The sample notebook uses methods from the scikit-optimize library. For more information, see the scikit-optimize API reference.

Using the sample notebook to evaluate the tuned model

The sample notebook has code that deploys the tuned model, inferences the tuned model, and then calculates the accuracy of the tuned model output. It also inferences the underlying foundation model and calculates the accuracy of the base model output, so that you can see a comparison.

If you want to use the sample notebook to tune and assess other models, you can replace the value of the model_id parameter in the following code.

base_model = ModelInference(
    model_id='ibm/granite-13b-instruct-v2',
    params=generate_params,
    api_client=client
)

For example, specify google/flan-t5-xl.

You must also replace the prompt text with a prompt from your own training data set.

response = tuned_model.generate_text(prompt="{your text here}")

If the accuracy score for your tuned model is low, review some ideas for ways to improve your training data in Addressing data quality problems in tuned model output.

Remember, optimization of the tuning parameters is specific to the model and training data that you are using. If you change either the model or the training data, you need to reassess the tuning experiment. Adjust the tuning parameters again to optimize them for your augmented data set.

Parent topic: Python library