When to tune a foundation model

When to fine tune a foundation model

Fine tuning can help you do the following things:

Customize the style of foundation model output

Maybe you want to augment a virtual assistant that helps customers with support requests. You can tune a foundation model to generate answers in a tone that is consistent with your company brand.

Introduce specialization

Fine tuning can hone the general capabilites of a model for specific tasks. For example, fine tuning was used to specialize base Granite foundation models to create derivative models for chat and code uses.

Add domain-specific knowledge

You can supplement the base model’s knowledge with vocabulary that is not represented in its pretraining data, such as arcane terminology from the legal or financial fields.

Incorporate proprietary data

Your company might have proprietary data that is highly relevant to your specific use case. By fine-tuning the model with examples that incorporate this data, you can confer some of this knowledge to the model without having to train it from scratch.

Address edge cases

Pretrained models are trained on common patterns. They don't know about situations that are specific to your business and therefore, were never covered in pretraining in a specific way. Fine-tuning a model on labeled examples of such situations is an effective way to ensure that edge cases are dealt with appropriately.

Create a foundation model that is tuned for many, diverse tasks

Although it is best to use full fine tuning only to tune a foundation model for a specific task, low-rank adaptation (LoRA) and quantized low-rank adaptation (QLoRA) techniques can be used to customize a model for multiple tasks. Because full fine tuning changes all of the weights of the base model, the tuned model might lose expertise in other generic tasks that it was trained to do previously due what is known as catastrophic forgetting. Catastrophic forgetting is a phenomenon where full fine-tuning causes the model to lose its core knowledge. With the LoRA or QLoRA techniques, the weights of the base model are not changed during the tuning experiment. Because these techniques are faster and require fewer resources, you can create multiple task-specific adapters that can be served using the same base model to save costs.

When to prompt tune a foundation model

Watch a video that explains when and why to prompt tune a foundation model.

Prompt tuning can help you do the following things:

Reduce the cost of inferencing at scale

Larger foundation models typically generate better results. However, they are also more costly because more resources are required to host them. By tuning a model, you can get similar, sometimes even better results from a smaller model that costs less to use. Deploying the smaller prompt-tuned model reduces long-term inference costs.

Adjust model output to use a certain style at a lower cost

Generate output in a reliable form in response to zero-shot prompts, which means consuming fewer tokens per call

When not to tune a model

Tuning a model is not always the right approach for improving the output of a model. For example, tuning a model cannot help you do the following thing:

Improve the accuracy of answers in model output

If you're using a foundation model for factual recall in a question-answering scenario, tuning will marginally improve answer accuracy. To get factual answers, you must provide factual information as part of your input to the model. Tuning can be used to help the generated factual answers conform to a format that can be more-easily used by a downstream process in a workflow. To learn about methods for returning factual answers, see Retreival-augmented generation.

Learn more

• Retreival-augmented generation
• Tuning methods

Parent topic: Tuning foundation models