When to tune a foundation model
Find out when tuning a model can help you use a foundation model to achieve your goals.
Tune a foundation model when you find a foundation model that meets your criteria, but needs to be customized a bit for your needs.
Fine tuning can help you do the following things:
- Customize the style of foundation model output
- Maybe you want to augment a virtual assistant that helps customers with support requests. You can tune a foundation model to generate answers in a tone that is consistent with your company brand.
- Introduce specialization
- Fine tuning can hone the general capabilites of a model for specific tasks. For example, fine tuning was used to specialize base Granite foundation models to create derivative models for chat and code uses.
- Add domain-specific knowledge
- You can supplement the base model’s knowledge with vocabulary that is not represented in its pretraining data, such as arcane terminology from the legal or financial fields.
- Incorporate proprietary data
- Your company might have proprietary data that is highly relevant to your specific use case. By fine-tuning the model with examples that incorporate this data, you can confer some of this knowledge to the model without having to train it from scratch.
- Address edge cases
- Pretrained models are trained on common patterns. They don't know about situations that are specific to your business and therefore, were never covered in pretraining in a specific way. Fine-tuning a model on labeled examples of such situations is an effective way to ensure that edge cases are dealt with appropriately.
- Create a foundation model that is tuned for many, diverse tasks
- Although it is best to use full fine tuning only to tune a foundation model for a specific task, low-rank adaptation (LoRA) and quantized low-rank adaptation (QLoRA) techniques can be used to customize a model for multiple tasks. Because full fine tuning changes all of the weights of the base model, the tuned model might lose expertise in other generic tasks that it was trained to do previously due what is known as catastrophic forgetting. Catastrophic forgetting is a phenomenon where full fine-tuning causes the model to lose its core knowledge. With the LoRA or QLoRA techniques, the weights of the base model are not changed during the tuning experiment. Because these techniques are faster and require fewer resources, you can create multiple task-specific adapters that can be served using the same base model to save costs.
Watch a video that explains when and why to tune a foundation model.
This video provides a visual method to learn the concepts and tasks in this documentation.
When not to tune a model
Tuning a model is not always the right approach for improving the output of a model. For example, tuning a model cannot help you do the following thing:
- Improve the accuracy of answers in model output
- If you're using a foundation model for factual recall in a question-answering scenario, tuning will marginally improve answer accuracy. To get factual answers, you must provide factual information as part of your input to the model. Tuning can be used to help the generated factual answers conform to a format that can be used more easily by a downstream process in a workflow. To learn about methods for returning factual answers, see Retreival-augmented generation.
- Get the model to use a specific vocabulary in its output consistently
- Large language models that are trained on large amounts of data formulate a vocabulary based on that initial set of data. You can introduce significant terms to the model from training data that you use to tune the model. However, the model might not use these preferred terms reliably in its output.
- Teach a foundation model to perform an entirely new task
- Experimenting with prompt engineering is an important first step because it helps you understand the type of output that a foundation model is and is not capable of generating. You can use tuning to tweak, tailor, and shape the output that a foundation model is able to return.