Deploying tuned models

You can tune a model to adapt it to a specific task, dataset, or use case . The tuning process adjusts the parameters or weights of a pre-trained model and improves the model's performance and accuracy. Deploy a tuned model so you can add it to a business workflow and start using foundation models in a meaningful way.

You can deploy a foundation model that is tuned with any of the following techniques:

  • Low-rank adaptation (LoRA) fine tuning
  • Quantized low-rank adaptation (QLoRA) fine tuning

Ways to work

You use various methods to deploy tuned foundation including:

Requirements

Before you deploy a tuned model, you must complete the following tasks:

  1. Tune a foundation model with the Tuning Studio or programmatically. For details, see Tuning foundation models.

  2. For LoRA or QLoRA fine-tuned models only: Review the requirements for deploying models tuned with LoRA or QLoRA, including supported models, hardware and software requirements, and deployment types. For details, see Requirements for deploying LoRA or QLoRA tuned models.

Workflow

Whichever way you choose to work, the high-level workflow for deploying a tuned foundation model remains the same. Deploying a tuned model involves the following tasks:

  1. Optional: Create repository asset for the tuned foundation model, if the asset was not automatically created when you tuned the model.
  2. Deploy the tuned model asset.
  3. Inference the tuned model deployment to test the tuned foundation model's responses to textual inputs.
  4. Retrieve the endpoint URL for the tuned model deployment that you can use to access the deployment from your applications. From the Deployments tab of your project or deployment space, click the deployment name. In the API reference tab, find the private and public endpoint links and code snippets that you can use to include the endpoint details in an application.

Learn more