Creating a detached deployment for an external prompt

Create a detached deployment for evaluating a prompt template for an externally-hosted large language model (LLM) without inferencing the model.

As part of your governance process, you can evaluate prompt templates for generative AI models that are not created or hosted by IBM. For example, you can evaluate a prompt template for a Google Vertex AI model, an Azure OpenAI model, or an AWS Bedrock model. For this type of evaluation, all inferencing is done on the remote model. Evaluations are run on the generated prompt output.

This type of asset is called a detached prompt template. It must be created programmatically. However, you can evaluate a detached prompt template in a deployment space by creating a detached deployment. Creating a detached deployment provides you with these benefits and capabilities:

• Evaluating a prompt template in a project or a space provides a richer experience for reviewing the results of evaluations.
• Use access control for projects and spaces to invite collaborators or restrict access, as needed.
• Track the results of your evaluations in factsheets in AI use cases as part of your governance solution.