How-to Use custom and community providers in Terraform Cloud and Enterprise

Problem

Introduction

HCP Terraform and Terraform Enterprise support several methods for using custom and community providers. For a general overview, refer to the official documentation on Custom and Community Providers.

This article outlines four primary methods:

1. Use the Private Provider Registry
2. Use Git Submodules
3. Use a Custom Terraform Bundle (Terraform Enterprise Only)
4. Use a Provider Network Mirror

Options

Option 1: Use the Private Provider Registry

Using the private provider registry is the recommended method. This feature, formerly known as the Private Module Registry (PMR), now supports versioning and provides a searchable list of providers and modules available to your organization. The private registry functions similarly to the public Terraform Registry and helps you share Terraform providers and Terraform modules internally.

• Review the guide on publishing providers to the registry.
• Refer to the documentation for using providers from the registry.

This feature is available in HCP Terraform and in Terraform Enterprise versions v202112-2 and later.

Option 2: Use Git Submodules

When you use the same custom provider across many repositories, you can use Git submodules to avoid duplicating the provider binary in each repository.

1. Create a separate Git repository for your custom provider (the “provider repository”).

2. Compile a linux_amd64 binary for your provider and make it executable.

   $ chmod +x <provider-binary-name>

3. Upload the binary to the root of your provider repository.
4. Navigate to the root of your configuration repository, or the specified Terraform Working Directory.

5. Add the provider repository as a submodule. Replace $ORGANIZATION and $REPOSITORY with your VCS organization and repository name.

   $ git submodule add \
   https://github.com/$ORGANIZATION/$REPOSITORY.git \
   terraform.d/plugins/linux_amd64

Note: HCP Terraform and Terraform Enterprise cannot pass HTTPS credentials during a run. The command above assumes a public repository. For private repositories, you must provide SSH keys in the VCS connection settings and clone over SSH.

Workspaces using this method must have the Include submodules on clone option enabled. You can find this setting in your workspace's Version Control settings. When the provider binary is updated, you must also update the submodule in any configuration repository that references it.

Option 3: Use a Custom Terraform Bundle (Terraform Enterprise Only)

Terraform Enterprise allows you to include custom and community providers in a custom Terraform bundle. This guide assumes you have already compiled the terraform-bundle CLI tool and it is available in your $PATH.

1. In your terraform-bundle.hcl configuration file, add the custom provider to the providers block.

   providers {
     my-custom-provider = ["1.0"]
   }

2. Create a ./plugins subdirectory where you run terraform-bundle, or specify a different location with the -plugin-dir flag.
3. Place the custom provider binary in the ./plugins directory. The binary must follow the naming convention terraform-provider-<NAME>_v<VERSION> and be compiled for the same operating system and architecture as your Terraform Enterprise instance (typically linux_amd64).

Option 4: Use a Provider Network Mirror

Provider network mirrors are supported in Terraform CLI v0.13.2 and later. This method requires you to create and maintain an HTTP-based service that follows the Provider Network Mirror Protocol.

Unlike services that use remote service discovery, a network mirror's location is a direct physical URL. You specify this URL in the CLI configuration.

provider_installation {
  network_mirror {
    url = "https://terraform.example.com/providers/"
  }
}

Your Terraform configuration source code must also be modified to use the mirror. This method is the least recommended because it requires significant setup and maintenance overhead.

For more details, refer to the Provider Network Mirror Protocol documentation.

Additional Information

For many use cases, publishing providers to the public or private Terraform Registry is the most effective solution. However, the alternative methods described here are available for infrastructures with specific constraints or requirements.