Considerations when setting the TFE_PARALLELISM environment variable

Problem

Introduction

You may need to adjust the TFE_PARALLELISM environment variable in Terraform Enterprise or HCP Terraform when infrastructure providers produce errors during concurrent operations or enforce non‑standard rate limiting.

This article outlines several approaches for managing provider rate limits and improving run performance.

Recommendation

Using custom values for TFE_PARALLELISM is highly dependent on your specific use case and infrastructure provider. Before making changes, review the following approaches to determine the best strategy for your environment.

Approach 1: Adjust TFE_PARALLELISM with Caution

The default value for TFE_PARALLELISM is 10. If you encounter rate limits, Terraform runs may take longer than with the default parallelism setting.

In general, changing TFE_PARALLELISM alone may not significantly decrease the time a large run takes. Performance depends on the composition of your resources and the specific API calls required to manage them.

Approach 2: Restructure Configurations

Consider breaking a large configuration into smaller, logically organized workspaces to reduce concurrent operations and avoid provider rate limits.

Approach 3: Use Multiple Provider Accounts

Creating multiple AWS accounts for Terraform runs provides independent API quotas, often eliminating rate‑limiting delays.

Leveraging multiple provider accounts in a single Terraform configuration provides separate API quotas, reducing throttling risk. This architectural approach often yields greater performance gains than adjusting the parallelism setting.

Additional Information

• For more details on how Terraform processes resources concurrently, refer to the documentation on Terraform's Graph Walking Mechanism.