June 30, 2021 By Janardhana Anipireddy
Geetha Sathyamurthy
3 min read

IBM Cloud Schematics is used to provision IBM Cloud resources by describing their configurations in a Terraform template file.

If you have tried to provision IBM Cloud resources by using the user interface or command line, you would have noticed that some of them take a very long time (e.g., IBM Event Streams). In such scenarios, HashiCorp recommends the use of timeout blocks. In any case, you must be aware of the typical time taken by IBM Cloud to provision or configure its cloud resources in order to avoid failures in your automation.

Now, the question is, what if IBM Cloud takes more time to provision or configure than the predefined timeout values in the timeout block? How do Terraform and IBM Cloud Schematics respond to such a situation?

In our experience with the terraform apply command, we have seen IBM Cloud Schematics exit the automation with a timeout error message. It sounds harmless, but the real problem is that the IBM Cloud service-broker has not stopped the provisioning operation. In other words, even though the IBM Cloud Schematics has reported the timeout error, the IBM Cloud resource (such as IBM Event Streams) will continue to be provisioned in your account, and you will be charged for that IBM Event Streams instance.

As an cloud engineer, once you see the timeout error message in the logs, your natural instinct is to retry the provisioning of the same Terraform template and the resources therein. For the second time, IBM Cloud Schematics will behave in an unexpected manner. It will use the previously saved Terraform state file in the workspace, to re-run the provisioning operation. Unfortunately, Terraform would have marked the IBM Event Streams instance as tainted in the terraform state file due to the timeout error. 

The default behaviour of the Terraform engine is to destroy the tainted resource and reprovision it, by default. In other words, the IBM Event Streams instance that is eventually provisioned by the IBM Cloud service-broker — even after IBM Cloud  Schematics reported the timeout error — is destroyed again, and IBM Cloud Schematics tries to provision another instance of the IBM Event Streams instance. This results in double your costs.  

How can you stop IBM Cloud Schematics from reprovisioning the IBM Cloud resources, due to the “tainted” flag in the Terraform state file?

Handling timeout errors

You should use the following procedure to handle such timeout errors in the IBM Cloud Schematics logs.

  1. Once you see the timeout block in the IBM Cloud Schematics workspace logs, view the Resources user interface page for the IBM Cloud Schematics workspace and verify whether the corresponding Terraform resource is marked as tainted.
  2. You should wait few hours and verify whether the IBM Cloud resource is actually getting provisioned in the account. Then, view the respective IBM Cloud resources user interface pages to confirm the provision.
  3. If the IBM Cloud resource is actually provisioned in the Cloud account after a delay, then you should check the IBM Cloud Schematics untaint CLI command to remove the resource’s taint flag in the state file.
  4. Finally, run the IBM Cloud Schematics plan and apply command from the user interface or command line to complete the provisioning of the template.

Note: In the last step, IBM Cloud Schematics idempotently provisions the remaining resources in that template.

Based on this experience, you must fine tune the timeout values in the Terraform template so that the same error does not repeat again in another environment. You must normally keep a watch on the following IBM Cloud resources for timeout errors in IBM Cloud Schematics:

  • IBM Event Streams
  • IBM Cloud Databases
  • IBM Cloud Kubernetes Service
  • IBM Cloud Satellite

We wish you good luck with your Terraform and Ansible automation with IBM Cloud Schematics.

Was this article helpful?
YesNo

More from Cloud

Attention new clients: exciting financial incentives for VMware Cloud Foundation on IBM Cloud

4 min read - New client specials: Get up to 50% off when you commit to a 1- or 3-year term contract on new VCF-as-a-Service offerings, plus an additional value of up to USD 200K in credits through 30 June 2025 when you migrate your VMware workloads to IBM Cloud®.1 Low starting prices: On-demand VCF-as-a-Service deployments begin under USD 200 per month.2 The IBM Cloud benefit: See the potential for a 201%3 return on investment (ROI) over 3 years with reduced downtime, cost and…

24 IBM offerings winning TrustRadius 2024 Top Rated Awards

2 min read - TrustRadius is a buyer intelligence platform for business technology. Comprehensive product information, in-depth customer insights and peer conversations enable buyers to make confident decisions. “Earning a Top Rated Award means the vendor has excellent customer satisfaction and proven credibility. It’s based entirely on reviews and customer sentiment,” said Becky Susko, TrustRadius, Marketing Program Manager of Awards. Top Rated Awards have to be earned: Gain 10+ new reviews in the past 12 months Earn a trScore of 7.5 or higher from…

IBM Tech Now: April 8, 2024

< 1 min read - ​Welcome IBM Tech Now, our video web series featuring the latest and greatest news and announcements in the world of technology. Make sure you subscribe to our YouTube channel to be notified every time a new IBM Tech Now video is published. IBM Tech Now: Episode 96 On this episode, we're covering the following topics: IBM Cloud Logs A collaboration with IBM watsonx.ai and Anaconda IBM offerings in the G2 Spring Reports Stay plugged in You can check out the…

IBM Newsletters

Get our newsletters and topic updates that deliver the latest thought leadership and insights on emerging trends.
Subscribe now More newsletters