April 5, 2021 By Dimitri Prosper 5 min read

Learn how to automate the configuration of instance storage in VPC.

Instance storage is a feature in IBM Cloud Virtual Private Cloud that offers one or more solid-state drives (SSD) directly attached to your virtual server instance (VSI). Instance storage provides low cost, low latency and higher IOPS when compared to Block Storage for VPC volumes. However, while the data stored in block storage volumes are persistent, the data stored in instance storage is temporary and is tied directly to the lifecycle of the instance:  

  • When a VSI is rebooted, the data is retained and available to your application.
  • When a VSI is shut down or moved to a different host due to a physical failure, the data is cryptographically erased and your application needs to recreate it or replicate it from another source. 

Instance storage is a useful feature for temporary storage and replicated data. However, due to its ephemeral nature, how do you ensure the storage is configured and ready for your application regardless of the two lifecycle events mentioned above?

This post will demonstrate how you can configure a simple service on a Linux-based VSI to do just that. 

Deploying and configuring instance storage

You can deploy a VSI with instance storage using the IBM Cloud Console, IBM Cloud CLI or Terraform/IBM Cloud Schematics. Once deployed, you can follow your operating system specific steps to configure your instance storage (i.e., format a disk and mount a file system, as you would for any other disks):

Selecting an instance storage profile in the IBM Cloud Console.

Creating a VSI with instance storage using the IBM Cloud CLI.

For this blog post, we will be using Terraform because it can automate all our manual steps:

  • Create the IBM Cloud VPC resources.
  • Deploy custom scripts used to configure our instance storage disk(s).
  • Deploy a simple application and configure it to start as a service with appropriate dependencies.

Creating the VPC and VSI

The Terraform template is available in our vpc-tutorials Github repository, which contains various companion templates to previous blog posts and solution tutorials under the vpc-instance-storage folder. If you have never used Terraform on IBM Cloud, check out the Getting Started with Terraform and Setting up your environment topics in our documentation.

A simplified view of our flow.

The deployment of the VPC resources along with a single virtual server instance (VSI) is handled through the main.tf:

  • To keep things simple in this example, the VSI is configured with a floating IP. To make things more secure, you can update the SSH inbound rules to restrict access based on your IP address or network CIDR or use a bastion host:
  • instance-storage-config-service.sh: This script is executed through the VSI user data (cloud-init) during first boot. It creates a systemd service that is configured to execute a script provided in the ExecStart directive on line 59:
  • In our main.tf, we use a Terraform null_resource to SSH into the VSI and copy our second script:
    • instance-storage.sh: The second script is executed when the instance-storage service runs, which is every time the VSI is started or the service is triggered through a start command:
      • The script starts by searching for instance storage disks that have a physical sector of 4096, a characteristic of instance storage disks. 
      • It then targets disks that have yet to be formatted. Instance storage disks will never be formatted under two conditions: new provisioning and stop and stop/start of the VSI. 
      • If any disks are found to meet these conditions, they are formatted and mounted. The mount point(s) used is based on the disk position (i.e., first = data0, second = data1 and so on). 

To help validate that this all works, we deploy a very simple application. In the app.tf, we use a Terraform null_resource to SSH into the VSI and, this time, copy two scripts:

  • app-config-config.sh: This is similar to the instance-storage-config-service.sh because it configures a service for our application (however, this script is executed by Terraform). If it fails, it returns an error that stops Terraform from proceeding. If successful, it creates a systemd service that is configured to execute a script (app.sh) provided in the ExecStart directive. The app.sh script can easily replaced by an actual application that is deployed on the VSI:
    • In line 46, through the Requires directive, we make sure that the data0 mount point is available before systemd attempts to start our application service. Otherwise, the service will fail and our application will not start.  
    • The service will retry every 30 seconds or so (RestartSec directive) and start only when the mount point is available. As a result, when the VSI is started for the first time or stopped and started, the expected mount point must be available before the app.service (i.e., our application is started):

The above should do it for our configuration. Below is our simple application used for testing:

  • app.sh: This script is meant to emulate an application that creates content inside the data0 mount point.  It will execute some write operations every 60 seconds or so:

Validate the resources/services deployed  

The README.md in the repository walks you through, step by step, in running the Terraform template and validating the deployment in more detail. 

With your deployment complete, you can confirm the working configuration:

  • List files under the /data0 mount point — there should be ~250 files. 
  • Reboot the VSI and again list files under the /data0 mount point — there should be ~250 files. 
  • Stop and start the VSI — the /data0 mount point should initially report 0 files, but eventually will report  ~250 files. 

Wrapping up

An enhancement can be made to the Terraform template to replace the Bash scripts with PowerShell scripts that could perform similar steps on a Windows operating system (i.e., configure Windows services and format/mount the new disks).  

If you have feedback, suggestions, or questions on this post, please reach out to me on LinkedIn. You can also open GitHub issues on the related code samples for clarifications. 

Was this article helpful?

More from Cloud

Enhance your data security posture with a no-code approach to application-level encryption

4 min read - Data is the lifeblood of every organization. As your organization’s data footprint expands across the clouds and between your own business lines to drive value, it is essential to secure data at all stages of the cloud adoption and throughout the data lifecycle. While there are different mechanisms available to encrypt data throughout its lifecycle (in transit, at rest and in use), application-level encryption (ALE) provides an additional layer of protection by encrypting data at its source. ALE can enhance…

Attention new clients: exciting financial incentives for VMware Cloud Foundation on IBM Cloud

4 min read - New client specials: Get up to 50% off when you commit to a 1- or 3-year term contract on new VCF-as-a-Service offerings, plus an additional value of up to USD 200K in credits through 30 June 2025 when you migrate your VMware workloads to IBM Cloud®.1 Low starting prices: On-demand VCF-as-a-Service deployments begin under USD 200 per month.2 The IBM Cloud benefit: See the potential for a 201%3 return on investment (ROI) over 3 years with reduced downtime, cost and…

The history of the central processing unit (CPU)

10 min read - The central processing unit (CPU) is the computer’s brain. It handles the assignment and processing of tasks, in addition to functions that make a computer run. There’s no way to overstate the importance of the CPU to computing. Virtually all computer systems contain, at the least, some type of basic CPU. Regardless of whether they’re used in personal computers (PCs), laptops, tablets, smartphones or even in supercomputers whose output is so strong it must be measured in floating-point operations per…

IBM Newsletters

Get our newsletters and topic updates that deliver the latest thought leadership and insights on emerging trends.
Subscribe now More newsletters