Step-by-step instructions for setting up enterprise-ready virtual private cloud (VPC) environments using IBM’s Infrastructure as Code solution, IBM Cloud Schematics.

The notion of Infrastructure as Code can be a compelling value proposition for organizations managing IT environments. A textual representation of cloud resources provides a single source of truth. Keeping the text in source code control makes it feel like computer program code. Software development practices can be applied, like code review by stakeholders from security, operations, and development. Finally, there is a centralized tool that takes the source description of resources and “makes it so” by comparing the actual resources to the truth before automatically creating resources.

This blog post provides step-by-step instructions for setting up enterprise-ready virtual private cloud (VPC) environments using IBM’s Infrastructure as Code solution, IBM Cloud Schematics. Sample code is provided, so you don’t need to be an expert to use this example.

IBM Cloud Schematics is a managed service that uses the Terraform open source project developed by HashiCorp as the source code. In this example, we’ll use a public GitHub repository. 

For an introduction and more information on IBM Cloud Schematics, please see “IBM Cloud Schematics: Enabling Infrastructure as Code.”

The steps

I developed sample code to make as easy as possible for you to experience Infrastructure as Code in an enterprise-grade environment. You’ll deploy virtual machines into an isolated virtual private cloud.

The following five steps create a proof-of-concept environment and should take about an hour to complete:

1. Fork my sample code repository in GitHub and clone locally.
2. Create a Schematics workspace that defines a virtual private cloud (VPC).
3. Use Schematics to create a VPC with a virtual server instance (VSI).
4. Use Schematics to install sample software on your VSI.
5. Use Schematics to create a second VSI on a new subnet in a different zone.

Steps 1, 2 and 3 will use Schematics to create the VPC resources in the following figure:

Step 1: Fork my sample code repository in GitHub and clone locally

The IBM Cloud Schematics managed service uses Terraform configuration files to declaratively define cloud resources. The Terraform configuration files are kept in a source code control system. This example uses the vpc-terraform-multicloud directory in the vpc-tutorials GitHub repository. To understand the full benefit of the Schematics service, it is best to fork this repository because during this tutorial, you will be making some changes.

If you are new to Git or GitHub,”Forking a repo” is a good place to start. My fork is https://github.com/powellquiring/vpc-tutorials (notice the name “powellquiring” in the link). From here on out, when you see powellquiring in a link, you will need to substitute your fork.

On you workstation, make a clone of your fork:

git clone https://github.com/powellquiring/vpc-tutorials
cd vpc-tutorials/vpc-terraform-multicloud/multicloud

Your working directory is the multicloud/ directory of your cloned fork. It has been initialized for you with the same contents as the part1a/ directory, so you are good to go. It contains the configuration files for the VPC resources below. Let’s take a look.

The ibm.tf file captures the configuration for the diagram in Figure 1. Open the file to see all the details (a summary is shown below). The data items are existing resources. The resource items will be created when the configuration is applied. Notice how closely it matches up with the diagram above.
 ibm.tf cut down:

provider ibm {
region       = "${var.ibm_region}"
data ibm_is_ssh_key "ssh_key" {
resource ibm_is_vpc "vpc" {
resource ibm_is_security_group "sg1" {
resource ibm_is_subnet "subnet1" {
resource ibm_is_instance "vsi1" {
resource ibm_is_floating_ip "vsi1" {
output ibm1_public_ip {
output ibm1_private_ip {

Also, notice the part1a.tf file has some Terraform locals that will be more interesting in future steps. Search for local.ibm_vsi1_security_group in ibm.tf to see how a local can be put to use.

part1a.tf:

locals {
  ibm_vsi1_security_groups = [ibm_is_security_group.sg1.id]
  ibm_vsi1_user_data = <<EOS
#!/bin/sh
EOS
}

Step 2: Create a Schematics workspace that defines a virtual private cloud (VPC)

Directly open the Schematics service or navigate to the top-left hamburger menu of your IBM Cloud dashboard and click Schematics:

Create a Schematics workspace attached to your fork by clicking the Create workspace button in IBM Cloud Schematics:

Fill out the Workspace name, Resource group, and Location (the defaults worked for me) and click Create.

Note: In the browser, navigate to your GitHub fork and then navigate within that repository to vpc-terraform-multicloud/multicloud. The browser URL is the value that Schematics is expecting, as shown in Figure 4.

Notice that in the created workspace, you are in the Settings tab:

• Copy the URL above into the GitHub or GitLab repository URL. For me it was: https://github.com/powellquiring/vpc-tutorials/tree/master/vpc-terraform-multicloud/multicloud
• Choose terraform_v0.12 from the Terraform version drop down
• Click Save template information.

Schematics reads the repository and displays several input variables—only the ssh_key_name is required, as shown in Figure 6.

Before continuing, you must have an existing SSH key in the cloud. Navigate to the SSH keys for VPC page and verify yours exists. If you do not have one, click Add SSH key and hit then View docs for more help. 

Step 3: Use Schematics to create a virtual private cloud (VPC) with a virtual server instance (VSI) 

Create the resources that are in the GitHub fork. Open the Activity panel and click Apply plan:

You will see a new log appear. It will take a few minutes to apply the plan. Click the View log link to follow the progress.

When you open the VPC infrastructure, you can see the resources that were created:

Verify your virtual server instance is powered on and then check out the VPC, Subnet, and other resources by clicking items on the left:

Step 4: Use Schematics to install sample software on your virtual server instance

Copy part1b/ into multicloud/

Let’s continue using the example by moving on to part1b. On your workstation, in the multicloud/  directory, you’ll see the following:

multicloud/ $ cd ..
vpc-terraform-multicloud $ ls -l 
total 16
-rw-r--r--   1 pquiring  staff  175 May 12 06:44 README.md
drwxr-xr-x   6 pquiring  staff  192 May 12 06:44 multicloud
drwxr-xr-x   6 pquiring  staff  192 May 12 06:44 part1a
drwxr-xr-x   8 pquiring  staff  256 May 12 06:44 part1b
drwxr-xr-x   9 pquiring  staff  288 May 12 06:44 part1c
drwxr-xr-x  10 pquiring  staff  320 May 12 06:44 part2
-rwxr-xr-x   1 pquiring  staff  849 May 12 06:44 publish.bash

The multicloud/ and part1a/ directories are identical. Schematics just applied the multicloud/ configuration. Let’s change the contents of multicloud/ to part1b/. On Linux or macOS, I do the following:

$ rm -rf multicloud
$ cp -r part1b multicloud
$ git add .
$ git commit -m 'multicloud is now part1b'
$ git push

Configuration changes

ibm_sg_app.tf contains the security group rules required to install software from the IBM supplied mirrors. Looking at the first few lines will allow you to decode the remaining contents. Endpoints available for VPC on IBM Cloud Docs describes the IP addresses. A security group is like a firewall. In this example, the rule is allowing only port 443 to the IP address 161.26.0.6.

ibm_sg_app.tf

resource ibm_is_security_group "install_software" {
  name     = "${var.basename}-install_software"
  vpc      = "${ibm_is_vpc.vpc.id}"
}
resource "ibm_is_security_group_rule" "egress_443_all" {
  group     = "${ibm_is_security_group.install_software.id}"
  direction = "outbound"
  remote    = "161.26.0.6"
  tcp = {
    port_min = 443
    port_max = 443
  }
}
...

part1b.tf also contributes security group rules to allow SSH access on port 3000. In addition, some output is declared that will be available in the log once Schematics applies the configuration.

Finally, check out the shared_app.tf. It may be simpler than you expected.

• The apt update and apt install nodejs commands install the runtime environment for the app.
• The cat > /app.js fills an app.js file with a program (you should check it out).
• The a-app.service, systemctl daemon-reload and systemctl start a-app scripts execute Linux commands that start up the app when executed and whenever the virtual server instance is rebooted.

shared_app.tf:

locals {
  shared_app_user_data = <<EOS
#!/bin/sh
apt update -y
apt install nodejs -y
cat > /app.js << 'EOF'
${file("./app/app.js")}
EOF
cat > /lib/systemd/system/a-app.service << 'EOF'
${file("./app/a-app.service")}
EOF
systemctl daemon-reload
systemctl start a-app
EOS
}

Back in the cloud console in the IBM Cloud Schematics Workspace and Settings panel, click Pull latest and Apply plan:

Check out the log that was generated. Scroll down to the bottom and notice the last view lines. These correspond to the output objects of the Terraform configuration files:

2020/05/15 16:05:10 Terraform show | 
 2020/05/15 16:05:10 Terraform show | Outputs:
 2020/05/15 16:05:10 Terraform show | 
 2020/05/15 16:05:10 Terraform show | ibm1_curl = <<~EOT
 2020/05/15 16:05:10 Terraform show |     curl 52.116.136.216:3000; # get hello world string
 2020/05/15 16:05:10 Terraform show |     curl 52.116.136.216:3000/info; # get the private IP address
 2020/05/15 16:05:10 Terraform show | EOT
 2020/05/15 16:05:10 Terraform show | ibm1_private_ip = "10.240.0.5"
 2020/05/15 16:05:10 Terraform show | ibm1_public_ip = "52.116.136.216"
 2020/05/15 16:05:10 Command finished successfully.
 2020/05/15 16:05:11 Done with the workspace action

Next, I copy/pasted the following two curl lines. Notice how they correspond to the Floating IP and Private IP addresses of vsi1. These are also visible in the log output.

Note: The curl may initially fail. It can take a few minutes for the software to install.

$ curl 169.61.245.24:3000;
{
   "hello": "world"
}
$ curl 52.116.136.216:3000/info
{
   "req_url": "/info",
   "os_hostname": "aaa-vsi1",
   "ipArrays": [
      [
         "10.240.0.5"
      ]
   ]
}

Step 5: Use Schematics to create a second virtual server instance on a new subnet in a different zone

In this step, you will configure the Terraform configuration for part1c. On your workstation, run the following commands:

$ rm -rf multicloud
$ cp -r part1c multicloud
$ git add .
$ git commit -m 'part1c'
$ git push

ibm_remote.tf introduces the vsi2 instance with its associated security group. Vsi2 is similar to the vsi1 configuration. One difference is that no floating IP is required. Your vsi1 will access vsi2 through its private IP.

part1c.tf includes a command that replace the REMOTE_IP address in the app.js file with the IP private address of the vsi2. We’ll use this technique to demonstrate vsi1-to-vsi2 connectivity.

In IBM Cloud Schematics, in the Setting tab, click Pull latest and click Apply plan again. The configuration changes will create a new vsi2 in a new subnet in a different zone, as shown in Figure 11.

Verify your work

Check out the IBM Cloud Schematics log. Here is the tail end of mine:

2020/05/15 16:57:26 Terraform show | ibm1_curl = <<~EOT
 2020/05/15 16:57:26 Terraform show |     curl 52.116.136.216:3000; # get hello world string
 2020/05/15 16:57:26 Terraform show |     curl 52.116.136.216:3000/info; # get the private IP address
 2020/05/15 16:57:26 Terraform show |     curl 52.116.136.216:3000/remote; # get the remote private IP address
 2020/05/15 16:57:26 Terraform show | EOT
 2020/05/15 16:57:26 Terraform show | ibm1_private_ip = "10.240.0.6"
 2020/05/15 16:57:26 Terraform show | ibm1_public_ip = "52.116.136.216"
 2020/05/15 16:57:26 Terraform show | ibm2_private_ip = "10.240.64.4"
 2020/05/15 16:57:26 Command finished successfully.
 2020/05/15 16:57:28 Done with the workspace action

Access to vsi2 from vsi1 is working as demonstrated by the final curl below, which I ran from my workstation:

$ curl 52.116.136.216:3000/info
{
   "req_url": "/info",
   "os_hostname": "aaa-vsi1",
   "ipArrays": [
      [
         "10.240.0.6"
      ]
   ]
}
$ curl 52.116.136.216:3000/remote
{
   "remote_info": {
      "req_url": "/info",
      "os_hostname": "aaa-vsi2",
      "ipArrays": [
         [
            "10.240.64.4"
         ]
      ]
   }
}

Congratulations! You have deployed a virtual server instance in a secure, isolated environment. Your highly isolated server is only accessible from another virtual server within the virtual private cloud that you control. It has a level of isolation appropriate for the most sensitive applications in the data center.

Conclusion

Cloud infrastructure is great for running workloads, and cloud resources often change incrementally over time.  The infrastructure source of truth was kept as code in source control. The application of the code was done centrally in the cloud and a known location (the Schematics workspace).

A Part 2 blog will connect the IBM cloud instances to AWS instances.

Learn more

• Getting started with Terraform
• Getting started with IBM Cloud Schematics
• IBM Cloud Virtual Private Cloud