Learn how to use the IBM Cloud CLI and Terraform to bring an encrypted cloud-init enabled image to the IBM Cloud.
Your company may have data-at-rest encryption requirements that include the images associated with provisioned Virtual Server Instances (VSIs). It is possible to encrypt an industry-standard Virtual Hardware Device (VHD) cloud-init image on-premises with your own data encryption key and provision an IBM VSI based on that encrypted image. The VHD encryption starts on one end—on-premises—and stays encrypted even as it is used on the other end—an IBM Cloud VSI. Hence, the name end-to-end encryption.
Migrating one virtual server instance
You can encrypt and migrate a cloud-init enabled image that you have built on-premises. My on-premises starting point is an IBM Cloud Virtual Server Instance. It may be helpful to follow these instructions as you are learning the process, or you can choose to skip my "on-premises" setup and feather in your own process.
At a high level, the steps will be as follows:
- Populate a running "on-premises" VSI with the software and data required.
- Encrypt the VHD image using an AES 512 bit key.
- Import the encrypted image from a bucket in IBM Cloud Object Storage.
- Provision a new VSI from the encrypted VSI template.
Using automation to avoid repetitive steps
It is possible to go through most of these steps using the GUI, with the exception of the steps described in
This is exactly what I did; once you go through the steps manually, it becomes clear that all those steps in the IBM Cloud console can be automated with their command line counterparts. Equipped with IBM Cloud CLI and Terraform, I decided to write a set of shell scripts and Terraform templates to prove it.
The files can be found in the Git repository. The scripts automate all steps you would find in the docs under Using End to End (E2E) Encryption to provision an encrypted instance:
- Create a Cloud Object Storage (COS) instance and a bucket to store the image to capture.
- Key Protect instance and Key to securely wrap the 512 bit AES encryption key.
- Encrypt the image and copy the encrypted image to COS object.
- Create an encrypted image template from the encrypted COS object.
- Create a test VSI from the encrypted image template.
On-premises mock up:
- Create an on-premises VSI with Nginx installed.
- Capture the VSI image and wait for the image to be ready.
- Copy the image to COS.
Running the automation scripts
To make it easier to understand what happens, I created several scripts mapping to the high-level steps:
Instructions on how to run the scripts can be found in the folder "README." You will need the right permissions to work with VSIs in Classic Infrastructure, with images, with Cloud Object Storage, and with Key Protect.
Once you have configured your environment with the local.env file, you can start running the scripts—
010-prepare-cos.sh, etc. Only call
090-cleanup.sh to remove all resources created by the previous scripts.
000-initialize-verify-prereqs.sh initializes the cli with the region and resource group configured in local.env and makes basic checks for the required IBM Cloud plugins and external tools.
010-prepare-cos.sh creates a Cloud Object Storage service and a bucket. The initial unencrypted VHD and, later, the encrypted VHD will be put into this bucket:
The VSIs, ssh keys for the VSIs, and image templates are found in the Classic section of the IBM Cloud console:
015-create-sshkey-in-cloud.sh uses Terraform to create an ssh key:
020-create-onprem-vm.sh uses Terraform to create a VSI that represents your on-premises computer. Load the boot disk with the data and programs you want to use in the cloud. I just put Nginx on mine:
030-capture-classic-to-cos.sh creates a VHD shown as an Image Template. It then copies the VHD into the COS bucket:
040-create-encrypter-vm.sh creates a VM that will be used to encrypt the VHD image.
050-use-vsi-encrypter-to-encrypt-cos-image.sh runs a script on the VM to get the VHD image from COS, then encrypts the VHD image and writes the encrypted VHD image back to the same bucket. The encrypting and image are done with the vhd-util command:
The dek is the file that contains the AES 512 bit encryption key. I created one using the dd command. You may have a corporate policy the defines the method used to create your key:
You will notice that the ibmcloud cos commands initialize an environment variable,
IBMCLOUD_TRACE=false, to work around a bug that may have been fixed by the time you read this:
060-prepare-kp.sh creates a Key Protect service instance, root key, and IAM authorization for block storage to access the Key Protect instance:
The VHD is encrypted at rest but it will be decrypted as it is read into memory and encrypted before writing to block storage. The next step demonstrates the use of Key Protect to wrap the data encryption key. In this step, IAM authorization is provided.
The authorization is visible in IAM:
070-create-encrypted-vm-image-template.sh creates a VSI template encrypted image from the encrypted VHD image that in cos.
A wrapped encryption key will be associated with image template. First, the binary data encryption key will be converted into ascii using base64.
Then, the ascii version will be wrapped. In the script code, notice the following:
DEK_BASE64 is the ascii version of the data encryption key.
WRAPPED_DEK_BASE64 is the encrypted representation of the ascii. The Key Protect service uses the root key identified as the key material to perform the wrap operation. This clear text string does not need to be kept secure since it can only be converted back to the DEK_BASE64 string by those that have access to the key protect instance.
Use the IAM service to restrict access to the Key Protect instance keys.
Finally, create the VSI encrypted image template:
Notice the parameters:
isEncrypted: This image is encrypted.
uri: Location of the encrypted image in COS.
wrappedDek: Data encryption key converted to base64 then wrapped by Key Protect.
crkCrn: The Cloud Resource Name (crn) of the Key Protect Key—it identifies both the Key Protect instance and the Key ID.
Here is a sample run:
Notice that the Image Type confirms it is encrypted:
Finally, a VSI can be started with the encrypted image. Nothing special here—just provision the instance:
In the Storage panel, confirm it is encrypted:
090-cleanup.sh deletes everything that was created—image templates, COS instance, Key Protect instance, VSI instances—using a mix of IBM Cloud CLI and Terraform.
The scripts listed here handle the most simple encryption use case but give the foundation of more complex setup
Keep in mind that only instances created with a cloud-init enabled image can be encrypted with this process. Also, only image templates with a single primary boot volume (or disk) can be encrypted. Secondary disks are not supported.