Instructions and code that show you how to use Ansible playbooks to manage the lifecycle of a secret in IBM Secret Manager.
IBM Cloud Secrets Manager is built on open-source HashiCorp Vault, and it allows you to dynamically create and manage secrets. For infrastructure or services provisioned from IBM Cloud, you need to rotate the secrets on a timely basis.
What is an Ansible playbook?
Many IT teams have a common requirement to manage the configuration for systems in an automated and repeatable way. Ansible is one of the leaders for configuration management tools, and it has a growing community of developers and industries adopting it as part of their standards.
An Ansible playbook is an automation blueprint used in a repeatable and consistent way across OS distributions. This tool is based on Python and YAML, and any developer who has familiarity with Python or YAML will find Ansible easy to use. Ansible makes use of reusable configuration tasks that are commonly called modules. We call a “play” to the combination of one or more modules and one or more plays creates an Ansible playbook. If we make an abstraction, a playbook is a list of tasks that we execute on a group of servers that are defined on an Ansible inventory.
Performing secret management functions with Ansible playbooks
In this blog post, we will discuss how to perform various functions for secret management in IBM Cloud Secrets manager, including creating a Vault token, looking up a secret group and creating/rotating secrets using an Ansible playbook. Although there are several examples provided here for various languages, this example code simplifies the process by building different tasks to accomplish the same thing using automation.
This playbook is oriented to work with a specific type of secret Key Value, but IBM Cloud Secrets Manager supports another type of secrets like arbitrary secrets, IAM credentials, SSL or TLS certificates.
Requirements
To use this Ansible playbook, you must have a valid IBM Cloud API key for services. These keys can also be stored, rotated, revoked or even leased if you only want to provide temporary access for other team members or services.
IBM Cloud API keys — combined with the right identity and access management (IAM) policy — enable access to cloud object storage, continuous delivery and other platform services. The IBM Cloud API Key is used to generate an IBM Cloud IAM token, which is an authentication token that lasts no more than one hour, by default. The IBM Cloud IAM token is used to authenticate with the IBM Secrets Vault API endpoints.
Setup
See the IBM Cloud Secrets Manager page to learn more about Secrets Manager and how to configure an instance:
---
- name: IBM Cloud secrets manager playbook
hosts: localhost
connection: local
vars:
api_key: "XXXXXXXXXXXXXX"
hostname_vault: "https://XXXXXXXXXXX.XXXXXXX.secrets-manager.appdomain.cloud"
secret_groups_name: "secrets_group_test"
secret_name: "test-kv-secret-playbook"
tasks:
- name: Create IAM Token
uri:
url: https://iam.cloud.ibm.com/identity/token
headers:
Content-Type: application/x-www-form-urlencoded
Accept: application/json
body_format: form-urlencoded
method: POST
body:
grant_type: "urn:ibm:params:oauth:grant-type:apikey"
apikey: "{{ api_key }}"
register: login
- block:
- name: Setting IAMtoken
set_fact:
iam_token: "{{ login.json.access_token }}"
- name: Create Vault Token
uri:
url: "{{ hostname_vault }}/v1/auth/ibmcloud/login"
headers:
Content-Type: application/json
Accept: application/json
body_format: json
method: PUT
body: '{"token": "{{ iam_token }}" }'
register: token_vault_rest_call
- name: Set vault token
set_fact:
vault_token: "{{ token_vault_rest_call.json.auth.client_token }}"
- name: Query available Secret groups
uri:
url: "{{ hostname_vault }}/v1/auth/ibmcloud/manage/groups"
headers:
Content-Type: application/json
X-Vault-Token: "{{ vault_token }}"
method: GET
register: secrets_group_rest_call
- name: Set secrets group id
set_fact:
secret_group_id: "{{ secrets_group_rest_call.json.data.groups | json_query(jmesquery) | join('') }}"
vars:
jmesquery: "[? name==`{{ secret_groups_name }}`].id"
- block:
- name: Create Secret group
uri:
url: "{{ hostname_vault }}/v1/auth/ibmcloud/manage/groups"
headers:
Accept: application/json
X-Vault-Token: "{{ vault_token }}"
Content-Type: application/json
body_format: json
method: PUT
body: '{ "name": "{{ secret_groups_name }}", "description": "Extended description for the secret group." }'
register: create_secret_group_rest_call
- name: Set secrets group id
set_fact:
secret_group_id: "{{ create_secret_group_rest_call.json.data.id }}"
when: secret_group_id | length == 0
- name: Check if the secret is already created
uri:
url: "{{ hostname_vault }}/v1/ibmcloud/kv/secrets/groups/{{ secret_group_id }}"
headers:
Accept: application/json
X-Vault-Token: "{{ vault_token }}"
body_format: json
method: GET
register: query_existing_secrets_rest_call
- name: Set secret id
set_fact:
secret_id: "{{ query_existing_secrets_rest_call.json.data.secrets | json_query(jmesquery) | join('') }}"
vars:
jmesquery: "[? name==`{{ secret_name }}`].id"
- name: Create a secret on specific group if not present
uri:
url: "{{ hostname_vault }}/v1/ibmcloud/kv/secrets/groups/{{ secret_group_id }}"
headers:
Content-Type: application/json
X-Vault-Token: "{{ vault_token }}"
body_format: json
method: POST
body: '{
"name": "{{ secret_name }}",
"description": "Extended description for my secret.",
"payload": {
"secret": "This a secret you cannot see this"
}
}'
register: create_secret_rest_call
when: secret_id | length == 0
- name: Set secrets ID
set_fact:
secret_id: "{{ create_secret_rest_call.json.data.id }}"
when: create_secret_rest_call.skipped is not defined
- name: Rotate the secret once
uri:
url: "{{ hostname_vault }}/v1/ibmcloud/kv/secrets/groups/{{ secret_group_id }}/{{ secret_id }}/rotate"
headers:
Content-Type: application/json
X-Vault-Token: "{{ vault_token }}"
body_format: json
method: POST
body: '{
"payload": {
"secret": "Now I have new value"
}
}'
register: rotate_secret_rest_call
when: login.status == 200
Understanding the tasks in the Ansible playbook
Let’s break down the code to understand each one of the modules in the playbook to manage the lifecycle of the secrets on the IBM Cloud Secrets Manager.
In the vars section, we define the values that are specific for each one of the Secrets Manager instances and specify the API Key to be used, the hostname of the Secrets Manager and the secrets group name and, finally, the secret name to be created/rotated:
vars:
api_key: "XXXXXXXXXXXXXX"
hostname_vault: "https://XXXXXXXXXXX.XXXXXXX.secrets-manager.appdomain.cloud"
secret_groups_name: "secrets_group_test"
secret_name: "test-kv-secret-playbook"
The first modules used in the playbook are responsible for creating an IAM Token using the API token. This is making use of the URI Ansible module to call the REST APIs on IBM Cloud and register the output from the API rest call in a variable called login. There is an Ansible block to manage the calls when the return from the API call is successful:
- name: Create IAM Token
uri:
url: https://iam.cloud.ibm.com/identity/token
headers:
Content-Type: application/x-www-form-urlencoded
Accept: application/json
body_format: form-urlencoded
method: POST
body:
grant_type: "urn:ibm:params:oauth:grant-type:apikey"
apikey: "{{ api_key }}"
register: login
- block:
. . .
when: login.status == 200
Once the IAM Token is created, the playbook creates an Ansible fact (Variable) that has the value of the IAM token. Then there is another URI module that uses the token to generate a Vault token to authenticate with the IBM Cloud Secrets Manager endpoint (HashiCorp compatible). The Vault token value is stored on a variable called vault_token:
- name: Setting IAMtoken
set_fact:
iam_token: "{{ login.json.access_token }}"
- name: Create Vault Token
uri:
url: "{{ hostname_vault }}/v1/auth/ibmcloud/login"
headers:
Content-Type: application/json
Accept: application/json
body_format: json
method: PUT
body: '{"token": "{{ iam_token }}" }'
register: token_vault_rest_call
- name: Set vault token
set_fact:
vault_token: "{{ token_vault_rest_call.json.auth.client_token }}"
Once the Vault token is generated, the playbook needs to get the secret group ID where the secret will be created/rotated. The playbook performs another API call to retrieve all the existent group IDs on the Secrets Manager instance, and based on the value of the variable set in the vars block, the playbook searches the value of the secret group ID using the name of the group ID. If the group is not found, the playbook creates the secret group (not a common scenario added for illustration purposes). The playbook stores the secret group ID in a variable called secret_group_id:
- name: Query available Secret groups
uri:
url: "{{ hostname_vault }}/v1/auth/ibmcloud/manage/groups"
headers:
Content-Type: application/json
X-Vault-Token: "{{ vault_token }}"
method: GET
register: secrets_group_rest_call
- name: Set secrets group id
set_fact:
secret_group_id: "{{ secrets_group_rest_call.json.data.groups | json_query(jmesquery) | join('') }}"
vars:
jmesquery: "[? name==`{{ secret_groups_name }}`].id"
- block:
- name: Create Secret group
uri:
url: "{{ hostname_vault }}/v1/auth/ibmcloud/manage/groups"
headers:
Accept: application/json
X-Vault-Token: "{{ vault_token }}"
Content-Type: application/json
body_format: json
method: PUT
body: '{ "name": "{{ secret_groups_name }}", "description": "Extended description for the secret group." }'
register: create_secret_group_rest_call
- name: Set secrets group id
set_fact:
secret_group_id: "{{ create_secret_group_rest_call.json.data.id }}"
when: secret_group_id | length == 0
The playbook has another module to verify if the secret is already created with the name defined on the vars section and stores the ID in a variable called secret_id. If the secret is not created, the variable value is null:
- name: Check if the secret is already created
uri:
url: "{{ hostname_vault }}/v1/ibmcloud/kv/secrets/groups/{{ secret_group_id }}"
headers:
Accept: application/json
X-Vault-Token: "{{ vault_token }}"
body_format: json
method: GET
register: query_existing_secrets_rest_call
- name: Set secret id
set_fact:
secret_id: "{{ query_existing_secrets_rest_call.json.data.secrets | json_query(jmesquery) | join('') }}"
vars:
jmesquery: "[? name==`{{ secret_name }}`].id"
Finally, the lasts two modules create the secret (if it doesn’t already exist) and rotate the value of the secret. The value of the secret is passed as part of the payload of the JSON call:
- name: Create a secret on specific group if not present
uri:
url: "{{ hostname_vault }}/v1/ibmcloud/kv/secrets/groups/{{ secret_group_id }}"
headers:
Content-Type: application/json
X-Vault-Token: "{{ vault_token }}"
body_format: json
method: POST
body: '{
"name": "{{ secret_name }}",
"description": "Extended description for my secret.",
"payload": {
"secret": "This a secret you cannot see this"
}
}'
register: create_secret_rest_call
when: secret_id | length == 0
- name: Rotate the secret once
uri:
url: "{{ hostname_vault }}/v1/ibmcloud/kv/secrets/groups/{{ secret_group_id }}/{{ secret_id }}/rotate"
headers:
Content-Type: application/json
X-Vault-Token: "{{ vault_token }}"
body_format: json
method: POST
body: '{
"payload": {
"secret": "Now I have new value"
}
}'
register: rotate_secret_rest_call
Dependencies
The dependencies are managed using a requirements file. It would consist of the following packages:
ansibleansible-lintpylintpyyamljmespath
GitHub code sample code
A GitHub repo with a copy of this code is available here.
Conclusion
IBM Cloud Secrets Manager is a SaaS offering that enables IBM Cloud customers to create, rotate, update and manage secrets from a central location. The instance is built on open-source HashiCorp Vault, which means that the Vault API is enabled to consume and manage the lifecycle of the secrets.
These APIs can be used to develop Ansible playbooks, and in this blog, we provided sample code that can be used as the base to create playbooks/roles that manage the lifecycle of a secret in IBM Secret Manager. The playbook allows you to create and look-up a secret group, create and look-up a secret and rotate the secret.