Table of contents

Managing secrets with the Credentials and Secrets API

Safeguard credentials and secrets with access control and audit logging by using the Credentials and Secrets Management APIs.

Introduction

Security is critical when services use secrets such as database access credentials, SSL certificates, API keys, and authentication tokens. Customers are also subject to regulatory compliance that requires reporting and auditing of privileged credential usage, and forensic analysis of access. Storing and accessing credentials in plain text on applications and services poses a serious security risk. You need tools that provide access and secrets management, both for system identities and user identities.

The Credentials and Secrets Management APIs introduce a centralized mechanism that is hardened by data encryption, secure access control, authorization checks, and auditing, which addresses credential management in Cloud Pak for Data across all services. You can securely store and tightly control access to secrets (such as passwords, tokens, API keys, encryption keys, and certificates) to protect access to sensitive data; store credentials such as passwords, API keys, tokens, certificates, or other sensitive information within a vault; and access vaults by using Python in Notebooks, scripts, and other code to store and retrieve secrets.

Users that need to connect to another service that requires credentials as a form of authentication can choose from a list of existing secrets instead of being forced to enter their credentials in multiple places. Users who need to update credentials can do so in one place and no longer need to update those credentials across services.

Currently, you can store, retrieve and manage your credentials, tokens, or certificates in the internal vault. Data stored in this vault is encrypted securely.

The Credentials and Secrets APIs

You can use the Credentials and Secrets APIs to complete the following tasks:

  • Generate an authorization token

    This API lets you generate an authorization token. This access token is used in subsequent API calls. For more information, see Get authorization token in the IBM® Cloud Pak for Data Platform API.

    Endpoint
    /icp4d-api/v1/authorize
    Method
    POST

    Authorization is needed, as with other APIs, to connect and invoke the appropriate methods of this service.

    curl -kiv -X POST \
      https://<hostname>/icp4d-api/v1/authorize \
      -H 'Content-Type: application/json' \
      -d '{
        "username": "<username>",
        "password": "<password>"
      }'
    

    The access token scoped to your user-profile is returned in the response.

    Sample request
    curl -k -X POST \
      https://<my-deployment-url>/icp4d-api/v1/authorize \
      -H 'Content-Type: application/json' \
      -d '{
        "username": "admin",
        "password": "password"
      }'
    
    Sample response
    {
        "_messageCode_": "200",
        "message": "Success",
        "token": "<authorization-token>"
    }

  • Store a secret

    This API lets you create a secret for your credentials and secure it in the internal vault.

    Endpoint
    /zen-data/v1/secrets
    Method
    POST
    Parameters
    Parameter Type Description
    secret_name String A name for your secret. This must be unique across all the secrets that you own and that are currently stored in the vault.
    description (Optional) String A short description for your secret.
    secret Free-form object (JSON) The secret is schema-less, which means you can enter any type of information as valid key-value pairs. The format that you use to store the secret is used when you retrieve your secret.
    type Enumerated type:
    • generic (default)
    • certificate
    • credentials
    • key
    • token
    The type of secret.
    vault_name string The name of the vault where you would like to store the secret. In Cloud Pak for Data 3.5, only internal vault is supported.
    Sample curl request and payload
    curl -k -X POST \
      https://<hostname>/zen-data/v1/secrets \
      -H 'Authorization: Bearer <authorization-token>' \
      -H 'Content-Type: application/json' \
      -d '{
        "secret_name": "<SECRET_NAME>",
        "description": "<DESCRIPTION>",
        "secret": {
          <free-form-object-to-hold-sensitive-information>
        },
        "type": "certificate/credentials/generic/key/token",
        "vault_name": "internal"
      }'
    
    Sample request
    curl -k -X POST \
      https://<my-deployment-url>/zen-data/v1/secrets \
      -H 'Authorization: Bearer <authorization-token>' \
      -H 'Content-Type: application/json' \
      -d '{
        "secret_name": "oracle_db_login_credentials",
        "description": "These are my login credentials for the Oracle instance with SSL",
        "secret": {
          "username": "testuser",
          "password": "BoYkaIevSIW6"
        },
        "type": "credentials",
        "vault_name": "internal"
      }'
    
    Sample request
    curl -k -X POST \
      https://<my-deployment-url>/zen-data/v1/secrets \
      -H 'Authorization: Bearer <authorization-token>' \
      -H 'Content-Type: application/json' \
      -d '{
        "secret_name": "artifactory_api_key",
        "description": "This is my api_key to artifactory",
        "secret": {
          "api_key": "VugP1PCcPyX6cbvfHYFQmnY0KJyR7w4aosjGpeDY8uY3mokfkXcy0hBvJbm9FRSQ"
        },
        "type": "key",
        "vault_name": "internal"
      }'
    
    Sample response
    {
        "_messageCode_": "success",
        "message": "Successfully added secret"
    }

  • Retrieve a list of secrets stored in a vault

    This API returns a list of all secret references that are stored in a particular vault. The response contains identifiers of the secrets that you can use in subsequent API calls to retrieve and manage the actual secret itself.

    Endpoint
    /zen-data/v1/vaults/<vault_name>/secrets
    Method
    GET
    curl -k -X GET \
      https://<hostname>/zen-data/v1/vaults/<vault_name>/secrets \
      -H 'Authorization: Bearer <authorization-token>'
    
    Parameters
    Parameter Type Description
    vault_name String A path parameter that identifies the vault where some or all of your secrets might be stored.
    Sample request
    curl -k -X GET \
      'https://<my-deployment-url>/zen-data/v1/vaults/internal/secrets' \
      -H 'Authorization: Bearer <authorization-token>'
    
    Sample response
    {
        "_messageCode_": "success",
        "data": [
            {
                "secret_name": "artifactory_api_key",
                "vault_name": "internal",
                "reference": "internal::artifactory_api_key",
                "type": "key"
            },
            {
                "secret_name": "oracle_db_login_credentials",
                "vault_name": "internal",
                "reference": "internal::oracle_db_login_credentials",
                "type": "credentials"
            }
        ],
        "message": "Successfully retrieved list of secrets"
    }

    From the response, you can capture the secret_name, vault_name, and the reference to your secret, which follows the <vault_name>::<secret_name> notation.

    You must provide this secret_reference in some of the subsequent API calls. This API is intended for UI experiences. You can always form the secret_reference notation by intuition if you know the name of secret and the name of the vault that you stored it in.

  • Retrieve a secret

    This API retrieves a secret that was stored in the vault previously.

    Endpoint
    /zen-data/v1/secrets/<secret_reference>
    Method
    GET
    curl -k -X GET \
      https://<hostname>/zen-data/v1/secrets/<secret_reference> \
      -H 'Authorization: Bearer <authorization-token>'
    
    Parameters
    Parameter Type Description
    secret_reference String This parameter identifies and associates the secret with the user. This parameter follows a specific notation - <vault_name>::<secret_name>. It is important to follow this notation when you retrieve a secret.

    Both vault_name and secret_name is what you used in the request body when you stored the secret.

    Sample request
    curl -k -X GET \
      'https://<my-deployment-url>/zen-data/v1/secrets/internal::oracle_db_login_credentials' \
      -H 'Authorization: Bearer <authorization-token>'
    
    Sample response
    {
        "_messageCode_": "success",
        "data": {
            "secret_name": "oracle_db_login_credentials",
            "vault_name": "internal",
            "type": "credentials",
            "description": "These are my login credentials for the Oracle instance with SSL",
            "secret": {
                "password": "BoYkaIevSIW6",
                "username": "testuser"
            },
            "created_at": "2020-10-19T01:00:45.075586Z",
            "updated_at": "2020-10-19T01:00:45.075586Z"
        },
        "message": "Successfully retrieved secret"
    }
    Sample request
    curl -k -X GET \
      'https://<my-deployment-url>/zen-data/v1/secrets/internal::artifactory_api_key' \
      -H 'Authorization: Bearer <authorization-token>' \
    
    Sample response
    {
        "_messageCode_": "success",
        "data": {
            "secret_name": "artifactory_api_key",
            "vault_name": "internal",
            "type": "key",
            "description": "This is my api_key to artifactory",
            "secret": {
                "api_key": "VugP1PCcPyX6cbvfHYFQmnY0KJyR7w4aosjGpeDY8uY3mokfkXcy0hBvJbm9FRSQ"
            },
            "created_at": "2020-10-19T01:05:28.118135Z",
            "updated_at": "2020-10-19T01:05:28.118135Z"
        },
        "message": "Successfully retrieved secret"
    }

    The data in the response body includes information such as the secret_name, vault_name, and secret.

  • Update a secret

    This API updates a secret that you stored previously. You can update the secret itself and the description. You cannot update the secret_name, vault_name or the type. These fields are immutable and cannot be edited.

    Endpoint
    /zen-data/v1/secrets/<secret_reference>
    Method
    PATCH
    curl -k -X PATCH \
      https://<hostname>/zen-data/v1/secrets/<secret_reference> \
      -H 'Authorization: Bearer <authorization-token>' \
      -H 'Content-Type: application/json' \
      -d '{
        "description": "<updated-description>",
        "secret": {
          <updated-secret-object>
        }
      }'
    

    Updating each of the two fields is optional. You can choose to update one or the other or both.

    Parameters
    Parameter Type Description
    secret_reference String This parameter identifies and associates the secret with the user. This parameter follows a specific notation - <vault_name>::<secret_name>. It is important to follow this notation when you update a secret.

    Both vault_name and secret_name is what you used in the request body when you stored the secret.

    Sample request
    curl -k -X PATCH \
      https://<my-deployment-url>/zen-data/v1/secrets/internal::oracle_db_login_credentials \
      -H 'Authorization: Bearer <authorization-token>' \
      -H 'Content-Type: application/json' \
      -d '{
        "description": "These are my login credentials for the Oracle instance with SSL - updated on Oct 18th",
        "secret": {
          "username": "testuser",
          "password": "kfmsp9mK1rpd"
        }
      }'
    
    Sample response
    {
        "_messageCode_": "success",
        "message": "Successfully edited secret"
    }

    When your secret is updated successfully, the next time you retrieve this secret you will see the updated secret content in the response.

  • Delete a secret

    This API deletes an existing secret from the vault. This action is irreversible. This service cannot revive a secret after it is deleted.

    Endpoint
    /zen-data/v1/secrets/<secret_reference>
    Method
    DELETE
    curl -k -X DELETE \
      https://<hostname>/zen-data/v1/secrets/<secret_reference> \
      -H 'Authorization: Bearer <authorization-token>'
    
    Parameters
    Parameter Type Description
    secret_reference String This parameter identifies and associates the secret with the user. This parameter follows a specific notation - <vault_name>::<secret_name>. It is important to follow this notation when you delete a secret.

    Both vault_name and secret_name is what you used in the request body when you stored the secret.

    Sample request
    curl -k -X DELETE \
      https://<my-deployment-url>/zen-data/v1/secrets/internal::oracle_db_login_credentials \
      -H 'Authorization: Bearer <authorization-token>'
    
    Sample response
    {
        "_messageCode_": "success",
        "message": "Successfully deleted secret"
    }

    If you try to retrieve or update a secret after the secret is deleted successfully, the API throws a 404 Not Found error.

Python methods

You can invoke Python methods in Jupyter runtime environments to retrieve secrets that you stored in the vault. Users sometimes need to establish a connection with an external data source in a Jupyter Notebook to train and test models on data in that external data source. To connect to the data source, users might need to enter their authentication information in clear text in the Notebook. This poses serious security risks, especially if the user wants to collaborate with other users in the same Notebook.

With the Python methods that are included in the Credentials and Secrets management service, you can store your credentials in the vault and then securely retrieve these credentials during run time of your Notebook without the need to enter your credentials in clear text.

You can invoke Python methods in Jupyter runtime environments to retrieve secrets:

  • Get all secret references

    Method
    get_my_secret_references()

    Returns a list of secrets you have stored in the internal vault. The list does not contain any sensitive information. It contains identifiers of the secrets and metadata only.

    Example
    from icpd_core import icpd_util
    import json
    
    # You can request to view references to all secrets stored in the internal vault
    my_internal_vault_secret_references = icpd_util.get_my_secret_references()
    
    # Debugging probes for printing the above output
    json_formatted_str = json.dumps(my_internal_vault_secret_references, indent=4)
    print(json_formatted_str)
        [
            {
                "secret_name": "hive_login_credentials",
                "vault_name": "internal",
                "reference": "internal::hive_login_credentials",
                "type": "credentials"
            },
            {
                "secret_name": "jfrog_artifactory_api_key",
                "vault_name": "internal",
                "reference": "internal::jfrog_artifactory_api_key",
                "type": "key"
            },
            {
                "secret_name": "oracle_db_login_credentials",
                "vault_name": "internal",
                "reference": "internal::oracle_db_login_credentials",
                "type": "credentials"
            }
        ]
    Returns

    The returned object contains a list of references to available secrets. Each reference includes the following information:

    Parameter Description
    secret_name The name of the secret that was added to the vault.
    vault_name The name of the vault where the secret was added. Currently, this is internal only.
    reference A concatenation of the <vault_name> and <secret_name> separated by ::.

    For example, internal::jfrog_artifactory_api_key.

    type Enumerated type:
    • generic (default)
    • certificate
    • credentials
    • key
    • token

  • Get a secret

    Method
    get_my_secret(secret_reference)

    Returns a secret that you have stored in the internal vault for a specified reference. This method returns the entire secret payload in the same manner that it was stored in the vault.

    Arguments

    The get_my_secret(secret_reference) method takes the secret_reference as a functional argument. You can access this secret_reference from the output of get_my_secret_references().

    Example

    The following example retrieves a secret with secret_reference of internal::oracle_db_login_credentials:

    # Here is an example that describes how you can request the username and password
    # to the Oracle DB from the vault and use references to them in code.
    # When you know the correct secret_reference, you can request to securely access 
    # the corresponding secret.
    my_oracle_db_credentials = icpd_util.get_my_secret('internal::oracle_db_login_credentials')
    
    # Since all the information is contained in memory, nothing is exposed to other users. 
    oracle_db_username = my_oracle_db_credentials['secret']['username']
    oracle_db_password = my_oracle_db_credentials['secret']['password']
    
    # You can directly reference the username and password using the above 
    # variables without having to enter anything in clear text.
    Returns

    The returned object, which contains the secret itself, is stored in-memory in the my_oracle_db_credentials Python variable. The username and password to the Oracle DB instance is contained in their corresponding keys in the returned object.

    Example

    The following example retrieves the artifactory api_key. If you know how the api_key was stored when the secret was added to the vault, you can retrieve it easily.

    # Here is another example that describes how you can request the api_key
    # that was also previously stored in the vault
    my_artifactory_api_key = icpd_util.get_my_secret('internal::jfrog_artifactory_api_key')
    
    api_key = my_artifactory_api_key['secret']['api_key']
    Returns
    The API key is returned in the my_artifactory_api_key Python variable. You do not have to enter the key in clear text and everything is contained in memory without the possibility of it becoming a security concern.