Keeping secrets – how your cloud application should access credentials and other private data

Keeping secrets – how your cloud application should access credentials and other private data

If you’re building a web app, you probably have secrets you have to deal with:

• database credentials

• session keys

• etc

So, where do you keep these secrets? Typical ways are:

• hard-code them into your source

• require them to be passed on the command-line of your program

• get them from a configuration file

• get them from environment variables

Developers using Cloud Foundry based systems have another option:

• get them from user-provided services

This blog post will go over the advantages and disadvantages of these approaches. Examples are provided for node.js, but are applicable to any language.

secrets via hard-coding

The documentation for the express-session package shows the following example of hard-coding your secrets into your code:

// from https://github.com/expressjs/session

var express = require('express')
var session = require('express-session')

var app = express()

app.use(session({secret: 'keyboard cat'}))

This is awful:

• If you need to change the secret, you need to change the code; apply some separation of concerns, and keep your code separate from your configuration.

• If you happen to check this code into a source code management (SCM) system, like GitHub, then everyone with access to that SCM will have access to your password. That might be literally everyone.

Please, DO NOT DO THIS!

Don’t be one of these people. Instead, use one of the techniques below.

secrets via configuration files

Here is an example using require() to get a secret from a JSON file:

fs = require("fs")

var fileName = "./secret-config.json"
var config

try {
  config = require(fileName)
}
catch (err) {
  config = {}
  console.log("unable to read file '" + fileName + "': ", err)
  console.log("see secret-config-sample.json for an example")
}

console.log("session secret is:", config.sessionSecret)

This example takes advantage of the node.js feature of being able to load a JSON file and get the parsed object as a result.

If you’re going to go this route, you should do the following:

• Do NOT store the config file in your SCM, because otherwise you may still be making your secret available to everyone who has access to your SCM.

• To keep the configuration file from being stored, add the file to your .gitignore file (or equivalent for your SCM).

• Create an example configuration file, say secret-config-sample.json, which other developers can copy to the actual secret-config.json file, and use as an example.

• Document the example configuration file usage.

You now have an issue of how to “manage” or save this file, since it’s not being stored in an SCM.

secrets via command-line arguments

Here is an example using the nopt package to get a secret from a command-line argument:

nopt = require("nopt")

var longOpts = {
  "sessionSecret": String,
}

var shortOpts = {
  "s": ["--sessionSecret"],
}

var parsed = nopt(longOpts, shortOpts, process.argv, 2)

console.log("session secret is:", parsed.sessionSecret)

You can then invoke your program using either of these commands:

<code>node secret-arg.js --sessionSecret "keyboard cat"<br>
node secret-arg.js -s "keyboard cat"<br>
</code>

This is certainly nicer than having secrets hard-coded in your app, but it also means you will be typing the secrets a lot. If you decide to “script” the command invocation, keep in mind your script now has your secrets in it. Use the “example file” pattern described above in “secrets via config files” to keep the secret out of your SCM.

secrets via environment variables

Here is an example using process.env to get a secret from an environment variable:

console.log("session secret is:", process.env.SESSION_SECRET)

You can then invoke your program using the following command:

<code>SESSION_SECRET="keyboard cat" node secret-env.js<br>
</code>

Like using command-line arguments, if you decide to create a script like this, keep in mind your secret will be visible to those having access to the script.

You likely have other ways of setting environment variables when you run your program. For instance, in Cloud Foundry, you can set environment variables via a manifest.yml file or with the cf set-envcommand.

If you decide to set the environment variable in your manifest.yml file, keep in mind your secret will be in the manifest. Use the “example file” pattern described above in “secrets via configuration files” to keep the secret out of your SCM. That is, put manifest.yml in your .gitignore file, and ship a manifest-sample.yml file instead.

secrets via Cloud Foundry user-provided services

Here is an example using the cfenv package to get a secret from a user-provided service:

cfenv = require("cfenv")

var localVCAP = require("./local-vcap.json")

var appEnv = cfenv.getAppEnv({vcap: localVCAP})
var creds  = appEnv.getServiceCreds(/session-secret/) || {}

console.log("session secret is:", creds.secret)

This is my favorite way to store secrets for Cloud Foundry. In the example above, the code is expecting a service whose name matches the regular expression /session-secret/ to contain the secret in the credentials property named secret. You can create the user-provided service with the cf cups command:

<code>cf cups a-session-secret-of-mine -p secret<br>
</code>

This will prompt you for the value of the property secret, and then create a new service named a-session-secret-of-mine. You will need to cf bind the service to your application to get access to it.

There are a number of advantages to storing your secrets in user-provided services:

• A service can be bound to multiple applications; this is a great way to store secrets that need to be shared by “micro-services”, if you’re into that kind of thing.

• Once created, these values are persistent until you delete the service or use the new cf uups command to update them.

• These values are only visible to users who have the appropriate access to the service.

• Using regular expression matching for services makes it easy to switch services by having multiple services with regexp matchable names, and binding only the one you want. See my project bluemix-service-switcher for an example of doing this.

secrets via multiple methods

Of course, for your all singing, all dancing wunder-app, you’ll want to allow folks to configure secrets in a variety of ways. Here’s an example that uses all of the techniques above – including hard-coding an undefined value in the code! That should be the only value you ever hard-code. 🙂

The example uses the defaults() function from underscore to apply precedence for obtaining a secret from multiple techniques.

fs = require("fs")

_     = require("underscore")
nopt  = require("nopt")
cfenv = require("cfenv")

config = _.defaults({},
  getConfigVCAP(),
  getConfigCommandLine(),
  getConfigEnvironment(),
  getConfigFile(),
  getConfigDefault()
)

var sessionSecret = config.sessionSecret
if (sessionSecret == null) {
  console.log("no session secret was configured")
}
else {
  console.log("session secret is:", sessionSecret)
}

//-------------------------------------
function getConfigVCAP() {
  var localVCAP

  try {
    localVCAP = require("./local-vcap.json")
  }
  catch (err) {
    localVCAP = {}
  }

  var appEnv = cfenv.getAppEnv({vcap: localVCAP})
  var creds  = appEnv.getServiceCreds(/session-secret/) || {}

  return {sessionSecret: creds.secret}
}

//-------------------------------------
function getConfigCommandLine() {
  var longOpts = {
    "sessionSecret": String,
  }

  var shortOpts = {
    "s": ["--sessionSecret"],
  }

  var parsed = nopt(longOpts, shortOpts, process.argv, 2)
  return {sessionSecret: parsed.sessionSecret}
}

//-------------------------------------
function getConfigEnvironment() {
  return {sessionSecret: process.env.SESSION_SECRET}
}

//-------------------------------------
function getConfigFile() {
  var fileName = "./secret-config.json"
  var config

  try {
    config = require(fileName)
  }
  catch (err) {
    config = {}
  }

  return {sessionSecret: config.sessionSecret}
}

//-------------------------------------
function getConfigDefault() {
  return {sessionSecret: undefined}
}