Managing the configuration of multiple environments is often the bane of many software delivery systems. Most organizations manage properties by using static property files (for example, local.properties, dev.properties, test.properties, stage.properties, and prod.properties) for different environments. I've designed many software delivery systems that way myself and advocate this approach in "Automation for the people: Speed deployment with automation." It's among the most simple approaches I've seen — but if you need to manage a software delivery system with hundreds or even thousands of individual properties for multiple applications and hundreds of instances, it won't scale and ultimately becomes unmanageable. Most failures result from someone entering a wrong value in any of what might be hundreds of different property files for a single software system.

The situation isn't hopeless, however. By effectively using automation, dynamic configuration, and a cloud infrastructure, your software delivery system can drastically reduce the need for static properties when moving software between environments. You'll learn how in this article.

Typical properties

Table 1 lists some of the typical properties that teams within an organization manage:

This isn't an exhaustive list, but it represents the more commonly used types in a typical software system.

Back to top

Traditional properties management

In most software organizations, which are composed of silos of development and operations teams, the development team (developers, testers, analysts, and so on) will often:

• Create a file (such as local.properties) to manage workstation-related properties.
• Modify properties for the "development" environment in a dev.properties file and commit it to a version-control repository. Because dynamic properties (such as IP addresses) can change, development must continually modify this file when such changes occur.
• Modify a test.properties file with the same property attributes, but with different values for testers, because they're likely using different IP addresses, files, and directories.

Configuration by the development team usually doesn't end here, because the development side of the organization typically manages other environments in addition to "development" and "test."

In a traditional organization, the development team will hand off the software package to the operations team (DBAs, systems/ops, release engineering, an so on) to deploy the software to downstream environments as part of the software delivery cycle. That team:

• Modifies properties for the "staging" environment in a stage.properties file and commits it to a version-control repository.
• Modifies similar properties for the "production" environment in a prod.properties file and commits it to a version-control repository. Because dynamic properties (such as IP addresses) can change, operations must continually update these property files when such changes occur.

I've seen teams that have a hierarchy of properties: local, environment (usually, with at least four environment property files, but often many more), application, subsystem, and instance(s). When they had hundreds of instances, they had hundreds of different property files, making it virtually impossible to manage the constant change in IP addresses, directories, and other properties. This resulted in frequent environment and deployment failures that were often extremely difficult to troubleshoot because the challenge of change management — who changed what and when? — makes it hard to identify the root cause of the error.

Externalized properties

In a well-designed software system that uses the traditional approach, all properties that can differ across environments are externalized into property files. In other words, no setting that could be variable is hard-coded into the application code. But the more properties there are, the more susceptible you are to introducing deployment errors. These errors are costly and difficult to troubleshoot and often cause delay in the "build promotion" process in getting software delivered to users.

Listing 1 is a simple example of a property file:

jboss.home=/usr/local/jboss
jboss.server.hostname=jenkins.example.com
jboss.server.port=8080
jboss.server.name=default

In nontrivial systems, a property file such as this one usually has hundreds of properties. And those properties are usually duplicated across development, test, staging, and production environments — leading to hundreds or thousands of properties to manage when delivering software to production.

Listing 2 shows a Java™ code snippet that retrieves the values from a properties file:

public PropertyReader(final String name, final ClassLoader loader) {
  try {
    InputStream is = loader.getResourceAsStream(convertName(name));
    if (is != null) {
      properties.load(is);
      is.close();
    } else {
      throw new IOException("Couldn't find property file: "
        + convertName(name));
      }
    } catch (IOException problem) {
        System.out.println("Property Reader: problem initializing");
  }
}

This is the most common approach that teams use in managing properties. But today's software delivery systems are capable of operating on an unprecedented scale. Thanks to hardware commoditization, virtualization, and cloud computing, you might be launching (and terminating) hundreds of environments when delivering and managing software systems. Your configuration-management approach must be able to scale commensurately. And (also thanks to hardware commoditization, virtualization, and cloud computing), it can.

Back to top

How dynamic configuration works

The approach I advocate in this article is to set and retrieve properties dynamically. When you define an environment using infrastructure automation tools (see "Agile DevOps: Infrastructure automation"), you can set configuration items — such as database name, and file and directory names — in a configuration database. You can do this in scripts, because the entire infrastructure is built from the ground up in scripts that are committed to a version-control repository. Furthermore, in a cloud or virtual infrastructure, you can dynamically set and retrieve configuration items such as IP addresses and domain names, which are often statically defined in traditional infrastructures.

The code in Listing 3 is a Ruby script that loads configuration items into a SimpleDB (NoSQL) database:

AWS::SimpleDB.consistent_reads do
  domain = sdb.domains["stacks"]
  item = domain.items["#{opts[:itemname]}"]
  
  file.each_line do|line|
    key,value = line.split '='
    item.attributes.set(
      "#{key}" => "#{value}")
  end
end

Username and password

Capistrano is a Ruby-based domain-specific language (DSL) for defining scripted deployments. The code in Listing 4 is from a Capistrano script that defines the database username and password based on the parameter that was defined once by the user:

set :dataSourceUsername do
  item = sdb.domains["stacks"].items["wildtracks-config"]
  item.attributes['dataSourceUsername'].values[0].to_s.chomp
end
set :dataSourcePassword do
  item = sdb.domains["stacks"].items["wildtracks-config"]
  item.attributes['dataSourcePassword'].values[0].to_s.chomp
end

Because you have complete control of the infrastructure through scripts, the database username and password can be automatically generated and stored in a secure configuration database.

This configuration is used throughout the rest of the software delivery system. The data pushed to this database is retrieved dynamically in upstream environments and used in downstream environments.

IP addresses

The code in Listing 5 is a portion of an AWS CloudFormation JSON script that dynamically sets and associates an external IP address:

"IPAddress" : {
  "Type" : "AWS::EC2::EIP"
},

"IPAssociation" : {
  "Type" : "AWS::EC2::EIPAssociation",
  "Properties" : {
    "InstanceId" : { "Ref" : "WebServer" },
    "EIP" : { "Ref" : "IPAddress" }
  }
},

You would commit a script such as this one to a version-control repository, along with corresponding automated tests.

Domain name

The CloudFormation JSON code in Listing 6 uses the AWS Route 53 service to dynamically set the A domain record for an application:

"JenkinsDNS" : {
  "Type" : "AWS::Route53::RecordSetGroup",
  "Properties" : {
    "HostedZoneName" : { "Fn::Join" : [ "", [ {"Ref" : "HostedZone"}, "." ]]},
    "RecordSets" : [
    {
      "Name" : { "Fn::Join" : ["", [ { "Ref" : "ApplicationName" }, ".", \
      { "Ref" : "HostedZone" }, "." ]]},
      "Type" : "A",
      "TTL"  : "900",
      "ResourceRecords" : [ { "Ref" : "IPAddress" } ]
    }]
  }
},

The script makes it unnecessary to configure this item manually with the Domain Name Service (DNS) provider, which is the traditional method.

Database name

The code in Listing 7 is from a CloudFormation script that defines a parameter called DatabaseName:

"Parameters": {
  "DatabaseName": {
    "Description" : "The name of the database",
    "Type": "String"
  }
}

The DatabaseName parameter is passed to an infrastructure script (such as a Puppet manifest) when the script creates a new database for the application. The entire script is committed to a version-control repository, and you write automated tests to verify that the database is accessible.

Port

The code in Listing 8 sets the external ports for the environment (and application):

"FrontendGroup" : {
  "Type" : "AWS::EC2::SecurityGroup",
  "Properties" : {
    "GroupDescription" : "Enable SSH and access to Apache and Tomcat",
    "SecurityGroupIngress" : [
      {"IpProtocol" : "tcp", "FromPort" : "22", "ToPort" : "22", "CidrIp" : "0.0.0.0/0"},
      {"IpProtocol" : "tcp", "FromPort" : "8080", "ToPort" : "8080",\
       "CidrIp" : "0.0.0.0/0"},
	  {"IpProtocol" : "tcp", "FromPort" : "80", "ToPort" : "80", \
	    "CidrIp" : "0.0.0.0/0"}
    ]
  }      
},

The only ports with external access in this example are port 22 (for SSH access), 8080 (for Apache Tomcat), and port 80 (for Apache HTTP Server). The access can be further restricted — through Classless Inter-Domain Routing (CIDR) notation — to networks or even individual nodes (see Resources).

Back to top

Dynamically speaking

In this article, you learned that configuration that is often statically defined in most software delivery systems can and should be dynamically defined so that you don't need to change it for every target environment. You saw that you can significantly reduce the number of configurable properties required for creating environments.

The next installment introduces an open source platform for continuous software delivery in the cloud. You'll learn the basic steps for running the platform, how to use the delivery pipeline, and how to run jobs to provision environments and run deployments.

Resources

Learn

• "Continuous Delivery in the Cloud: Dynamic Configuration": (Brian Jakovich, Stelligent, October 2012): Dynamically configure properties for a delivery pipeline in a cloud infrastructure.
  
  
• "Automation for the people: Speed deployment with automation" (Paul Duvall, developerWorks, January 2008): Read about externalizing properties for deployments.
  
  
• AWS CloudFormation: Learn about this AWS service for managing, provisioning, and updating resources.
  
  
• CIDR Notation: Wikipedia describes CIDR notation.
  
  
• Stay current with developerWorks technical events and webcasts focused on a variety of IBM products and IT industry topics.
  
  
• Attend a free developerWorks Live! briefing to get up-to-speed quickly on IBM products and tools as well as IT industry trends.
  
  
• Follow developerWorks on Twitter.
  
  
• Watch developerWorks on-demand demos ranging from product installation and setup demos for beginners, to advanced functionality for experienced developers.
  
  

Get products and technologies

• IBM Tivoli® Provisioning Manager: Tivoli Provisioning Manager enables a dynamic infrastructure by automating the management of physical servers, virtual servers, software, storage, and networks.
  
  
• IBM Tivoli System Automation for Multiplatforms: Tivoli System Automation for Multiplatforms provides high availability and automation for enterprise-wide applications and IT services.
  
  
• Evaluate IBM products in the way that suits you best: Download a product trial, try a product online, use a product in a cloud environment, or spend a few hours in the SOA Sandbox learning how to implement Service Oriented Architecture efficiently.
  
  

Discuss

• Get involved in the developerWorks community. Connect with other developerWorks users while exploring the developer-driven blogs, forums, groups, and wikis.
  
  
• The developerWorks Agile transformation community provides news, discussions, and training to help you and your organization build a foundation on agile development principles.
  
  

About the author

Paul Duvall is the CTO of Stelligent. A featured speaker at many leading software conferences, he has worked in virtually every role on software projects: developer, project manager, architect, and tester. He is the principal author of Continuous Integration: Improving Software Quality and Reducing Risk (Addison-Wesley, 2007) and a 2008 Jolt Award Winner. He is also the author of Startup@Cloud and DevOps in the Cloud LiveLessons (Pearson Education, June 2012). He's contributed to several other books as well. Paul authored the 20-article Automation for the people series on developerWorks. He is passionate about getting high-quality software to users quicker and more often through continuous delivery and the cloud. Read his blog at Stelligent.com.

Rate this article

Comments

Back to top

Table of contents

• Typical properties
• Traditional properties management
• How dynamic configuration works
• Dynamically speaking
• Resources
• About the author
• Comments

Next steps from IBM

Learn more, join the cloud community

---

• Community: Cloud computing conversations led by IBM community members
• Buy: Try IBM software on IBM SmartCloud Enterprise

IBM SmartCloud trial. No charge.

Unleash the power of hybrid cloud computing today!

---

Public and private solutions in one trial.

Special offers