Introduction

Today, IT data centers must provide and maintain stable, secure, and scalable services that run on multiple types of server hardware and operating systems, across several operating system versions. Efficiency is important, and administrators want to minimize the support of these services and make them a fully automated, or "lights-out," operation. In this two-part "Automating infrastructure management with Cfengine" series, we'll examine Cfengine, an automation framework for system administration and IT management.

Part 1 of this series will provide you with a Cfengine V3 Community Edition overview and information on how to build and configure a Cfengine policy/distribution server and client. Part 2 will continue with examples for using Cfengine to do many of the day-to-day tasks involved in supporting services.

This article assumes that you have already installed the base UNIX® or Linux® operating system and you are familiar with Cygwin and Linux packaging commands.

Back to top

Configuration management and Cfengine

Good configuration management establishes and maintains consistencies based on the performance, functional, and physical attributes of IT platforms and products, and their environments. It is also used to determine appropriate security features and ensure that they are properly applied. It's a complex job, and there are many configuration management tools available.

Some tools, such as the Arusha Project (ARK), can be used with Cfengine. Others use similar system management concepts, but are written in different programming languages, like Puppet, written in Ruby and using REST calls, Synctool written in Python, and SmartFrog, which is based on Java™ technology. Others like opsi (Open PC Server Integration) are written just for Windows®. IBM® Systems Director and the recently acquired IBM BigFix also provides configuration management.

Cfengine is a GNU open source configuration management framework for computer system automation. The framework is lightweight and can be built for almost any platform. It runs natively on all common platforms, which include AIX, Linux, UNIX, Apple, and Windows.

The IT Infrastructure Library, which is the most widely adopted approach for IT service management in the world, defines knowledge management as a key process for delivering services. Knowledge management ensures that a person, with the right knowledge, delivers and supports services required by the business. The premise is that services are delivered efficiently with a clear understanding of the value provided by the services with relevant information available as required. Cfengine V3 focuses on knowledge management in every phase of the management life cycle. Using a defined configuration, Cfengine can ensure that you have the proper packages, configuration files, file permissions, and processes are running in your environment.

Cfengine has two editions: a long-standing community edition and a commercial enterprise edition. For the commercial edition, there are several offerings: Nova, Constellation, and Galaxy (see Resources).

Back to top

Life-cycle management

Cfengine follows the Build-Deploy-Manage-Audit (BDMA) model of system life cycle management. BDMA comprises four phases of the system life cycle: build, deploy, manage, and audit. In the build phase, you plan policy changes, formulate desired state promises, and build a template of proposed promises, so that if the all machines make and keep these promises, the system will function seamlessly.

In the deploy phase, you publish your policy to all the autonomous clients, and each client runs an agent capable of implementing the policies and maintaining them over time, without assistance.

In the manage phase, the autonomous agent manages the system so you only need to deal with rare events that cannot be handled automatically.

Finally, in the audit phase, changes are locally audited and maintained. Decision outcomes are ensured by the design in Cfengine and maintained automatically.

Cfengine should not be thought of as a rollout system, where one attempts to force out absolute changes and reverse them in case of error. With Cfengine, you publish sequences of policy revisions, always moving forward. The state changes are managed locally by each individual client and continuously repaired for compliance with policy.

Dependencies

Cfengine requires OpenSSL and BerkeleyDB V3.2 or later. Optionally, you can build in support for the Perl Compatible Regular Expression library (PCRE), OpenLDAP, and PostgreSQL or MySQL. Some database features are only available on the commercial editions. On Windows, Cfengine requires a Cygwin environment. Also, version-control software like Subversion (SVN) can be used to control changes to configuration files.

Terms

• Promises — Actions or rules defined for a system to follow.
• Bundles — Groups of promises.
• Policies — Bundles defining the tangible knowledge we can manage.
• Classes — Attributes/variables defined as set "on" or "off" as not defined.
• Policies consist of bundles of promises.
• Keys — Can be public or private, used as authentication for client access to the server, generated on each client using cf-key, then copied to the key store on the server. In IBM using server keys alone has provided enough security.

Getting started

Cfengine can run on any UNIX server, as well as, on Windows with the help of the Cygwin environment. On a UNIX server, begin by installing OpenSSL, BerkeleyDB, and any additional packages discussed above. Cfengine is usually installed from a distribution package (such as rpm or deb), or the source can be downloaded and compiled (see Resources).

If you plan to build Cfengine from the source code, you will also need to install additional tools like gcc, flex, and bison on your UNIX/Linux server. On a Windows server, begin by installing Cygwin, then install the gcc, flex, and bison tools to compile and install the code.

The base Cfengine files are installed in subdirectories of the Cfengine work directory, /var/cfengine. This work directory is used by Cfengine for its own purpose and is defined at build time. This path is also considered to be a secure local file system. As configuration files are coded, they are placed in a configuration distribution point (/var/cfmasterfiles, for example) on all Cfengine servers. This path is what is generally placed under version control.

Building Cfengine from source code

Download the source code from the Cfengine repository. Log in to the system as a non-root user and run the following commands.

tar –zxf cfengine-x.x.x.tar.gz
cd cfengine-x.x.x
./configure --prefix=/var/cfengine 
   --sbindir=/var/cfengine/bin --localstatedir=/var/cfengine
   --with-workdir=/var/cfengine --with-openssl=/usr make

Now switch to the root user and run the following commands (binary files are placed in /var/cfengine/bin).

make install 
make installcheck

Installing Cfengine on the Policy/Distribution Server

Log in to the system as the root user and execute the following commands:

mkdir –p /var/cfengine/masterfiles
cp cfengine-x.x.x/inputs/update.cf to /var/cfengine/masterfiles
cp cfengine-x.x.x/inputs/promises.cf to /var/cfengine/masterfiles

Update the "mailto" option in the file, promises.cf, with the proper email address. Add the networks your server should accept connections from in the "body server control" section of promise.cf:

            
allowconnects => { "192.", "10.", "127.0.0.1" , "::1" };
allowallconnects => { "192.", "10.", "127.0.0.1" , "::1" };
trustkeysfrom => { "192.", "10.", "127.0.0.1" , "::1" };

Update the policy server in all configuration files to the new server environment:

cp cfengine-x.x.x/inputs/failsafe.cf to /var/cfengine/masterfiles
cp cfengine-x.x.x/inputs/cfengine_stdlib.cf to /var/cfengine/masterfiles

Cfengine components

Here is a list of noteworthy components of the Cfengine installation. (Note: If you installed Cfengine from a binary package or did any customization during a compiled installation, these base directories may be different.)

Directories

• /var/cfengine/bin — Directory with Cfengine binaries
• /var/cfengine/inputs — Directory with Cfengine configuration files
• /var/cfengine/outputs — Directory with Cfengine run reports
• /var/cfengine/ppkeys — Directory with authentication keys
• /var/cfmasterfiles — Directory with master files on the policy server
• /var/cfengine/repository — Directory containing a backup of important Cfengine files for recovery (configurable name/location)

Binary files

• /var/cfengine/bin/cf-promises — Command that checks promise syntax
• /var/cfengine/bin/cf-agent — Command that maintains promises made in common and agent bundles about the state of the system
• /var/cfengine/bin/cf-serverd — The server (daemon) used to distribute policy or data files to clients and responds to requests from cf-runagent
• /var/cfengine/bin/cf-execd — Scheduling daemon responsible for running cf-agent
• /var/cfengine/bin/cf-runagent — Command that runs cf-agent on a remote machine
• /var/cfengine/bin/cf-monitord — Daemon responsible for collecting information about the status of your system
• /var/cfengine/bin/cf-report — Command that generates summary and other reports from Cfengine embedded databases
• /var/cfengine/bin/cf-know — Command that generates an ISO-standard Topic Map from a number of promises (the Knowledge modeling agent)
• /var/cfengine/bin/cf-key — Key generation tool run once on every host to create public/private key pairs for secure communication

Configuration files

• /var/cfengine/inputs/promises.cf — Main configuration file used by cf-agent
• /var/cfengine/inputs/library.cf — Community library that contains a collection of reusable code; for V3.1.x releases, it is best to use the community library

Start the Cfengine Policy Server for the first time

Run the following commands on the server:

/var/cfengine/bin/cf-key
mkdir /var/cfmasterfiles/ppkeys
cp /var/cfengine/ppkey/localhost.pub /var/cfmasterfiles/ppkeys/root-<servername or ip>.pub
cp /var/cfengine/ppkey/localhost.priv \
/var/cfmasterfiles/ppkeys/root-<servername or ip>.priv
cp /var/cfmasterfiles/inputs/update.cf /var/cfengine/inputs/
cp /var/cfmasterfiles/inputs/failsafe.cf /var/cfengine/inputs/
/var/cfengine/bin/cf-agent –bootstrap

The Cfengine Policy server is now configured. To prepare Cfengine to serve remote clients, execute the following command: start cf-server.

In most environments, you can run cf-serverd via the configurations. Other times, it's good to have a stand-alone configuration file for your server configurations. This provides a location for external processes to relaunch cf-serverd if the process dies unexpectedly. We can also have a watchdog script to relaunch cf-serverd. To run it from the command line:

/var/cfengine/bin/cf-serverd
/var/Cfengine/bin/cf-serverd –f
/var/cfmasterfiles/prod/server/server.cf

Initial setup of a client

1. /var/cfengine/inputs/failsafe.cf
2. /var/cfengine/inputs/update.cf
3. /var/cfengine/ppkeys/root-<server>.pub

Start the Cfengine client for the first time by executing the following commands:

/var/cfengine/bin/cf-key
/var/cfengine/bin/cf-agent –K –bootstrap

Back to top

Create a binary package for your environment

At this point, you may want to create APT or RPM packages to be used for configuring your clients. One package contains the Cfengine binaries, and another package contains the minimal Cfengine configuration files and server public keys to bootstrap Cfengine, along with the cf-key and cf-agent commands listed above. With the two packages built, it is simple to configure a client with Cfengine by installing the packages.

We will demonstrate creating binary packages for the Red Hat environment. (To create a Debian package, please refer to UbuntuForums.org). Different distributions may have different requirements for creating a binary package. Refer to the documentation for your environment to see how to properly prepare this information.

Preparing an RPM

RPMs begin with a spec file that contains information about the package. Listing 1 shows an example of the Red Hat RPM Spec file.

%define debug_package %{nil}
%define bin_path \
"/bin:/usr/bin:/usr/sbin:/usr/bin/X11:/sbin:\
/opt/cfengine/bin:/
opt/cfengine/bin:/opt/freeware/bin/:/usr/gnu/bin"
Name : cfengine
Summary : A tool to maintain complicated networks
Version : 3.1.4
Release : 1
URL : http://www.cfengine.org
Vendor : %{__spec_vendor}
License : GPL
Group : System Environment/Client Management
Packager : <your name> <your email address >
Distribution : %{__spec_distribution}
Source : %{name}-%{version}.tar.gz
BuildRoot: %{_tmppath}/%{name}-%{version}-%{release}-root-%(%{__id_u} -n)
%ifos linux
Requires : db4 openssl coreutils pcre
#libacl zlib libattr e2fsprogs-libs keyutils-libs
BuildRequires : db4 db4-devel openssl openssl-devel pcre-devel
#libacl libacl-devel openssl openssl-devel pcre-devel zlib zlib-devel
                libattr libattr-devel e2fsprogs-libs keyutils-libs libselinux libsepol
ExclusiveArch : i386 x86_64 ppc ppc64
%endif
%descriptionCfengine is the standalone, open-source datacenter 
management platform run by leading enterprises since 1993. 
Customize your IT systems, align to network, business and 
regulatory needs, monitor problems, automatically repair 
and maintain systems more effectively than with proprietary 
software. Cope with new and legacy systems and adopt on a 
phased basis. Cfengine yields reduced costs, improved 
efficiency and higher return on investment for the lowest 
cost of adoption in the industry!
Authors:
--------
Mark Burgess
%prep
[ -d %{buildroot} -a "%{buildroot}" != "" ] && rm -rf %{buildroot}
mkdir -p %{buildroot}
%setup -q
%build
export CFLAGS="-g -O2 -D_FORTIFY_SOURCE=0"
%configure \
--prefix=/var/cfengine \
--sbindir=/var/cfengine/bin \
--localstatedir=/var/cfengine \
--with-workdir=/var/cfengine \
--libdir=%{_libdir} \
--with-berkeleydb=%{_libdir} \
--with-openssl=/usr \
--with-pcre
make
make DESTDIR=${RPM_BUILD_ROOT} install
%files
%defattr(-,root,root)
%{_mandir}/man?/*
/var/cfengine/*
%{_libdir}/libpromises.la
%{_libdir}/libpromises.so*
%pre
export PATH=$PATH:%bin_path
%post
export PATH=$PATH:%bin_path

RPM build process

1. Download the source code to the /usr/src/redhat/SOURCES/ folder.
2. Place the spec file in the /usr/src/redhat/SPECS/ folder.
3. Run the command rpmbuild –ba /usr/src/redhat/SPECS/cfengine.spec to build the binary and the source RPMs.
4. Install and test your new RPM by running rpm –ivh <RPM name>. Test the removal of the RPM by running rpm –e <RPM name>.

First steps with Cfengine

On initialization, cf-agent self-identifies many attributes on the host. From these attributes, "hard classes" are defined. As the process runs, other classes to be defined are known as "soft classes." Initial execution begins with a call of cf-execd –F from cron or the command line. cf-execd reads the cf-promise file, which does syntax checking. If any errors are found, cf-agent aborts the current configuration and will look to run the /var/cfengine/inputs/failsafe.cf configuration.

Figure 1 diagrammatically explains the Cfengine process.

Sample configuration

To familiarize yourself with Cfengine, listings 2-6 show sample configuration files to be placed in /var/cfengine/inputs. To verify that the syntax is correct, run cf-promises –f ./test.cf. To execute with this configuration, run cf-agent –Kiv -f ./test .cf.

########################################################
# failsafe.cf
########################################################
body common control
{
 bundlesequence => { "update" };
 inputs => { "update.cf" };
 version => "1.2.3";
}
bundle agent failsafe
{
 classes:
  "failsafe" not => "bootstrap_mode";
}

#######################################################
# Copyright (C) Cfengine AS
# This file is part of Cfengine 3 - written and maintained by Cfengine AS.
#######################################################
# promises.cf
#######################################################
body common control
{
 bundlesequence => { 
   "update",
   "garbage_collection",
   "main"
};
inputs => {
 "update.cf",
 "site.cf"
};
}
#######################################################
body agent control
{
 ifelapsed => "15";
}
#######################################################
body executor control
{
 splaytime => "1";
 mailto => "username@localhost.localdomain";
 smtpserver => "localhost";
 mailmaxlines => "30";
 # Instead of a separate update script, now do this
 exec_command => "$(sys.workdir)/bin/cf-agent -f failsafe.cf && 
         $(sys.workdir)/bin/cf-agent";
}
#######################################################
body reporter control
{
 reports => { "performance", "last_seen", "monitor_history" };
 build_directory => "$(sys.workdir)/reports";
 report_output => "html
}
#######################################################
body server control 
{
 allowconnects => { "192.", "10.", "127.0.0.1" , "::1" };
 allowallconnects => { "192.", "10.", "127.0.0.1" , "::1" };
 trustkeysfrom => { "192.", "10.", "127.0.0.1" , "::1" };
 # Makes updates and runs happen in one
 cfruncommand => "$(sys.workdir)/bin/cf-agent -f failsafe.cf && 
         $(sys.workdir)/bin/cf-agent";
 allowusers => { "root" };
}
#######################################################
# Server configuration
#######################################################
bundle server access_rules()
{
 access:
  "/var/cfmasterfiles"
  admit => { "192.", "10.", "127.0.0.1" , "::1" };
 roles:
  ".*" authorize => { "root" };
}
body action local_immediate
{
 ifelapsed => "0";
 action_policy => "fix";
}
#######################################################
## To avoid namespace conflict and reduce file footprint
#######################################################
body depth_search local_recurse(d)
{
 depth => "$(d)";
 xdev => "true";
}
body delete local_tidy
{
 dirlinks => "delete";
 rmdirs => "true";
}
body file_select local_days_old(days)
{
 mtime => irange(0,ago(0,0,"$(days)",0,0,0));
 file_result => "mtime";
}
body classes local_define(class,alert)
{
 promise_repaired => { "$(class)" };
 repair_failed => { "$(alert)" };
}

            
#######################################################
### update.cf – config file used by the base delivery system
#######################################################
bundle agent update
{
vars:
 "master_inputs" string => "/var/cfmasterfiles/inputs";
 "master_scripts" string => "/var/cfmasterfiles/scripts";
 "master_ppkeys" string => "/var/cfmasterfiles/ppkeys";
 "master_server" slist => { "localhost" };
 redhat|centos::
 "update_crontab" string => "/var/spool/cron/root";
 SuSE::
 "update_crontab" string => "/var/spool/cron/tabs/root";
 #define others as needed (darwin, macOSX should support below)
 (!SuSE).(!redhat)::
 "update_crontab" string => "/var/spool/cron/crontabs/root";
classes:
 "exec_fix" not => regline(".*cf-execd.*","$(update_crontab)");
files:
 "/var/cfengine/inputs" 
 handle => "update_inputs",
 comment => "Update the base inputs directory for client",
 perms => u_p("600"),
 copy_from => update_scp("$(master_inputs)","$(master_server)"),
 depth_search => recurse_svn("inf"),
 file_select => cf3_config,
 action => update_immediate;
 "/var/cfengine/scripts" 
 handle => "update_scripts",
 comment => "Update the base scripts directory for client",
 perms => u_p("750"),
 copy_from => update_scp("$(master_scripts)","$(master_server)"),
 depth_search => recurse_svn("inf"),
 file_select => cf3_scripts,
 action => update_immediate; "/var/cfengine/ppkeys" 
 handle => "update_ppkeys",
 comment => "Update the base ppkeys directory for client",
 perms => u_p("600"),
copy_from => update_scppubs("$(master_ppkeys)","$(master_server)"),
 depth_search => recurse_svn("inf"),
 file_select => cf3_pub,
action => update_immediate;
 exec_fix::
 "$(update_crontab)"
 handle => "update_cron",
 comment => "Ensure that cron entry exists",
 create => "true",
 action => update_immediate,
 edit_line => update_add2cron,
 classes => update_repaired("updated_cron");
commands:
 bootstrap_mode::
 "/bin/echo"
 args => "Running Bootstrap, version: $(sys.cf_version) Workdir is: $(sys.workdir) ",
 handle => "callback_bootstrap",
 comment => "Callback Bootstrap happened",
 action => update_immediate;
 failsafe::
 "/bin/echo"
 args => "Running Failsafe, version: $(sys.cf_version) Workdir is: $(sys.workdir) ",
 handle => "callback_failsafe",
 comment => "Callback Failsafe happened",
 action => update_immediate;
 !bootstap.!failsafe::
 "/bin/echo"
 args => "Running Normal, version: $(sys.cf_version) Workdir is: $(sys.workdir)",
 handle => "callback_normalrun",
 comment => "Callback Normal Run Happened",
 contain => update_root,
 action => update_immediate;
}
############################################
body perms u_p(p)
{
 mode => "$(p)";
}
body depth_search recurse_svn(d) 
{ 
 depth => "$(d)"; 
 exclude_dirs => { "\.svn" }; 
}
############################################
 add_cron::
 "Added a 15 minute schedule to crontab";
}
body file_select cf3_config
{
 leaf_name => { "^.svn", ".*\.cf" , ".*\.sh" };
 file_result => "leaf_name";
}
body file_select cf3_scripts
{
 leaf_name => { ".*\.sh", ".*.py" };
 file_result => "leaf_name";
}
body file_select cf3_pub
{
 leaf_name => { "^localhost*", ".*\.pub" };
 file_result => "leaf_name";
}
#########################################################
body copy_from update_scp(from,server)
{
 source => "$(from)";
 servers => { "$(server)" };
 compare => "digest";
 verify => "true";
 purge => "true";
 trustkey => "true";
}
body copy_from update_scppubs(from,server)
{
 source => "$(from)";
 servers => { "$(server)" };
 compare => "digest";
 verify => "true";
 purge => "false";
 trustkey => "true";
} body action update_immediate
{
 ifelapsed => "0";
}
body classes update_repaired(class)
{
 promise_repaired => { "$(class)" };
}
body action update_background
{
 ifelapsed => "0";
 action_policy => "fix";
}
body contain update_root
{
 exec_owner => "root";
 useshell => "true";
}
#########################################################
# bundle for bodies
#########################################################
bundle edit_line update_add2cron {
 classes: 
 "add_cron" not => regline("^#*[CF3 normal run]","$(edit.filename)");
 insert_lines:
 add_cron::
 "5,15,30,45 * * * * /var/cfengine/bin/cf-execd -F #CF3 normal run";
 reports:
add_cron::
 "Added a 15 minute schedule to crontab";
}

#######################################################
# Copyright (C) Cfengine AS
# This file is part of Cfengine 3 # site.cf - Site specific promises
#######################################################
bundle common g
{
vars:
 !failsafe||!bootstrap::
 "message" string => "All Looks good";
 bootstrap::
 "message" string => "Running bootstrap";
 failsafe::
 "message" string => "Running Failsafe";
}
#######################################################
# General site issues can be in bundles like this one
#######################################################
bundle agent main
{
### This would be a place to add something new!
commands:
 cfengine_3_1_4::
 "/bin/echo"
 args => "Example Command with message param: '$(g.message)'",
 handle => "echo_command",
 comment => "Example of the echo command",
 action => local_immediate,
 classes => local_define("cmd_1","life");
}
#######################################################
# Garbage collection issues
#######################################################
bundle agent garbage_collection
{
files:
 "$(sys.workdir)/outputs" 
 delete => local_tidy,
 file_select => local_days_old("5"),
 depth_search => local_recurse("inf");
}

############################################################
#
# Simple test execution – test.cf
#
###########################################################
body common control 
{ 
bundlesequence => { "testbundle" }; 
} 
###########################################################
bundle agent testbundle 
{ 
vars: 
 "size" int => "46k"; 
 "rand" int => randomint("33","$(size)");
commands: 
 "/bin/echo" 
 args => "Hello world - $(size)/$(rand)", 
 contain => standard, 
 classes => cdefine("followup","alert");
 followup:: 
 "/bin/ls"
 contain => standard;
reports:
 alert::
 "What happened?";
}
###########################################################
body contain standard
{
 exec_owner => "mark";
 useshell => "true";
}
###########################################################
body classes cdefine(class,alert)
{
 promise_repaired => { "$(class)" };
 repair_failed => { "$(alert)" };
} 

Back to top

Summary

In this article, you have learned how to initialize a Cfengine V3 policy/distribution server and Cfengine V3 client. You either installed binary packages or compiled from source code. You also saw how to build binary packages for your environment from the downloaded code. And you saw sample configuration files for running Cfengine, how to check them for accuracy and how to apply them to your running environment.

The example, while very simple, allowed you to test your environment. Part 2 of this "Automating infrastructure management with Cfengine" series will include more examples of system administration tasks.

Resources

Learn

• Cfengine information is available from Cfengine.com.
  
  
• Download the source files from the Cfengine source archive.
  
  
• Find about more about RPMs from RPM.org.
  
  
• Here is more detailed information from FedoraProject.org about creating RPM packages.
  
  
• Find a detailed guide to creating APT packages.
  
  
• Check out the IT Infrastructure Library (ITIL).
  
  
• Wikipedia provides a comparison of open source configuration management software, including Cfengine.
  
  
• Learn more about Information Management at the developerWorks Information Management zone. Find technical documentation, how-to articles, education, downloads, product information, and more.
  
  
• Stay current with developerWorks technical events and webcasts.
  
  
• Follow developerWorks on Twitter.
  
  

Get products and technologies

• Distribution packages are located in the cfengine download area.
  
  
• Build your next development project with IBM trial software, available for download directly from developerWorks.
  
  

Discuss

• Check out the developerWorks blogs and get involved in the developerWorks community.
  
  

About the authors

Rate this article

Comments

Back to top

Table of contents

• Introduction
• Configuration management and Cfengine
• Life-cycle management
• Create a binary package for your environment
• Summary
• Resources
• About the authors
• Comments

Dig deeper into Linux on developerWorks

IBM SmartCloud trial. No charge.

Unleash the power of hybrid cloud computing today!

---

Public and private solutions in one trial.

Special offers