Creating a custom integration node setup on a virtual machine using Chef
ImranShakir 2700063YK3 Visits (4965)
Following on from the
The power of using infrastructure technologies such as Chef, is threefold:
The Hypervisor Edition of IBM Integration Bus Version 9 was released at the same time as the base product. You can use the Hypervisor Edition to deploy Integration workloads with technologies such as PureApplication Systems and IBM Workload Deployer. By using Chef, you can achieve a more flexible way of building a virtual image that includes IBM Integration Bus. The drawback of creating this flexibility is the extra time required to obtain the base operating system image, and package the final image for whichever platform you want to run it on. The benefit of using Chef over the Hypervisor Edition in this situation is the ability to customize the image with a specific integration node configuration and the option to include other software components in the image as well.
To demonstrate the flexibility of the Chef approach, consider this example, in which you have two teams, each requiring 3 integration nodes to be set up. One for development, one for test and one for production. Amongst the provisioning options available are two options (respectively displayed in the diagrams below):
There are good reasons to choose either option depending on your situation. In this article, you will see that the integration nodes can still be described as Chef artefacts before that decision is made. Because of the “easily disposable” nature of virtual machines, you're therefore able to provision using one way to start off with, and then dispose of the virtual machines and provision the other way if you change your mind.
The scenario that you will walk through in this article will be a little simpler than this, involving two integration nodes and just one virtual machine. After walking through the tutorial, I will discuss how you may extend the concepts learned in the walk through to fit the scenario described above.
How do I build virtual images?
The prerequisites are:
The steps are:
The steps that I wrote are specific to VMWare Workstation 10 and Ubuntu, but I have tested this process on RedHat on VMWare Workstation 10 as well, so feel free to get in contact if you're attempting that.
Create the virtual machine
Install chef-solo on the virtual machine
Once the guest operating system has installed and booted, install chef-solo on it, by following all the instructions below on the guest operating system.
Host the IBM Integration Bus for Developers image
There are instruction on where to get the image in the
Any of these locations need to be accessible from the virtual machine. The quickest way to check if this is true is to open the browser in the virtual machine and type in the location of the image, which will start with ftp://, http:// or file:// depending on the hosting location you chose above. If the package starts to download, you know you've got it right!
Get the ibm_integration_bus cookbook
Now you need to set up your environment for running chef solo.
Note that now running `ls` shows you the directory, 'ibm
Create integration node descriptions as JSON
Consider now a simpler scenario than the one described in the introduction. You want to create two test integration nodes for our developer 'Tom'. Tom has been working with the IBM Integration Bus for Developers up until now, but now he is ready to put his solutions into test. He's been working on two separate projects, called 'Co-ordinated Request Reply' (CRR) and 'Batch Processing' (BP).
These two scenarios require two different integration node setups. The cookbook uses Chef a databag to describe integration node setups as JSON documents (data items). Take a look at the sample provided in the root directory of the ibm_integration_bus cookbook, named 'def
For now, navigate up to your 'chef' directory, and create a directory named 'data_bags' which contains a directory named 'iib_nodes':
Each databag item in the 'iib_nodes' data bag directory corresponds to a description of an integration node that could be provisioned during your Chef run. You're going to need to create two integration nodes, so copy across the template file to your 'iib_nodes' directory, by using either a file browser or the commands:
Now you're going to customize these descriptions to suit Tom's needs. Use a text editor (I used gedit from the command line `gedit filename`) to edit these descriptions.
You wish to set the following properties for integration node one:
To achieve these properties, I edited the “crr_node.json” document to contain the content:
Integration node two requires a slightly different setup to TOM_CRR_NODE. Note that you need to set all the ports to values that are not the default, so that the two integration nodes don't clash:
The JSON for this is:
Run the cookbook
You can now run the cookbook using chef-solo. This creates a user for IBM Integration Bus, installs the IBM Integration Bus package, tunes the operating system, and sets up the integration nodes as described in the above JSON documents.
solo.rb is a configuration file for the Chef executable itself. In it, specify three directory paths.
You configure Chef by using Ruby files. This is what my solo.rb looks like:
In order to generate the “account_password”, you need to create a shadow_hash of the password. Use the command:
In this scenario:
Now execute the Chef process. Run:
Feel free to make a cup of tea or similar at this stage – the time taken to download the file will depend on the speed of your network connection. Once the IBM Integration Bus for Developers image has been downloaded by the Chef client, the rest of the process should take 20-30 minutes.
Verify the integration node setup
To verify some of the other properties, try making a REST call to one of the integration nodes. To do this I used RESTClient – which I installed on the default FireFox installation on the virtual machine. The HTTPRequest should look like:
Note the structure of the response by clicking in “Response Body (Highlight)”. The structure here contains more information than the structure that is provided as input to the Chef script, but still has the same basic structure. A small part of the full description of an integration node can be re-created by using the Chef scripts, by using an integration node's REST response as input to the Chef script! (Although, the properties URL still needs to be called separately, and copy/pasted into the databag item.)
After running through these instructions you should have a working virtual machine with two integration nodes and a working installation of IBM Integration Bus. This leaves you free to utilize IBM Integration Bus with all the freedoms that virtualization gives you: easy transfer to different hardware, simple exploitation of snapshot functionality, or use in a larger IaaS provisioning framework.
This article has demonstrated the two key concepts of the Chef scripts:
You can see how both of these allow large amounts of flexibility in the scenario where you have two teams, each requiring a development, test and production integration node. Not only can you create the node descriptions before deciding which machines to actually provision them on, virtualization and automation means that both setups can be attempted and scrapped easily before a final conclusion is reached.
Extending to the original scenario
To extend the custom integration node scenario so that it applies to the original scenario, which is described with six integration nodes and two or three virtual machines, it becomes clear that one thing is necessary: a central area to store all of the information that is kept in the Chef directory.
The cookbook, the data bag items and JSON configuration can no longer be stored on just the single machine, as multiple machines (or Chef "nodes") will need access to them. The prev
You can also store your data bag items in the Chef server. First create the data bag, and from the work station run:
Then to create a data bag item (on Windows, use a similar text editor on Linux):
This gives you a stub of a JSON document in a text editor, into which you can copy and paste the example data bag item and edit it to your liking.
Once you have six data bag items in your Chef server, let's call them:
You are free to create role definitions that describe the 5 potential virtual machines that could be created in the original scenario. The only real difference between them needs to be the 'iib_nodes' attribute for the 'ibm
In the situation where you want to split the nodes by type, you would have three role definitions, with different 'iib_nodes' attributes, which are:
Then to create this configuration, you would need only to bootstrap the 2 or 3 nodes, specifying those role definitions in your run list. If these are virtual machines, they're easy to destroy if you feel the decision was wrong, and you can then provision another set of virtual machines and bootstrap those instead.