A question we in CICS and CICS Explorer development often hear from customers is:
"how do I manage my CICS Explorer artifacts?"
The artifacts in question are generally CICS bundles, applications and platforms, and their contained resources.
This blog post will tell you:
- how to keep your artifacts under source control.
- how to share between different developers and deployers.
- how to progress from platform to platform, through development, test and production.
A Health Warning
First of all, I'd better make one thing clear - the canonical version of your artifacts should be those that you see in your CICS Explorer workspace, locally - not those that you see on zFS once you've exported your CICS bundles, applications or platforms. The export process performs some transformations that means that some information is lost, compared to what you have in your workspace. Therefore, if you ensure that you back up the resources in your CICS Explorer workspace and put them under source control, you've got the right idea - and can re-export them if something happens to your zFS copy or make further modifications if required. If you just back up what's on zFS, you can't put it back into the workspace and carry on modifying the bundle as if nothing happened.
And, of course, I'm sure you're all aware that backing up and source control are good things.
Installing Source Control into CICS Explorer
A great thing about CICS Explorer is that it's built on the open-source environment Eclipse. This means that it's possible to install many of the plug-ins that make the whole Eclipse ecosystem, and, for that matter, to write your own. These plug-ins include integration with Source Control systems, including Git, Mercurial, CVS, Subversion, and Bazaar. For the purpose of this blog post, I'm going to talk about working with Rational Team Concert (RTC), which also allows you to integrate task tracking, agile planning and continuous builds. We use it in the development of CICS Explorer and CICS TS, and, put simply, it's awesome. You can even try it out using the online JazzHub.
Let's get started by installing the RTC client. How you do this depends on how you installed CICS Explorer - if you installed it using IBM Installation Manager, then I recommend that you install RTC using IBM Installation Manager too. If you installed CICS Explorer by downloading it directly or adding it into an existing Eclipse installation using the Update Site method, I recommend that you install RTC using the p2 Update Site. Choose the corresponding download from the RTC downloads site. Then, either install via IBM Installation Manager or the Help -> Install New Software menu in CICS Explorer.
I downloaded CICS Explorer directly so I'm using the p2 Update Site method. I used the Help -> Install New Software menu to bring up the Install Dialog, and I then added a new update site and chose the downloaded archive file as the update site.
Once CICS Explorer has analysed that repository, it will present a list of one thing to install - the Rational Team Concert Client Feature. Tick it, and allow it to install and for CICS Explorer to restart.
RTC is now installed! Let's get it configured. Open the Work Items perspective (Window -> Open Perspective -> Other...). In the Team Artifacts view on the left, you can add new repository connections and connect to particular RTC projects. You can also accept a Team Invitation - a few lines of properties often emailed by RTC when you're added to a project, which does it all in one. I'm going to do it the manual way. First, click to Create a Repository Connection, and fill in the dialog. Here, I'm connecting to an RTC server which is running locally - great for evaluating RTC!
Once you've made the connection to the RTC repository, connect to your project. In the same view, click Manage Connected Project Areas, and tick the one you want to connect to. The view will be populated with your project, and within it you'll be able to see Builds, Enterprise Extensions, Reports, Source Control and Work Items.
That's it - your source control in CICS Explorer is now set up.
Now we're going to dive further into using RTC with CICS Explorer - it's not my place to explain the concepts of source control in RTC, but you can find this in the RTC documentation and there's an excellent comparison of the concepts between RTC and Subversion if you're more familiar with that SCM.
Getting Existing Artifacts into Source Control
You might already have a bunch of CICS bundles, applications, application bindings and platforms in your workspace, that you want to get into RTC. At this point, let me introduce my platform and application, that I'll be working with.
- I have a platform called test with two region types - java and data. The platform provides some resources for the java and data region types to work with, such as some basic useful logging and data parsing programs. I've also added a bundle called strict.policies to the platform, which will apply some strict policies so I can assert behaviour, while I might actually use less strict policies in production.
- I've got a payroll application which uses the java and data region types, with a bundle in each.
- I've got an application binding called payroll.test.binding, which provides some URI maps for the application that are specific to the test platform (they use particular ports).
The setup can be visualised as follows:
Sharing into RTC
All these resources are currently in my workspace, but I need to get them into RTC:
Select all the projects that you wish to be added to a particular component in your workspace, right-click and select Team -> Share Projects. If you have a choice of repository plug-ins, use Jazz Source Control (Jazz being the engine that RTC is based on).
On the next page, create a new repository workspace for your code changes:
Now, flow that repository workspace with your stream. I'm starting flowing with the Test stream, which means that my repository workspace will be pre-populated with the contents of that stream, and I'm initially just sharing the projects that relate to my payroll application. Now complete the wizard. You'll discover that the Pending Changes view starts tracking your repository workspace, and in it you've got a new change set called 'Share', containing all the projects you just shared. Rename it to something useful before you deliver it.
Go through the same process with your platform-related projects, reusing your new repository workspace and adding them to the Platform component this time. You'll discover your workspace eventually looks like this:
Your Project Explorer shows the projects with decorations for which workspace and component they are loaded from, and the pending changes show two outgoing change sets. If we rename them and right-click -> Deliver, the change sets are delivered to the Test stream and there are no outgoing change sets any more.
Committing Modifications to Source Control
We're now in a much better position than we were, back when it was just lonely us and our fragile workspace. Now, the projects have been backed up in our source code repository and it's also available to other people to work on and modify.
For example, Mr Alter Ego has spotted that I've not actually added any policies to the strict.policies bundle - so he's loaded his own workspace repository (flowed with the Test stream) into his local workspace and made the change, as below (note the new changes with a yellow background), to add a storage-limiting policy to the platform. Now, an abend will occur whenever a task uses more than 1MB of 64-bit storage:
He can then check those unresolved changes (the new policy file and the altered cics.xml) into a new change set, and deliver them to the stream. We've just started the process of collaborative working, and our Test stream is now ready to test our application.
From Development to Test to Production to...
You probably have a number of different platforms in your environment, which track the various different stages of your applications - development, test, and production are a subset, but they might include different testing environments at different loads or with different topologies, or a number of different production environments for different brands of your organisation. Having a form of source code management lets you manage all these different platforms and manage your applications in them.
Separating Production from Test
Here's one of the simplest layouts you could have, that gives you separation between test and production by using a different stream for each:
Here you see the repository workspaces owned by two developers, which flow their application changes into the Payroll component of the Test stream. From the Test stream, the testers can check out the current application, export it to zFS, get it installed and start testing a workload. Once the application has passed all the required checks, the change sets from Test can be delivered to Production, exported to the Production location on zFS, and be stood up against a production workload.
Dealing with Multiple Application Versions
As your application develops, you will probably find that you have several releases of the application. You may even have more than one release of the application in production at once. How do you go about making fixes to those previous releases? The key is to ensure that you take snapshots of your code at pertinent points, which then enables you to create streams that diverge from your 'latest' code at that point. Have a look at this diagram:
Here, we have our Production and Test streams as before. But we also have a Test and Production stream for the previous release, and a Test and Production stream for the release before that. Every time a new release is made public, a new Test and Production have been created to look after the code for that release. Now imagine you find a bug in Production - 2. You can deliver a fix to Test - 2, ensure that it's safe, and then flow it to Production - 2 when ready and put the fixed code into use. You can also flow that bug fix from Test - 2 to Test - 1 and to Test, if required, so that all releases get fixed.
Keeping Platform Development Separate
What you'll probably notice in this latest diagram is that I've not included the 'Platform' component in the streams any more. This is because your platforms are likely to develop in an orthogonal fashion to your applications, or you may even be using a vendor's application with your own platform. Your 'Production' platform, for example, is likely to go through its own test and production stages, allowing the latest modifications to your platform to be checked before they go live. The streams for your platform might look a little like this:
Here, the production platform has its own flow, and the test platform has its own individual flow too, from a 'test' state to a 'production' state.
These are all just examples of how you might set up a source control mechanism for a CICS cloud-based environment, and your environment is likely to differ. However, this hopefully allows you to see how CICS Explorer integrates with source control systems, and best practices for managing your code.
I've not had time to mention all the other aspects of source control integration with Eclipse, but they include:
- History view
- Side-by-side comparison between versions
- Integration with issue tracking systems, especially RTC
- Editor-based annotation of individual changes
- ...and much more
I'm sure that I've barely scraped the surface of the questions that can be answered, so feel free to comment with further suggestions.