![Close [x]](http://dw1.s81c.com/i/c.gif){kind=small}
Often industry solutions are deployed on a complicated and distributed topology, that involve the installation and integration of multiple middleware, and several resource configurations and enterprise application deployments. This familiar scenario can make the industry solution deployment complex and produces a large number of various loggings during the deployment and configuration. The loggings are generated to different directories and on distributed environments. It can be hard for the customer to find the logging for a specific middleware installation or a specific configuration, and if the deployment failed troubleshooting is more difficult.
Fortunately, it is possible to handle and manage logging in a centralized way during distributed industry solution deployment. In this article, we demonstrate an approach to centralize logging management for industry solution deployment that will not only make the deployment clean and neat, but also reduce overall effort for troubleshooting or deployment verification.
The IBM Software Assembly Toolkit was designed to simplify the process of creating and deploying industry solutions. It provides an open platform that you can use to integrate custom applications, products, and services with middleware components to create and distribute a total business solution as a single deliverable. The toolkit automates the installation and configuration of middleware products and software applications by assembling them into simple, fast, and reliable deployment packages. This helps developers deliver and package a complete business solution to one or more target computers in the way they want.
The toolkit provides flexible, easy-to-use packaging and deployment tools for creating and distributing complete solutions across multiple platforms.
You might consider the IBM Software Assembly Toolkit as an industry solution deployment toolkit if you are going to build your own industry solution on top of a complex topology instead of a single deployment, as remote agents are deployed on each target for the life of the installer and deployment. The toolkit also provides support for multiple operation systems, which makes your solution install development flexible and efficient. The Eclipse-based IDE may help you quickly become a "solution deployment master".
The advantages of using the toolkit for industry solution deployment include:
- You can use built-in remote agent capability, and execute relevant tasks remotely through those remote deployment agents. The remote deployment agent is a very important function in that enables the installer capabilities for distributed deployment, which makes it different from many other toolkits.
- You can use multiple technical approaches to implement the application deployment accelerators, including Java™, Ant, Shell, Command Line, and IBM Installation Manager.
- You can use Eclipse-based IDE and GUI interfaces that make both development and testing easier.
- You can share variables such as user names, installation locations, passwords, and port numbers within the entire installer.
Additionally, it supports multiple platforms, including Windows(reg/>, HP-UNIX, and Linux. Because of these advantages, we chose the IBM Software Assembly Toolkit as our industry solution deployment toolkit. We used the IBM Software Assembly Toolkit V3.2 to develop the samples and code for this article. For more features and details of the IBM Software Assembly Toolkit see Resources.
Although there are many advantages of using the IBM Software Assembly Toolkit, it is not perfect yet, especially from a logging management perspective. It doesn't manage the install and configuration logs in an centralized way, all the logs generated during deployment are delivered randomly.
Two main drawbacks in the toolkit logging are:
- All of the logs are generated by geography. During a multiple-node deployment, each node has its own logs generated according to the tasks running on this node.
- The toolkit creates temporary log files randomly, and removes them after installation terminates. You cannot find those files after you terminate or close the installer.
These drawbacks make it difficult to check and manage the toolkit logging during deployment, especially if any troubleshooting work is needed. Now, let's investigate the drawbacks more and learn how you can eliminate them and improve your solution deployment.
Begin with a simple industry solution deployment scenario
You can download a sample installer (see Download) that checks the status of IBM WebSphere Application Server in a WebSphere Application Server Network Deployment distributed environment. Assume that we have two nodes in this environment, one hosts the deployment manager, the other hosts the managed application server.
Figure 1 shows how the sample installer should appear based on this scenario.
Figure 1. The sample installer tasks screen
In the SAMPLE_INSTALLER download included with this article (see Download), there is only one task group named Check Server Status defined, and only one Install Task inside. In this install task, we import the application Sample_Server_Status, and execute server status check.
Listing 1 contains the sample code of the application development accelerator Sample_Server_Status, implemented using Ant.
Listing 1. Sample code of the application development accelerator Sample_Server_Status
<?xml version="1.0" encoding="UTF-8"?>
<project basedir="." default="default">
<!-- Make the properties from the application deployment accelerator available. -->
<property file="${IRU_propFile}"/>
<target name="default">
<echo message="Unpacked directory: ${IRU_unpacked}" />
<echo message="Logs directory: ${IRU_logsDir}" />
<echo message="Log file name: ${IRU_logFile}" />
<echo message="Deployment Accelerator properties file name: ${IRU_propFile}" />
<mkdir dir="${logLocation}"/>
<!-- execute wasadmin with the specified jython script and property file -->
<exec executable="${WASHome}/bin/serverStatus.bat">
<arg line="-all > '${logLocation}/serverStatus.log'" />
</exec>
<!--validate execution result-->
<loadfile property="logcontent" srcFile="${logLocation}/serverStatus.log"/>
<condition property="status.finished">
<or>
<contains string="${logcontent}" substring="ADMU0508I" casesensitive="no"/>
<contains string="${logcontent}" substring="ADMU0509I" casesensitive="no"/>
</or>
</condition>
<echo message="Success=${status.finished}" />
<fail message="Failed to verify install task. Please refer to $
{logLocation}/serverStatus.log on target node for detailed information.">
<condition>
<isfalse value="${status.finished}" />
</condition>
</fail>
</target>
</project>
|
The code in Listing 1 executes the serverStatus.bat command under the WebSphere Application Server home bin directory and generates a log file under a log file folder. After the execution is finished, it loads the log file and validates if there is a successful string output into the log file. If not, the Ant script will notify you to refer to the logging on a remote target node for detailed information. Both WASHome and logLocation are user input variables defined in the application development accelerator.
As shown in Figure 2, the sample installer also has two target groups, the deployment manager node and managed node. Each target node has the Sample_Server_Status application development accelerator running on top of it.
Figure 2. Sample installer targets
After you finish running the sample installer on your environment, a logging file is created.
On your deployment manager you see C:\Samples\ServerStatusLogs\serverStatus.log:
DMU0116I: Tool information is being logged in file: C:\Program Files\IBM\WebSphere\AppServer\profiles\SampleDM\logs\serverStatus.log ADMU0128I: Starting tool with the SampleDM profile ADMU0503I: Retrieving server status for all servers ADMU0505I: Servers found in configuration: ADMU0506I: Server name: dmgr ADMU0508I: The Deployment Manager "dmgr" is STARTED |
On your managed node you see C:\Samples\ServerStatusLogs\serverStatus.log:
ADMU0116I: Tool information is being logged in file: C:\Program Files\IBM\WebSphere\AppServer\profiles\SampleManaged\logs\serverStatus.log ADMU0128I: Starting tool with the SampleManaged profile ADMU0503I: Retrieving server status for all servers ADMU0505I: Servers found in configuration: ADMU0506I: Server name: Sample.AppSvrNode01.0 ADMU0506I: Server name: nodeagent ADMU0509I: The Application Server " Sample.AppSvrNode01.0" is STARTED ADMU0509I: The Node Agent "nodeagent" is STARTED. |
Note that in our sample, the managed node has two WebSphere instances, the nodeagent and the application server. Sample.AppSvrNode01.0 is the application server name.
The serverStatus.log files are generated because developers explicitly add logger creation statement in their application development accelerator's code, so we call them application related logs. In addition to this type of log, the toolkit temporarily generates log files that record the runtime status and execution process, thus we call them temporary logs.
There are two kinds of temporary logs generated at runtime. To find their locations, you have to open the deployment task and read the detailed messages, as shown in Figure 3, which is a screen capture of the Sample_Server_Staus task in the sample installer.
Figure 3. Deployment task detailed messages
The unpacked directory outlined first (in red) is named IRU_unpacked, while the log directory outlined second (in green) is named SolutionEnabler.
Before your installer terminates, find a temporary folder named "iru" under c:\tmp. (If you are using a Linux operation system find it under /tmp.) Figure 4 shows the iru directory file structure.
Figure 4. File structure of the iru directory
In that folder you will find your application script (MainProgram.xml), the ant binaries (iru_ant) and other IBM Software Assembly Toolkit built-in runtime support jar files (extenalSupportJars). The toolkit creates the iru folder during runtime because it is in the runtime environment where your customer's install scripts execute.
Note that you will only find the iru folder while running your installer, after the install tasks are finished, the folder is removed automatically.
Unlike the IRU_unpacked log, the location of SolutionEnabler log is random, and for each time you run a toolkit deployment task, a new SolutionEnabler log folder is generated, with a different number appended. Figure 5 shows the file structure of the SolutionEnabler log.
Figure 5. File structure of the SolutionEnabler log
There is a log folder under SolutionEnabler folder. The log folder contains all the logs that record the progress and execution status of each task.
At this point you might be confused by the logging mechanism of the IBM Software Assembly Toolkit.
First, all the loggings (including application related logs and temporary logs) are generated by geography. Each node that has the server status check application development accelerator running has serverStatus logging created. Thus you have to navigate among different physical machines frequently to gather or validate logs. The work effort is increased if the installer fails and you need to troubleshoot.
Second, the temporary logs can be troublesome. The IRU_unpacked logs that are created during runtime, are removed after the deployment terminates, regardless on whether the deployment succeeds or fails. The SolutionEnabler logs are created during runtime, but the logs are placed randomly, and are not removed after termination. Both of these problems make it is hard to manage the logs and use to troubleshoot or trace the deployment status.
Improve the deployment with logging centralization features
Generally, most industry solution installers generate and manage loggings using a stage server that dispatches and executes deployment packages. Unfortunately, the logging is dispersed.
Figure 6. A common logging mechanism
However, we offer an improved sample installer with logging centralization features. In our improved version, a stage server dispatches and executes deployment packages, but all the deployment related logging, including the application related logs, IRU_unpacked logs and SolutionEnabler logs, are retrieved and migrated to a specific logging server, with no logging left dispersedly on target servers.
The logging server can be a separate node, or it can be one of the target machines, as shown in Figure 7.
Figure 7. Suggested logging mechanism
First you want to collect all the deployment logging on a specific logging server. We recommended that you add an extra user input to the toolkit installer wizard for your logging server. As shown in Figure 8, the variable logHost (highlighted in red) points to the host of the centralized logging server. You can download the sample code to get more about the improved installer.
Figure 8. Tasks of improved sample installer
Next, to add logging centralization feature, you want improve the sample code implementation as shown in Listing 2.
Listing 2. Add logging centralization feature
<?xml version="1.0" encoding="UTF-8"?>
<project basedir="." default="default">
<!-- Make the properties from the application deployment accelerator available. -->
<property file="${IRU_propFile}"/>
<target name="default">
<echo message="Unpacked directory: ${IRU_unpacked}" />
<echo message="Logs directory: ${IRU_logsDir}" />
<echo message="Log file name: ${IRU_logFile}" />
<echo message="Deployment Accelerator properties file name: ${IRU_propFile}" />
<mkdir dir="${logLocation}"/>
<!-- execute wasadmin with the specified jython script and property file -->
<exec executable="${WASHome}/bin/serverStatus.bat">
<arg line="-all > '${logLocation}/serverStatus.log'" />
</exec>
<!-- get local hostname -->
<exec executable="hostname" outputproperty="hostName"/>
<!--4 Steps migrate logging to logging server -->
<!--Step1: migrate Application Related Logging to logging server -->
<copy file="${logLocation}/serverStatus.log" todir="//${logHost}/c$
/Samples/${hostName}Log/"/>
<!--Step2: migrate IRU Unpacked Logging to logging server -->
<copy todir="//${logHost}/c$/Samples/${hostName}Log/IRU_Unpacked/">
<fileset dir="${IRU_unpacked}"/>
</copy>
<!--Step3: migrate Solution Enabler Logging to logging server -->
<copy todir="//${logHost}/c$/Samples/${hostName}Log/Solution_Enabler/">
<fileset dir="${IRU_logsDir}"/>
</copy>
<!--Step4: clear Logging from target server -->
<delete dir="${logLocation}"/>
<!--validate execution result -->
<loadfile property="logcontent" srcFile="//${logHost}/c$/Samples/
${hostName}Log/serverStatus.log"/>
<condition property="status.finished">
<or>
<contains string="${logcontent}" substring="ADMU0508I" casesensitive="no"/>
<contains string="${logcontent}" substring="ADMU0509I" casesensitive="no"/>
</or>
</condition>
<echo message="Success=${status.finished}" />
<fail message="Failed to verify install task. Please refer to //${logHost}
/${logLocation}/../${hostName}Log/serverStatus.log for detailed information.">
<condition>
<isfalse value="${status.finished}" />
</condition>
</fail>
</target>
</project>
|
In this improved implementation, we reuse the status check logic. The Ant script executes wsadmin command to get the server status as before, and then, the logs are generated.
To manage and migrate those logs in an centralized approach, we added a four-step logging migration to copy the logging from target node to logging server, that logging server can be a separate one from the target machines, or it can be one of the target machines.
The loggings on the centralized logging server can be categorized by the folder name. We name the folder by target machine’s host name. That means the logging gathered from different targets are put into different folders.
As commented, we migrate the application related logs, server status logs, to the logging server in step 1. We migrate the IRU_Unpacked logs and SolutionEnabler logs respectively in step 2 and step 3. Then in step 4, the last step, we clear all the logging from target servers.
By doing so, the validation logic can be executed on the centralized logging server, and if the validation fails, users can navigate to the logging server to refer to detailed information about the failuer.
Now, when you finish running the improved sample installer in your environment, the logging files are no longer generated on your target machines. All the loggings are swept and migrated to the centralized logging server.
Figure 9 shows the file structure on logging server. Test4DM is the host name of deployment manager node, while Test4Mgd is the host name of the managed node.
Figure 9. File structure as a result of improved logging
In this article, you learned about a logging centralization approach that can greatly reduce the cost of managing logging when deploying your industry solution.
With the help of this logging centralization approach, all the install and configure logs can be managed in a centralized way, for developers and customers, there is no need to worry about where to look up related deployment logs, no need to worry about when the temporary logs would be deleted, the only thing they need to do is provide a centralized logging directory. After that, all the logs can be collected from distributed target machines and put in to the centralized directory automatically.
| Description | Name | Size | Download method |
|---|---|---|---|
| Sample installer and improved installer | sampleCode.zip | 59KB | HTTP |
Information about download methods
Learn
- IBM Software Assembly Toolkit Information Center: Find more
toolkit resources.
-
Apache Ant website: Learn more about
this Java library and command-line tool.
- developerWorks
Industries: Find more industry-specific technical resources for
developers.
- developerWorks
WebSphere: Explore more WebSphere-related technical resources for
developers.
- developerWorks on
Twitter: Join today to follow developerWorks tweets.
- developerWorks podcasts: Listen to interesting interviews and
discussions for software developers.
-
developerWorks on-demand
demos: Watch demos ranging from product installation and setup for
beginners to advanced functionality for experienced developers.
Get products and technologies
-
Evaluation
software: Download a trial version, work with product in an
online, sandbox environment, or access it in the cloud.
Discuss
-
developerWorks profile: Create your profile today and set up a watchlist.
- developerWorks
community: Connect with other developerWorks users while exploring
the developer-driven blogs, forums, groups, and wikis.
Zhuo Zhao is a software engineer working in the IBM China Development Lab. She has many years of experience in Industry Solution development and deployment. Currently, Zhuo is working on automated deployment for IBM Integrated Information Core.
Jing Fan is a staff software engineer on the Industry Solution Development Team. She is responsible for Integrated Information Core installer development and deployment. Jing is experienced working with the IBM Solution Assembly Toolkit and Install Anywhere.
