Introduction

Management Beans (MBeans) provide a framework within which application developers can easily provide local and remote management of their applications. It can also provide a standardized interface through which an application’s client and other 3^rd-party clients can interact with an application. Websphere Application Server supports applications that utilize MBeans on both non-z/OS platforms (also referred to as distributed platforms) and the z/OS platform. Although this support is uniform across platforms, you may need to make some adjustments on the z/OS platform for your MBeans to work as expected.

This article assumes some familiarity with developing applications using MBeans for WebSphere Application Server. The error messages were generated using Websphere Application Server Version 6.1.0.19 running on z/OS Version 1 Release 9.

Back to top

MBean descriptor files

MBean descriptor files describe the type of MBean as well as the operations supported by the MBean. Listing 1 shows an example of a descriptor file for a simple MBean called BackupServiceMBean. We will use this example throughout our discussion.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE MBean SYSTEM "MbeanDescriptor.dtd">
<MBean type="BackupServiceMBean">

<operation name="listBackups" role="operation" type="java.util.Collection"
 targetObjectType="objectReference" impact="INFO">
    <signature>
      <parameter name="backupDirectory"
          description="backup directory" type="java.lang.String" />
    </signature>
  </operation>

</MBean>

Back to top

Fully qualify all MBean descriptor type (class) names

On the distributed platforms, when specifying Java language type (class) names, you may omit the package name and simply use the name of the class. On z/OS however, the fully-qualified type name is required. Using the example in Listing 1 above, if the parameter type was simply String instead of java.lang.String, on the z/OS platform you would receive an error similar to that shown in Listing 2a when you attempt to activate the MBean.

com.ibm.websphere.management.exception.AdminException: 
ADMN0005E: The service is unable to activate MBean: type BackupServiceMBean, 
collaborator com.backup.mbeans.impl.BackupServiceMBeanImpl@29c029c, 
configuration ID BackupServiceMBeanId, descriptor BackupServiceMBean.xml.

This error may show up in the servant job log and/or may be part of the exception message that your code receives. Additionally, the ffdc directory, in <WAS_INSTALL_ROOT>/AppServer/profiles/<PROFILE_NAME>/logs/ffdc will contain one or more log files related to the error. Listing 2b shows a snippet of one of those log files. (<WAS_INSTALL_ROOT> is the location where you installed WebSphere Application Server in the file system and <PROFILE_NAME> is the name of the applicable profile.)

Exception = java.lang.NoClassDefFoundError
Source = com.ibm.ws390.management.proxy.ServantMBeanInvokerProxy.newInstance
probeid = 555
Stack Dump = java.lang.NoClassDefFoundError: String
   at java.lang.Class.getDeclaredMethodsImpl(Native Method)
   at java.lang.Class.getDeclaredMethods(Class.java:664)
   at java.lang.Class.getInterfaceMethodCountImpl(Class.java:946)
   at java.lang.Class.getInterfaceMethodsImpl(Class.java:961)
   at java.lang.Class.getMethods(Class.java:905)
   at sun.misc.ProxyGenerator.generateClassFile(ProxyGenerator.java:425)
   at sun.misc.ProxyGenerator.generateProxyClass(ProxyGenerator.java:322)
   at java.lang.reflect.Proxy.getProxyClass(Proxy.java:514)
   at java.lang.reflect.Proxy.newProxyInstance(Proxy.java:594)
   at com.ibm.ws390.management.proxy.ServantMBeanInvokerProxy.newInstance(
ServantMBeanInvokerProxy.java:621)
   at com.ibm.ws.management.ControlAdminServiceImpl.registerProxyMBean(
ControlAdminServiceImpl.java:724)
   at com.ibm.ws.management.ControlAdminServiceImpl.activateProxyMBean(
ControlAdminServiceImpl.java:307)

Back to top

Ensure all methods in MBean interface have a matching signature in the descriptor

On the distributed platforms, you can invoke a method on an MBean object even if an entry does not exist for that method in the MBean descriptor. Using the example in Listing 1, if you did not define the listBackups method in the descriptor file, but instead defined it only in the MBean implementation, the call would still succeed. On z/OS however, any method on an MBean object that is to be invoked through the MBean interface must have a matching method signature in the MBean descriptor. Listings 3a and 3b depict the failure you might see on z/OS when you attempt to invoke the method. The error in Listing 3a might show up in the servant job log and/or be part of the exception message that your code receives.

javax.management.ServiceNotFoundException: Operation listBackups not in ModelMBeanInfo
at javax.management.modelmbean.RequiredModelMBean.invoke(RequiredModelMBean.java:87  
at com.sun.jmx.mbeanserver.DynamicMetaDataImpl.invoke(DynamicMetaDataImpl.java:231)
...
Caused by: javax.management.MBeanException: Operation listBackups not in ModelMBeanInfo
at javax.management.modelmbean.RequiredModelMBean.invoke(Req

com.ibm.ws.management.AdminServiceImpl.invoke 679
Exception = javax.management.MBeanException
Source = com.ibm.ws.management.AdminServiceImpl.invoke
probeid = 679
Stack Dump = javax.management.MBeanException: 
  Operation listBackups not in ModelMBeanInfo
  at javax.management.modelmbean.RequiredModelMBean.invoke(RequiredModelMBean.java:879)
...
Caused by: javax.management.ServiceNotFoundException: 
Operation listBackups not in ModelMBeanInfo

Back to top

Ensure the MBean security policy loads successfully

Each MBean has a security policy associated with it. You either explicitly define the security policy, or the system defines a default policy based on the MBean type if a security policy is not found. Listing 4 shows a sample security policy for the BackupServiceMBean.

<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE role-permission SYSTEM "RolePermissionDescriptor.dtd" >
<role-permission>
  <resource>
    <resource-name>BackupServiceMBean</resource-name>
    <class-name>com.backup.mbeans.BackupServiceMBean</class-name>
    <description>This is the Backup Service MBean security descriptor.</description>
  </resource>
  <security-role>
    <role>
      <role-name>BackupAdministrator</role-name>
    </role>
  </security-role>

  <method-permission>
    <description>BackupServiceMBean method permission table</description>
    <role-name>BackupAdministrator</role-name>
    <method>
       <resource-name>BackupServiceMBean</resource-name>
       <method-name>*</method-name>
    </method>
  </method-permission>

</role-permission>

On distributed systems, the MBean security policy XML files usually reside inside an EAR (Enterprise Archive) or some other similar container alongside the descriptor file, so that the system will be able to find it. On a z/OS system, this may not be the case. You may need a couple extra tweaks before your MBean security policy is loaded successfully.

You usually notice that your security policy is not being loaded successfully when either authorized users fail to access MBean methods they should have access to, or when non-authorized users are granted access to methods they should not have access to. To confirm that the problem is indeed with the loading of the custom security policy, enable the trace option com.ibm.ws.management.*=all.

One way to turn on this option is through the Integrated Solutions Console (ISC):

1. After logging into the ISC, go to Troubleshooting > Logs and Trace > {SERVER_NAME} > Change Log Detail Levels, where {SERVER_NAME}is the name given to the server you are running.
2. If your MBean is loaded at WebSphere Application Server startup, then you should add the option to the Configuration tab.
3. Save the changes to the Master Configuration, and restart WebSphere Application Server.

When the Mbean loads with this option turned on, if WebSphere Application Server cannot load your security policy you get an error similar to that in Listing 5 in your control region’s job log.

Trace: 2009/06/25 16:32:12.295 01 t=6C5E88 c=UNK key=S2 (13007002)
 ThreadId: 0000001a
 FunctionName: com.ibm.ws.management.descriptor.MBeanDescriptorManager
 SourceId: com.ibm.ws.management.descriptor.MBeanDescriptorManager
 Category: FINEST
 ExtendedMessage: can not find the system descriptor file; 
 BackupServiceMBeanSecurity.xml

Even if you don’t suspect that your security policy is not being loaded, it is still a good idea to turn on the trace option to verify this as the default security policy might be very similar to yours. Remember to turn off the trace when it is no longer needed because it will generate an enormous amount of messages in the logs and could negatively impact performance.

After you’ve confirmed that your security policy isn’t being loaded, follow these steps to resolve it:

1. Place a copy of your MBean XML files (security policy and descriptor files) in a JAR file that you will install into WebSphere Application Server’s plugins directory (<WAS_INSTALL_ROOT>/AppServer/plugins). If your application already has a JAR file installed in the plugins directory, then you may use that JAR file. Otherwise, you will need to create a new JAR file for your application. The XML files should be located at the root level of the JAR file hierarchy. Listing 6 shows the contents of a sample JAR file for the BackupServiceMBean.
   
   Listing 6: Sample JAR contents
   

   META-INF/MANIFEST.MF META-INF/ BackupServiceMBeanSecurity.xml BackupServiceMBean.xml plugin.xml

   
   
2. In your plugin.xml file, add an extension point for com.ibm.wsspi.extension.mbean-provider as in Listing 7.
   
   Listing 7: Sample plugin.xml
   

   <?xml version="1.0" encoding="UTF-8"?> <?eclipse version="3.0" ?> <plugin id="com.backup" name="BackupServiceExtensions" version="1.0.0" provider-name="Backup Corp."> <extension point="com.ibm.wsspi.extension.mbean-provider"> </extension> </plugin>

   
   
3. If you created a new JAR file just for this purpose, then you must reinitialize OSGI (Open Service Gateway Initiative) after you deploy your JAR file to the plugins directory or your changes will not be picked up. You can reinitialize OSGI via the osgiCfgInit.sh script located in <WAS_INSTALL_ROOT>/AppServer/bin. Alternatively, you can restart WebSphere Application Server to reload the plugin JAR file.

After this, your security policy should be loaded successfully. This difference is certainly the most serious as it poses a grave security risk. Assuming that the server does not find your security policy and uses a default policy, if the default policy happens to be less restrictive than your intended security policy then you have a security exposure and the resources that are under that application’s control are at risk. This is all the more dangerous given its somewhat silent nature; there may not be any indication of an error anywhere.

Back to top

Serialization

When you pass Java objects between systems, they must be serialized in some way. Serialization involves converting an object into a stream of bytes for sending, and rebuilding the object from the stream of bytes on the receiver side. There are a number of ways of doing this. However, if your application and objects are to integrate fully into the Java serialization framework, then you must either define your classes as Serializable or define them as Externalizable and provide an implementation.

On a distributed system, your application can invoke your MBeans locally (both client and application are running in the same WebSphere Application Server) and not have to worry about serialization. This is not so on z/OS. On z/OS, there is a daemon, a control, and a servant region, and they each run in their own address space (analogous to a process on a distributed platform). For more information, see WebSphere Application Server for z/OS process model overview.

Applications typically run in the servant region, while MBeans run in the control region. Because of this, MBean calls, though originating in the same application server, end up crossing address space boundaries. This is where the need for serialization comes in because the objects need to be sent back and forth between the servant and control regions. Using our example MBean from Listing 1, let’s assume that the objects making up the Collection are of type com.backup.Backup. If this class was not serialized, you would receive an error similar to Listings 8a and 8b when you invoke the method:

javax.management.MBeanException: Exception thrown in RequiredModelMBean while trying to 
invoke operation listBackups:
java.io.NotSerializableException: com.backup.Backup
  at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1113)
  at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:325)
  at java.util.ArrayList.writeObject(ArrayList.java:595) 

com.ibm.ws.management.AdminServiceImpl.invoke 679
Exception = javax.management.MBeanException
Source = com.ibm.ws.management.AdminServiceImpl.invoke
probeid = 679
Stack Dump = javax.management.MBeanException: Exception thrown in RequiredModelMBean  
while trying to invoke operation listBackups
  at javax.management.modelmbean.RequiredModelMBean.invokeMethod(
    RequiredModelMBean.java:1119)
  at javax.management.modelmbean.RequiredModelMBean.invoke(RequiredModelMBean.java:973)
...
Caused by: java.io.NotSerializableException: com.backup.Backup
  at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1113)
  at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:325)
  at java.util.ArrayList.writeObject(ArrayList.java:595)

A more sinister error occurs when the class being passed back and forth is serialized, but one or more classes in the class hierarchy are not. Using our example again, let’s assume that com.backup.ZosBackup inherits from the super class com.backup.Backup; class ZosBackup is serialized but class Backup is not. In this scenario, unless ZosBackup provides a serialization implementation, the data that are a part of class Backup will not get written to the stream when you pass the object on the MBean call. Therefore, when you rebuild the ZosBackup object, the data from Backup get initialized to their default values.

Depending on the application, this problem could show up way downstream with no indication that it stemmed from the MBean call. Note that this is not a problem with the MBeans themselves but is instead the nature of Java serialization. The real issue here is that you might not be aware that serialization is happening under the covers and hence would not know to pursue that avenue in problem diagnosis.

This seems easy enough, right? Simply make sure that all classes that are used in the MBean definition are Serializable and we’re done. Well, not so easy. The difficulty stems from using 3^rd-party code or libraries, including those in the Java Developer’s Kit (JDK), coupled with widespread use of the recommended object-oriented practice to use general types as opposed to specific types wherever possible.

Having the javadoc (Java Documentation) available can sometimes help in determining if a class is Serializable or not, but does not eliminate the problem. Going back to our BackupServiceMBean example, let’s say the MBean’s listBackups implementation received the java.util.Collection type to be returned from a 3^rd-party library as opposed to creating it. Since the Collection type is an interface, inside the listBackups method we have no way of knowing if the underlying (specific) type is Serializable or not. The solution here is whenever you have an object for which you can’t say definitively that it is Serializable, convert it to one that is.

Performance

Given the fact that MBean calls within the application server traverse address space boundaries, and serialization and deserialization have to occur for all objects sent back and forth, there is a potential for a performance penalty. This penalty is probably negligible for most MBean methods in most applications, but could be a real problem for performance-sensitive code paths.

For each application, you need to determine if the performance characteristics of its MBean invocations are consistent with its performance requirements. Assuming that the performance cost is undesirable or prohibitive, one way to reduce this would be to use Externalizable and provide your own serialization implementation.

However, 3^rd-party objects that are already Serializable could pose a problem. Here, you need to investigate to see if it’s worth it to convert any such object into a new one with a more efficient serialization implementation. Another option would be to not use an MBean interface for performance-sensitive code paths. The drawback here is that you don’t get to take advantage of the MBean framework, particularly the security roles and authorizations.

Back to top

Conclusion

These subtle differences between WebSphere Application Server for z/OS and distributed regarding MBeans can make for a significant increase in effort to get your MBeans working on z/OS. As you can see, the solutions to the problems are fairly simple and straightforward. Moreover, the solutions can be implemented on both sets of platforms, thus reducing the amount of platform-specific code in your application. The differences discussed above are not meant to be exhaustive, but provide you with a good starting point of things to avoid when porting MBeans to the z/OS platform.

Back to top

Acknowledgements

The author would like to recognize the following individuals for their contributions to this article: Steven Hart, Jason Katonica, and David Connelly.

Resources

Learn

• WebSphere Application Server for z/OS process model overview: details the differences between the non-z/OS WebSphere Application Server process models, and how to customize the WebSphere z/OS process model.
  
  
• Overview of Java Management and Monitoring: provides an overview of MBeans and JMX technology.
  
  
• Extending the WebSphere Application Server administrative system with custom MBeans: describes how to create MBeans on WebSphere Application Server for z/OS.
  
  
• Discover the secrets of the Java Serialization API: provides an overview of Java serialization.
  
  
• WebSphere Application Server on z/OS : provides product and support information
  
  
• developerWorks WebSphere on System z zone: provides resources for WebSphere products on System z.
  
  

Get products and technologies

• IBM Enterprise Modernization Sandbox for System z : evaluate IBM's latest solutions for modernizing and deploying core System z applications in a "tinker, test, and try" environment without downloading and installing them.
  
  

Discuss

• Destination Z mainframe forum: a place to discuss anything to do with mainframe software development.
  
  

About the author

Roan Dawkins is a Software Engineer in IBM's z/OS organization. He has performed application development in various areas of Java and related technologies including JMX, JSF, JSP, OSGI, and Websphere. He currently develops z/OS Cryptographic software.

Comments (Undergoing maintenance)

Back to top

Trademarks  |  My developerWorks terms and conditions

Table of contents

• Introduction
• MBean descriptor files
• Fully qualify all MBean descriptor type (class) names
• Ensure all methods in MBean interface have a matching signature in the descriptor
• Ensure the MBean security policy loads successfully
• Serialization
• Conclusion
• Acknowledgements
• Resources
• About the author
• Comments

Next steps from IBM

• Try: WebSphere Application Server
• Webcast: Using Rational Application Developer with WebSphere Application Server to increase productivity
• Briefing: Building SOA solutions and managing the service lifecycle
• Buy: WebSphere Application Server

My developerWorks community

Dig deeper into WebSphere on developerWorks

DB2 pureScale

See how true transparent scaling architecture can be achieved on distributed platforms

Special offers