Java Technology Community

Hi All,

We have to setup SSO between WebShere Portal V8.5 and SAP Netweaver V7 using Tivoli Federated Identity Manager V6.2.2 using SAML 2.0.

Could you please guide on the above.

I am excited to announce that the IBM® SDK, Java™* Technology Edition is now available as an official repo on DockerHub. The images in this repository contain IBM® SDK, Java™ Technology Edition, version 1.8.0_sr3fp10 (8.0-3.10). If you have been using the IBM Java Docker images provided earlier, you would have to make a minor update to your Dockerfiles in the following way

Eg. If you have been using the SFJ image, you'll need to change from this

    FROM ibmcom/ibmjava:sfj

to this

    FROM ibmjava:sfj

All the variants of the image provided earlier will continue to be available with the same set of tags (jre, sdk, sfj, jre-alpine and sfj-alpine). Images for the s390x and ppc64le architectures can be accessed as before and there will be no change needed to access them. As before, if you have issues relating specifically to this Docker image, please use the GitHub issue tracker. For more general issues relating to IBM® SDK, Java™ Technology Edition you can ask questions in the developerWorks forum: IBM Java Runtimes and SDKs.

The s390x and ppc64le Docker images for IBM® SDK, Java™ Technology Edition are now available on DockerHub. This is in addition to the x86_64 images that are already available. Click on the links below for more information on these images.

•     x86_64
•     s390x
•     ppc64le

We have three types of images for all the three architectures, the Software Developers Kit (SDK), and the Java Runtime Environment (JRE) and a small footprint version of the JRE (SFJ). Please see my earlier blog for more details on the types of images and how to use them.

To run the latest s390x JRE and SFJ docker images.

$ docker run --rm s390x/ibmjava:8-jre java -version
java version "1.8.0"
Java(TM) SE Runtime Environment (build pxz6480sr3-20160428_01(SR3))
IBM J9 VM (build 2.8, JRE 1.8.0 Linux s390x-64 Compressed References 20160427_301573 (JIT enabled, AOT enabled)
J9VM - R28_Java8_SR3_20160427_1620_B301573
JIT  - tr.r14.java.green_20160329_114288
GC   - R28_Java8_SR3_20160427_1620_B301573_CMPRSS
J9CL - 20160427_301573)
JCL - 20160421_01 based on Oracle jdk8u91-b14

$ docker run --rm s390x/ibmjava:8-sfj java -version
java version "1.8.0"
Java(TM) SE Runtime Environment (build pxz6480sr3-20160428_01(SR3) Small Footprint)
IBM J9 VM (build 2.8, JRE 1.8.0 Linux s390x-64 Compressed References 20160427_301573 (JIT enabled, AOT enabled)
J9VM - R28_Java8_SR3_20160427_1620_B301573
JIT  - tr.r14.java.green_20160329_114288
GC   - R28_Java8_SR3_20160427_1620_B301573_CMPRSS
J9CL - 20160427_301573)
JCL - 20160421_01 based on Oracle jdk8u91-b14

To run the latest ppc64le JRE and SFJ docker images.

$ docker run --rm ppc64le/ibmjava:8-jre java -version
java version "1.8.0"
Java(TM) SE Runtime Environment (build pxl6480sr3-20160428_01(SR3))
IBM J9 VM (build 2.8, JRE 1.8.0 Linux ppc64le-64 Compressed References 20160427_301573 (JIT enabled, AOT enabled)
J9VM - R28_Java8_SR3_20160427_1620_B301573
JIT  - tr.r14.java.green_20160329_114288
GC   - R28_Java8_SR3_20160427_1620_B301573_CMPRSS
J9CL - 20160427_301573)
JCL - 20160421_01 based on Oracle jdk8u91-b14

$ docker run --rm ppc64le/ibmjava:8-sfj java -version
java version "1.8.0"
Java(TM) SE Runtime Environment (build pxl6480sr3-20160428_01(SR3) Small Footprint)
IBM J9 VM (build 2.8, JRE 1.8.0 Linux ppc64le-64 Compressed References 20160427_301573 (JIT enabled, AOT enabled)
J9VM - R28_Java8_SR3_20160427_1620_B301573
JIT  - tr.r14.java.green_20160329_114288
GC   - R28_Java8_SR3_20160427_1620_B301573_CMPRSS
J9CL - 20160427_301573)
JCL - 20160421_01 based on Oracle jdk8u91-b14

To run the latest x86_64 JRE and SFJ alpine docker images. (Alpine Linux is only available for x86_64)

$ docker run --rm ibmcom/ibmjava:8-jre-alpine java -version
java version "1.8.0"
Java(TM) SE Runtime Environment (build pxa6480sr3-20160428_01(SR3))
IBM J9 VM (build 2.8, JRE 1.8.0 Linux amd64-64 Compressed References 20160427_301573 (JIT enabled, AOT enabled)
J9VM - R28_Java8_SR3_20160427_1620_B301573
JIT  - tr.r14.java.green_20160329_114288
GC   - R28_Java8_SR3_20160427_1620_B301573_CMPRSS
J9CL - 20160427_301573)
JCL - 20160421_01 based on Oracle jdk8u91-b14

$ docker run --rm ibmcom/ibmjava:8-sfj-alpine java -version
java version "1.8.0"
Java(TM) SE Runtime Environment (build pxa6480sr3-20160428_01(SR3) Small Footprint)
IBM J9 VM (build 2.8, JRE 1.8.0 Linux amd64-64 Compressed References 20160427_301573 (JIT enabled, AOT enabled)
J9VM - R28_Java8_SR3_20160427_1620_B301573
JIT  - tr.r14.java.green_20160329_114288
GC   - R28_Java8_SR3_20160427_1620_B301573_CMPRSS
J9CL - 20160427_301573)
JCL - 20160421_01 based on Oracle jdk8u91-b14

IBM® SDK, Java™* Technology Edition is now available on Docker Hub! See the Docker website for more information on containers and how you can use Docker for your application. For more information about IBM® SDK, Java™ Technology Edition and API documentation, see the IBM Knowledge Center. For tutorials, recipes, and Java usage in Bluemix, see IBM developerWorks.The base Java image gets auto-updated as and when there are new Java releases as well as for PSIRTs and fixes. Since the base Image is updated frequently and incrementally, the cost to adopt newer versions of Java are incremental and minimal as well. This reduces the need to have a “major” effort to upgrade to newer versions of Java making it seamless. Performance improvements come free of cost !

There are three types of Docker images here: the Software Developers Kit (SDK), and the Java Runtime Environment (JRE) and a small footprint version of the JRE (SFJ). These images can be used as the basis for custom built images for running your applications.

To run the latest JRE docker image

$ docker run --rm ibmcom/ibmjava java -version
java version "1.8.0"
Java(TM) SE Runtime Environment (build pxa6480sr3-20160428_01(SR3))
IBM J9 VM (build 2.8, JRE 1.8.0 Linux amd64-64 Compressed References 20160427_301573 (JIT enabled, AOT enabled)
J9VM - R28_Java8_SR3_20160427_1620_B301573
JIT  - tr.r14.java.green_20160329_114288
GC   - R28_Java8_SR3_20160427_1620_B301573_CMPRSS
J9CL - 20160427_301573)
JCL - 20160421_01 based on Oracle jdk8u91-b14

Small Footprint JRE

The Small Footprint JRE (SFJ) is designed specifically for web developers who want to develop and deploy cloud-based Java applications. Java tools and functions that are not required in the cloud environment, such as the Java control panel, are removed. The runtime environment is stripped down to provide core, essential function that has a greatly reduced disk and memory footprint. To run a pre-built jar file with the SFJ image, you can use the following dockerfile:

FROM ibmcom/ibmjava:sfj
RUN mkdir /opt/app
COPY japp.jar /opt/app
CMD ["java", "-jar", "/opt/app/japp.jar"]

You can build and run the Docker Image as shown in the following example:

docker build -t japp .
docker run --rm japp

Alpine Linux

Consider using Alpine Linux if you are concerned about the size of the overall image. Alpine Linux is a stripped down version of Linux that is based on musl glibc and Busybox, resulting in a Docker image size of approximately 5 MB. Due to its extremely small size and reduced number of installed packages, it has a much smaller attack surface which improves security. However, because the IBM SDK has a dependency on gnu glibc, installing this library adds an extra 8 MB to the image size. The following table compares Docker Image sizes based on the latest JRE 8.0-3.0.

Note: Alpine Linux is not an officially supported operating system for IBM® SDK, Java™ Technology Edition.

To run the alpine SFJ docker image

$ docker run --rm ibmcom/ibmjava:sfj-alpine java -version
java version "1.8.0"
Java(TM) SE Runtime Environment (build pxa6480sr3-20160428_01(SR3) Small Footprint)
IBM J9 VM (build 2.8, JRE 1.8.0 Linux amd64-64 Compressed References 20160427_301573 (JIT enabled, AOT enabled)
J9VM - R28_Java8_SR3_20160427_1620_B301573
JIT  - tr.r14.java.green_20160329_114288
GC   - R28_Java8_SR3_20160427_1620_B301573_CMPRSS
J9CL - 20160427_301573)
JCL - 20160421_01 based on Oracle jdk8u91-b14

To test your existing application jar files directly from the host file system, you can mount the host path onto the container by using the following commands:

FROM ibmcom/ibmjava:jre
CMD ["java", "-jar", "/opt/app/japp.jar"]

docker build -t japp .
docker run -it -v /path/on/host/system/jars:/opt/app japp

Using the Class Data Sharing feature

IBM SDK, Java Technology Edition provides a feature called Class data sharing. This mechanism offers transparent and dynamic sharing of data between multiple Java virtual machines (JVMs) running on the same host thereby reducing the amount of physical memory consumed by each JVM instance. By providing partially verified classes and possibly pre-loaded classes in memory, this mechanism also improves the start up time of the JVM.

To enable class data sharing between JVMs that are running in different containers on the same host, a common location must be shared between containers. This requirement can be satisfied through the host or a data volume container. When enabled, class data sharing creates a named "class cache", which is a memory-mapped file, at the common location. This feature is enabled by passing the -Xshareclasses option to the JVM as shown in the following Dockerfile example:

FROM ibmcom/ibmjava:jre
RUN mkdir /opt/shareclasses
RUN mkdir /opt/app
COPY japp.jar /opt/app
CMD ["java", "-Xshareclasses:cacheDir=/opt/shareclasses", "-jar", "/opt/app/japp.jar"]

The cacheDir sub-option specifies the location of the class cache. For example /opt/sharedclasses. When sharing through the host, a host path must be mounted onto the container at the location the JVM expects to find the class cache. For example:

docker build -t japp .
docker run -it -v /path/on/host/shareclasses/dir:/opt/shareclasses japp

When sharing through a data volume container, create a named data volume container that shares a volume.

docker create -v /opt/shareclasses --name classcache japp /bin/true

Then start your JVM container by using --volumes-from flag to mount the shared volume, as shown in the following example:

docker run -it --volumes-from classcache japp

See Also

See the Websphere-Liberty image, which builds on top of this IBM docker image for Java.

License

The Dockerfiles and associated scripts are licensed under the Apache License 2.0.

Licenses for the products installed within the images:

    IBM® SDK, Java™ Technology Edition: International License Agreement for Non-Warranted Programs.

Issues

For issues relating specifically to this Docker image, please use the GitHub issue tracker.

For more general issues relating to IBM® SDK, Java™ Technology Edition you can ask questions in the developerWorks forum: IBM Java Runtimes and SDKs.

For general information on Troubleshooting with the SDK, please do take a look at our How Do I ...? page.

*Java and all Java-based trademarks and logos are trademarks or registered trademarks of Oracle and/or its affiliates.

Avoid Multiple ORB initialization.

Technote (FAQ)

Question

Why should I avoid multiple ORB initialization within a single JVM?

Answer

Multiple ORB inits are known to cause several problems:

1. WLM functionality and cluster failover does not work with multiple ORBs.

2. Native Heap OOM.
   Each ORB creates a number of Listener, Reader and Worker Threads. Too many ORBs (hundreds or thousands) may lead to exhaustion of the JVM's native memory.

3. ORB properties may not be honored for subsequent ORBs if the ORB.init()
   no-arguments constructor is used, or different properties are passed rather than those with which the first ORB was init'ed. Typical problems seen:

   1. Timeouts are not honored.

   2. Trace is not generated from the other init'ed ORBs.

4. ListenerThread creation failures. Subsequent ORBs experience these failures when static ports are employed as each subsequent ORB will attempt to bind to the ports already being used by ORB #1.
   http://www-01.ibm.com/support/docview.wss?uid=swg21248645
    

5. WebSphere Application Server does not officially support multiple ORBs.
   http://www-01.ibm.com/support/knowledgecenter/SSAW57_8.5.5/com.ibm.websphere.nd.multiplatform.doc/ae/corb_maino.html
   The ApplicationServer runtime inits its own ORB, and any applications running within the server should access that already-existing ORB via the InitialContext object rather than init'ing a new ORB. See “How to avoid multiple ORB inits” below.

    

Question

How do I identify whether an application is init'ing multiple ORBs?

Answer

1. ORB Trace

   1. Search for “ORB.init call sequence”. The entire stack for each ORB.init is logged in ORB traces, which will reveal which code is invoking ORB.init.

   2. Search for “O=1” (or greater). ORB ReaderThreads and ListenerThreads designate the ORB instance number (O=<#>) in their thread name. “O=1” or greater indicates multiple ORBs. e.g.
      “LT=1:P=417656:O=2:port=9409”
      “RT=3:P=511663:O=13:WSSSLTransportConnection[addr=9.76.140.7,port=3227,
      local=9502]”

2. Javacore or thread dump

   1. A typical WebSphere server will have 4-5 ORB Listeners (basic standalone server will have 2 ORB Listeners). A search for the number of ORB ListenerThreads (“LT=<#>...”) as well as the ORB instance numbers will reveal multiple ORBs.

Question

What are ways in which ORBs are initialized?

Answer

ORBs can be initialized when:

1. Calling ORB.init() explicitly.

   1. Servers or clients using CosNaming (rather than JNDI) will usually call ORB.init directly.
      
      Properties props = new Properties();
      props.put("org.omg.CORBA.ORBClass", "com.ibm.CORBA.iiop.ORB");
      props.put("com.ibm.CORBA.ORBInitRef.NameService",
      "corbaloc:iiop:myhost.mycompany.com:2809/NameService");
      props.put("com.ibm.CORBA.ORBInitRef.NameServiceServerRoot",
      "corbaloc:iiop:myhost.mycompany.com:2809/NameServiceServerRoot");
      ORB orb = ORB.init((String[])null, props);
      
      See http://www-01.ibm.com/support/knowledgecenter/SSAW57_8.5.5/com.ibm.websphere.nd.multiplatform.doc/ae/rnam_example_corba1.html
       

2. Getting an Initial Context

   1. Using com.sun.jndi.cosnaming.CNCtxFactory.
      This inits an ORB each time, regardless of whether ORB.init is called explicity or not.
      
      Server
      Hashtable env = new Hashtable();
      env.put(Context.INITIAL_CONTEXT_FACTORY,
      "com.sun.jndi.cosnaming.CNCtxFactory");
      env.put(Context.PROVIDER_URL, "iiop://myhost.mycompany.com:2809");
      javax.naming.Context initialContext = new javax.naming.InitialContext(env);
      try {

      org.omg.CORBA.ORB orb = (ORB) initialContext.lookup("java:comp/ORB");
      } catch (Exception e) {
      e.printStackTrace();
      }
      
      Client
      Hashtable env = new Hashtable();
      env.put(Context.INITIAL_CONTEXT_FACTORY,
      "com.sun.jndi.cosnaming.CNCtxFactory");
      env.put(Context.PROVIDER_URL, "iiop://myhost.mycompany.com:2809");
      javax.naming.Context initialContext = new javax.naming.InitialContext(env);
      org.omg.CORBA.ORB orb =
      (org.omg.CORBA.ORB
      javax.rmi.PortableRemoteObject.narrow(ctx.lookup("java:comp/ORB"),
      org.omg.CORBA.ORB.class);
       

   2. Using com.ibm.websphere.naming.WsnInitialContextFactory
      -- Preferred over com.sun.jndi.cosnaming.CNCtxFactory.
      -- Has mechanism in place to determine if ORB already exists and thus avoids init'ing additional ORBs.

Server

Hashtable env = new Hashtable();

env.put(Context.INITIAL_CONTEXT_FACTORY,

"com.ibm.websphere.naming.WsnInitialContextFactory");

env.put(Context.PROVIDER_URL, "iiop://myhost.mycompany.com:2809");

javax.naming.Context initialContext = new javax.naming.InitialContext(env);

try {

org.omg.CORBA.ORB orb = (ORB) initialContext.lookup("java:comp/ORB");

} catch (Exception e) {

e.printStackTrace();

}

Client

javax.naming.Context ctx = new javax.naming.InitialContext();

org.omg.CORBA.ORB orb =

(org.omg.CORBA.ORB)
javax.rmi.PortableRemoteObject.narrow(ctx.lookup("java:comp/ORB"),

org.omg.CORBA.ORB.class);

3. Creating a JMX Connector

   1. Creating a new RMI Connector using a JVMServiceURL can potentially generate a new ORB.

   2. JMXServiceURL designates the address of a JMX API connector server. The address is an Abstract Service URL for SLP, as defined in RFC 2609 and amended by RFC 3111. It must look like this: service:jmx:protocol:sap
      -- protocol is the transport protocol to be used to connect to the connector server.
      -- sap is the address at which the connector server is found.

   3. The protocol is a short string such as "rmi", "iiop", "jmxmp" or "soap". If the protocol is “iiop” then an ORB will be initialized internally and it will be responsible for the transport of JMX requests and responses.
      JMXServiceURL url = new JMXServiceURL("service:jmx:iiop://host:port");

   4. The new ORB will be initialized with the default properties, hence it is imperative that the orb.properties file from the <JAVA_HOME> is updated with optimal values are set for the ORB properties based on the application's requirements.

   5. For details on JMX: https://docs.oracle.com/javase/7/docs/api/javax/management/remote/rmi/package-summary.html
      http://mx4j.sourceforge.net/docs/ch03s15.html
      For details on ORB properties:
      http://www-01.ibm.com/support/knowledgecenter/SSAW57_8.5.5/com.ibm.websphere.nd.multiplatform.doc/ae/rorb_setg.html

Question

How to avoid multiple ORB inits?

Answer

1. Do NOT call ORB.init() directly within application code (client or server) running inside a WebSphere Application Server.

   1. A J2EE application should use the already-existing ORB instance provided by the Application Server.

   2. Use the “JNDI Approach” as shown in "Preferred API for obtaining an ORB instance":
      http://www-01.ibm.com/support/knowledgecenter/SSAW57_8.5.5/com.ibm.websphere.nd.multiplatform.doc/ae/rorb_orbprog.html

2. Avoid multiple ORB.init calls within client code not running within Application Server.

3. Use the com.ibm.websphere.naming.WsnInitialContextFactory rather than the com.sun.jndi.cosnaming.CNCtxFactory for InitialContext creation.

4. Limit the number of JMX RMIConnectors created if possible.

What is Security?

- Authentication, Authorization
- Confidentiality, privacy
- Non-repudiation
- Intrusion prevention and detection

Java Security

Java Security Model has evolved over the years from a sandbox restricted model to a more robust model that is controlled through permissions and policies.

Sample policy file

   grant codeBase "file://c:/myfiles" {
        permission java.io.FilePermission "<<ALL FILES>>", "read";
    };

The above code grants read file permission to the code whose origin is from the codebase. Writing a policy file manually can be tedious and error prone, hence we can use policytool that comes with the JDK to create a policy file.

Default Policy

<JAVA_HOME>/jre/lib/security/java.security file specifies the policy file locations

policy.url.1=file:${java.home}/lib/security/java.policy
policy.url.2=file:${java.home}/lib/security/java.pol
policy.url.3=file:///${user.home}/.java.policy

We can either statically add our policy file location to the end of this list or dynamically specify it using the system property -Djava.security.policy=/home/test/mypolicy. It is good to specify the policy file using the system property as it would be applicable only to the JVM instance to which it is passed

Security Manager

java.lang.SecurityManager allows applications to implement a security policy. It allows the application to check if the operation being performed is safe or not and throws a security exception if found to be unsafe. By default Security Manager is enabled for an applet but for a java application it has to be enabled explicitly by using the system property -Djava.security.manager

Cryptography

Encryption and Decryption

Encryption is the process of transforming information so it is unintelligible to anyone but the intended recipient. Decryption is the process of transforming encrypted information so that it is intelligible again. This is done using a key and a cipher or cryptographic algorithm which is a mathematical function. Key is a number that enables to keep encrypted information secret.

Symmetric key encryption
 

plain text ====> Encrypted/Ciphered text ====> Decrypted/Deciphered text

Some of the symmetric key algorithms are DES, Triple DES (DESede), AES

Asymmetric key encryption

plain text ====> Encrypted/Ciphered text ====> Decrypted/Deciphered text
               public key                      private key

Compared to Symmetric key encryption, this requires more computation and hence not used for bulk data encryption/decryption. Instead it is used in SSL for initial authentication and during the secret key exchange or agreement phase.  Usually public key is used for encryption and private key is used for decryption, as data encrypted with private key can be decrypted by anyone who has the public key.  But private key is used for encryption in digital signing.

NOTE: Encryption strength depends not just on the key length but on the cipher algorithm used.

Another issue with Asymmetric key encryption is impersonation, which is resolved by using Certificates.

Certificate

A certificate is an electronic document used to identify an individual, a server, a company, or some other entity and to associate that identity with a public key and this certificate has to be signed by a CA. Certificate authorities (CAs) are entities that validate identities and issue certificates and are independent trusted third parties or organizations who digitally sign the certificate. In addition to a public key, a certificate always includes the name of the entity it identifies, an expiration date, the name of the CA that issued the certificate, a serial number, and other information. Most importantly, a certificate always includes the digital signature of the issuing CA. We can have any no. of CAs in the certificate chain, the top CA is called the root CA, whose certificate would be a self signed Certificate.

For example, in this chain of certificates, Root CA - > CA1 -> CA2 -> User, the user certificate is validated and trusted only if all the certificates in the chain have not expired and the digital signature is verified on each certificate starting from the Root CA. 

Digital Signature

Digital signature is the encrypted value of a one-way hash of the data (that needs to be signed) along with the hashing algorithm used. Here the encryption of the one-way hash is done using the private key of the Signer and can be verified using the public key of the Signer.

Some of the Security Tools shipped with the JDK

keytool - is a key & certificate management utility. It is used to create pairs of public and private keys, to import and display certificate chains, to export certificates, and to generate X.509 v1 self-signed certificates and certificate requests that can be sent to a certification authority. Ikeyman is an equivalent IBM specific GUI tool to keytool

Jarsigner -  signs JAR (Java ARchive format) files and verifies the authenticity of the signature(s) of signed JAR files.

Policytool - creates and modifies the policy configuration files that define your installation's security policy

Generating a RSA keypair:

keytool -genkeypair -alias myalias -keyalg RSA -validity 1000 -keypass passw0rd -keystore teststore -storepass passw0rd -dname "CN=MyRSACert,O=ORG,C=IN“

Signing a jar file using jarsigner and the RSA key generated above:

jarsigner -keystore teststore -storepass passw0rd -keypass passw0rd –signedjar mysigned.jar my.jar myalias

In the above command my.jar is signed using the RSA key and a new signed jar - mysigned.jar is created, if you do not want a new jar created you can remove -signedjar mysigned.jar from the above command

Verifying the above signed jar using jarsigner

jarsigner -verify -verbose -certs mysigned.jar

IBM Java 7.0, 7.1 and 8.0 recently changed its fixpack naming convention. Now the regular fixpacks are incremented by ten instead of one. This is to incorporate custom fixpacks (like monthly fixpacks for Bluemix) or unplanned fixpacks for handling security vulnerabilities. With the new naming convention, a maximum of nine interim fixpacks can be incorporated between regular fixpacks. The interim  fixpacks will be incremented by one. There is no change in Service Refresh naming convention and it will continued to be incremented by one.

The new IBM JDK fixpack naming convention is also in alignment with Oracle JDK update naming convention as they also provide room to incorporate unplanned/security fixes.

The upcoming Java release is 7.0 SR9, 7.1 SR3 and 8.0 SR1 and the next release is a regular fixpack. Hence, it will be 7.0 SR9 FP10, 7.1 SR3 FP10 and 8.0 SR1 FP10. Any interim fixpacks (custom/ unplanned fixpacks) before next regular fixpack  will be FP1, FP2, etc.

If the next release of Java after 7.0 SR9 FP10, 7.1 SR3 FP10 and 8.0 SR1 FP10 is again a regular fixpack, then it will be 7.0 SR9 FP20, 7.1 SR3 FP20 and 8.0 SR1 FP20.

If the next release after 7.0 SR9 FP10, 7.1 SR3 FP10 and 8.0 SR1 FP10 is a service refresh, then it will be 7.0 SR10, 7.1 SR4 and 8.0 SR2.

If the next release after 7.0 SR9 FP10, 7.1 SR3 FP10 and 8.0 SR1 FP10 is a custom/unplanned fixpack, then it will be 7.0 SR9 FP11, 7.1 SR3 FP11 and 8.0 SR1 FP11.  
 

From the Classroom,  Object Serialization and Reflection

(This is the next in a series of articles I'll be doing under the  banner of "From the Classroom".  The topics of these articles stem from common questions, insights, misunderstandings and the occasional epiphany that my introductory Java programming students have raised over the years.)


Every time I teach my students how to use serialization with ObjectInputStreams and ObjectOutputStreams someone will ask how the writeObject and readObject methods know what fields exist and how to access them.  Instead of wading through the intricacies of ObjectOutputStream.java and ObjectInputStream.java I usually give them several short examples demonstrating the capabilities built into Java's Class, Field, Modifier, and Method classes. The techniques I describe in the examples fall under the general umbrella of Java's Reflection API, click here for the official tutorial.

Examples:

1.  Given an object stored in a generic Object reference variable, how do you learn what its "real" class is?
2.  How can you display its entire class hierarchy?
3.  What about the object's data, can I get a list of variable names, types, and values?
4.  What if a field references an array?  How can I display its values and determine its data type and dimensions?
5.  If a field is a primitive type, how do I know which type and how can I process it as a primitive type instead of a wrapper object?
6.  How do I list and call methods of the class?
7.  Finally, and most importantly, when would I ever need to do this?  Wouldn't I always know the field names and data types my program will be using?

1.  Given an object stored in a generic Object reference variable, how do you learn what its "real" class is?

This one is easy,  every class automatically inherits the Object class.  getClass() is a method of the Object class and will return a reference to a Class object.  This reference can then be used to determine the object's class name, superclass and other information.  E.g.

 public static void displayObjectInfo(Object theObject) {
        Class theClass = theObject.getClass();  // get the Class object
        System.out.println("Class name = "+theClass.getName());         // display its name    
        System.out.println("Package = "+ theClass.getPackage().toString());  // display its package
        System.out.println("Superclass = "+theClass.getSuperclass().getName());  // display the name of its superclass
    }


Running the method for a simple Integer  

Integer myInt = new Integer(42);
displayObjectInfo(myInt);

displays:

Class name = java.lang.Integer
Package = package java.lang, Java Platform API Specification, version 1.8
Superclass = java.lang.Number

2.  How can you display its entire class hierarchy?

This is just a matter of repeatedly calling the getSuperclass() method, either in a recursive routine or in a while loop:

While loop method:

  public static void displayObjectInheritance(Object theObject){
     Class theClass  = theObject.getClass();
   
     while(theClass != null){
         System.out.println(theClass.getName());
         theClass = theClass.getSuperclass();
     }
   }

Integer myInt = new Integer(42);
displayObjectInheritance(myInt);

displays:

java.lang.Integer
java.lang.Number
java.lang.Object

Recursive routine :

public static void displayObjectInheritanceRecursion(Object theObject){
        Class theClass = theObject.getClass();
        displayObjectInheritanceRecurse(theClass, theObject);
       
}

public static String displayObjectInheritanceRecurse(Class theClass, Object theObject){
  
  String indent = "";  // indent the names in the hierarchy 
  Class superClass = theClass.getSuperclass();
    
  if(superClass == null) { // is this the Object class?
      System.out.println(indent + theClass.getName()); // yes, print its name and stop recursion
      return "  ";  //
     }
     else {
        indent = displayObjectInheritanceRecurse(superClass, theObject);
        System.out.println(indent + theClass.getName() );  
        return indent + "  ";
    }
}

Integer myInt = new Integer(42);
displayObjectInheritanceRecursion(myInt);

displays:

java.lang.Object
  java.lang.Number
    java.lang.Integer

3.  What about the object's data, can I get a list of variable names, types, and values?

The Class class provides two different ways to get an object's variables, getFields() and getDeclaredFields().  getFields() will return an array of all fields within the class and its super classes that are declared as public.  getDeclaredFields() will return all fields (public, private, and otherwise) for that class, but not for any super classes.  Both methods will return an array of Field objects.  In order to get a list of all variables for a given object reference you first need to get its class object, then iterate over all its super classes calling the getDeclaredFields() method for each.  The Field class provides methods to access information about a field; this includes it's name, value, data type, modifiers. The Field class also provides methods to change a field's value. 

In the example below, the getModifiers() method is called to retrieve the fields modifiers, the getType() is used to retrieve the data type, the getName() method is used to access a field's name,  and  the get() method is used to access the value of a field for a given object.  The getModifiers method will return an integer with all the field's modifiers encoded in it,  the Modifier class is used to decode this integer.  If the value of the field is a primitive type, then the get() method will wrap the data in an object of the appropriate wrapper class.  If the field were declared as private, then the get() method will generate an IllegalAccessException.  In this case this exception was prevented by setting the field as accessible using the setAccessible(true) method.  Using setAccessible(true) suppresses Java language access checking. This will not work if a security manager is present.

  public static void displayObjectFields(Object theObject) throws IllegalAccessException
   {
     Class theClass  = theObject.getClass();
     while(theClass != null){
         Field[] fields = theClass.getDeclaredFields();
         for(Field field : fields) {
           field.setAccessible(true);
           System.out.println(Modifier.toString(field.getModifiers())+ " "+
                           field.getType().toString()+ " " +field.getName() + " = " +
                           field.get(theObject).toString());
         }
         theClass = theClass.getSuperclass();
     }
   }

Integer myInt = new Integer(42);
displayObjectFields(myInt);

displays:

public static final int MIN_VALUE = -2147483648
public static final int MAX_VALUE = 2147483647
public static final class java.lang.Class TYPE = int
static final class [C digits = [C@52e922
static final class [C DigitTens = [C@25154f
static final class [C DigitOnes = [C@10dea4e
static final class [I sizeTable = [I@647e05
private final int value = 42
public static final int SIZE = 32
public static final int BYTES = 4
private static final long serialVersionUID = 1360826667806852920
private static final long serialVersionUID = -8742448824652078965

4.  What if a field references an array?  How can I display its values and determine its data type and dimensions?

To handle arrays (and the previous example did not),  you first need to get the data type of the field and use the isArray() method to determine if it is an array( isPrimitive() and isEnum() methods also exist).  If the field is an array, then getComponentType() can be called to get the data type of the array. To access the array's data, first use the field's get() method to get a reference to the array itself.  Once you have this reference you can call methods of the java.lang.reflect.Array class to retrieve the array's dimensions (getLength()) and data items (get()).   The previous example was modified to distinguish between array data types and all others.  If the field holds an array, then the displayArray method is called with the data type of the array and a reference variable to the array itself.

displayArray first gets the length of the array by calling java.lang.reflect.Array.getLength(), then it checks the array's type to see if it is an array of arrays or an array of data items.  If it is an array of arrays, then displayArray is recursively called for each array in the array. If it is an array of data items, then a for loop is used to get and display each item by calling java.lang.reflect.Array.get(). 

   public static void displayObjectFieldsArray(Object theObject) throws IllegalAccessException
   {
     Class theClass  = theObject.getClass();
     while(theClass != null){
         Field[] fields = theClass.getDeclaredFields();
         for(Field field : fields) {
             field.setAccessible(true);
             Class fieldType = field.getType();
             System.out.print(Modifier.toString(field.getModifiers())+ " "+
                           field.getType().toString()+ " " +field.getName() + " = " );
             if(fieldType.isArray())
               displayArray(fieldType.getComponentType(), field.get(theObject));
             else
               System.out.print(field.get(theObject).toString());
             System.out.println();
         }
         theClass = theClass.getSuperclass();
     }

   }

    public static void displayArray(Class arrayType, Object theArray)
    {
         int length =  java.lang.reflect.Array.getLength(theArray);
  
          // is this an array of arrays?
            if(arrayType.isArray())  {   
             // array of arrays, make a recursive call
             System.out.print("{"); 
             for(int j = 0; j < length; j++)
             {
               Object arr2 = java.lang.reflect.Array.get(theArray, j);
               displayArray(arrayType.getComponentType(),arr2);
               if(j!= length-1)
                  System.out.print(",");
             }
             System.out.print("}");
            }
            else
            {
             // array of data, just display it
             System.out.print("{");
             for(int j = 0; j < length; j++)
             {
              System.out.print(java.lang.reflect.Array.get(theArray, j).toString());
              if(j!= length-1)
                  System.out.print(",");
              }
              System.out.print("}");
             }
     }

Integer myInt = new Integer(42);
displayObjectFieldsArray(myInt);

displays:

public static final int MIN_VALUE = -2147483648
public static final int MAX_VALUE = 2147483647
public static final class java.lang.Class TYPE = int
static final class [C digits = {0,1,2,3,4,5,6,7,8,9,a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y,z}
static final class [C DigitTens = {0,0,0,0,0,0,0,0,0,0,1,1,1,1,1,1,1,1,1,1,2,2,2,2,2,2,2,2,2,2,3,3,3,3,3,3,3,3,3,3,4,4,4,4,4,4,4,4,4,4,5,5,5,5,5,5,5,5,5,5,6,6,6,6,6,6,6,6,6,6,7,7,7,7,7,7,7,7,7,7,8,8,8,8,8,8,8,8,8,8,9,9,9,9,9,9,9,9,9,9}
static final class [C DigitOnes = {0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9}
static final class [I sizeTable = {9,99,999,9999,99999,999999,9999999,99999999,999999999,2147483647}
private final int value = 42
public static final int SIZE = 32
public static final int BYTES = 4
private static final long serialVersionUID = 1360826667806852920
private static final long serialVersionUID = -8742448824652078965

5.  If a field is a primitive type, how do I know which type and how can I process it as a primitive type instead of a wrapper object?

When you call the Field object's getType() method it will return a reference to a Class object.  You can then call the Class object's isPrimitive() method to determine if the data type is a primitive (short, int, double, byte, etc).  Each of Java's wrapper classes (Integer, Double, Short, Byte, etc.) has a field that holds a class instance for that type, the field is called TYPE. You can compare this field to the value returned by getType() to determine the exact data type. For example:

Class fieldType = field.getType();
if(fieldType.isPrimitive()){
  if(fieldType == Integer.TYPE)
    // then this is an integer primitive
  else if(fieldType == Double.TYPE)
    // then this is a double primitive
...
}

The Field class also provides set and get methods for each primitive type, e.g.  getInt(), setInt().  You can use these instead of the get() and set() methods which use wrapper objects. The following example retrieves the Field object for the value field in an Integer object, then sets it to a value of 33.

       Object myInteger = new Integer(42); // create an object for demonstration purposes
      
       Class theClass = myInteger.getClass();  // get the class of the object
       Field valueField = theClass.getDeclaredField("value");  // retrieve a reference to the "value" field
       valueField.setAccessible(true);   // make it accessible since it is declared as private
       valueField.setInt(myInteger, 33);  // change the value from 42 to 33.
       System.out.println(myInteger);   // output for verification

That seems like a lot of trouble, and it is for an object of type Integer, but this same technique could be used on any an object of any type.

6.  How do I list and call methods of the class?

The Class class provides methods to retrieve a list of methods just like it does for fields.  And just like fields you can call getMethods() to retrieve an array of Method objects for the public methods of the class and its super classes, or call getDeclaredMethods() to get a list of all methods (public and otherwise) for that class (but not the super class).  Like the Field class, the Method class has a getModifiers() and a getName() method to retrieve the method's modifiers and name.  Method also has a getReturnType() method to get the method's return data type and a getParameterTypes() method to get an array of parameter types.  The Method object also has an invoke() that will allow you to call the method. The following method displays all the declared methods for a given object, including methods in all the super classes.


         public static void displayDeclaredMethods(Object theObject) throws Exception {
    
      Class theClass = theObject.getClass();
     
      while(theClass != null)
      {
       Method[] methods = theClass.getDeclaredMethods();
  
      for(int i = 0; i < methods.length; i++){
        methods[i].setAccessible(true);  // allow us to access it
        System.out.print(Modifier.toString(methods[i].getModifiers())+ " "+
                           methods[i].getReturnType().toString()+ " " +
                           methods[i].getName());
        Class[] params = methods[i].getParameterTypes();
        System.out.print("(");
        for(int j = 0; j < params.length; j++){
          System.out.print(params[j].getTypeName());
          if(j < params.length-1)
              System.out.print(",");
         }
           
         System.out.println(");");
        }
        theClass = theClass.getSuperclass();
       }
     }

Integer myInt = new Integer(42);
displayDeclaredMethods(myInt);

displays:

public static int numberOfLeadingZeros(int);
public static int numberOfTrailingZeros(int);
public static int bitCount(int);
public boolean equals(java.lang.Object);
public static class java.lang.String toString(int,int);
public class java.lang.String toString();
public static class java.lang.String toString(int);
public static int hashCode(int);
public int hashCode();
public static int min(int,int);
public static int max(int,int);
public static int reverseBytes(int);
public int compareTo(java.lang.Integer);
public volatile int compareTo(java.lang.Object);
public byte byteValue();
public short shortValue();
public int intValue();
public long longValue();
public float floatValue();
public double doubleValue();
public static class java.lang.Integer valueOf(java.lang.String,int);
public static class java.lang.Integer valueOf(int);
public static class java.lang.Integer valueOf(java.lang.String);
public static class java.lang.String toHexString(int);
public static int compare(int,int);
public static class java.lang.Integer decode(java.lang.String);
static void getChars(int,int,char[]);
public static int reverse(int);
static int stringSize(int);
public static int sum(int,int);
public static int parseInt(java.lang.String);
public static int parseInt(java.lang.String,int);
public static long toUnsignedLong(int);
public static int compareUnsigned(int,int);
public static int divideUnsigned(int,int);
static int formatUnsignedInt(int,int,char[],int,int);
public static class java.lang.Integer getInteger(java.lang.String,java.lang.Integer);
public static class java.lang.Integer getInteger(java.lang.String,int);
public static class java.lang.Integer getInteger(java.lang.String);
public static int highestOneBit(int);
public static int lowestOneBit(int);
public static int parseUnsignedInt(java.lang.String);
public static int parseUnsignedInt(java.lang.String,int);
public static int remainderUnsigned(int,int);
public static int rotateLeft(int,int);
public static int rotateRight(int,int);
public static int signum(int);
public static class java.lang.String toBinaryString(int);
public static class java.lang.String toOctalString(int);
public static class java.lang.String toUnsignedString(int);
public static class java.lang.String toUnsignedString(int,int);
private static class java.lang.String toUnsignedString0(int,int);
public byte byteValue();
public short shortValue();
public abstract int intValue();
public abstract long longValue();
public abstract float floatValue();
public abstract double doubleValue();
protected void finalize();
public final void wait();
public final void wait(long,int);
public final native void wait(long);
public boolean equals(java.lang.Object);
public class java.lang.String toString();
public native int hashCode();
public final native class java.lang.Class getClass();
protected native class java.lang.Object clone();
public final native void notify();
public final native void notifyAll();
private static native void registerNatives();

7.  Finally, and most importantly, when would I ever need to do this?  Wouldn't I always know the field names and data types my program will be using?

For most Java application you would never need to use these techniques as you would be aware of the object's class, fields, and methods.   But what if you are writing a specialized API (your own serialization) for I/O or an object stress test engine, debuger tool,  or perhaps a class browser for a visual development environment?  You'd want code that could handle any sort of object. Some examples of Java classes that make use of these techniques include ObjectInputStream, ObjectOutputStream, XMLEncoder, and XMLDecoder.  Oracles reflection tutorial provides more information on these techniques and some pros and cons of using reflection.

From the Classroom,  the debate on the most efficient way to increment a variable.

Mark Pendergast, Florida Gulf Coast University.

(This is the first in a series of articles I'll be doing under the  banner of "From the Classroom".  The topics of these articles stem from common questions, insights, misunderstandings and the occasional epiphany that my introductory Java programming students have raised over the years.) 

Chapter two of nearly every Java programming textbook centers on assignment statements, expressions, and operators. The first hurdle students face is the difference between an assignment statement and a mathematical equation. Both can use the same characters and look the same, but one calculates and stores a new value whereas the other specifies a relationship between variables. Just as they grasp this they are faced variable type checking and operator precedence.  Finally, you hit them with the fact that there are multiple ways to do the same thing.  One of the most perplexing is "why are there four ways to increment a variable by one?"
 

i++;
++i;
i += 1;
i = i + 1;

Why do they exists and where did they come from?  Java inherited much of its language syntax from C and C++.    Dennis Ritchie [1] explained that C was developed in the late 1960s and early 1970s at Bell Labs and evolved from the BCPL and B languages.  These languages inherited constructs from other languages such as Algol, FORTRAN, and PL/1. The += operator assignment came from Algol 68.  ++ and -- operators were developed for the B language and it is thought that they were created in order to take advantage of the DEC's PDP-7's minicomputer's auto-increment memory cells.  Others say that the creators of the B compiler found that the translation of i++ to machine code was smaller than that of i = i+1.  Memory was a valuable commodity in those days with the early mini computers having only 8k 18 bit words, so every word counted. 

Given that these operators exist, many programmers (especially my students) take delight in using them to write very compact, cryptic code, that uses the fewest number of keystrokes and characters. Call it a warped sense of pride.  Most of the style guides I've perused (Google, Oracle, etc) are silent on the issue of which to use.   One of the my early idols, Donald Knuth[2], once wrote that

"Programmers waste enormous amounts of time thinking about, or worrying about, the speed of noncritical parts of their programs, and these attempts at efficiency actually have a strong negative impact when debugging and maintenance are considered."

But if one were to disregard sage advice, coding style, and maintainability standards and only concentrate on performance, which of the four is the most efficient?

One way to check efficiency is to examine the byte code that each creates. To do this I wrote four short test programs:

public class Method1 {

   public static void main(String[] args)
    {
      int i = 0;
      i++;
      System.out.println(" i = "+i);   
    }
}


public class Method2 {
   public static void main(String[] args)
    {
      int i = 0;
      ++i;
      System.out.println(" i = "+i);   
    }
}


public class Method3 {
   public static void main(String[] args)
    {
      int i = 0;
      i += 1;
      System.out.println(" i = "+i);   
    }
}

public class Method4 {
   public static void main(String[] args)
    {
      int i = 0;
      i = i + 1;
      System.out.println(" i = "+i);   
    }
}

I then used the javap tool with the -c option from the java development kit[3] to examine the byte code each generated for incrementing the variable i.  Method 1, 2, and 3 all generated identical byte code, consisting of 1 increment instruction. Method 4, i = i+1, generated 4 instructions, one to load i onto the operand stack, one to place the constant 1 onto the operand stack, one to add the top two entries on the operand stack, and a final one to store the new value of i [4].

Clearly if methods 1, 2, 3 all generated the same byte code, then they ought to be equally efficient.  Method 4 on the other hand requires four times as many byte code instructions.  Can we conclude that method 4 is four times slower, or slower by any amount?

In early versions of Java you would be correct in thinking that.  JRE version 1.1 used a byte code interpreter to execute Java programs, that is, a program that read, interpreted and executed one line of byte code at a time.  Since JRE version 1.2, the interpreter was replaced by a just-in-time compiler (JIT) [4].  A java virtual machine with a JIT converts Java byte code to highly optimized machine code for a specific platform.

Since it would be difficult to intercept and examine this machine code, I created a second set of test codes that used a call to System.nanoTime()to retrieve the starting system time,  performed 10000 increments, and called System.nanoTime()a second time to retrieve the  ending time. 10000 increments were used because one increment would not take long enough to register an elapsed time (other than zero). The difference in times was then displayed.

              int i=0;
      long stime=0, ftime=0;
    
      stime = System.nanoTime();
      for(int j = 0; j < 10000; j++)
         i++;   // or ++i, i += 1, i = i+1;
      ftime = System.nanoTime();
     
      System.out.println("time = "+(ftime-stime));

 After running one of the methods a dozen times on my venerable overclocked quad processor (Intel(R) Core I7, Windows 7) the results were (as many of you already suspect)  spectacularly inconsistent, ranging from 15999 nano seconds to 84218 nanoseconds. Inconsistent data is, of course, the bane of any devoted fan of MythBusters.  So what happened?  Context switching during the runs.  Since there doesn't seem to be a method called System.holdContextSwitches() I had to get a little creative in capturing the time it takes to perform the operations.  My next test consisted of measuring the time it took to perform 10000 increments a million times, then reporting the minimum elapsed time. My hope is to capture a test run without any context switches.    Was it necessary to do the test a million times?  Probably not, but as many have often said, "any worth doing is worth overdoing". 

       int i=0;
      long stime=0, ftime=0;
      long min = Long.MAX_VALUE;
      for(int k=0 ; k < 1000000; k++)
      {
       stime = System.nanoTime();
       for(int j = 0; j < 10000; j++)
         i++;  // or ++i, i += 1, i = i+1;
       ftime = System.nanoTime();
       if(ftime-stime < min)
           min = ftime-stime;
      }
      System.out.println("time = "+min); 

This time a dozen runs produced a result of 15696 nanoseconds for the i++ method. Consistent data at last. I repeated this for the ++i, and the i += 1 methods.  As expected, all reported 15696 nanoseconds.  But what about the i = i+1 method, the one that had four times as many byte code instructions?  It too reported a consistent 15696 nanoseconds.  The skeptics among you would rightly point out that the overhead of the for-loop dwarfed the actual time used by the increment instructions. In fact, the for-loop has a j++ increment instruction in it.  To account for that possibility, I decreased the for-loop from 1000 to 10, and repeated the increment instruction within the for-loop 1000 times (thank heavens for cut and paste).   Again, the time taken was consistent for all four methods. This time it was lower, 2414 nanoseconds, since the for- loop overhead was only incurred 10 times instead of 10000 times.

So from a performance standpoint, these tests did not show any difference between the four methods. That doesn't mean that there isn't some scenario or platform where the JIT optimizer wouldn't favor one method over another, but we can be fairly safe in saying it is OK to use whatever one you like. What do I prefer?  I tend to use i++ whenever an increment of one is called for and the += for any other value.  I advise my students against placing the ++ within another expression, e.g. I would prefer that they not write code like  j = i++ * 4; instead use two statements, j = i*4; and i++;  And in case you are curious, they produce the same byte code (slightly different ordering) and execute in the same amount of time using the testing method previously discussed.

[1] Ritchie, Dennis, M. "The development of the C Language" retrieved from http://cm.bell-labs.com/who/dmr/chist.html.

[2] Rubens, Paul "10 Bad Coding Practices that Wreck Software Development Projects", retrieved from http://www.cio.com/article/2448952/developer/10-bad-coding-practices-that-wreck-software-development-projects.html

[3] Oracle, "javap - The Java Class File Disassembler", retrieved from  http://docs.oracle.com/javase/8/docs/technotes/tools/windows/javap.html

[4] Oracle, "Understanding Just-In-Time Compilation and Optimization", retrieved from http://docs.oracle.com/cd/E15289_01/doc.40/e15058/underst_jit.htm

[5] Oracle, "Chapter 6, The Java Virtual Machine Instruction Set", retrieved from http://docs.oracle.com/javase/specs/jvms/se7/html/jvms-6.html
 

I have an application that needs to notify the user when some job is done. It uses JSF and Primefaces, so this sort of notification could be implemented using atmosphere (aka Push).

But another funny approach is to use a XMPP server embedded in your java web application. Ok, ok, you don't have to embed it, you can just run an instance of a industrial-strenght XMPP server such as Openfire and Tigase. But hey, we're just playing a little, so I'll show you how to do it using Vysper, a little proof-of-concept developed using Apache Mina, that is simple enough for a few minutes play.

Before showing how to do it, it's nice to remember that threads and JEE applications usually don't mix, so we can play, but we must be aware of what are we doing.

First, you're going to need these JARs, most of them come from Vysper. Just a few huh?

• aopalliance-1.0.jar
• commons-codec-1.4.jar
• commons-collections-3.1.jar
• commons-io-1.4.jar
• commons-lang-2.5.jar
• commons-logging-1.1.jar
• concurrent-1.3.4.jar
• derby-10.2.1.6.jar
• dnsjava-2.0.8.jar
• ehcache-core-2.2.0.jar
• fontbox-0.1.0.jar
• jackrabbit-api-1.5.0.jar
• jackrabbit-core-1.5.3.jar
• jackrabbit-jcr-commons-1.5.3.jar
• jackrabbit-spi-1.5.0.jar
• jackrabbit-spi-commons-1.5.0.jar
• jackrabbit-text-extractors-1.5.0.jar
• jcl-over-slf4j-1.5.3.jar
• jcr-1.0.jar
• jempbox-0.2.0.jar
• jetty-continuation-7.2.1.v20101111.jar
• jetty-http-7.2.1.v20101111.jar
• jetty-io-7.2.1.v20101111.jar
• jetty-security-7.2.1.v20101111.jar
• jetty-server-7.2.1.v20101111.jar
• jetty-servlet-7.2.1.v20101111.jar
• jetty-util-7.2.1.v20101111.jar
• jetty-websocket-7.2.1.v20101111.jar
• log4j-1.2.14.jar
• lucene-core-2.3.2.jar
• mina-core-2.0.2.jar
• nbxml-0.7.jar
• nekohtml-1.9.7.jar
• pdfbox-0.7.3.jar
• poi-3.0.2-FINAL.jar
• poi-scratchpad-3.0.2-FINAL.jar
• primefaces-4.0.jar
• servlet-api-2.5.jar
• slf4j-api-1.5.3.jar
• slf4j-log4j12-1.5.3.jar
• smack-3.1.0.jar
• smackx-3.1.0.jar
• spec-compliance-0.7.jar
• spring-aop-3.0.5.RELEASE.jar
• spring-asm-3.0.5.RELEASE.jar
• spring-beans-3.0.5.RELEASE.jar
• spring-context-3.0.5.RELEASE.jar
• spring-core-3.0.5.RELEASE.jar
• spring-expression-3.0.5.RELEASE.jar
• vysper-core-0.7.jar
• vysper-websockets-0.7.jar
• xep0045-muc-0.7.jar
• xep0060-pubsub-0.7.jar
• xep0124-xep0206-bosh-0.7.jar
• xercesImpl-2.8.1.jar
• xml-apis-1.3.03.jar

Now, copy from Vysper the bogus certificate, so your XMPP server can "work" under a "secure" channel. It's called bogus_mina_tls.cert.

My xhtml looks like this

<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml"
    xmlns:f="http://java.sun.com/jsf/core"
    xmlns:h="http://java.sun.com/jsf/html"
    xmlns:ui="http://java.sun.com/jsf/facelets"
    xmlns:p="http://primefaces.org/ui">
<h:head>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
    <title>Messaging Prototype</title>
    <link rel="icon" type="image/png" href="favicon.ico"></link>   
</h:head>
<h:body>   
    <h:outputStylesheet library="css" name="style.css"  />

    <p:ajaxStatus style="width:16px;height:16px;" id="ajaxStatusPanel"> 
        <f:facet name="start"> 
            <h:graphicImage value="./ajaxloading.gif" /> 
        </f:facet> 
         
        <f:facet name="complete"> 
            <h:outputText value="" /> 
        </f:facet> 
    </p:ajaxStatus> 

    <h:form>   
        <p:messages id="messages" showDetail="true" autoUpdate="true" closable="true" />
        <p:spacer height="10" />
        <p:panel>
            <h:panelGrid columns="2">
                <p:commandButton value="Enter" action="#{messagingMB.sendMessage}" />
            </h:panelGrid>
        </p:panel>
        <p:spacer height="10" />
    </h:form>
</h:body>
</html>

Pretty simple huh? The Managed Bean is also easy.



import java.io.Serializable;

import javax.annotation.PostConstruct;
import javax.ejb.EJB;
import javax.faces.bean.ManagedBean;
import javax.faces.bean.ViewScoped;

import org.jivesoftware.smack.XMPPException;

@ManagedBean
@ViewScoped
public class MessagingMB implements Serializable {

    private static final long    serialVersionUID    = -9092497421080796430L;
   
    @EJB
    private JSFUtilEJB jsfUtilEJB;
   
    @PostConstruct
    public void init() {
    }

    public void sendMessage() {
        try {
            new BasicClient().test();
        } catch (XMPPException e) {
            jsfUtilEJB.addErrorMessage(e,"Could not send");
        }
    }
}

and of course, the EJBs



import javax.ejb.Stateless;
import javax.faces.application.FacesMessage;
import javax.faces.application.FacesMessage.Severity;
import javax.faces.context.FacesContext;

@Stateless
public class JSFUtilEJB {

    @SuppressWarnings("unchecked")
    public <T> T findBean(String beanName) {
        FacesContext context = FacesContext.getCurrentInstance();
        return (T) context.getApplication().evaluateExpressionGet(context, "#{" + beanName + "}", Object.class);
    }

    public long addErrorMessage(String msg) {
        return addMessage(null,FacesMessage.SEVERITY_ERROR,msg);
    }
    public long addErrorMessage(Exception e,String summary){
        return addMessage(e,FacesMessage.SEVERITY_ERROR,summary);
    }
    public long addFatalErrorMessage(Exception e,String summary){
        return addMessage(e,FacesMessage.SEVERITY_FATAL,summary);
    }
    public long addInfoMessage(String summary){
        return addMessage(null,FacesMessage.SEVERITY_INFO,summary);
    }
    public long addWarnMessage(Exception e,String summary){
        return addMessage(e,FacesMessage.SEVERITY_WARN,summary);
    }

    public long addErrorMessage(Exception e) {
        return addMessage(e,FacesMessage.SEVERITY_ERROR,e.getMessage(),e.getClass().getSimpleName());
    }

    private long addMessage(Exception e,Severity severity, String summary) {
        FacesContext context = FacesContext.getCurrentInstance();
        String clientId = null;
       
        long id = -1;
       
        if (e != null){
            id = printStackTrace(e);
            FacesMessage facesMessage = null;
            if (e.getCause() instanceof org.apache.openjpa.persistence.EntityExistsException){
                facesMessage = new FacesMessage(severity,"[Error: #"+id+"] "+summary,"You are trying are to add a new object that already exists or your're trying to violate a unique constraint)" );   
            }else{
                facesMessage = new FacesMessage(severity,"[Error: #"+id+"] "+summary,e.getMessage() );
            }
            context.addMessage(clientId , facesMessage );
        }else{
            FacesMessage facesMessage = new FacesMessage(severity,summary," ");
            context.addMessage(clientId , facesMessage );
        }

        return id;
    }

    private long addMessage(Exception e,Severity severity, String summary, String detail) {
        FacesContext context = FacesContext.getCurrentInstance();
        String clientId = null;
       
        long id = -1;
        if (e != null){
            id = printStackTrace(e);       
            FacesMessage facesMessage = new FacesMessage(severity,"["+id+"] "+summary,detail );
            context.addMessage(clientId , facesMessage );
        }else{
            FacesMessage facesMessage = new FacesMessage(severity,summary,detail );
            context.addMessage(clientId , facesMessage );
        }
       
        return id;
    }

    public long printStackTrace(Exception e){
        long uniqueId = System.currentTimeMillis();
        return uniqueId;
    }

}

and



import java.io.File;
import java.io.Serializable;

import javax.annotation.PostConstruct;
import javax.ejb.Singleton;
import javax.ejb.Startup;

import org.apache.vysper.mina.TCPEndpoint;
import org.apache.vysper.storage.StorageProviderRegistry;
import org.apache.vysper.storage.inmemory.MemoryStorageProviderRegistry;
import org.apache.vysper.xmpp.addressing.EntityImpl;
import org.apache.vysper.xmpp.authorization.AccountManagement;
import org.apache.vysper.xmpp.server.XMPPServer;

@Startup
@Singleton
public class XmppEJB implements Serializable {
   
    /**
     * <br>06/09/2014
     */
    private static final long serialVersionUID = 1L;
    private boolean started;

    @PostConstruct
    public void init() {
        try {
            // choose the storage you want to use
    //        StorageProviderRegistry providerRegistry = new JcrStorageProviderRegistry();
            StorageProviderRegistry providerRegistry = new MemoryStorageProviderRegistry();
   
            final AccountManagement accountManagement = (AccountManagement) providerRegistry.retrieve(AccountManagement.class);
   
            if(!accountManagement.verifyAccountExists(EntityImpl.parse("user1@vysper.org"))) {
                accountManagement.addUser(EntityImpl.parse("user1@vysper.org"), "password");
            }
            if(!accountManagement.verifyAccountExists(EntityImpl.parse("user2@vysper.org"))) {
                accountManagement.addUser(EntityImpl.parse("user2@vysper.org"), "password");
            }
           
            XMPPServer server = new XMPPServer("vysper.org");
            server.addEndpoint(new TCPEndpoint());
            server.setStorageProviderRegistry(providerRegistry);
            server.setTLSCertificateInfo(new File("/path/to/bogus_mina_tls.cert"), "boguspw");
           
            server.start();
            System.out.println("server is running...");
        } catch (Exception e) {
            e.printStackTrace();
        }
       
        started = true;
    }

    public boolean isStarted() {
        return this.started;
    }
   
   
   
}

and the basic client, which comes from Vysper.

import java.util.Date;

import org.jivesoftware.smack.Chat;
import org.jivesoftware.smack.ConnectionConfiguration;
import org.jivesoftware.smack.MessageListener;
import org.jivesoftware.smack.PacketListener;
import org.jivesoftware.smack.Roster;
import org.jivesoftware.smack.SASLAuthentication;
import org.jivesoftware.smack.XMPPConnection;
import org.jivesoftware.smack.XMPPException;
import org.jivesoftware.smack.filter.PacketFilter;
import org.jivesoftware.smack.packet.IQ;
import org.jivesoftware.smack.packet.Message;
import org.jivesoftware.smack.packet.Packet;
import org.jivesoftware.smack.packet.PacketExtension;
import org.jivesoftware.smack.packet.Presence;
import org.jivesoftware.smackx.packet.Time;
import org.jivesoftware.smackx.packet.Version;

public class BasicClient {

    static class IQListener implements PacketListener {

        public void processPacket(Packet packet) {
            IQ iq = (IQ) packet;
            String iqString = iq.toString();
            System.out.println("T" + System.currentTimeMillis() + " IQ: "
                    + iqString + ": " + iq.toXML());
        }
    }

    static class PresenceListener implements PacketListener {

        public void processPacket(Packet packet) {
            Presence presence = (Presence) packet;
            String iqString = presence.toString();
            final PacketExtension extension = presence
                    .getExtension("http://jabber.org/protocol/caps");
            if (extension != null)
                System.out.println("T" + System.currentTimeMillis() + " Pres: "
                        + iqString + ": " + presence.toXML());
        }
    }

    public void test() throws XMPPException {

        String me = "user2@vysper.org";
        String to = "user1@vysper.org";

        try {
            ConnectionConfiguration connectionConfiguration = new ConnectionConfiguration(
                    "localhost");
            connectionConfiguration.setCompressionEnabled(false);
            connectionConfiguration.setSelfSignedCertificateEnabled(true);
            connectionConfiguration.setExpiredCertificatesCheckEnabled(false);
//            connectionConfiguration.setDebuggerEnabled(true);
            connectionConfiguration
                    .setSecurityMode(ConnectionConfiguration.SecurityMode.required);
//            XMPPConnection.DEBUG_ENABLED = true;
            XMPPConnection connection = new XMPPConnection(
                    connectionConfiguration);
            connection.connect();

            SASLAuthentication saslAuthentication = connection
                    .getSASLAuthentication();
            saslAuthentication.authenticate(me, "password", "test");
            connection.login(me, "pqssword");

            connection.getRoster().setSubscriptionMode(
                    Roster.SubscriptionMode.accept_all);

            connection.addPacketListener(new IQListener(), new PacketFilter() {
                public boolean accept(Packet packet) {
                    return packet instanceof IQ;
                }
            });

            connection.addPacketListener(new PresenceListener(),
                    new PacketFilter() {
                        public boolean accept(Packet packet) {
                            return packet instanceof Presence;
                        }
                    });

            Chat chat = null;
            if (to != null) {
                Presence presence = new Presence(Presence.Type.subscribe);
                presence.setFrom(connection.getUser());
                String toEntity = to;
                presence.setTo(toEntity);
                connection.sendPacket(presence);

                chat = connection.getChatManager().createChat(toEntity,
                        new MessageListener() {
                            public void processMessage(Chat inchat,
                                    Message message) {
                                System.out.println("log received message: "
                                        + message.getBody());
                            }
                        });
            }

            connection.sendPacket(new Presence(Presence.Type.available,
                    "pommes", 1, Presence.Mode.available));

            Thread.sleep(1000);

            // query server version
            sendIQGetWithTimestamp(connection, new Version());

            // query server time
            sendIQGetWithTimestamp(connection, new Time());

            chat.sendMessage("Hello " + to + " at " + new Date());

            connection.disconnect();
        } catch (Throwable e) {
            e.printStackTrace(); // To change body of catch statement use File |
                                    // Settings | File Templates.
        }
        System.out.println("bye");
    }

    private static void sendIQGetWithTimestamp(XMPPConnection connection, IQ iq) {
        iq.setType(IQ.Type.GET);
        connection.sendPacket(iq);
        System.out.println("T" + System.currentTimeMillis()
                + " IQ request sent");
    }
}

We're almost set. Of course, now we need a XMPP client, such as Pidgin

First, let me just say thanks to this blog, because I don't know why, Vysper site has little to no information about how to configure Pidgin, so this blog post was really useful.

Let me show you how my pidgin user looks like
 

I know, it's in Portuguese.


And that's it. We're all set. Start your JSF web application and play.

Notice that the communication is bidirectional, so you can just use your XMPP client to send commands to the server. To do that, you just have to change this listener

                chat = connection.getChatManager().createChat(toEntity,
                        new MessageListener() {
                            public void processMessage(Chat inchat,
                                    Message message) {
                                System.out.println("log received message: "
                                        + message.getBody());
                            }
                        });

I wonder if we could just create a DSL to process some commands and if we could find some autocomplete pidgin plugin to write commands using this DSL. Suggestions are welcome. ;-)

ps. The EJB does not shutdown the server gracefully. But I bet there's some EJB annotation to do that in an event of a server shutdown.

Continuous learning from the best literature is the best way to grow as a Java developer and craftsman. Learning from other experienced developers is also a good way to learn. And anyone that would like to get all of this will now have a chance to do so. We are just starting up a global reading group fro reading of Refactoring to patterns and we will meet in a phone meeting every or every second Wednesday at 12:00 am GMT from the middle or end of January to discuss what we have read and learn from each other. There are still a chance to join this and the information about how it works and how to raise an interest for this can be found in the Remote learning community.

This is an activity that have been run two times before within the IBM internal Java developers community but this year will we run it with this developerWorks community as our home and open for anyone. There is no requirement on any skills in advance. Some basic knowledge of Java and development is enough. We will be developers with different level of skills and experiences that want to learn from each other. The interest to learn is what unite us. There is neither any requirement to participate in every meeting. Anyone join as often as they can. And there is neither any requirement to say anything. It is okay to just listen on what the others say also.

If you need the TCL expect functionality in your java software, you basically have 4 options

1. Expect4J (http://code.google.com/p/expect4j/)
2. ExpectJ (http://expectj.sourceforge.net/)
3. Expect-for-java (https://github.com/ronniedong/Expect-for-Java)
4. Enchanter (http://code.google.com/p/enchanter/)

TCL-Java bridges such as Jacl/TclBlend (http://tcljava.sourceforge.net) are useful to run small pieces of TCL code, but since expect contains some native C libraries, jacl is not an option. In the best case, you can write a TCL wrapper with TclBlend that calls your java code instead of the opposite.

So, from these 4 options, only ExpectJ supports expect "interact" mode. In expect, the "interact" command gives back the session control to the user. So, if the idea is to connect to a remote site after multiple hops, and get the session back (instead of running fire-and-forget scripts), ExpectJ is the way to go.

Now, you can also expose ExpectJ directives in a nice way (DSL) using groove. This is specially useful if you want to let the user write his/her own "expect" scripts.

Allowing the user to run his/her own scripts from inside an application also has security issues. For example, would you allow the user to add a "system.exit()" command? Not a good idea. We have to limit this too.

Groovy has some syntactic sugar features that lets the developer to write DSLs in a much faster and simpler way that it would be if you decide to write a full language from the scratch, with tools like ANTLR and JavaCC. You can also use eclipse to edit and run your Groovy code in a nice IDE and the learning curve, for Java developers, is nice.

To demonstrate a simple proof-of-concept, let's assume these 4 steps

1. Wrap ExpectJ engine in a java class
2. Wrap java class into a Groovy class
3. Add Groovy syntatic sugar
4. Sandbox Groovy script interpreter

We're also using Jsch (http://www.jcraft.com/jsch/), a pure java-based SSH client.

Step #1 - Wrap ExpectJ engine in a java class

public class Expect {
    
    private ExpectJ expectinator;
    private Spawn spawn;
    private int lastPos;

    public Expect() throws IOException{
        expectinator = new ExpectJ(5);
        spawn = expectinator.spawn("localhost", 22, "leoks", "xyz");
    }
    
    public void send(String s) throws IOException{
        spawn.send(s+"\n");
    }
    
    public String expect(String s) throws IOException, TimeoutException{
        spawn.expect(s);
        return getReply();
    }
    
    private String getReply(){
        String all = spawn.getCurrentStandardOutContents();
        int newLastPos = all.length();
        String reply = all.substring(lastPos);
        lastPos = newLastPos;
        return reply;
    }
    
}

Step #2 - Wrap java class into a Groovy class

class ExpectGroovy {
    
    def Expect expectJava = new Expect()
    
    def expect(String expression){
        expectJava.expect(expression)
    }
    
    def send(String command){
        return expectJava.send(command)
    }
}

Step #3 - Add Groovy syntatic sugar

class Example {
    static main(args) {
        def binding = new Binding(exp: new ExpectGroovy())
        
        ExpectSandbox sandbox = new ExpectSandbox();
        
        def config = new CompilerConfiguration()
        config.scriptBaseClass = ExpectBaseScriptClass.class.name
        config.addCompilationCustomizers(new SandboxTransformer())
        sandbox.register()
        
        def shell = new GroovyShell(this.class.classLoader, binding, config)
        String script = ...
        try{
            shell.evaluate(script)
        }catch(e){e.printStackTrace()}
        
        sandbox.unregister()
        print "OK"
    }
}
abstract class ExpectBaseScriptClass extends Script{
    void expect(String expression){
        this.binding.exp.expect expression
    }
    
    void send(String command){
        this.binding.exp.send command
    }
}

Step #4 - Sandbox Groovy script interpreter

Now, the groovy script may look like this

expect "\\$"

//try to uncomment - the script does not kill the process, instead this instruction
//is blocked by groovy security restrictions
//see http://groovy.codehaus.org/Advanced+compiler+configuration
//System.exit(0)

x = 0
while (x < 5){
    result = send "hostname"
    expect "\\$"
    x++
}

Please notice that

• The “expect” object is implicit (binding)
• Groovy loops and variable assignment for free (mallet for example needed a special code just for loop evaluation)
• Groovy allows to restrict certain undesired commands, as well as java packages restrictions
• No explicit grammar defined
• Groovy can be coded with eclipse, with code completion, etc

References

• http://groovy.codehaus.org/Writing+Domain-Specific+Languages
• http://www.slideshare.net/glaforge/groovy-dsls-from-beginner-to-expert-guillaume-laforge-and-paul-king-springone2gx-2011
• http://www.ibm.com/developerworks/library/j-pg02179/

The parse method in SimpleDateFormat converts an input String to a Date format. It uses the Date-time patterns while constructing the SimpleDateFormat for parsing the input string. The pattern letters are defined in the API specification [ http://docs.oracle.com/javase/6/docs/api/java/text/SimpleDateFormat.html].

There are 4 different letter patterns which can be used to parse the hours in the string input.

H - represents Hour in day (0-23) format.
k - represents Hour in day (1-24) format. Here, 24 is the start of a day.
K - represents Hour in am/pm (0-11) format.
h - represents Hour in am/pm (1-12) format.Here, 12am is the start of a day.

The letter pattern 'a' represents the am/pm marker which can be used in conjunction with 12-hour format.

If an input hour of '24' is specified, it will be parsed as '00' of the next day with 'HH' pattern where as it will be parsed as '00' of the same day with 'kk' pattern as 24 is the start of the day for 'kk' format.

If an input hour of '12' is specified, it will be parsed as '12' of the same day with 'KK' pattern where as it will be parsed as '00' of the same day with 'hh' pattern as 12am is the start of the day and am is the default am/pm marker if not specified.

If an input am/pm marker of 'pm' is specified, an hour offset of '12' will be added to the input hour. For example, if '15' is specified as the input hour, it will be parsed as '15' of the same day with 'am' where as it will be parsed as '03' of the next day  with 'pm' ( 15 + 12 = 27 ; 27-24 = 3 of next day).


The following code snippet can be used to try out different date-time patterns.

public static void main(String[] args) {
    String dateformats = "yyyyMMddhhmmss";
    
    String inputDate="20130313150000";
    
    SimpleDateFormat dateFormat = new SimpleDateFormat(dateformats);
    try {
    System.out.println("Date-Time Pattern = " + dateformats + " : Input Date-Time string = " + inputDate + " : Parsed Date output = "+ dateFormat.parse(inputDate));
    } catch (ParseException e) {
    e.printStackTrace();
    }
}

    
For example, the parsed Date output for some sample input date-time string with different Date-Time pattern is given below

Contributed by:

Sai Hema, Java Technology Center