WebSphere Portal Services team

If you configured a syndicator / subscriber pair for automatic syndication but the automatic syndication does not start and instead of syndicating the content the syndicator goes to pending state then check the property "connect.moduleconfig.syndication.update.subscribers.on.start" in the WebSphere Application Server WAS Admin console ( Resource environment providers > WCM WCMConfigService > Custom properties ).

If this property is set to "connect.moduleconfig.syndication.update.subscribers.on.start=false" on either the syndicator or the subscriber delete this property from the resource environment provider, synchronize the cell and restart the servers.

In a customer project we were migrating from IBM WebSphere Portal v8 to IBM WebSphere Portal Version 9. After the migration log messages like

com.ibm.ws.exception.RuntimeWarning: com.ibm.ws.exception.RuntimeError: com.ibm.ws.exception.RuntimeError: java.lang.RuntimeException: com.ibm.ws.cdi.CDIRuntimeException: com.ibm.ws.cdi.CDIDeploymentRuntimeExcep
tion: org.jboss.weld.exceptions.DeploymentException: org.apache.wink.client.ClientConfig

...

Caused by: org.jboss.weld.exceptions.DeploymentException: org.apache.wink.client.ClientConfig
...
Caused by: java.lang.NoClassDefFoundError: org.apache.wink.client.ClientConfig

were shown in the SystemOut.log. It turned out that some of the installed applications use Apache Wink Java APIs to communicate with RESTful services.

The following article in the knowledge center shows how to use Apache Wink Client applications with WebSphere Application Server.

If you migrate to WebSphere Application Server v9 and you see exceptions like the ones above in your SystemOut.log and your applications use the Apache Wink Java APIs too, you might want to check your JAX-RS version.

You can do this in the WebSphere Application Server Console -> WebSphere Application Server -> Select Server -> Container Services -> Default JAXRS provider settings.

The default Jaxrs Implementation in WebSphere Application Server v9 is the Apache CXF implementation, provider version 2.0. However, the Apache Wink implementation is supported by the Jaxrs provider Version 1.1.

Like described in the knowledge center you can switch you Jaxrs Implementation to Version 1.1 in which the Apache Wink APIs are supported . An alternative is to update the Jaxrs Implementation of your custom applications.

Starting with Digital Experience - WebSphere Portal v8.5 IBM switched to a continuous delivery model providing new features with every new Combined Cumulative Fix. The currently latest available CF14 is no exception to that. The following new features and improvements are available via CF14:

• IBM WebSphere Portal Version 8.5 and 9.0
• Integration with IBM Watson Content Hub (requires IBM WebSphere Portal Version 9.0)
• Performance improvements for IBM Web Content Manager
• Rich Media Edition Server support
• Web Application Bridge improvement
• IBM Social Rendering Templates for Digital Data Connector

Please see https://www.ibm.com/support/knowledgecenter/SSHRKX_8.5.0/mp/overview/new_cf14.html for more details on these enhancements.

If you are interested in what was delivered in older CFs start here: https://www.ibm.com/support/knowledgecenter/SSHRKX_8.5.0/mp/overview/new_cf.html

Analyzing a problem in WebSphere Application Server often requires looking at the javacore file. Once you are identifying an interesting thread there it often requires to look at the SystemOut/SystemErr log for entries originated from that thread. But how to find those? Well in WAS v8.5.5 it is quite easy as you can see in the the below example.

Looking at the javacore file locate the 3XMJAVALTHREAD eye-catcher line for the thread and take the value shown next to the getId: which is 0x7CB in our example.

3XMTHREADINFO      "WebContainer : 16" J9VMThread:0x000000000A967100, j9thread_t:0x0000000027D4A6F0, java/lang/Thread:0x00000006E81BB378, state:R, prio=5
3XMJAVALTHREAD            (java/lang/Thread getId:0x7CB, isDaemon:true)
3XMTHREADINFO1            (native thread ID:0x600, native priority:0x5, native policy:UNKNOWN, vmstate:R, vm thread flags:0x01000001)
3XMCPUTIME               CPU usage total: 4274.125000000 secs, user: 3905.828125000 secs, system: 368.296875000 secs, current category="Application"

Now switch to the SystemOut/SystemErr log which by default lists the id of the thread after the timestamp. This thread id matches the id we identified before in the javacore ignoring leading zeros and the case for the hex characters:

[2017-05-10 06:32:22:735 CEST] 000007cb SRTServletRes W com.ibm.ws.webcontainer.srt.SRTServletResponse setIntHeader CWSRV8094W: WARNING: Cannot set header. Response already committed.

Just downloaded the WebSphere Application Server Performance Tuning Toolkit and tried to start it on my RHEL7 workstation. However the start of ./PerfTuningToolkit  failed with the following error:

(PerfTuningToolkit:26327): Gtk-WARNING **: Unable to locate theme engine in module_path: "adwaita",

(PerfTuningToolkit:26327): Gtk-WARNING **: Unable to locate theme engine in module_path: "adwaita",
Gtk-Message: Failed to load module "pk-gtk-module"
27  PerfDB  INFO   [main] openjpa.Runtime - Starting OpenJPA 2.0.1
87  PerfDB  INFO   [main] openjpa.jdbc.JDBC - Using dictionary class "org.apache.openjpa.jdbc.sql.DerbyDictionary".
Job found still running after platform shutdown.  Jobs should be canceled by the plugin that scheduled them during shutdown: org.eclipse.ui.internal.progress.ProgressViewUpdater$1

As there are multiple errors here started to install the missing packages first (32bit compatibility packages were already installed):

yum -y install PackageKit-gtk3-module.i686 PackageKit-gtk3-module.x86_64

yum -y install gnome-themes-standard.x86_64

yum -y install adwaita-gtk2-theme.i686 adwaita-gtk2-theme.x86_64 adwaita-icon-theme.noarch adwaita-cursor-theme.noarch adwaita-icon-theme-devel.noarch adwaita-qt.x86_64 adwaita-qt-common.noarch adwaita-qt4.i686 adwaita-qt4.x86_64 adwaita-qt5.i686 adwaita-qt5.x86_64

But after that this the WebSphere Application Server Performance Tuning Toolkit still did not start. Found in the file <install_home>/workspace/.metadata/.log that libxpcom.so could not be found by the <install_home>/configuration/org.eclipse.osgi/bundles/104/1/.cp/libswt-mozilla-gtk-3655.so (ldd <install_home>/configuration/org.eclipse.osgi/bundles/104/1/.cp/libswt-mozilla-gtk-3655.so).

Hence installed the following xulrunner-compat packages:

yum -y install xulrunner-compat.i586 xulrunner-compat-flash-plugin.i686

and edited <install_home>/PerfTuningToolkit.ini by adding the line "-Dorg.eclipse.swt.browser.XULRunnerPath=/usr/lib/xulrunner-compat/" after the line "-Dosgi.requiredJavaVersion=1.5"

Recently we got an inquiry from a customer how he can monitor the progress of the "./ConfigEngine.sh export-wcm-data" task. Unfortunately there is no progress information available during the export.

However I learned from development that the progress can be monitored if all (and only if all) libraries are exported namely by comparing the number of *.node file in the export directory with result of the following SQL query: "SELECT COUNT(1) FROM jcrSchema.ICMSTJCRLINKS WHERE WSID = 1"

Since I can think of using the WebSphere Application Server plug-in (and that's for a while now ) the plug-in running in the HTTP server was falling back from HTTPS to HTTP protocol if an HTTPS request was received but no SSL communication could be established to the WebSphere Application Server web-container. This changed in WebSphere Application server v8.5. See http://www-01.ibm.com/support/docview.wss?uid=nas8N1019946 for details.

After years of supporting only Jython v2.1 in the wsadmin scripts with the latest version of WebSphere Application Server namely v9.0 the wsadmin scripting framework supports Jython v2.7 now!

At the same time the old Jython v2.1 can still be used in case you are facing compatibility issues with the new Jython version. To run your wsadmin scripts with Jython v2.1 library either specify "-usejython21 true" at the command line at invocation time or set the properity "com.ibm.ws.scripting.usejython21=true" in "<profile_home>/properties/wsadmin.properties".

Please see https://www.ibm.com/support/knowledgecenter/SSAW57_9.0.0/com.ibm.websphere.nd.multiplatform.doc/ae/cxml_jython.html for further details.

In a project we were using the IBM UX Screen Flow Manager to implement portlet based dialogs.

After migrating from WebSphere Portal 8.0 to WebSphere Portal 8.5 the following error occurred when importing the xmlaccess script which defines the access control on the virtual resource.

EJPXA0067E: The following configuration data is needed to create a virtual-resource resource: resource-parentref. [virtual-resource name=DIALOG_DEFINITIONS]

Solution:

Replace the name of the virtual-resource DIALOG_DEFINITONS with the name PCM_DIALOGS.

<virtual-resource action="update" domain="rel" name="PCM_DIALOGS">
  <access-control externalized="false" owner="undefined" private="false">
    <role actionset="User" update="set">
      <mapping subjectid="all authenticated portal users" subjecttype="user_group" update="set"/>
    </role>
  </access-control>
</virtual-resource>

If you need to check the content of the keytab for the SPNEGO implementation on WebSphere you can list the content by running:

"<WAS_HOME>/java/jre/bin/java com.ibm.security.krb5.internal.tools.Klist -e -k <keytab>"

Today we were working on an SPNEGO SSO authentication problem which did not work. We had actually two "flavours" of the problem. In one environment SPNEGO SSO did not work at all and in the other environment SPNEGO worked fine at the primary node but did not work on the second cluster member.

In first problem scenario we always got the login page prompted asking for user-id and password but all setup seemed correct. We enabled the following trace on WebSphere Application Server: "com.ibm.ws.security.spnego.*=all" and recreated the problem.

In the trace file we found the following lines:

--- snip ---

[1/17/17 10:23:55:821 CET] 00000139 SpnegoHandler 2 com.ibm.ws.security.spnego.SpnegoHandler isAuthHeaderNotSPNEGO Client sent back a non-SPNEGO authentication header: 0000:  53535000 01000000                      SSP. ....
-0000:  06062b06 01050502                      ..+. ....

Request: 0000:  4e544c4d 53535000 01000000 978208e2    NTLM SSP. .... ....
0010:  00000000 00000000 00000000 00000000    .... .... .... ....
0020:  0601b11d 0000000f                      .... ....

[1/17/17 10:23:55:822 CET] 00000139 SpnegoHandler < com.ibm.ws.security.spnego.SpnegoHandler isAuthHeaderNotSPNEGO RETURN true

--- snip ---

The section SPNEGO issues and their possible solutions in the WebSphere Application Server Information Center lists several possible causes for this problem. In our case it was quite a simple cause ( which is quite at the end of the list ) namely that the DNS entry for the hostname we used to access the site was an ALIAS and not a HOST record.

You can check this by either using the native Linux "host <host-name>" command which gives the following result for an alias:

--- snip ---

[xxx@yyy 20170117]$ host baddnsentryforspnego.acme.com
baddnsentryforspnego.acme.com is an alias for realhost.sub.acme.com.
realhost.sub.acme.com has address 192.168.150.66.185

--- snip ---

but for a HOST record the result looks like the following:

--- snip ---

[xxx@yyy 20170117]$ host intranetmigrdep.sncf.fr
gooddnsentryforspnego.acme.com has address 192.167.77.174

--- snip ---

or - alternatively you can use "nslookup" with "set debug" enabled. On Linux the output looks like the following:

--- snip ---

[xxx@yyy 20170117]$ nslookup
> set debug
> baddnsentryforspnego.acme.com
Server:        192.168.17.159
Address:    192.168.17.159#53

------------
    QUESTIONS:
    baddnsentryforspnego.acme.com, type = A, class = IN
    ANSWERS:
    ->  baddnsentryforspnego.acme.com
    canonical name = realhost.sub.acme.com.
    ttl = 1469
    ->  realhost.sub.acme.com
    internet address = 192.168.150.66.185
    ttl = 184
    AUTHORITY RECORDS:
    ADDITIONAL RECORDS:
------------
Non-authoritative answer:
baddnsentryforspnego.acme.com    canonical name = realhost.sub.acme.com.
Name:    realhost.sub.acme.com
Address: 192.168.150.66.185
>
>
> gooddnsentryforspnego.acme.com
Server:        192.168.17.159
Address:    192.168.17.159#53

------------
    QUESTIONS:
    gooddnsentryforspnego.acme.com, type = A, class = IN
    ANSWERS:
    ->  gooddnsentryforspnego.acme.com
    internet address = 192.167.77.174
    ttl = 1197
    AUTHORITY RECORDS:
    ADDITIONAL RECORDS:
------------
Non-authoritative answer:
Name:    gooddnsentryforspnego.acme.com
Address: 192.167.77.174

--- snip ---

It's quite easy to test is that is your problem - just add an entry to the clients hosts file for the baddnsentryforspnego.acme.com name with the correct IP address and test.

For the second problem namely that SPNEGO worked on the primary cluster node but failed on the secondary cluster node we found that "Use the alias host name for the application server" was checked in the WAS Administrative Console (Global security -> SPNEGO web authentication) without proper aliased being set-up. Unchecked this setting fixed the problem for us.

If you are setting up SPNEGO with WebSphere Application Server I can definitely recommend the following link: http://www-03.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP101065 and http://planetlotus.org/profiles/dave-hay_137448

The fallback of the Localize-Plugin that is documented here is wrong:

http://www.ibm.com/support/knowledgecenter/SSYJ99_8.5.0/wcm/wcm_dev_storing_translated_text.html

In there it says:

"At render time, if the requested language does not exist in the list of elements, the first element is used as the default language."

Instead the following fallback is valid:

a) locale is determined by the user-profile -> fallback is the portal default language
b) locale is determined by the state -> fallback is the portal default language
c) locale is determined by the browser-language only ->
        c.1) If one of the languages within the list of languages in the browser settings matches with the name of one of the text elements within the TextProvider sitearea/content -> take the first matching language as the fallback
       c.2) If none of the languages within the list of languages in the browser settings matches with the name of one of the text elements within the TextProvider sitearea/content -> fallback is the portal default language

A WCM library exported on a WebSphere Portal v7 system using export-wcm-data might fail during the import with the following error:

--- snip ---

org.xml.sax.SAXParseException: An invalid XML character (Unicode: 0x3) was found in the element content of the document

--- snip ---

The reason for this exception is that low hex characters are in the exported ".node" file(s). These characters are usually shows as ^A, ^B etc.

For this problem there is actually a fix included in WebSphere Portal v7.0.0.2 CF24 (was back-ported from v8) but needs to be activated. Perform the following steps to remove these invalid characters:

1. Stop WebSphere Portal
2. Open icm.properites file in <profile_home>/PortalServer/jcr/lib/com/ibm/icm
3. Add the following property: jcr.string.property.control.character.filtering=true
4. Start WebSphere Portal again
5. Rerung the export of the WCM library
6. Verify the .node file which failed before

As last week I wanted to use the "no-cache" environment variable in the IBM HTTP Server configuration for a more fine grained caching configuration I learned that this one is only available in Apache 2.2.12 and later. As the IBM HTTP Server 8.5 is based on Apache 2.2.8 I failed to configure caching for specific mime types only. Just checked the IBM HTTP Server v9.0 documentation and found that IBM HTTP Server is based on Apache HTTP Server 2.4.12. This should allow us to make use of the "no-cache" environment variable from now on when using IBM HTTP Server.