Bring information and collaboration into the context of your daily business processes using composite applications
An article I wrote for Expeditor 6.1 just posted to Developer Works.
Get an introduction to the IBM Lotus Expeditor Property Broker and learn how you can create a component that contributes to the declarative communication offered by the broker. We show you how to wire components declaratively using extension points, the IBM WebSphere Portal Wiring Tool, and the Property Broker APIs.
As we start talking about the possibilities for composition of "reusable" services (even UI ones), it's important to take a look around the industry and understand what some barriers to this could be.
I just happened to come across this article by David Chappell who was pessimistic about the reality of reuse with SOA. This article talks about the problem being one of larger context demands for business services that are not present for the very successful object level reuse of rich frameworks like Java and .Net.
In the chain of other responses that I ended up following, was this excellent IEEE article talking about The Social Side of Services that reflects the challenges that any reuse effort faces due to non-technical social reasons and approaches you can take from any level of the organization.
Points to ponder,
Here are a couple of Lotusphere sessions related to Composite Applications:
One of the main benefits of composite applications is that they allow defining new situational applications easily by re-using existing components in a new context. This support of business flexibility and agility is achieved through loosely coupled components. Components can be wired together in composite applications making this model the front-end of Service Oriented Architecture (SOA).
This model will become really powerful once there will be a set of components available that can be used for future applications. Everyone can obviously implement his/her own re-usable components. Additionally I would like to provide so called out of the box components in Lotus Notes. These components could then be reused without any code changes in custom composite applications. This would essentially be a new type of API (an UI API or property broker API) for some of the Lotus applications/components.
Here are some ideas. None of this is final yet, but I'd like to get some ideas out there to get feedback in this BLOG. I'd like to provide different components that publish properties which custom components can consume and I'd like to provide actions (UI services) that other components can invoke.
A sample application that was demonstrated at the DNUG 2007 in Germany recently shows these new capabilities.
In this customer interests application you can select another person in the contacts component in the upper left corner which causes the Notes database component 'customer forum' and the Eclipse tag cloud component to be updated. See here for more and bigger screenshots: 01, 02, 03, 04.
To implement this sample application the Notes Contacts View component could be reused from the out of the box Notes contacts application. This component publishes a property 'PersonCommonName' whenever the selection in the view changes. In order to implement this application two further components were implemented that provide actions that consume this property to update the information they are displaying. In order to implement this sample no template change in the names.nsf store of the contacts application was required. The wiring was done via the Composite Application Editor and the wires are stored in the composite application rather than in any component.
Before I describe other ideas let me define some key terms you need to understand. A component in the Notes terminology is a NSF. I call the rectangles in the UI on the rich client 'panes'. The panes are interaction services or presentation services that provide views into components. Technically a pane is on the rich client an implementation of an Eclipse ViewPart which optionally has an interface for inter component/pane communication via property broker.
Additional out of the box components candidates
Another reusable component could be the mail/calendar component which has different panes like a Mail View, Calendar Navigator and Calendar View. The Mail View pane could publish the properties 'PersonCommonName', 'PersonDN', 'EMailAddress' etc. when selections in the view change. The Calendar View pane could publish 'DisplayedDate' and 'DisplayedDateRange'. For more advanced scenarios both Mail View and Calendar View could also publish some key of the currently selected entry. For example the NotesURL pointing to the currently selected Notes document could be published so that other components can consume it and use it as callback to access other fields in this currently selected document.
Vice versa the Mail View pane could provide some 'new mail memo' actions that can be invoked with different input properties. In the simplest case an 'EMailAddress' property could be passed in. In order to also pre-populate some other fields like the 'cc' and 'subject' in the mail the standard RFC2368 'mailto' property could be passed in. For the calendar pane actions could be provided to change the currently displayed entries when other components on the page change their currently displayed dates (e.g. if you have your own group calendar component on the same page as the standard Notes calendar component).
Another example of a re-usable component has already been published in Lotus Expeditor 6.1, an embedded internet browser. It provides an action that displays a new URL that another component can pass in.
There could also be a component for the word processing, spreadsheet and presentation editors. An action 'OpenDocument' could accept an URL pointing to document as input that is supposed to be displayed.
So there are lots of exciting possibilities here. Please let us know what types of components/panes and properties/actions you would like to see us expose so you can include Lotus components in your own composition.[Read More]
Today I want to talk about the different types of composite application frameworks that are available with WebSphere Portal 6. For some background: here is a "What's New in WebSphere Portal 6" developerWorks article and here is the Portal 6.0 InfoCenter main topic for Composite Applications (which is really talking about Instance Based Composite Apps, but I'm getting ahead of myself...)
Disclaimer: These are NOT official names, and I'm just experimenting on you brave readers to see how they resonate. Please provide us feedback on whether you love them or hate them.
Page Based Composite Applications
The traditional way that WebSphere Portal applications are created is by assembling portlets on a collection of related pages that are integrated into the primary MyPortal navigation hierarchy. These applications are either created by using the Portal based web composition tools or manually created using XML Access. Once they are integrated into the topology there is no real way to tell them apart from any of the other other pages, although there is usually a clear distinction at development and UX design time. Of course this style of creating pages is still fully supported in Portal 6. It's very important to note that there is no reason to stop developing these kinds of applications if your needs are being met (i.e. the new capabilities are not a replacement for what you are doing now, but additional tools to solve additional problems). Access control, page level properties and portlet entity level properties can be used to provide a little bit of local context, but all these applications are essentially running in the global Portal context and leveraging global user roles (e.g. user profile attributes for someone being a manager and/or global group membership). I'd like to call this type of application a "Page Oriented Composite Application" to reflect their focus on portal pages as a fundamental unit of aggregation. It doesn't mean that there can't be multi-pages though. I assume that readers are familiar with these from earlier releases of WebSphere Portal and will not go into more detail here. (For those who what a high level description, have a look at these articles written about Portal 5.1 by the Lotus software CTO Doug Wilson Part 1: Overview, Part 2: Cooperative Portlets, Part 3: Advanced Patterns)
Instance Based Composite Applications
In WebSphere Portal 6 we have added a new context level, an application instance. It augments and extends the capabilities of Page Based Composite Applications, and still supports everything you could do before. More precisely, I like to say that an "Instance Based Composite Application" ISA "Page Based Composite Application" in the conceptual OO inheritance sense. Instance based applications can do everything that page based ones can do and more. The difference is more significant than just tracking a new set of preference data though. An application instance is managed as a single unified entity (e.g. created, deleted, backed up, restored, access controlled, customized). Instance lives within the application instance catalog and are grouped by category. As part of implementing the ISA relationship, each application instance also manages it's own unique branch of the portal topology, with portlet entity and page level properties settable on a per instance basis. Due to configuration split (a major new feature of Portal 6), the instance specific branches are not in the MyPortal topology branch, but can be referenced from there via page links. There are also programming model capabilities that are associated only with the instance based model . These are application level roles with component role mapping, creation of new instances from templates (including component instance lifecycle), creating templates from existing instances, template parameters and policy driven administration.
Application level Roles
Application level roles can be defined using the application domain language (e.g. for a club: President, Treasurer, Member, etc). Application instance access control is done using these application level role names, and membership is instance specific. Thus, I can be the President of one club, and just a Member of three others. My user experience and allowed capabilities depends on my role in the specific instance (even if they were all created from the exact same template). Transitivity of access control can be accomplished by having these application level roles be mapped to roles defined by the component, using the component's domain language. This allows the template assembler to hide the details of the application domain to component domain mapping from the application instance owner and allows component's to be reused in many contexts.
Instances from Templates
Application instances are created from templates. A Template is a single XML file serialization of the important aspects of an application instance (and transitively all the contained component instances). Each component gets to decide what "important" means to it. The Portal component automatically serializes the entire portal page topology of the application instance along with all portlet entity properties. Thus, most standard portlets that use portlet entity properties to store customization state are already template enabled "out of the box". That means that you don't need to rewrite your existing portlets to leverage them in instance based composite apps. I'll go into the patterns for component creation ("By Reference" and "By Value") in a different blog entry, but, before then, here is a developerWorks article by Thomas Stober, Peter Fischer and Hendrik Haddorp. It walks through an example of creating a "By Value" component.
Templates are important to enable "self-service IT". They allow the ability to capture a common pattern of component composition and then allow the creation and management of new instances to be delegated to end users. The creator becomes the owner/administrator of these new application instances. The owner is in control of the membership of the application and can optionally be given rights to modify the instances that they own. (adding co-owners is also permitted)
Templates from Instances
Templates can also be used to capture subsequent organizational learning. As an owner customizes their instance to more closely match their particular business problem, it may evolve a long way from the original template. This new instance can then be "Saved as a Template" to capture that learning for reuse in other contexts. Templates can even be used as a unit of deployment by importing and exporting them from the template catalog.
Template parameters are a useful tool to allow for instantiation time customization of a new application instance by the eventual owner. When a new instance is created, a dialog is presented that asks the creator to specify one or more values (maybe a customer id, project id or other information applicable to the new context being created). Components use these instantiation time parameters to affect how they create their own instances and can automatically adapt their configuration so they are up and running right away (e.g. use the customer id to automatically filter a dashboard portlet view for just that customer). Expanding the richness of capabilities in this area is one of the things we are looking to do in future Portal releases.
Policy Driven Management
Policy driven application instance management is being added in Portal 6.0.1 to help to enforce instance based quota's and relevance (e.g. by archiving instances that have not been used in a long time), without manual IT intervention. This is done by sending email to owners and automatically locking the application instance if they go over quota and automatically archiving instances based on last used time. This helps an IT organization manage the infrastructure impact of proliferation of instances. It was developed based on administration experiences with the viral adoption of early versions of Lotus QuickPlace in big customers. (Note: Similar solutions have also been developed for more recent QuickPlace releases). More details of this capability will be available when 6.0.1 is released next year.
Conceptual Model for Instance Based Composite Applications:
Note: that the "Portal Topology Branch" ISA "Component Instance". That is because we have implemented a Business Component Adapter for the Portal itself and drive the lifecycle and templating of the portal topology in the exact same way as any component BC adapter that you could write. If you look in the resulting template.xml you will see that the portal topology XML is just component template instance data like any other.
Now that I've covered the high level conceptual architecture, I'd like to describe a taxonomy for use cases for this new instance based model.
Use Case Categories for "Instance from Templates"
The primary use for instance based applications is where there are going to be multiple instances of some entity, concept or process that share some common structure. (singleton instances are a special case of this) Furthermore, an instance can be further customized to better fit it's exact context (which over time can lead to singleton instances...). In truth, these three primary classifications categories blend together at the edges, but they have some differences that it makes sense to talk about them separately. Let's look at each in turn.
1) Multiple Concrete Things (need a flashy name here, any suggestions? :-)
Examples in this category are stores, departments, business partners, customers, clubs, promotional offerings. What they have in common is an association with some concrete "thing" that provides a external reason for the creation of a context. These "thing" domains also generally provides a natural terminology for the application level roles. (e.g. the roles in a club, the roles when interacting with a store, business partner, etc). Each kind of domain would likely be represented by a separate application instance catalog category and define one or more templates (e.g. there may be different levels of relationships with business partners that determines what the default set of components and capabilities would be for a new instance). The instances usually are usually oriented to interact with multiple existing systems of record (e.g. eCommerce, CRM) and the portlets are filtered for the specific context (e.g. showing just the orders of the customer associated with the instance). They could also mix in some of the team collaboration components like Wiki's, Discussions, Document Libraries (see below), but these are usually not the primary purpose of creating the instance. In a future post I'll explore the "offering" scenario in more detail. The primary components are of a "By Reference" type.
2) Team Collaboration
Collaboration in teams is similar to the above, but a team or project is a somewhat less concrete thing. I use this category to reflect uses where the primary purpose of the instance is to facilitate the collaboration of the team members towards a shared purpose (a project or other defining feature that creates a team). Another term for this could be community based collaboration. The instances are usually oriented around collaboration components (e.g. document library, discussion, Wiki, team calendar, etc) that have their data scoped to the instance. They could also bring in contextually relevant data from external systems of record (e.g. a defect dashboard summary from the bug tracking system). Thus the majority of components are the "By Value" types. In order to facilitate optimal collaboration in the team, individual instances may be customized by adding instance specific components (e.g. tying into to a team specific build system) or customizing specific components (e.g. specifying instance specific categories for a document library). After the holiday's I'll blog about a concrete example in this category.
I also split this category out because Lotus' collaboration heritage is in facilitating team collaboration using Lotus Notes and Lotus QuickPlace. The patterns we have learned in creating those products have been applied to the instance based infrastructure we have created for WebSphere Portal 6.
3) Business Process Centric
In this case, an instance is created to explicitly help in the execution of a complex business process. "Request For Proposal", "Employee Onboarding" "Statistical Instrument" are good examples of this. What makes this category different from the Team Collaboration category is the use of instance specific state (or state maintained in an associated Process Server workflow instance) to affect what is seen within the composition. This is based on the current process state and also allows the components to adapt themselves to the state via a "workflow state" component and cooperative portlets. You could even imagine peoples roles automatically changing as the process progresses. Many people use Lotus Notes to manually create these kinds of applications today and our vision is to allow business users to safely customize the processes without needing to be a developer. Preliminary steps towards this goal has been made generally available in Portal 126.96.36.199 this past Monday. Here is the online documentation for this feature. We expect to greatly enhance the capability in this space in future releases. I have a great scenario that covers this case that the Australian Bureau of Statistics has allowed me to share with you. I'll talk about that in another post (probably in the new year, given the backlog I've been accumulating as I write each post :-).
In summary, we've talked about the two kinds of user facing composite applications supported in Portal 6. They are 1) Page Based Composite Applications and 2) Instance Based Composite Applications. I talked about how Instance Based adds another level of context around an instance and a unified lifecycle and management story. I also talked about how instance based is a specialization of a page based composite application.
I finished up with a rough categorization of use cases for instance based composite apps. They are 1) Multiple Concrete Things, 2) Team Collaboration and 3) Business Process Centric.
In future posts I'll talk about patterns for component creation (e.g. the "By Reference" and "By Value" terms mentioned above) as well as giving some concrete examples for each of the three instance based use case categories.
Questions to the readers:
We had a wide ranging internal debate on the names for the two types of composite applications supported in Portal 6.0. In the end I picked the names that had pretty good internal consensus and resonated with me (since I was writing this blog entry :-).
What is your opinion on these names? Do they help clear up why there are two models and what some of the use cases are for the instance based one? Do they just confuse you more? Is there too fine a distinction being made? (and thus only of interest to us philosophically oriented architects)?
Do you like the use of the UML conceptual model? I really like the UML for it's information density and precision.
What about the categorization of the use cases? Are the names meaningful? Helpful? Anyone have a better name for the "Multiple Concrete Things" category?
Thanks for reading,
The IBM Education Assistant site now has the Lotus Expeditor 6.1 on-line tutorials posted. The Composite Application tutorial can be viewed here, however there are tons of freely available online presentations and demos for all areas of the Lotus Expeditor. This medium is a great way to get educated quickly on the features available in Expeditor.
Some other presentations relative to Composite Applications:
I am often asked what it means to be a component in a composite application in the Lotus Expeditor and Hannover. In general, because components are wired at the UI level, the component in Eclipse terms is called a ViewPart. The ViewPart is what makes up the different boxes in an Eclipse application. If you want to start creating components in a manner that is compatible with RCP applications and Composite Applications, then start making your views independent from other code. For instance, your view should not require other plug-ins or views unless it is necessary. Think of your view as a first class ActiveX control and you can not have any dependencies on other ActiveX controls. Write your view with the mindset that others may use this piece of real-estate in another context. Declare your interfaces to the component through the Property Broker and WSDL files. This means if any other component wants to interact with or receive information from your ViewPart you do it over the Property Broker.
Why you ask?
You can easily argue that the Eclipse selection model listener is already in place for this purpose, and yes it is. The major difference is a developer has to take those components, place them on a perspective and code the two together. In the Property Broker model, a novice user or administrator (not a developer) can bring these components together using Portal Admin or one of the other tools being released by IBM - Component Designer, Composite Application Editor, etc.
Tying in with Portal
In Portal, the parts are called portlets and are considered the same level of componentry as a ViewPart. You use the portal wiring tool to wire the components together on a portal page. This exact same model is used for Composite Applications coming from Portal in the Lotus Expeditor. You can have your portlets map to actual portlets (that run in the local container) or they can map to an Eclipse based SWT view. There are a series of new portlets that will show in portal when the Expeditor's Network Client Installer (NCI) is installed on top of the portal. You will get a new Rich Client tab when modifying pages and portlets. These portlets assist in describing how your portlets should render on the client. When the Expeditor client requests an application from portal the portal responds with Composite Application XML - or CA XML. This CA XML describes the entire application to the client; it describes the navigation, the layouts for the pages, and any preferences set on the pages and portlets by the administrator. The Rich Client tab in the portal administrator gives a user interface that makes it easier to set these preferences which tell the Expeditor how to project the application.
Running in Expeditor
Since Expeditor is Eclipsed based, any application running in it must be described as Eclipse Features and Plug-ins. The portlet preferences is where these features are listed for the Expeditor to resolve. When the Expeditor sees a feature requirement on a particular portlet it tells the Expeditor that it must attempt to resolve that feature prior to launching the application - meaning, see if the feature is currently installed and if not look to the remote site, download, and install the feature. If the feature resolves then the application UI is dynamically created by the Composite Application Infrastructure and displayed.
Hopefully this answers some questions and sparks some good discussion and questions.