Several (read too many) years ago I made a decision about my career. I wanted to build world class web sites! This was in the mid to late 90s when the dot.com boom was still booming, and building the next ebay or toys.com was the thing to do. Razorfish (c) was the company to be, and every IT company from coast to coast was building up its consultancy. Anyway, I left my cushy job as director of IT at a small company and became a consultant. I suppose many years later I am pretty much doing what I set out to do, and for the most part am pretty happy about it. The travel can be a bit wearing from time to time, but my manager and I work hard to maintain a decent work/life balance and still help our customers to be successful.
OK, this is not my usual blog fodder, but I thought I would mention that the group I work for, IBM Software Services for Lotus is looking for a few good men and women to join our elite force. The portal space is exploding and as fellow blogger Bobby Wolf pointed out awhile back WebSphere Portal is the number one product in this space.
If you have some experience in Portals, WebSphere Portal, or Workplace, maybe some J2EE or WebSphere administration experience, you may want to send me a note (or a resume) at firstname.lastname@example.org and I will forward it to the right folks. Software Services for Lotus has THE best WebSphere Portal practitioners in the business with a focus on deep product expertise, so if you are interested in being one of the best? You can also check out openings at IBM Jobs.
Who knows, this might be a way for me to tell if anyone is actually reading my blog. : )
Portal in Action
When is your portal performance, good enough? What I mean by this is
1. You have squeezed 80% of the performance out of the machine, you think that might be enough and getting the additional 20% might take weeks to tweak and fine tuning code and options.
2. Your system is crashing or bogged down with only a small load, and you are not sure what steps to take next?
So before we get too deep, I do not consider myself a great performance expert, IBM has some great performance folks and I find myself going to them for help with many questions. Not surprisingly however I do spend a lot of time helping customers with performance tuning and problems. One of the most common questions asked of performance engineers is, "What kind of performance should I be getting?" Unfortunately the all too common answer ends up being, "it depends".
Yes, I know this sounds vague, but it is not a trick or an attempt to be evasive. One of the main reasons for this misunderstanding is that performance is very much tied into the application, and since every (yes every) application is different; performance will vary. I have had customers mention that IBM should provide a base application to perform portal testing, but I tend to disagree. The reason being that even in what you might call a base system or an "out of the box" portal, there can be variations. Different databases and customer repositories, hardware variations, network latency, lots of different things can contribute to performance issues. So often the numbers can become a bit fuzzy within each organization. In a previous blog I talked a little about this subject,
What about testing the infrastructure, and how you might get started with the testing process. Before we get too far into what steps you might take, I will try and dispel a few misunderstandings around performance tuning that often confuse new portal teams.
Usually targeting performance numbers begins at the sizing stage. IBM has a group called TechLine that offers assistance in sizing your system based on your expected user load and your application parameters. The drawback here is that you need to know in advance, the number of users and the rate at which they might access the system, and information about the design of your portlets that will be deployed to the system. Usually this is measured in terms of Low, Medium, and High complexity portlets. I've talked with TechLine folks about this and some of this is subjective. There are some guidelines that I will provide in a later entry. Your best bet to take advantage of TechLine is to work with your sales support team to get the questionnaire filled out and submitted to the TechLine team correctly.
I talked with a TechLine rep earlier this year to see if there was any feedback from portal projects where they had provided estimates. Unfortunately that data is not regularly collected but it would be interesting data. I would encourage anyone out there who has results to begin this feedback cycle with their sales reps, or even send comments directly to me. I will gladly feed this data back to technline to see if it can be used to help improve future estimates. Of particular interest is how your original sizing estimates were in comparison to what the application actually looks like live and how that might have affected your estimates.
OK, back on point, usually there becomes a point when one of several things happen. I'll try and look at each of these situations in turn and see if in my humble experience I can offer some thoughts on how you might approach each issue.
1. Performance is poor and you need to dig in and see where the bottleneck is occurring
2. Performance appears to be "good enough" and continued testing just confirms your results. Continued tuning can be done, but given time and cost constraints there might not be enough runway in your project plan to fine tune the system to the nth degree.
3. Performance is good, but you need it to be better before you turn the system loose on your real users. How much better often varies, but most of the time you are pretty close, but not what you were expecting.
With the first situation, where performance is just plain bad and you are not even close to the numbers you need to reach, there are several things you might try. First read my previous blog (mentioned above) for ideas on how to narrow down the problem areas, but sometimes you just have to dig in and look for the bottle neck. Thread dumps are really my favorite tool for seeing where activity is hanging up in the JVM. Thread Analyzer can be downloaded from the IBM site to connect with your portal and initiate a dump. There are otherwise to initiate a dump that you can learn by checking out the WAS InfoCenter.
Some people are afraid of thread dumps, and I agree they can be hard to read, but many times the problem can just jump out at you. If you see every thread seems to be waiting on the LDAP or a database connection for example, then it is a pretty good bet that you have an expensive query, or maybe some indexing could be done. Likewise, if the threads are waiting on a single object or service, then you might have a caching or syncronization problem with that service. Some issues are harder to read, maybe they are WAS related, and may require assistance from IBM support. Additionally, thread dumps are snapshots in time, so several such snapshots may need to be taken to see how activity is moving around in the system as the load progresses on your system.
OK, I'm going to stop here and continue this conversation in my next blog. Sometimes I get going and forget where to stop. : ) 'till next time.[Read More]
It is interesting how some of the discussions or questions that I get involved with, lead me to writing about some topic. Several recent discussions have got me to thinking about some of the issues around portal and portlet design. A colleague recently asked me if there was a list of questions one could ask to help determine how an application might be broken down into a single or many portlets. To the best of my knowledge there is no published set of questions or best practices around these types of design practices. I thought I would explore the issue a little and see if it helps move us forward on this topic.
Here are some of the questions one might explore as the flush out the design of their portal application.
What does the user expect to see on the screen? This question could really be turned around to ask, "what SHOULD the user see on the screen"? Information Architects (IAs) and graphic designers are well versed in taking content and applications and putting them in a format that makes the most sense to the user. This means building a navigation categories and a taxonomy that focuses on usability as well as functionality. IAs need to be well versed in portal technology however to ensure that their design does not put undue constraint on the development team. On the other hand it may be tempting for the development team to complain that a design approach is too hard and can't be implemented in time. My suggestion is for some compromise, letting the architect or lead developer work together with the IA and graphic designer to come up with the right solution for your project.
What parts of the application can be reused? If there is a high amount of potential reuse then it may make sense to break your application down into several portlets and think about using portlet messaging or some other way to achieve the application flow. Portlet messaging can be a very useful approach. There is a growing trend toward breaking portlets down into very fine-grained components which then all communicate with each other via messaging. This is consistent with the SOA and Web Services trend that is currently sweeping through IT organizations. The idea being that each portlet is a reusable component and can be combined with other portlets to create new functionality. While I applaud the vision, one needs to look closely at how often a component will really be reused. Spending extra time and effort to create components that will only be used once might be considered a waste of resources. If you do decide to look more into this approach, spend a lot of up-front time developing the naming conventions and communication standards for your environment. Knowing the rules up-front for portlets to be designed and communicate so they can be combined within your environment, will save a lot of possible rework over time for your team.
Are there parts of the application that need to be shared across portlets? This question came up recently in a conversation. What if I have some properties or code that needs to be shared between different portlets? One approach is to put all your portlets in a single war file to share the information. Instance variable are generally a bad thing in portlets, but for general properties it can be useful to load them on init() and maintain them for the life of the portlet. As long as a property is specific to the portlet and not to someone consuming the portlet it should not cause any problems. Shared libraries are a more common approach for this issue, Singletons, Portal Services, even EJB's can be used to feed a group of portlets or to cache commonly needed or expensive data. One aspect of using shared libraries that I try to insist upon, is consistency with whichever approach you choose. If you are building portlet services then stick with that approach unless some exception case is made. When each developer uses a different approach for a service (portlet service, singleton, EJB, Web Service), then it can lead to confusion when other developers try and use the services, and make deployment of all the different services more complex.
Are there portions of the application that have specific security requirements? This is an interesting approach to mapping security or access control across several portlets. You may be able to simplify the development effort if ACLs can be mapped to different portlets rather then being built into the portlets themselves. If J2EE security is needed or some custom or external security authorization integration then additional design and development work needs to be done to integrate a different approach into your environment. If you can break your portlet into several portlets to take advantage of available portlet security functionality, it might be the right approach.
How many different views or pages does my application need? This question poses some interesting possibilities. At first glance if a portlet has a complex set of pages or views then I would lean toward using an additional framework such as the Struts-Portal Framework or JavaServer Faces in my portlet. As most of you know I suggest caution when embracing a add on framework, but in some cases the choice is well warranted. Even if your application does not have complex view requirements there may be reasons to take this route. If you have decided upon using JSF as a standard or there is some other compelling functionality that your application requires.
Finally, are there any obvious patterns that can be applied to the application? Generally patterns emerge as the design and development continues, as you get more experienced and learn more about patterns you will start to see the value of including them into your design. There may be some obvious ones such as Facade or Factory, or some less then obvious ones that could be used in the design. Some day I'll spend a few blog entries and talk about how patterns might be incorporated into your portlet design.
OK, so admittedly this is a pretty high level discussion, but one that warrants some thought before your next portal project. One of my overall goals with this blog is to get people to think about how their vision really affects the reality of their project. The effort and the dollar value result of that effort, as well as hopefully giving you some tips and tricks along the way.
As a final comment, remember that a portal effort is not really a single project. It is usually more like an ongoing (living) entity with your company? This implies that it really makes sense to start small and learn as your portal grows. This helps keep you from making bad design decisions at the start that may haunt you for years to come.[Read More]
Just a quick update from my last entry. I came across these tidbits today from my (what seems like hourly) baseline briefing.
Here is a nice set of blog entries provided by someone who lived through the experience of manning their data center during a crisis.
For some info on the cost effectiveness of disaster planning check out another article from baseline.
As a final bonus here are some additional articles on disaster planning from CIO Insight.