My name is Scott Vrusho, and I manage Notes/Domino Maintenance. I have been the Development Manager since April 2004, but I assisted for the two years prior to that.
During recent years, we have changed the way we create Maintenance Releases, Fix Packs, and so on. I'm going to take this opportunity to explain the way the process works, but first let me start off by giving a bit of history. In the early 5.x days, we had QMRs, which stood for Quarterly Maintenance Releases. As the name indicates, we shipped these every three months. The three months, though, didn't allot enough time for test, causing an increased regression rate. We dropped the "Q" and switched to Maintenance Releases, shipping every four months and spending the bulk of the extra time in test. That change occurred in 2001.
Shipping maintenance every four months worked well when feature releases were shipping every three years and there was only one current stream to support. With the more rapid onset of feature releases, and lengthier commitments for support of various codestreams, adjustments were made to do more frequent updates for the latest codestream and less frequent updates to the older codestreams. For example, seven months between 5.0.11 and 5.0.12 and another thirteen months before 5.0.13. It was around this time, mid-2002, that a four-, eight-, and twelve-month schedule was put in for the three most recent codestreams. So, 6.5x was every four months, 6.0x every eight months, and 5.x every twelve months.
Critical fixpacks were a delivery vehicle we used for a short period of time in 2003. These were hot-off-the-press fixes for severe issues. We discontinued critical fixpacks in their current incarnation, and we moved to fixpacks in 2005. More on this below...
In late 2004/early 2005, with increased concerns surrounding quality, we took a number of corrective actions. These included bolstering testing in certain areas, increasing interoperability testing with other products, stress testing simulations, and tactical Fix Packs for 6.5.2 and 6.5.3.
In early 2005, we found that the number of hotfixes being built was at an all-time high, however, a small percentage (roughly 5 percent) of the SPRs affected a broader set of customers. This rate was concerning because hotfixes receive limited testing, and re-packaging the same fix over and over is not a good use of our time. Customers were also telling us that they could not deploy maintenance releases as fast as we were putting them out. In fact, we would be lucky to have a server upgrade once a year, let alone a client upgrade every four months. We were shipping maintenance releases faster than customers could deploy them, and instead people wanted hotfixes on the older releases.
This brings us to our current Fix Pack and Maintenance Release strategy. The solution we implemented was to increase the time between Maintenance Releases and thus ship more frequent Server Fix Packs. The goal of the Server Fix Pack is to deliver a smaller number of safe fixes that help address the issues the broader customer base is experiencing or is likely to experience. Instead of getting the 500 or 1000 fixes that an MR delivers, the customer gets 20-50 focused fixes. The fix pack content is focused on bad regressions, crash, hang, security, and data loss issues, but we include them only if the fix is safe, has been deployed in a production environment, doesn't affect translation, and has no new features. Because we deliver a limited number of fixes in a smaller package, customers can deploy with less testing and less risk than they could with a bigger maintenance release. Customers can also spend the time they need to validate a maintenance release before deploying it. Instead of upgrading three times a year, they can now deploy a new MR once a year at most, and they can use Fix Packs to address the bulk of the more pervasive issues. You can read more about these choices in the "Notices" view in the Fix List database on the Web or on the What's New tab. There have also been multiple technotes outlining our strategy and providing more granular detail about the individual fix packs. For example:
- "Information about Lotus Domino 7.0.3 Fix Pack 1"
- "Upgrade Central: Planning your upgrade to Lotus Notes / Domino 8.0.2"
- "Upgrade Central: Planning your upgrade to Lotus Notes / Domino 7.0.3 (including Fix Packs for Domino 7.0.3)"
- "Maintenance Releases, Fix Packs and Additional Fixes for Lotus Domino and Notes" (A comprehensive list of maintenance releases, fix packs, and additional fixes for IBM® Lotus® Domino and Notes sorted by version)
But what about the client? The big elephant in the room is the client. It's an interesting problem. We encourage customers to deploy Server Fix Packs proactively so that they have fixes in place for problems before they run into them. The customer response to date, when asked if they would proactively deploy a Client Fix Pack (that, BTW, would work using smart upgrade), has been "No." Now, if the customer wanted or needed a fix for a specific issue, their answer, of course, is different. Admittedly to date, our focus has been on servers and server fix packs. Client Fix Packs are being introduced for the first time in late 2008 starting with 8.0.2 Fix Pack 1. Client Fix Packs are not planned for earlier releases. For earlier releases, we have another delivery mechanism called a CCH, or Cumulative Client Hotfix. A CCH is a hotfix for the client, but instead of just a single fix, we include a set of fixes that multiple customers are asking for. We do more pointed testing on a CCH than on a typical hotfix to ensure that it is a quality deliverable. We do not perform extensive regression testing on the CCH, but the same fix is merged into the next maintenance release where we do perform that level of testing. The CCH gives us an avenue to provide fixes sooner than a Maintenance Release or a Client Fix Pack. The one drawback is that the CCH must be obtained through support channels as it is not posted publicly. On a positive note, CCHs can be deployed using smart upgrade.
Finally, the schedule. You have heard me ramble on a lot in the preceding paragraphs, and I hope I have done a decent job of outlining the challenges and solutions. A sound byte indicating MRs will be every xx months is no longer possible, and, in fact, has not been possible since 2002. But we do post the delivery schedule of the upcoming MR(s) and Fix Pack(s) in the Fix List database on the Upcoming Releases tab.
I hope this information was valuable, and I welcome any feedback.
Scott Vrusho, a native Bostonian, has been working at Lotus and IBM as an Engineer and Development Manager since 1987. Scott was a Notes technical advocate for IBM at the Nagano Japan and Sydney Australia Olympic games. Since 2002, Scott has been a manager in Notes development, where he has been responsible for Notes/Domino Maintenance, Serviceability and more recently the Security response team.





