Flemming T Christensen on Quality Collaboration
email@example.com 100000NMQP Tags:  smartcloud education lotuslive hybrid notes 3 Comments 958 Visits
News tip: Our education team has just released a free, self-paced course on SmartCloud Notes in a hybrid environment. (The course link was originally posted in the blog Apr 26th 2011. I have updated it on May 21st 2013 after a reader notified me it was broken. The change is a result of the rebranding from LotusLive to SmartCloud).
A hybrid environment allows integration between your on-premises Domino systems and the cloud. Replicating your Domino Directory to the cloud provides for a seamless integration between environments. So rather than replacing existing Domino infrastructure with cloud based offerings, you can leverage the cloud based offerings as an extension of your existing on-premises environment. Your Domino administrators continue to administer on-premises Domino servers and applications, while IBM administers and maintains the SmartCloud Notes mail servers in the cloud.
Hoping we don't tempt fate with our timing, on Friday the 13th of this month, we quietly turned on an automated fault analysis capability for Notes System Diagnostics (NSD) files uploaded to our Technical Support file repository, called ECUREP (for Enhanced CUstomer REPository). In other words, any time an entitled customer uploads an NSD file, our systems will automatically - without delay - perform an analysis to determine what type of incident is reflected (crash, hang, out-of-memory condition, user killed processes, etc), and for a crash whether the crash stack contained in the NSD file matches any known problems in our database. In cases where a customer encounters a problem already seen & solved elsewhere, the system will be able to point to the known defect and the associated technote. In cases where the crash stack contained in the uploaded NSD file does not match any known problems, no result is returned, but per standard Support process, a new defect is opened to track the further analysis. The Support Engineer along with Development may manually apply other internal tools like MemCheck or Laza, to analyze the incident. Fault Analyzer has shipped with the Domino product since version 7.0 to process data captured with the Automated Data Collection (ADC) feature. Local analysis at the customer site can determine general disposition, or incident type, but local analysis won’t match the crash stack against our in-house database of known issues. That database is comprised of all NSD submissions to the ECUREP system, plus similar data captured in IBM’s internal worldwide environment with over 400,000 employees.
The new automated support analysis leverages the same Fault Analyzer tool available in Domino itself, and runs against our latest database of known issues. It can handle compressed archive files in zip, tar, tar.gz, tar.bz2, ar, jar, dump and cpio formats up to 225 MB in size. There is no logical limitation determining the 225 MB cutoff; it's a cautionary, self-imposed limit we have set to avoid slowing down related processes. Once we get a sense of how the analysis system operates, we may alter the limit. The system offers several key advantages. From a customer perspective, a first possible answer is returned much faster in cases, where the crash stack signature is known. From a vendor perspective, it provides our engineers a quick first analysis of the diagnostic data. The system 'stamps' the information into the Problem Management Record (PMR) visible to the entitled customer via the Service Request tool on the Web. This helps keep all information related to the customer's issue in one central thread visible to both the customer and the support representative.
Given that we have just launched this automated use of the Fault Analyzer in our support process, we fully expect that we will have opportunities to tune and improve the process as we learn from initial submissions and analyses. A key design concern has been – and continues to be - minimizing false positives. Returning an incorrect defect match could potentially waste time for both our customer and our support representative, so to start with we have set match criteria that we believe are specific enough to minimize false positives, but we continue to review and tune the algorithms. As we learn from the initial submissions, we will look for ways to refine the match criteria to allow more submissions to find a match, but only in cases where we can assure ourselves the identification can be done with sufficient accuracy. Experience from the first couple of weeks show that less than half the NSD submissions find a match. However, finding matches for 100% of the submissions is not our success criterion for automated fault analysis. New problems, e.g. from interaction with newly released 3rd party components, will obviously have no matches the first time they are submitted by any customer. If we were able to find matches for all submissions, it would mean that all problems were known. And that would mean either that we were terribly lagging in delivering maintenance releases, or that our customers were terribly back level in applying the maintenance. So to improve fault match identification, we're not focused on achieving matches in a specific percentage of cases, but rather on identifying those additional circumstances (crash stack specifics) that allow us to positively match with additional known issues and extend our logic to cover those circumstances as well.
I hope you agree that with the new automated Fault Analysis, we have taken yet one more step to provide more efficient support to our customer base. Crashes, hangs and resource exhaustion should be rare events, but when they do occur, rapid problem identification is key to minimizing business impact.
firstname.lastname@example.org 100000NMQP Tags:  notes testimonial domino sametime portal quickr connections 1,498 Visits
As a quality engineer, it's important to explore select customer success stories, so we can work to replicate the associated success factors across all deployments. In that vein, I'd like to offer a list of IBM Collaboration Solutions customer success stories for you to enjoy. I’m not including any analysis here; just sharing a collection of testimonials.
Colgate-Palmolive, PSC Group, INSUREtrust, Australian Bureau of Statistics (Notes 8.5)Portal:
Sametime (& Connections):
Finnish Defense Forces (pdf)
These testimonials demonstrate part of the value our software brings to our clients across our portfolio. Enjoy every one. This list actually comes from an internal blog post I wrote last year, so most of the linked videos are about a year old, but I like the collection covering most of our key on-premises products. I’ll naturally continue to share additional, more recent success stories along the way, as I already have with for example Signature Mortgage and Colleagues in Care.
Continuing in the vein of prior posts with the ‘better’ tag, I want to describe quality improvements made in recent releases. Notes/Domino 8.5.3 has just been released, and you can read about new features in the announcement. There’s plenty to like. The embedded Symphony version has been updated to 3.0, and the embedded Sametime to 8.5.1, both key advances. There are enhancements to XPages and the Domino Designer, and much more. But quality is not just about new features. It’s about all features working well.
The overall quality objectives for the 8.5.3 maintenance release were to significantly reduce the outstanding defect backlog, to improve integration with companion products like Connections, Sametime and Symphony, and to expand test coverage and test automation. The team has delivered on all of those objectives. All major components (Domino, Notes, Designer, Traveler) reduced their deferred defect backlogs by considerable amounts, some by more than half. The vast majority of those defects had not been reported by customers. They were found in house. Removing them eliminates the risk our customers will run into them. Reducing internal defect backlogs is always an objective for a modification release (a.k.a. maintenance release), but release 8.5.3 has achieved reductions that are greater than is typical for most maintenance releases.
Security is a high priority for any release from IBM. In Notes/Domino 8.5.3 we moved systematically forward with further detailing of our threat model and the adoption of Rational AppScan Enterprise Edition for testing of the full attack surface across the Notes client. Similar efforts were done for Traveler and for iNotes. (Domino did this work previously). All the components had security testing in the past; what’s changing is that we’re adding Rational AppScan testing across all of our portfolio. And of course resolving all security defects before releasing.
The Domino team also focused on memory related improvements in release 8.5.3, delivering new NSD macros, and an administrator capability to track and drop 'bad' IMAP sessions, which can cause server crashes. A very key improvement is a substantial reduction in use of shared 16-bit handles, which will reduce the type of conflicts that can cause potential hangs or crashes. The aggregate result is an even more stable solution. For the Domino Configuration Tuner (DCT), we continue to deliver additional rules to help you ensure your environment is optimized. If you use DCT, be sure to download new rules regularly. We add new rules at least quarterly, and some times monthly. For iNotes, we continue to focus on achieving full parity between the Notes and iNotes client experiences, delivering important improvements to sorting by subject, to auto-processing of calendar entries, and the option to not expand personal groups when sending.
Less visible to our customers is the continued progress on test automation. The more of our standard test scenarios are automated, the more time our engineers can devote to specialized, exploratory testing around new features. Some critical areas have doubled the number of automated tests this year, freeing up engineers to expand coverage, all part of our continuous improvement effort.
Release 8.5.3 is the next global deployment candidate for IBM’s own internal environment of nearly 400,000 users around the globe. Prior to release, the IBM CIO’s Office deployed it to over 4,000 IBM employees, and our Services Division deployed a pre-release build for over 14,000 users. That means over 18,000 people were using it daily before we declared it ready to ship. The CIO servers are primarily AIX and zLinux servers. Although the majority run the client on Windows, there are a few hundred running on Linux and Mac platforms as well.
In summary, there’s a lot to like about Notes/Domino 8.5.3. I’ve described a few highlights of the quality effort here, but of course the proof is in the pudding, or more accurately, in the released software. Enjoy the new release. As always, feedback is welcome.
tags: domino, exchange, migration, notes, outlook
Enjoy Peter Presnell's insight gathered over a number of migration projects. Replacing an advanced messaging, application development, and web server platform like Domino is far from trivial. And the author has seen how it often goes down. If you care about quality of your environment, take note of this experience. Thanks to Peter for sharing.
Anyone considering a migration project would be wise to very carefully estimate both the full scope and budget of the effort, as well as the ability to continue support for existing usage patterns. Map how your users are actually leveraging the platform they have today. Don't assume you know.
Years ago, one of my first projects for IBM was to run beta tests for departmental cutsheet network printers. I'll never forget a particular customer: Great and very cooperative IT staff, but when they pointed me to their 'main' print queue, I found limited traffic. To make a long story short, they didn't realize many employees had switched their print jobs to alternate queues on alternate server clusters. Keeping up with all areas of usage is no small task. Know your users, especially when planning change projects, whether migrations, expansions, or other.
I serendipitously ran into unexpected behavior of my embedded Sametime client on a machine running Notes 8.5.1. I locked out the screen by pressing F5 as I was getting up to go to lunch. After pressing F5, my Notes client will not let me open or send mails, nor will it let me access or write anything in any of the already open Sametime chats. That's expected behavior. But before leaving my desk, I noticed that my manager opened a chat window with a question. Not wanting to let her wait until after lunch, I instinctively typed my answer in the chat window and sent it back successfully. But the client was still 'screen locked', so this was not expected behavior to me.
Now, the Sametime preferences include a section on Auto-Status changes, which can determine what happens in response to changes in your calendar, Notes client, or operating system. I don't have the setting for 'Locking Lotus Notes' selected. Why? Because it forces the status to Do Not Disturb (DND). What I want, when I'm Away from the workstation, is really an Away status, so buddies can leave me messages in new chat windows, which I'll see when I return. With DND, no new chats can be initiated. So I don't check 'Locking Lotus Notes'; Instead, I set my Sametime status to Away and lock my Notes screen with F5. That's just my personal preference. You can see my Auto-Status preference settings below.
The observation above took place yesterday. This morning, I started working from home, then screen locked Notes and put the laptop in standby mode, drove to the office, and woke up the laptop to continue. My Notes client is still screen locked with the empty password dialog box showing. I replaced my notes \\notes\data\workspace recently to eliminate an issue caused by a non-released plug-in I was working with. This replacement naturally causes me to loose Eclipse based settings I had saved, like the geographic locations Sametime associates with each wireless access point I use. So when my laptop automatically attaches to the wireless network in the office, Sametime pops up a dialog for me to enter my 'new' geographic location. I filled it out and applied it successfully. All while the Notes client was still screen locked. Again, that doesn't strike me as 'expected' behavior, but our Sametime security architect confirmed for me that this is 'Working as Designed'. Sametime itself, meaning the standalone Connect client, doesn't have the lock-out concept. You're either contactable (logged on as Available or Away) or not contactable (DND or not online). The intersection with the Notes client is such that the above behavior is what results.
My observations lead me to two suggestions I'd like to hear your take on:1. Should there be a choice between Away and DND, when you select the 'Locking Lotus Notes' option?
2. In the absence of the Away option, should the behavior when locked out, and 'Locking Lotus Notes' is not checked, prevent responses in not just existing chats, but new chats as well, and in dialogs like the one for geographic location, until the password is re-entered?
An important aspect of guiding quality improvement is to ensure we focus on the things that matter to our clients. This topic arose out of my own observations, and out of my own subjective expectations for how I would like to see the product work. So I need your help to find out what our clients think. Is this just a pet peeve of mine, or is this a change we should pursue? How important is this issue to you? Please share your thoughts in comments.