The From the Field column is brought to you by Lotus Support, and features technical articles based on our experiences with helping customers to find solutions in the field. This article describes a performance problem that was traced back to the design of the application, specifically, the use of Reader Names fields in combination with an All documents view with thousands of documents.
Planning an application's security is a critical step in designing any application. With Notes and Domino, you can choose from security features ranging from server security to database security and on down to secure views, documents, sections, paragraphs, and fields. Each of these features has a specific purpose, and each ensures that a different level of your application is safe from unauthorized access. However, when implementing appropriate security for an application, you should not have to make trade-offs in server performance.
This article examines a real-world Domino 4.5.7 application, designed to be secure at the document level, that nearly ground to a halt because of the database design.
In this instance, the application was designed in-house for a newspaper company publishing four separate papers. The same application design was used across all the newspapers, but each functioned essentially as a separate company, meaning that each had its own hardware, software, databases, and wire feeds. However, because each newspaper used the same database design, each ultimately met with the same server performance problems.
The newspaper company used Release 4 and so solved its problem with a Release 4 fix. However, new features in Release 5 could have been used to enhance server performance for the application. The last section of this article discusses these new features and their implementation.
The application consisted of four different databases, which could be accessed by Windows 95, Macintosh, and Unix clients. The first three databases were stored on the same server, a Quad CPU Sun Enterprise 4000 Server, with 2 GB RAM, and a RAID 1 disk configuration. The server was Running Solaris 2.5.1. The databases on this server were:
- A wire database to store news feeds. The wire database was continuously populated with news items via a LotusScript agent. The news feeds added 10,000 to 15,000 new documents a day to the wire database. The database also sent e-mail and pager alerts to reporters and editors when a news story arrived that met certain criteria.
- An editorial database used by reporters to create articles for the paper. The editorial database included a field called ReaderNames, based on the Reader Names field type. This field type ensures that only authorized users can read and edit a given document. The database was populated with documents in two ways: by journalists, who searched for relevant news items in the wire database and moved them to the editorial database, and by a LotusScript agent that searched each news wire for specific keywords and then sent the item to the relevant "basket" in the editorial database. Baskets included standard newspaper section names, such as Sports, City, and Entertainment.
- A history database. This database was an intermediary database used to store older versions of the same story as it was rewritten and edited. Although drafts of a story could have been stored in the editorial database, the history database was designed to limit the overall number of documents in the editorial database and thus enhance-or at least not impede-server performance.
A separate server, a Quad CPU Sun Enterprise 4000 Server, with 2 GB RAM, and a RAID 1 disk configuration, contained the last database:
- An archive database. This database had the same design as the editorial database, except that the documents it contained were read-only. Each night, a LotusScript agent archived published or inactive stories by moving them out of the editorial and history databases and into the archive database. How a story was archived was determined by the basket it was in and which options the reporter chose when saving the story. Options included "Don't archive," "Archive x number of revisions," and "Archive current only." The archive database was also organized by basket, so that anyone wishing to read an archived story could limit his or her search to a particular basket.
Because it was the editorial database that caused the bulk of server performance problems, this article focuses on the design of that database and explains how the performance problems were ultimately overcome.
The editorial database: What went wrong
Because of the document locking feature used in the application (which was designed to keep two copies of a document in the editorial database from being worked on at the same time), a single server handled the load, with a clustered server for failover. Depending on the time of day, up to 200 people might access the editorial database-and therefore the single server-simultaneously. However, even when as few as 80 people accessed the database, server performance was an issue.
The editorial database consisted of a single, non-categorized view, with a Basket column for sorting articles by newspaper section (for example, Sports, Entertainment, Top Stories). The Basket column was sorted alphabetically:
Figure 1. The single, non-categorized view
This database could include up to 20,000 documents at a time. The design goal was to allow up to 250 people to access the database at one time without affecting server performance.
However, the design of the editorial database didn't work well primarily because of its size. Here's why: When Notes/Domino displays a view, it attempts to retrieve a full page (64K) of data. When the database uses Reader Names fields, the server must check the Reader Names field in each document until it finds enough documents to display a full page of data. As a result, if a user with access to a limited number of documents in the view tried to navigate in that view, the server would search hundreds or even thousands of documents before returning a single page full of documents. During this process, if other users attempted to navigate the view, they were required to wait until the search process for the first user was complete. In fact, the more concurrent users waiting for the Reader Names fields to be scanned, the slower the server performance.
Using Release 4, the company had a number of options for fixing the application, as explained in the following sections.
Remove the Reader Names fields
This option would certainly improve server performance, but it would also result in no document security.
Split the database load across multiple servers
This option would have caused more problems than it solved. Because the application included document locking, it would have been extremely difficult to ensure that the same document was not locked on separate servers at the same time.
Use private views that contain only the documents a user has access to
This option would provide fast access and easy navigation, but it would also present tremendous maintenance problems, as it would require hard coding each replication formula for each user. The size of the database would also increase considerably, since there would be a great deal of duplication across all views. In other words, each document would be indexed in a separate view for each user who had access to it. As a result, rather than having the document indexed once in the database, it would be indexed for as many times as there were authorized readers, due to the one private view for each user.
Categorize and collapse the view
This option would be extremely fast at the category level, since Reader Names aren't checked at that level. However, if a user who didn't have access to a majority of the documents in the view tried to expand a category, that user would cause the same problem with server performance for everyone.
Split up the single, large view into smaller views by basket
This option would certainly increase the speed at which documents were displayed. However, the documents would no longer be in a centralized location. Users would have to change views to access articles in different baskets. For example, an editor working with the Metro Sports reporter on one article and the Northwest Sports reporter on another would have to change views to go from document to document.
Categorize the view by the ReaderNames field.
This option would result in fast performance, provided the user stuck to his or her own Reader Names category. However, the user would have to navigate to his or her name and expand that category before working with any of the documents. This fix provides a rather cumbersome way to work. In addition, because the quick search feature only works for the first sorted column, users would lose that functionality.
The server performance problems were ultimately overcome by redesigning the database to combine two of the options listed above:
- Categorize and collapse the view
- Split up the single, large view into smaller views by basket
In the original database, the first column of the view was the Basket column. This column was sorted so that related articles were grouped together, but not categorized. The redesign involved first categorizing the Basket column so that it became a collapsible Basket category. This change gave users an immediate performance benefit when they accessed the view displaying collapsed categories.
Figure 2. Collapsed Basket categories
However, when a user with no access to any of the documents in a particular basket tried to expand that category, he or she would find that the category was not expandable. In other words, a user might see 50 categories, but only be able to expand a handful. Also, if a user tried to issue the Expand All command, performance problems would result as Notes/Domino checked the ReaderNames field in thousands of documents.
With these problems in mind, the database designer looked for an additional fix.
The second redesign step was to split up the view-which, as you recall, contained all of the baskets and all of the documents. Again, the database designer was faced with several issues: If the original view was split up to include only one basket per view, the result would be over 100 views. To reduce the number of views to a more manageable size, the designer chose to organize the baskets into six broad topical areas with a view for each. For example, baskets containing articles about sports would be grouped together in the Sports view. In this way, most users could work within a single view containing a limited number of documents. In addition, a user would have access to the majority of documents in his or her working view.
Remember that one of the main causes of the server slowdown was that it had to scan potentially thousands of documents before returning a full page of documents. By grouping the documents in this way, the server could find 64K worth of documents very quickly. As a result, the server load decreased, and server performance improved.
Figure 3. Grouping documents in views
Finally, the designer left intact the original view, which contained all of the documents. However, the designer restricted access to this view to administrators and editors who had permission to work with the majority of documents in the database.
Release 5 features offer an even more efficient solution
The Release 4 fix implemented by the newspaper company served them well. However, if they had been using Release 5, its new features would have given the database designer the opportunity to improve the design of the editorial database even more and to fine-tune it for even better performance and usability.
The Release 5 solution to this problem starts with a different choice in the list of options mentioned above: Categorize the view by the ReaderNames field. However, if this were the only fix, imagine the steps that you, as the user, would have to go through to access a document with your name in its ReaderNames field:
- Find your name (as a category) and navigate to that section of the view.
- Expand the category.
- Navigate to the desired basket.
- Find the document.
Although this design would avoid major server performance problems, it does not provide an ideal way to work. There are too many steps between opening the database and finding the document you want to work with.
The following steps will create a more efficient Release 5 database design. The new design uses the "Show single category" feature so that when you, the user, open the editorial database, you see only the category assigned your name, and therefore, only the documents you have access to.
The first set of steps implements and sets properties for a category embedded in a form. This functionality is new to Release 5.
- In the original view, insert a column at the beginning that is sorted and categorized by the ReaderNames field.
Note: The ReaderNames field in all documents must contain user names, not group names.
- Create a new form, and embed the new view in the form. (This step is accomplished by choosing Create - Embedded Element - View and then selecting the name of the view to embed.)
- Once the view is embedded, select "Show single category" under the Embedded View Element.
- Enter @Username for the category formula. This ensures that when the form is opened, it will automatically calculate the current user's name and display the appropriate documents beneath the user's name in the view.
Note: To maximize the display area, the designer might also choose the "Fit to window" property for the embedded view.
- Since this form is only for display purposes, add a computed text field called SaveOptions with a value of 0 to the form as well. This field will prevent the document from being saved and prevent the user from being prompted to save the document.
Figure 4. Form with the embedded view
The next set of steps the designer can take to fine-tune the database uses the new frameset feature. Inserting the form you created above into a frameset provides a clean method of serving up the form. (A frameset also gives you the ability to use multiple frames; for example, to use the left-hand frame for a navigation bar.)
To insert the form in a frameset:
- Create a new frameset in the database.
- Right-click the frame where you want to insert the form containing the embedded view and choose Frame Properties.
- In the Frame properties box, select Named Element and then Form as the Type.
- Set the Value to the form with the embedded view. (You can click the Folder button and use the Locate Object dialog box to find the right form.)
Figure 5. Frame properties box
- Close and save the frameset with a name of your choice.
- Choose File - Database - Properties to open the Database properties box.
- Click the Launch tab, select "Open Designated Frameset," and select your frameset name from the drop-down list.
Figure 6. Launch tab of Database properties box
The screen below shows the Reader Names categorized view embedded in a form and frameset. (In this example, the designer has also used Edit mode to delete one of the frames in the default two-frame frameset.) Because the designer chose the "Open Designated Frameset" option, this is the view the user defined by @Username will see when he or she opens the database.
Figure 7. View embedded in form and frameset
Not only is this database more convenient to use because it immediately displays only those categories and documents the user has access to, it also solves the problem the original application had with slow server performance. Furthermore, it does all of this without sacrificing any of the functionality built into the original application.
Keep in mind that many of the new features in Domino Designer let you achieve greater functionality in your application and enhance your application's performance-without sacrificing security. In the application discussed in this article, the server was essentially bogged down by a single, large database with one large, non-categorized view. By spending some time examining Domino features and how to use them in this custom application, we were able to maximize performance, serve up the data in an improved interface, and provide a better experience for users who work with the application every day.
-
Designing a secure Domino application
-
Performance Considerations for Domino Applications
-
Application Development Best Practices Guide
-
Lotus Notes Knowledge Base
Scott Vrusho, a native Bostonian, has been working at Lotus in various service and support functions for 13 years. The last 7 have been in Notes Support, where he has led the Appletalk, Large Application Performance, and Database Design Product Area Expert Teams. Scott is a member of the Support Engineering Team in Notes Support, which focuses on resolving high severity, high impact customer issues; and he has been an on-site support resource at both the Nagano and Sydney Olympic Games.
Jane Hosie-Bounar started her freelance technical writing and editing career at Lotus 13 years ago, developing courses for a variety of products. Although her client list has grown over the years, she continues to be a frequent contributor at Lotus, writing or editing Help, Users Guides, and other documentation for products like LearningSpace and Notes/Domino.
Table of contents
Local resources
My developerWorks community
Tags
Use the slider bar to see more or fewer tags.
Popular tags shows the top tags for this particular content zone (for example, Java technology, Linux, WebSphere).
My tags shows your tags for this particular content zone (for example, Java technology, Linux, WebSphere).
Popular article tags |
My article tagsSkip to tags list
Popular article tags |
My article tags
