Monitoring performance with cn=monitor

You can retrieve statistics from the server by issuing a search request with a search base of cn=monitor and a filter of (objectclass=*). These are the only values accepted for search base and filter on the monitor search. However, any of the possible scope values are accepted.

The z/OS® LDAP server presents monitor data in multiple entries:

Server-wide statistics are contained in an entry whose distinguished name is cn=monitor.
Each configured backend has statistics contained in its own entry named cn=backendXXXX,cn=monitor, where XXXX is the backend name specified on the database configuration option in the LDAP server configuration file. If no backend name is specified on the database configuration option, the LDAP server generates a name. The naming contexts pertaining to the specific backend are also included in the entry to identify which server backend is being reported.
Several entries contain statistics for backends that are created by the LDAP server:
- cn=backendMonitor,cn=monitor - Statistics for the backend handling cn=monitor searches
- cn=backendSchema,cn=monitor - Statistics for the backend managing the schema
- cn=backendRootDSE,cn=monitor - Statistics for the backend handling root DSE searches
If the operations monitor is on (the operationsMonitorSize configuration option is not set to zero), the cn=operations,cn=monitor entry contains statistics on search patterns.

For a scope of:

base: Only the cn=monitor entry is returned containing server-wide statistics.
one (one-level search): All backend-specific entries are returned and the operations monitor entry is returned (if configured).
sub (subtree search): All entries are returned, including the operations monitor entry (if configured).

The statistics reported on the cn=monitor subtree search can also be displayed by using the LDAP server DISPLAY operator modify command. Operations monitor statistics cannot be displayed by using the DISPLAY operator command. The command is:

f dssrv,display monitor

See the description of the DISPLAY MONITOR output in Displaying performance information and server settings for details.

Statistics generally reflect data that is gathered since the LDAP server was started. However, many of the counters can be reset by using the LDAP server RESET operator modify command. The command is:

f dssrv,reset monitor

In this case, the values reflect data that is gathered since the last reset.

The monitor search returns the following attributes:

Table 1. Server statistics
`Attribute`	Description
`currentconnections`	Current number of client connections.
`currenttime`	Current date and time on the server.
`livethreads`	Configured number of communication threads (`commThreads`).
`maxconnections`	Configured maximum number of concurrent client connections (`maxConnections`).
`maxreachedconnections`	High water mark for concurrent client connections.
`pagedentriescached`	Number of entries currently cached for paged search requests.
`pagedsearches`	Number of paged search requests completed. This value is incremented only after the last page of a paged search request is returned.
`resets`	Number of times statistics were reset.
`resettime`	Date and time statistics were last reset.
`searchpagessent`	Number of pages sent for paged search requests. This value is incremented after each page of a paged search request is returned.
`sortedsearches`	Number of sorted search requests completed. If a paged search request was specified along with a sorted search request, this value is incremented only after the last page of the paged search request is returned.
`starttime`	Date and time the server was started.
`sysmaxconnections`	System-defined maximum number of connections.
`timedoutconnections`	Number of idle client connections closed based on the `idleConnectionTimeout` setting.
`timedoutpagesets`	Number of paged search result sets abandoned based on the `idleConnectionTimeout` setting.
`totalconnections`	Number of client connections that have been made to the server.
`version`	Version of the LDAP server.

The statistics reported for the maxconnections, sysmaxconnections, totalconnections, currentconnections, and maxreachedconnections attribute values only contain information for network connections. PC connection statistics are not included in these attribute values.

The sysmaxconnections value might be lower than the maxconnections value because of system limits. If the value for the maxConnections configuration option is not valid, the maxconnections attribute value on cn=monitor search reflects the system maximum connection limit. For information about how the maximum number of client connections is set in the LDAP server, see the maxConnections configuration option at maxConnections num-connections.

When statistics are reset, resettime is set to the value of currenttime, resets is incremented, maxreachedconnections is set to the value of currentconnections, and timedoutconnections, timedoutpagesets, pagedsearches, searchpagessent, and sortedsearches are set to 0. None of the other server statistics that were previously listed are affected by a reset.

Table 2. Server and backend-specific statistics
Statistics	Description
`abandonsrequested`	Number of abandon operations requested.
`abandonscompleted`	Number of abandon operations completed.
`addsrequested`	Number of add operations requested.
`addscompleted`	Number of add operations completed.
`bindsrequested`	Number of bind operations requested.
`bindscompleted`	Number of bind operations completed.
`bytessent`	Number of bytes of data sent,
`comparesrequested`	Number of compare operations requested.
`comparescompleted`	Number of compare operations completed.
`deletesrequested`	Number of delete operations requested.
`deletescomplketed`	Number of delete operations completed.
`entriessent`	Number of search entries sent.
`extopsrequested`	Number of extended operations requested.
`extopscompleted`	Number of extended operations completed.
`groupgatheringevents`	Number of group gathering events completed.
`modifiesrequested`	Number of modify operations requested.
`modifiescompleted`	Number of modify operations completed.
`modifydnsrequested`	Number of modifyDn operations requested.
`modifydnscompleted`	Number of modifyDn operations completed.
`opscompleted`	Number of operations completed.
`opsinitiated`	Number of operations initiated.
`searchreferencessent`	Number of search references sent.
`searchesrequested`	Number of search operations requested. If a paged search request was specified, this value is incremented only when the initial paged search request is received.
`searchescompleted`	Number of search operations completed. If a paged search request was specified, this value is incremented only after the last page of a paged search request has been returned.
`unbindsrequested`	Number of unbind operations requested.
`unbindscompleted`	Number of unbind operations completed.
`unknownopsrequested`	Number of unrecognized operations completed.
`unknownopscompleted`	Number of unrecognized operations completed.

When statistics are reset, all of the server and backend-specific statistics that were previously listed are set to zero.

Table 3. Backend-specific statistics
`Statistics`	Description
`acl_source_cache_size`	Configured maximum size (in entries) of the ACL Source cache (`aclSourceCacheSize`).
`acl_source_cache_current`	Current size (in entries) of the ACL Source cache.
`acl_source_cache_hit`	Number of lookups that have hit the ACL Source cache.
`acl_source_cache_miss`	Number of lookups that have missed the ACL Source cache.
`acl_source_cache_percent_hit`	Percent of lookups that have hit the ACL Source cache.
`acl_source_cache_refresh`	Number of times the ACL Source cache was invalidated.
`acl_source_cache_refresh_avgsize`	Average number of entries in the ACL Source cache at invalidation.
`dn_cache_size`	Configured maximum size (in entries) of the DN cache (`dnCacheSize`).
`dn_cache_current`	Current size (in entries) of the DN cache.
`dn_cache_hit`	Number of lookups that have hit the DN cache.
`dn_cache_miss`	Number of lookups that have missed the DN cache.
`dn_cache_percent_hit`	Percent of lookups that have hit the DN cache.
`dn_cache_refresh`	Number of times the DN cache was invalidated.
`dn_cache_refresh_avgsize`	Average number of entries in the DN cache at invalidation.
`dn_to_eid_cache_size`	Configured maximum size (in entries) of the DN to Entry ID cache (`dnToEidCacheSize`).
`dn_to_eid_cache_current`	Current size (in entries) of the DN to Entry ID cache.
`dn_to_eid_cache_hit`	Number of lookups that have hit the DN to Entry ID cache.
`dn_to_eid_cache_miss`	Number of lookups that have missed the DN to Entry ID cache.
`dn_to_eid_cache_percent_hit`	Percent of lookups that have hit the DN to Entry ID cache.
`dn_to_eid_cache_refresh`	Number of times the DN to Entry ID cache was invalidated.
`dn_to_eid_cache_refresh_avgsize`	Average number of entries in the DN to Entry ID cache at invalidation.
`entry_cache_size`	Configured maximum size (in entries) of the Entry cache (`entryCacheSize`)
`entry_cache_current`	Current size (in entries) of the Entry cache.
`entry_cache_hit`	Number of lookups that have hit the Entry cache.
`entry_cache_miss`	Number of lookups that have missed the Entry cache.
`entry_cache_percent_hit`	Percent of lookups that have hit the Entry cache.
`entry_cache_refresh`	Number of times the Entry cache was invalidated.
`entry_cache_refresh_avgsize`	Average number of entries in the Entry cache at invalidation.
`entry_owner_cache_size`	Configured maximum size (in entries) of the Entry Owner cache (`entryOwnerCacheSize`).
`entry_owner_cache_current`	Current size (in entries) of the Entry Owner cache.
`entry_owner_cache_hit`	Number of lookups that have hit the Entry Owner cache.
`entry_owner_cache_miss`	Number of lookups that have missed the Entry Owner cache.
`entry_owner_cache_percent_hit`	Percent of lookups that have hit the Entry Owner cache.
`entry_owner_cache_refresh`	Number of times the Entry Owner cache was invalidated.
`entry_owner_cache_refresh_avgsize`	Average number of entries in the Entry Owner cache at invalidation.
`filter_cache_size`	Configured maximum size (in entries) of the Filter cache (`filterCacheSize`).
`filter_cache_current`	Current size (in entries) of the Filter cache.
`filter_cache_hit`	Number of lookups that have hit the Filter cache.
`filter_cache_miss`	Percent of lookups that have hit the Filter cache.
`filter_cache_percent_hit`	Percent of lookups that have hit the Filter cache.
`filter_cache_refresh`	Number of times the Filter cache was invalidated.
`filter_cache_refresh_avgsize`	Average number of entries in the Filter cache at invalidation.
`filter_cache_bypass_limit`	Configured Filter cache bypass limit (`filterCacheBypassLimit`).
`namingcontexts`	Suffixes managed by this backend.

Not all of the preceding cache statistics appear for each backend. A backend reports statistics for those caches that it supports. The schema backend reports dn_cache statistics. The LDBM and CDBM backends report filter_cache statistics. A TDBM backend reports statistics for all caches except the dn_cache. A Db2®-based GDBM backend reports statistics for all caches except the dn_cache, while a file-based GDBM backend only reports filter_cache statistics.

When statistics are reset, the cache_hit, cache_miss, cache_percent_hit, cache_refresh, and cache_refresh_avgsize for each cache are reset to zero. Resetting the statistics has no effect on the cache_size for each cache, nor on the filter_cache_bypass_limit, since these are configured values. Resetting the statistics also has no effect on the cache_current for each cache, since the contents of the caches are not altered by a reset of statistics. Some caches may get invalidated and refreshed because of directory update operations. When this occurs, cache_refresh is incremented and cache_current is set to zero to reflect the refreshed (empty) cache. The cache_hit, cache_miss, and values cache_percent_hit are accumulated across cache invalidation and refresh until a RESET MONITOR command is issued or the server ends.

Table 4. Operations monitor statistics
Statistics	Description
`cachesize`	Configured maximum number of search patterns in the operations monitor (`operationsMonitorSize`) .
`currenttimestamp`	Current date and time in Coordinated Universal Time stamp format.
`entries`	Total number of search patterns in the operations monitor entry.
`numtrimmed`	Number of search patterns trimmed from the operations monitor.
`resets`	Number of times the operations monitor statistics were reset.
`resettimestamp`	Date and time in Coordinated Universal Time stamp format of last reset or server startup if the reset command was never issued.
`searchStats`	Search statistics for search patterns based on the search parameters. (search base, scope, filter, and attributes to be returned) and status (success or failure).
`searchIPStats`	Search statistics for search patterns consisting of the same elements as the `searchStats` pattern, but also including the client IP address.

When statistics are reset, resettimestamp is set to currentTimestamp, resets is incremented by one, entries is set to zero, numtrimmed is set to zero, and all search patterns are deleted.

The Coordinated Universal Time stamp format used in the currentTimestamp and resettimestamp attribute values is:

yyyymmddhhiiss.uuuuuuZ

Where,

yyyy is year, mm is month, dd is day, hh is hour, ii is minutes, ss is seconds, uuuuuu is microseconds, Z is a character constant meaning that this time is based on Coordinated Universal Time, also known as GMT.

The searchIPStats and searchStats attribute values contain search rates and other search activity that are being monitored. Depending upon the LDAP server configuration, there can be searchIPStats and searchStats attribute values returned in the cn=operations,cn=monitor entry for each search executed against the LDAP server. The searchStats attribute values contain the total of all data collected for all searches matching this search pattern no matter the IP address of the client.

The format of the searchIPStats and searchStats attribute values is:

ldap://clientIP/baseDN?attributes?scope?filter-string?status,numOps=numOps,avg=avg,
rate=rate,maxRate=maxRate,maxRateTimeStamp=maxRateTimeStamp,
createTimeStamp=createTimeStamp,ID=opid

The following describes the LDAP search pattern parts:

attributes: List of attributes to be returned.
avg: Average elapsed time for each occurrence of search pattern in microseconds.
baseDN: Distinguished name of the base of the search, with _v substituted for attribute values.
clientIP: Client IP address (omitted for searchStats search patterns).
createTimeStamp: Date and time this search pattern was first added, in Coordinated Universal Time stamp format.
filter-string: Search filter with substitutions for literal attribute values. Excluding the * character, all strings in values are substituted with _v. For example: (cn=*bob*bah*) would be (cn=*_v*_v*). There is no substitution on objectclass equality values when the objectclass is defined in the schema.
maxRate: The highest rate on this entry.
maxRateTimeStamp: Date and time maxRate was last set, in Coordinated Universal Time stamp format.
numOps: Total number of times this search pattern has occurred.
opid: A unique integer value that distinguishes each operations monitor search pattern.
rate: Number of search operations processed in the previous one minute interval. Starting with server startup or the last reset command, rate is recalculated for each search pattern every 60 seconds.
scope: base for base object searches, one for one-level searches, and sub for subtree searches.
status: success for any search operation that results in return code LDAP_SUCCESS, LDAP_PARTIAL_RESULTS, or LDAP_REFERRAL. Any other return codes result in status being set to failure.

See Table 4 for the time stamp format.

In addition to the preceding syntax, the following character escaping is performed:

Comma = %2C
Percent = %25
Question mark = %3F
Space = %20

Note: The comma, percent, and question mark characters are not escaped when they are used as metacharacters in the search pattern.

For information about monitoring performance with the LDAP server DISPLAY MONITOR operator command, see Displaying performance information and server settings.

Note: DISPLAY MONITOR output does not display cn=operations,

cn=monitor
data

Monitor search examples

Following is an example of a monitor search using scope=base. This returns only statistics related to the entire server:

ldapsearch -h ldaphost -p ldapport -b cn=monitor -s base objectclass=*

cn=monitor
version=z/OS Version 2 Release 2 IBM Tivoli Directory Server LDAP Server 
livethreads=10
maxconnections=24982
sysmaxconnections=25000
totalconnections=20709
currentconnections=1
maxreachedconnections=15
timedoutconnections=12 
timedoutpagesets=0
pagedentriescached=0
pagedsearches=5
sortedsearches=2
opsinitiated=62126
opscompleted=62125
abandonsrequested=0
abandonscompleted=0
addsrequested=2318
addscompleted=2318
bindsrequested=20709
bindscompleted=20709
comparesrequested=0
comparescompleted=0
deletesrequested=2228
deletescompleted=2228
extopsrequested=0
extopscompleted=0
modifiesrequested=11501
modifiescompleted=11501
modifydnsrequested=440
modifydnscompleted=440
searchesrequested=4222
searchescompleted=4221
unbindsrequested=20708
unbindscompleted=20708
unknownopsrequested=0
unknownopscompleted=0
groupgatheringevents=20641
entriessent=4221
bytessent=1564656734
searchreferencessent=0
searchpagessent=18
currenttime=Thu May 23 16:33:00.187846 2013
starttime=Thu May 23 15:52:21.693392 2013
resettime=Thu May 23 15:52:21.693392 2013
resets=0

Following is an example of output of a monitor search with scope=one for a server configured with TDBM and LDBM backends. This returns backend specific statistics and operations monitor statistics. The cache statistics shown would only be included for TDBM, LDBM, GDBM, CDBM, and schema backends, since the other backend types do not implement caches. Operations monitor statistics are included for all backends.

Note that not all operational statistics for each backend are shown in the example below. They have been omitted from the example only, and appear in full for a cn=monitor search.

ldapsearch -L -h ldaphost -p ldapport -b cn=monitor -s one objectclass=*

dn: cn=backendMyTDBM,cn=monitor
namingcontexts: C=CA
namingcontexts: C=TDBM
namingcontexts: CN=MOVER
namingcontexts: CN=MOVING
opsinitiated: 3013
opscompleted: 3013
abandonsrequested: 0
abandonscompleted: 0
addsrequested: 380
addscompleted: 380
bindsrequested: 0
bindscompleted: 0
comparesrequested: 0
comparescompleted: 0
deletesrequested: 365
deletescompleted: 365
extopsrequested: 0
extopscompleted: 0
modifiesrequested: 1645
modifiescompleted: 1645
modifydnsrequested: 63
modifydnscompleted: 63
searchesrequested: 560
searchescompleted: 560
unbindsrequested: 0
unbindscompleted: 0
unknownopsrequested: 0
groupgatheringevents: 3580
unknownopscompleted: 0
entriessent: 560
bytessent: 105692
searchreferencessent: 0
acl_source_cache_size: 100
acl_source_cache_current: 1
acl_source_cache_hit: 3012
acl_source_cache_miss: 1
acl_source_cache_percent_hit: 99.97%
acl_source_cache_refresh: 0
acl_source_cache_refresh_avgsize: 0
dn_to_eid_cache_size: 1000
dn_to_eid_cache_current: 555
dn_to_eid_cache_hit: 195263
dn_to_eid_cache_miss: 4035
dn_to_eid_cache_percent_hit: 97.98%
dn_to_eid_cache_refresh: 0
dn_to_eid_cache_refresh_avgsize: 0
entry_cache_size: 5000
entry_cache_current: 562
entry_cache_hit: 381420
entry_cache_miss: 1259
entry_cache_percent_hit: 99.67%
entry_cache_refresh: 0
entry_cache_refresh_avgsize: 0
entry_owner_cache_size: 100
entry_owner_cache_current: 1
entry_owner_cache_hit: 3012
entry_owner_cache_miss: 1
entry_owner_cache_percent_hit: 99.97%
entry_owner_cache_refresh: 0
entry_owner_cache_refresh_avgsize: 0
filter_cache_size: 5000
filter_cache_current: 0
filter_cache_hit: 0
filter_cache_miss: 0
filter_cache_percent_hit: 0.00%
filter_cache_refresh: 2446
filter_cache_refresh_avgsize: 0
filter_cache_bypass_limit: 100

dn: cn=backendLDBM-002,cn=monitor
namingcontexts: C=AU
namingcontexts: C=LDBM
...
searchreferencessent: 0
filter_cache_size: 5000
filter_cache_current: 0
filter_cache_hit: 0
filter_cache_miss: 0
filter_cache_percent_hit: 0.00%
filter_cache_refresh: 16487
filter_cache_refresh_avgsize: 0
filter_cache_bypass_limit: 100

dn: cn=backendMonitor,cn=monitor
namingcontexts: CN=MONITOR
...

dn: cn=backendSchema,cn=monitor
namingcontexts: CN=SCHEMA
...
searchreferencessent: 0
dn_cache_size: 1000
dn_cache_current: 1000
dn_cache_hit: 123743
dn_cache_miss: 22017
dn_cache_percent_hit: 84.90%
dn_cache_refresh: 0
dn_cache_refresh_avgsize: 0


dn: cn=backendRootDSE,cn=monitor
...

dn: cn=operations,cn=monitor
searchStats: ldap:///OU=_v,O=_v,C=_v?telephoneNumber,postalAddress,mail,uid?o
 ne?(objectclass=inetOrgPerson)?failure,numOps=51,avg=230,rate=32,maxRate=32,
 maxRateTimeStamp=20130524132741.415477Z,createTimeStamp=20130524132628.36161
 8Z,ID=2737
searchStats: ldap:///OU=_v,O=_v??sub?(|(&(sn=_v)(cn=_v*))(description=*_v*))?
 success,numOps=42,avg=246,rate=5,maxRate=37,maxRateTimeStamp=20130524132626.
 545031Z,createTimeStamp=20130524132615.953823Z,ID=2738
searchStats: ldap:///RACFGROUPID=_v+RACFUSERID=_v,PROFILETYPE=_v,CN=_v?racfco
 nnectowner,racfconnectgroupauthority,racfconnectgroupuacc?base?(objectClass=
 *)?success,numOps=4,avg=240,rate=0,maxRate=4,maxRateTimeStamp=20130524132628
 .047031Z,createTimeStamp=20130524132626.878552Z,ID=2739
searchIPStats: ldap://9.12.47.208/OU=_v,O=_v,C=_v?telephoneNumber,postalAddre
 ss,mail,uid?one?(objectclass=inetOrgPerson)?failure,numOps=51,avg=230,rate=3
 2,maxRate=32,maxRateTimeStamp=20130524132741.415477Z,createTimeStamp=20130524
 132628.361618Z,ID=2740
searchIPStats: ldap://fe00::f4f7:0:0:7442:750f/OU=_v,O=_v??sub?(|(&(sn=_v)(cn
 =_v*))(description=*_v*))?success,numOps=42,avg=246,rate=5,maxRate=37,maxRat
 eTimeStamp=20130524132626.545031Z,createTimeStamp=20130524132615.953823Z,ID=
 2741
searchIPStats: ldap://127.0.0.1/RACFGROUPID=_v+RACFUSERID=_v,PROFILETYPE=_v,C
 N=_v?racfconnectowner,racfconnectgroupauthority,racfconnectgroupuacc?base?(o
 bjectClass=*)?success,numOps=4,avg=240,rate=0,maxRate=4,maxRateTimeStamp=2013
 0524132628.047031Z,createTimeStamp=20130524132626.878552Z,ID=2742
currenttimestamp: 20130524132836.785259Z
resettimestamp: 20130524132615.369362Z
resets: 0
numtrimmed: 0
entries: 6
cachesize: 1000