Optimizing replication performance with volatile attributes

Edit online

Improve replication performance by separating volatile attribute replication from regular attribute replication. This ensures that frequent changes to volatile attributes do not delay the replication of critical non-volatile attributes.

In replication environments, some attributes such as last login timestamps or previous bind timestamps change very frequently. These high-frequency updates can fill the replication queue and delay replication of more critical attributes.

To address this performance issue, IBM Directory Server provides a mechanism to separate volatile attribute replication from regular attribute replication. By marking frequently changing attributes as volatile and enabling separate replication handling, you can ensure that important attribute changes are replicated promptly without being delayed by volatile updates

Scenario
Consider a directory with the following attributes:
  • User password (critical, changes infrequently)
  • Last login timestamp (non-critical, changes on every login)

In a traditional/regular replication setup, all attribute changes share the same replication queue. When users log in frequently, the last login timestamps generate many replication changes. These high-frequency changes fill up the replication queue, and critical changes like passwords get queued behind volatile attribute changes. The important attributes experience replication delays. There could be an inconsistent state for the directory server across replicas.

Solution: Volatile Attribute Replication
The volatile attribute replication feature provides:
  • Separate replication threads for volatile attributes
  • Independent replication queues for volatile vs regular attributes
  • Dedicated monitoring for volatile attribute replication
Configure volatile attribute replication
  1. Identify volatile attributes
    • Change very frequently (multiple times per minute/hour)
    • Are not critical for security or access control
    • Can tolerate slightly delayed replication
    • Generate high replication traffic

    Example: Last login timestamps

  2. Mark attributes as volatile in the schema

    Add the VOLATILE keyword to the IBMAttributetypes definition in your schema file.

    Syntax: 
    IBMAttributetypes=( <OID> DBNAME( '<dbname1>' '<dbname2>' ) VOLATILE [other-keywords] )
    Example: 
    IBMAttributetypes=( latestbindtimeoid DBNAME( 'latestBindTime' 'latestBindTime' ) VOLATILE ACCESS-CLASS normal LENGTH 30 EQUALITY ORDERING )

    To apply such schema changes, create an LDIF file "schema_update.ldif" and use the idsldapmodify command to update the IBMAttributetypes definition.

    Example LDIF file:

    dn: cn=schema
changetype: modify
replace: attributetypes
attributetypes: ( latestbindtimeoid NAME ( 'latestBindTime' ) DESC 'The last login time for a user.' EQUALITY generalizedTimeMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.24 USAGE userApplications )
-
replace: IBMAttributetypes
IBMAttributetypes: ( latestbindtimeoid DBNAME ( 'latestBindTime' 'latestBindTime' ) VOLATILE ACCESS-CLASS normal LENGTH 240 EQUALITY )

    Command to apply the LDIF:

    -h <hostname> -p <port> -D <adminDN> -w <password> -i schema_update.ldif
  3. Update schema on all servers
    • Update the schema file on the supplier server.
    • Restart the directory server or reload the schema.
    • Update the schema on all consumer servers
    • Restart consumer servers or reload their schemas
  4. Verify configuration

    Run this command to confirm volatile attributes are recognized

    idsldapsearch -h hostname -p port -D <admin_user> -w <password> -b cn=schema -s base objectclass=* IBMAttributetypes | grep -i volatile
  5. Use volatile attribute in an entry
Once the attribute is made volatile, its data will be stored in the attribute table, not in the ldap_entry table. In this case, only its reference will be stored in the ldap_entry table. Updates to ldap_entry are restricted when:
  • The volatile attribute already exists and the same attribute is added.
  • A replace operation is performed for the volatile attribute.
Environment Variable for Volatile Handling
You can enable or disable volatile attribute handling using the IBMSLAPD_VOLATILE_DISABLED environment variable. This is useful for:
  • Troubleshooting
  • Backward compatibility testing
  • Temporary disabling without schema changes
Disable Volatile Handling
export IBMSLAPD_VOLATILE_DISABLED=TRUE
Effect when disabled:
  • The VOLATILE keyword is parsed but ignored
  • All attributes are treated as regular attributes
  • Changes are tracked in the regular replication change table
  • Volatile-specific monitoring attributes are not populated
Adding the variable to configuration file using the LDAP client utility:
  • Issue the idsldapmodify command of the following format: 
    idsldapmodify -p <port> -D <adminDN> -w <adminPW>
dn: cn=Front End, cn=Configuration
changetype: modify
add: ibm-slapdSetenv
ibm-slapdSetenv: IBMSLAPD_VOLATILE_DISABLED=TRUE
    and restart the directory server instance
  • or from command prompt set the environment variable IBMSLAPD_VOLATILE_DISABLED
On AIX® and Linux™ systems (ksh shell) 
export IBMSLAPD_VOLATILE_DISABLED=TRUE
On Windows™ systems 
set IBMSLAPD_VOLATILE_DISABLED=TRUE

For the set value of the environment variable to be effective, restart the directory server instance from the same shell where the IBMSLAPD_VOLATILE_DISABLED environment variable was set.

To enable volatile handling (default)
Update configuration file, set: 
ibm-slapdSetenv: export IBMSLAPD_VOLATILE_DISABLED=FALSE

From the command prompt:

unset IBMSLAPD_VOLATILE_DISABLED

or

export IBMSLAPD_VOLATILE_DISABLED=FALSE
Note: Set this environment variable before starting the directory server.
Monitoring volatile replication
Monitoring Attributes

IBM Directory Server provides operational attributes to monitor volatile attribute replication. For details, see Replication.

Example of monitoring attributes of both regular and volatile replication 
ibm-replicationLastChangeId=2
ibm-replicationPendingChangeCount=0
ibm-replicationFailedChangeCount=0
ibm-replicationLastVolatileChangeId=5
ibm-replicationPendingVolatileChangeCount=0
ibm-replicationFailedVolatileChangeCount=0