Replication

Content Engine object replication is the ability to selectively copy and maintain synchronization of replicable objects between repositories; specifically, between a Content Engine object store repository and an external repository. Content Engine object replication supports the following operations:

  • Copying an object to or from a Content Engine object store repository and an external repository.
  • After an object is copied, synchronizing changes that are made to either the original object or its copy.

An object that is copied to one or more repositories by the replication framework is referred to a replicated object. A copy of a replicated object is referred to as a replica.

The Content Engine Java™ and .NET APIs include several classes and properties that support Content Engine object replication. However, replication is typically initiated and configured by using the administrative tools. For more information, see Setting up Content Federation Services for Image Services or Setting up Content Federation Services for Content Manager OnDemand.

Replicable objects

Replication is supported for Content Engine objects that are instantiated from a subclass of the Replicable class (referred to as replicable objects).

Each Replicable object includes the following properties:

Replicated objects

Each object in a Content Engine object store has a unique ID that is controlled by the Content Engine. However, a replicated object in an external repository possesses a different unique ID from the original object over which the Content Engine has no control. An ExternalIdentity object represents the identity of a replicated object in an external repository. This object can be obtained from the ExternalIdentityList collection of a Replicable object. Each ExternalIdentity object has the following properties:

  • ExternalObjectIdentity: A string that specifies the unique identity of this external replicated object, relative to an external repository in which a replica of the object exists. The combination of the value of the ExternalObjectIdentity property and the external repository name must be unique within the partner object store of the external repository.
  • ExternalRepository: An ExternalRepository object that represents the external repository in which the replicated object exists.
  • ReplicationData: Specifies provider-specific data for this external replicated object. You cannot query for data that is stored in this property.

Initiating replication

To initiate replication for a replicable Content Engine object, set its ReplicationGroup property to a replication group (represented by a ReplicationGroup object). A class can be configured to automatically replicate new instances by setting a default value for the ReplicationGroup property. To stop replication on a replicable Content Engine object, set its ReplicationGroup property to null. When replication of an object is stopped, its existing replicas are not deleted, but update synchronization is canceled.

Repositories

Content Engine objects replication occurs between an object store repository, which is represented by an ObjectStore object, and an external repository, which is represented by an ExternalRepository object. Both of these objects are instances of subclasses of the base Repository class.

External repositories

Unlike an object store repository, an external repository is a repository that is external to the Content Engine. All of the external repositories that IBM® supports are also capable of federating content. Federating content is the process of creating a Content Engine document that references a document in an external repository and mapping the source document metadata to Content Engine document properties. By using content federation, you can integrate data that is generated by multiple applications and stored in disparate databases or on multiple servers. A specific external repository is represented by one of the following objects, which are instantiated from subinterfaces of the ContentFederatingRepository interface:

  • CMODRepository: An IBM Content Manager OnDemand external repository that is capable of federating content.
  • ImageServicesRepository: An IBM FileNet Image Services external repository that is capable of federating content.

Each external repository maintains a list of ExternalClassDescription objects. Each of these objects provides information about a replicable class in an external repository that can be replicated. This information can be used for defining class and property mappings. Each class can be configured to be replicated from the external repository to an object store, as a target for objects that are replicated from the object store, or both. The ExternalClassDescription objects are stored in the ExternalClassDescriptionList collection that is returned by the ExternalClassDescriptions property of an ExternalRepository object.

Replication groups

A replication group is a group of object stores and external repositories among which replication can be enabled. Each replication group is represented by a ReplicationGroup object. Each replicable object in an object store that is enabled for replication specifies the replication group to which it belongs with its ReplicationGroup property.

An object store or external repository can participate in multiple replication groups. Although each replicated object must belong to a single replication group, the replicated objects within a given object store or external repository do not need to belong to the same replication group.

Each ReplicationGroup object has a ReplicationParticipants property that specifies a ReplicationParticipantList collection. This collection defines the replication participants of the group. It is composed of a mix of ObjectStoreParticipant and ExternalParticipant objects, each of which is derived from the base ReplicationParticipant class. To add an object store or external repository as a replication participant of a replication group, add an ObjectStoreParticipant or ExternalParticipant object to the list of participants. Conversely, to remove a repository from a replication group, delete the participant from the list of participants. After a participant is removed from a replication group, the remaining participants no longer generate or process outbound journal entries for the removed object store or external repository. For an external repository, inbound changes are no longer processed.

Replication participants

A replication participant is the participation of an object store or external repository in a replication group and is represented by the base ReplicationParticipant interface. This interface has a ReplicationMode property, which specifies the replication mode of the participant.

The ReplicationParticipant interface has the following subinterfaces:

Replication mode

In addition to specifying a repository, each replication participant also specifies a replication mode, which determines how objects are replicated in the replication group. The replication mode is controlled by the ReplicationMode property of the participant, which specifies one of the ReplicationMode constants. Two modes of replication are supported:

  • MASTER: In this mode, replicated objects in the repository can be modified and any modifications are then propagated to other participants in the replication group.
  • SLAVE: In this mode, replicated objects in the repository are read-only, and can change only as a result of the propagation of modifications from another repository operating in master mode in the replication group.

Partner object store

Each external repository participant in a replication group must be a partner of an object store in that replication group. This object store partner is the only recipient of changes from that external repository and the only object store that delivers changes to that external repository. In a multi-site configuration, it is preferable to choose an object store in the same site as the external repository; however, any object store can be chosen as a partner. A partner object store affects an external repository and the replication group to which it belongs as follows:

  • When changes are pulled from the external repository into the partner object store, those changes are batched by using a journal entry for onward propagation to the other members of the group, even if the partner object store is operating in slave mode.
  • When changes are made in the partner object store, those changes are batched by using a journal entry for propagation to all other members of the group, including the external repository.
  • When changes are made in a non-partner object store, those changes are batched by using a journal entry for propagation to all other object stores. External repositories are excluded.

Class and property mappings

Before an object can be replicated from an object store to an external repository, its class and properties must first be mapped. A mapping must be maintained between the object ID of the object in the external repository and the Content Engine object of the object in the object store so that inbound updates that carry the external identifier can locate the correct Content Engine object to which to apply updates.

External class mappings are stored as a list of ExternalClassAlias objects in a ReplicableClassDefinition object that is associated with the mapped Content Engine class. This list can be accessed in the ExternalClassAliasList collection of ExternalClassAlias objects that are returned by the ExternalAliases property. Each ExternalClassAlias object has the following properties:

  • ExternalRepository: An ExternalRepository object that represents the external repository in which the mapped external class is defined.
  • ClassIdentity: The identity of the mapped external class.
  • AliasDirection: The direction in which mapping applies for the class alias. This property specifies one of the following ReplicationDirection constants:
    • INBOUND: Specifies that instances of an external class can be replicated from an external repository to a Content Engine object store, which results in the creation of instances of the mapped Content Engine class.
    • OUTBOUND: Specifies that instances of the mapped Content Engine class can be replicated to an external repository, which results in the creation of instances of the specified external class.
    • BOTH: Specifies that instances of an external class can be replicated from an external repository to a Content Engine object store and can also be a target for objects that are replicated from a Content Engine object store to an external repository.

External property mappings are stored as a list of ExternalPropertyAlias objects in the PropertyDefinition object within a ReplicableClassDefinition object that is associated with the mapped Content Engine class. This list can be accessed in the ExternalPropertyAliasList collection of ExternalPropertyAlias objects that are returned by the ExternalAliases property. Each ExternalPropertyAlias object has the following properties:

  • ExternalRepository: An ExternalRepository object that represents the external repository in which the mapped external property is defined.
  • ClassIdentity: The identity of the mapped external class that contains the mapped external property.
  • PropertyIdentity: The identity of the mapped external property.
  • AliasDirection: The direction in which mapping applies for the property alias. This property specifies one of the following ReplicationDirection constants:
    • INBOUND: Specifies that a value of the external property is copied into the mapped Content Engine property when external repository changes are propagated to an instance of the mapped Content Engine class.
    • OUTBOUND: Specifies that a value of the mapped Content Engine property is copied to the specified external repository property when Content Engine changes are propagated to an instance of the external class.
    • BOTH: Specifies that property values can be copied in either an inbound or an outbound direction, depending on the direction of propagation.

Journal entries

Each time that a Content Engine object is initially associated with a replication group (replication of the object is initiated) or a Content Engine that is already associated with a replication group is modified, a journal entry is written to a queue to be processed. A server background task periodically queries for new journal entries, and failed entries that are ready to be retried. It submits those entries in batches to the appropriate external repository provider, which is responsible for replicating the object to the external repository or propagating the updates of the object. This type of processing is referred to as outbound processing.

Each journal entry is represented by a ReplicationJournalEntry object. This object has the following properties:

  • DestinationRepository: The external repository that is the destination for the source object of this journal entry operation.
  • HeldUntilDate: The date and time when the current lease on this journal entry expires or when to try it again following a failure.
  • LastFailureReason: A description of the latest failure condition, if any, for the journal entry operation.
  • ReplicationGroup: A ReplicationGroup object that represents the replication group to which the source object of this journal entry operation belongs. In the special case of when a ReplicationGroup property is modified from one non-null value to another, the following types of journal entries are generated, depending on the location of the repository:
    • For a repository in the original replication group but not in the new group, a dereplicate journal entry is generated.
    • For a repository in the new group but not in the original, an initiate journal entry is generated.
    • For a repository in both the original and the new group, an update journal entry is generated to propagate the property change (and any other updates).
  • ReplicationOperation: Specifies the type of operation of this journal entry. This property can have one of the following values:
    • 0 (initiate): This type of journal entry is written when the ReplicationGroup property of an object is first set to a non-null value, which requires the object to become replicated.
    • 1 (update): This type of journal entry is written for any change, including delete, of a replicated object.
    • 2 (dereplicate): This type of journal entry is written when the property is set back to null, which stops replication for that object.
    • 3 (class confirm): This type of journal entry is generated when the replication provider is required to notify the external repository of changes in class mappings for the repository.
    • 4 (group confirm): This type of journal entry is generated when the replication provider is required to notify the external repository of changes in replication group configuration that involves the repository.
  • ReplicationStatus: A ReplicationJournalStatus constant that specifies the processing status of this journal entry. The constant can have one of the following values:
    • FAILED: The processing of the replication journal entry has failed multiple times and has been abandoned.
    • IN_PROGRESS: The processing of the replication journal entry is in progress.
    • RETRY_WAIT: The processing of the replication journal entry has failed at least once and is waiting to be tried again.
    • WAITING: A new replication journal entry is waiting for initial processing.
  • RetryCount: Specifies the number of remaining attempts to process this journal entry in the queue.
  • SourceObject: The source object in the object store of the replication operation that generated this journal entry.