Technology preview of Asynchronous Reconciliation in Registry Services

Body

The registration of a large number of resources may take days to complete. A large part of this time (up to 40%, depending on the complexity of your data) is used for the reconciliation of the resources. One way to speed the registration is to perform the reconciliation of the resources asynchronously, in a separate task that is independent of the registration itself.

Another additional benefit of asynchronous reconciliation, also called continuous reconciliation, specially during bulk-load operations, is that we avoid conflicts and retries due to different products registering information about the same resources at the same time. Also, by managing reconciliation separately, it can be optimized to avoid repeated operations if the same registration record is touched multiple times during a short interval. The registration records can also be processed in optimal order, with resources that provide naming context being processed before the ones that depend on them for context.

You can get a grip on asynchronous reconciliation by using it in a test environment, and provide us some feedback. To enable it, you must set the following configuration parameter, by using the config CLI:

frs.[bat|sh] config -set continuous.reconciliation=on

Once asynchronous reconciliation is enabled, we can disable the inline reconciliation, which is the reconciliation that would be performed in the same thread of the registration:

frs.[bat|sh] config -set inline.reconciliation=off

The settings for asynchronous and inline reconciliation does not affect the update or deletion of resources, since these operations are always performed in the same thread of the HTTP request.

The default interval between each verification whether there are new resources to be reconciled since last verification is defined by the configuration parameter reconciliation.pollingInterval, which has a default value of 30 seconds. If the reconciliation of new resources takes more than 30 seconds to be performed, there will no delay before next verification. You can customize this value by using the config CLI, as follows:

frs.[bat|sh] config -set reconciliation.pollingInterval=60

Asynchronous reconciliation also updates automatically the reconciled records whenever resource shapes or cleansing rules are updated, so there is no need to run recomputeReconciledState CLI after these updates when it is enabled.

The RFE that originated this feature can be accessed at Technology Preview of Asynchronous Reconciliation for New Resource Registration, Cleansing Rules and Resource Shapes. Come review the RFE and provide your feedback.