Data import process

The data synchronization component uses the ChangeDataImport agent to import the contents of the compressed file that is exported from the source database.

In a full export, database deletes are not included in the compressed file. At the end of the import, the ChangeDataImport agent deletes the database entities that have a timestamp older than the starting of the import. A full export includes every entity that belongs to a profile in the source database. Therefore, an import considers all the entities that belong to the profile in the compressed file. However, if any entity is not included in the import process, it implies that the entity is deleted in the source database.

Consider the scenario where the source database is the Staging server and the target database is the Production server. When the import agent runs in this situation, files are imported according to their Change Project ID rather than according to a profile. The entities that are modified during this import are then logged in the YFS_ENTITY_CHANGE table on the Production server as part of the DEFAULT change project. Therefore, all the changes are recorded on the Production server and they can be exported as part of a Delta export by the data synchronization process.

The ChangeDataImport agent raises the ON_COMPLETE event when it finishes processing all the files. If none of the profiles had errors, the event is raised with success. If one or more profiles had errors, the ON_COMPLETE event is raised with failure set for the overall status, and the individual zip files are named inside the event with either SUCCESS or FAILURE, as applicable. In the event of a failure, the ON_FAILURE event includes the name of the xml file that had the failure. If you run the ChangeDataImport agent again after correcting the issue, the ON_COMPLETE event will not include the names of the zip files because zip files are no longer available to process; only the .xml files are available. Details about any failures are recorded in the system logs.

Entity transformation for data import process

When data is imported, the entity transforms for the import of records is defined and stored in a resource XML file, entityTransforms.xml. Note the following about the configuration of the entityTransforms.xml file to ensure that data is synchronized correctly in the data import process.

If you have a column inside a database record that is an XML file, the XML needs to be transformed in the entityTransforms.xml file because the configuration parameters for the XML (such as URLs and IP addresses) will differ in the target and source databases. For example, in the following entityTransforms.xml file, CONFIG_XML will be transformed.

<Table Name="YFS_SUB_FLOW">
	<Columns>
		<Column Name="CONFIG_XML">
			<Transform Match="t3://localhost:7001" Replace="${PROVIDER_URL}" 
				XPath="xml:/SubFlowConfig/Link/Properties/@ProviderURL"/>
		</Column>
	</Columns>
</Table>