Export node

In Synthetic Data Generator, you can use the Export node to save the synthetic data that you generated to a database or file.

Description
Use the Export node in a Synthetic Data Generator flow to save the generated synthetic dataset. You can pick the format and location to save the synthetic data that is generated.
Using the node
The Export node is usually one of the last nodes in a Synthetic Data Generator flow. It takes the output from the Generate node and writes the dataset in the format and location that you choose. You can save the dataset as a file or to a database.
Although you can add several Export nodes to a Synthetic Data Generator flow, you cannot use multiple nodes to export data to different locations.
Mandatory or optional
The Export node is mandatory if you want to save the synthetic dataset that is generated.

Scripting with the Export

You can use scripting languages, like Python, to progammatically set properties for nodes.

Export node properties

The following properties are specific to the Export node. For information about common node properties, see Properties for flows and nodes.

Table 1. Node properties for scripting
Property name Data type Property description
asset_type DataAsset, Connection Specify your data type: DataAsset or Connection.
asset_id String When DataAsset is set for the asset_type, this is the ID of the asset.
asset_name String When DataAsset is set for the asset_type, this is the name of the asset.
connection_id String When Connection is set for the asset_type, this is the ID of the connection.
connection_name String When Connection is set for the asset_type, this is the name of the connection.
connection_path String When Connection is set for the asset_type, this is the path to the table in the connection. Depending on the database connection, the path includes the catalog and schema, for example catalog_name/schema_name/table_name.
user_settings String Escaped JSON string containing the interaction properties for the connection, for example:
user_settings: "{\"interactionProperties\":{\"write_mode\":\"write\",\"file_name\":\"output.csv\",\"file_format\":\"csv\",\"quote_numerics\":true,\"encoding\":\"utf-8\",\"first_line_header\":true,\"include_types\":false}}"
These values will change based on the type of connection you're using.

Example

The following is an example of the properties used in a scriipt.

import json
stream = sdg.script.stream()
dataassetexport = stream.findByID("<import nodeId>")

# loads the string settings as a json object
userSettings = json.loads(dataassetexport.getPropertyValue("user_settings"))

userSettings["interactionProperties"]["file_name"] = "<new file name>"
dataassetexport.setPropertyValue("user_settings", json.dumps(userSettings))