Export node
In Synthetic Data Generator, you can use the Export node to save the synthetic data that you generated to a database or file.
- Description
- Use the Export node in a Synthetic Data Generator flow to save the generated synthetic dataset. You can pick the format and location to save the synthetic data that is generated.
- Using the node
- The Export node is usually one of the last nodes in a Synthetic Data Generator flow. It takes the output from the Generate node and writes the dataset in the format and location that you choose. You can save the dataset as a file or to a database.
- Although you can add several Export nodes to a Synthetic Data Generator flow, you cannot use multiple nodes to export data to different locations.
- Mandatory or optional
- The Export node is mandatory if you want to save the synthetic dataset that is generated.
Scripting with the Export
You can use scripting languages, like Python, to progammatically set properties for nodes.
Export node properties
The following properties are specific to the Export node. For information about common node properties, see Properties for flows and nodes.
| Property name | Data type | Property description |
|---|---|---|
asset_type |
DataAsset, Connection |
Specify your data type: DataAsset or Connection. |
asset_id |
String | When DataAsset is set for the asset_type, this is the ID of the asset. |
asset_name |
String | When DataAsset is set for the asset_type, this is the name of the asset. |
connection_id |
String | When Connection is set for the asset_type, this is the ID of the connection. |
connection_name |
String | When Connection is set for the asset_type, this is the name of the connection. |
connection_path |
String | When Connection is set for the asset_type, this is the path to the table in the connection. Depending on the database connection, the path includes the catalog and schema, for example catalog_name/schema_name/table_name. |
user_settings |
String | Escaped JSON string containing the interaction properties for the connection, for example: user_settings: "{\"interactionProperties\":{\"write_mode\":\"write\",\"file_name\":\"output.csv\",\"file_format\":\"csv\",\"quote_numerics\":true,\"encoding\":\"utf-8\",\"first_line_header\":true,\"include_types\":false}}"These values will change based on the type of connection you're using. |
Example
The following is an example of the properties used in a scriipt.
import json
stream = sdg.script.stream()
dataassetexport = stream.findByID("<import nodeId>")
# loads the string settings as a json object
userSettings = json.loads(dataassetexport.getPropertyValue("user_settings"))
userSettings["interactionProperties"]["file_name"] = "<new file name>"
dataassetexport.setPropertyValue("user_settings", json.dumps(userSettings))