What's new and changed in Synthetic Data Generator

You can now use PDFs as reference documents for the knowledge builder pipeline. The knowledge builder pipeline uses the PDFs as background information when making its question and answer pairs.

For more information about other file types that the knowledge data builder pipeline uses, see Generating synthetic unstructured data.

You can now use Synthetic Data Generator within projects that are integrated with Git. You can use the Git repository to track changes, collaborate with others, and manage your data assets for Synthetic Data Generator.

For more information, see Projects with default Git integration.

You can now create unstructured text datasets that mimic your organization's data. The synthetic data generation API creates synthetic data by using one of the data builder pipelines. You can pick the builder pipelines to use with your unstructured data to tune and evaluate foundation models for your specific use cases.

For more information, see Generating unstructured synthetic data.

Now when you use the Anonymize node to mask real data, the replacement values look more realistic while still replacing the real data. For example, first names, last names, and email addresses look more believable, and zip codes follow patterns like real region codes.

For more information, see Using differential privacy.

You can now set how many categorical values can be detected before the field is identified as Typeless. Previously, if a Categorical field had more than 250 categories, it would often be identified as a Typeless field. You can adjust this setting in the "Flow Properties" dialog.