Executors

An executor stage triggers a task when it receives an event.

Use executors as part of a dataflow trigger in an event stream to perform event-driven, flow-related tasks, such as moving a fully-written file when a target closes it.

For more information about the event framework, see Dataflow triggers overview.

Standalone flows

In standalone flows, you can use the following executors:
  • ADLS Gen2 File Metadata - Changes file metadata, creates an empty file, or removes a file or directory in Azure Data Lake Storage Gen2 upon receiving an event.
  • Amazon S3 - Creates new Amazon S3 objects for the specified content, copies objects within a bucket, or adds tags to existing Amazon S3 objects.
  • Databricks Job Launcher - Starts the specified Databricks job upon receiving an event record.
  • Databricks Query - Runs Spark SQL queries on Databricks upon receiving an event.
  • Email - Sends custom email to the configured recipients upon receiving an event.
  • Google BigQuery - Runs SQL queries on Google BigQuery upon receiving an event.
  • Google Cloud Storage - Creates new Google Cloud Storage objects for the specified content, copies or moves objects, or adds metadata to existing Google Cloud Storage objects.
  • JDBC Query - Runs user-defined SQL queries upon receiving an event record.
  • Pipeline Finisher - Stops and transitions the flow to a Finished state upon receiving an event record.
  • SFTP/FTP/FTPS Client - Moves or removes files from an SFTP, FTP, or FTPS server.
  • Shell - Executes a shell script upon receiving an event record.
  • Snowflake - Loads whole files staged by the Snowflake File Uploader target to Snowflake tables.