Managing Connections
Managing connections includes filtering the list of displayed connections, testing connections, upgrading connections, and editing connection values. You can also view the pipelines and fragments that use each connection.
When needed, you can delete a connection when it is not used in any pipelines or pipeline fragments.
Filtering Connections
In the Connections view, you can filter the list of displayed connections by connection type or tag. Or you can search for connections by name or description.
Testing Connections
When you create or edit connections, you can test that the configured connection values are valid.
When you test a connection, you verify that the selected authoring Data Collector can connect to the external system using the configured values. Testing a connection does not guarantee that a running pipeline using the connection can also connect to the external system.
For example, let's say that you use credential functions in connection properties to retrieve sensitive values from a credential store. When you test the connection, you verify that the selected authoring Data Collector can retrieve the values from the credential store and connect to the external system. A successful test verifies that the credential store is correctly configured for the selected authoring Data Collector. If the same credential store is not correctly configured for the execution Data Collector, then when you start a job that includes a pipeline using this connection, the pipeline fails to connect to the external system.
When a connection type supports multiple stage libraries, such as a Kafka or Kudu connection, you select the stage library installed on the selected authoring Data Collector to use for testing. For example, let's say that both Apache Kafka and Cloudera CDH stage libraries are installed on the selected authoring Data Collector. To test a Kafka connection to brokers running in the Apache Kafka cluster, you select the Apache Kafka stage library.
Viewing Connection Usage
You can view the pipelines and pipeline fragments that use a connection. You might want to view the list of pipelines and fragments before editing, upgrading, or deleting a connection.
Editing Connections
You can edit connections to change most connection properties. You cannot change the connection type.
Edit connections from the Connections view. Hover over a connection, and
click the Edit icon: .
- Name and description
- Authoring Data Collector - When editing a connection, you can select
a different authoring Data Collector.
Ensure that the stage library required by the connection is installed on the selected authoring Data Collector.
If you select an authoring Data Collector of a later version that includes changes to connection properties, you can choose to upgrade the connection.
- Tags - You can assign and remove tags from the job to identify the connection in a different way.
In the Configure Connection section of the dialog box, you can change configuration properties for the connection type. For a description of each property, see Overview.
Upgrading a Connection
The Data Collector version associated with a connection determines the properties available in a connection. A later Data Collector version can include changes to connection properties.
When you edit a connection and choose a later authoring Data Collector version that includes changes to connection properties, you can upgrade the connection to use the changed properties. Alternatively, you can create a copy of the connection and then upgrade the copy to use the changed properties.
Before upgrading a connection, you might want to view the connection usage to determine the pipelines and fragments using the connection.
Deleting Connections
- In the Navigation panel, click Connections.
-
Select connections in the list, and then click the
Delete icon:
.
- Click OK to confirm the deletion.