Pipeline deployments
Pipelines can be installed on a single server or multiple servers, depending on the system requirements and the server resources.
When deploying pipelines, consider the following performance factors:
- Pipelines can be run in single form, or configured to run concurrent parallel processing threads.
- Each CPU can handle 1.5 to 2 pipelines or parallel processing pipeline threads.
- Parallel processing pipelines can receive data from multiple data sources at once, so you do not need to split the files manually to equal the number of single pipelines.
When deploying pipelines, also consider the following factors:
- Pipelines can be executed on any supported hardware and operating system configuration.
- Although possible, do not run the pipelines on the machine where the database is located.
- Parallel processing pipelines are less work to configure than multiple pipelines.
- Multiple server configurations require more work and maintenance to administer.
- Single server configurations require expensive hardware that increases exponentially with the number of CPUs.