Contribute in GitHub: Edit online

Manta Launcher

Purpose of This Document

This document provides guidance to IBM Manta Data Lineage administrators on how to use Manta Launcher, enabling them to launch all Manta Data Lineage products in the correct order using a single utility.

Manta Launcher ships with a dedicated web server that presents a web interface showing runtime info about the whole installation. The Manta Launcher dashboard is available at http://localhost:8282/manta-launcher/ by default. The actual port can be changed during installation.

The dashboard shows a simple table with a list of applications and their statuses together with some other useful data.

Launcher Dashboard

The dashboard contains following data points.

• App—name of the component

• App Directory—installation directory of the component

• URL—local URL on which the component can be reached

• Status—status of the component; this column has one of three possible values

  • Running: The component is started and running.

  • Unavailable: This is a transitional state. The component is either starting or stopping. It is not available for use.

  • Stopped: The component is not running.

• PID—process ID as it is reported by the operating system

• Logs—location of the logs in the file system

Integration to the Operating System

Manta Data Lineage is installed as a single service called MantaLauncher. This service starts Manta Launcher, which then starts up all the components in the correct order. To restart an individual component use the control scripts provided in the installation directory.

Components

Manta Data Lineage is composed of three standalone applications (Manta Flow Server, Manta Admin UI, and Keycloak) and CLI, controllable by Manta Process Manager. Because of this, we provide a simple utility for Manta administrators that will launch all long-running, standalone applications (Manta Flow Server, Manta Admin UI, and Keycloak) in the correct order and tell the user whether they have been launched correctly. An application is deemed to have started correctly when the utility can successfully verify the following conditions.

• The exit code of the command used to start the application indicates success.

• The process is running with a valid process ID.

• The health endpoint of the application returns a correct response indicating that the application is running and ready to receive commands.

• Manta Flow Server in an on-premises installation runs in a container of Apache Tomcat web server.

• Manta Admin UI in an on-premises installation runs in a container of Apache Tomcat web server.

• Manta Keycloak in an on-premises installation is built on Quarkus framework.

• Manta ActiveMQ Artemis in an on-premises installation runs as a separate message broker

Manta Launcher

Manta Launcher is a Java command-line utility that launches all installed Manta Data Lineage components using the preferred startup mode. If the components are installed as services, Launcher will start them as OS services, if not, Launcher will start the components using the startup scripts. Manta Launcher should be used with the provided bash scripts for Unix-based operating systems and batch scripts for Windows. The scripts are located in <MANTA_HOME>/launcher/bin/*.[sh|bat].

Since the application is a command-line application, it runs in a shell. The user can find out whether or not the application has successfully started from the printed messages and the log files.

For ease of use, Windows operating systems have shortcuts that start the shell or symbolic links in Unix-based systems pointing to the scripts starting Manta Launcher in the installation folder. The shortcuts/symlinks are created based on the mode of installation of the Manta Data Lineage components.

• If all the components (Manta Flow Server, Manta Admin UI, and Manta Keycloak) are installed as services, then the following shortcuts/symlinks are created in the installation folder.

  • Manta Launcher - Start services / launchmantaservices

  • Manta Launcher - Shutdown services / shutdownmantaservices

  • Manta Launcher - Kill processes / killmantaprocesses

• If any of the components are not installed as a service, then the following shortcuts/symlinks are created in the installation folder.

  • Manta Launcher - Start / launchmantaapplications

  • Manta Launcher - Shutdown / shutdownmantaapplications

  • Manta Launcher - Kill processes / killmantaprocesses

Administration of Manta Data Lineage Component Processes

This section discusses the administration of Manta Data Lineage component processes. All of the following administration procedures are automated by Manta Launcher, but they can also be performed manually, if required.

Dependencies between Manta Data Lineage Components

Some Manta Data Lineage components need other components to run in order to be fully functional. Manta Launcher is aware of these dependencies and starts the applications in the correct order so that all the applications are fully functional during startup and afterwards.

If you need to start up the Manta Data Lineage components manually, the correct order is:

1. Manta Keycloak / Manta ActiveMQ Artemis

2. Manta Configuration service

3. Manta Flow Server / Manta Admin UI / Open Manta Designer

To shut down gracefully, the order of these applications should be reversed.

Starting Manta Data Lineage Components Using Startup Scripts

Each of the Manta Data Lineage components has a custom startup script that will start the application as a new process. This method of starting a Manta Data Lineage application is generally used when the application is not installed as a service. Manta Launcher launches the application in the correct dependency order—it launches an application, waits until it is responsive, and then launches the next application.

The list of scripts used to start the applications is in the table below.

Shutting Down Manta Data Lineage Components Using Scripts

Each of the Manta Data Lineage components has a bash/batch script that should be used to shut down the applications gracefully. This method of shutting down applications is generally used when the application is not installed as a service. Manta Launcher shuts the applications down in the correct dependency order, which is the reverse of the startup order—it gracefully shuts the application down, waits until it is not responsive, and then gracefully shuts down the next application.

The list of scripts used to shut down the applications is in the table below.

Starting Manta Data Lineage Components as OS Services

Each Manta Data Lineage component can also be started as a service, if it is installed as such. Manta Launcher uses the command net start <SERVICE_NAME> (see the documentation for the net command, including exit codes and other possible errors that would be outputted to the appropriate stdout log file) to start the service on Windows and the command service <SERVICE_NAME> start (see the documentation for the service command used to administer systemd services) on Unix systems. The algorithm for starting a service is the same as it is when starting an application using a script—Launcher starts the service, waits until it is responsive, and then launches the next service.

Shutting Down Manta Data Lineage Services

Each Manta Data Lineage service can also be gracefully shut down using Manta Launcher. Manta Launcher uses the command net stop <SERVICE_NAME> to gracefully shut the service down on Windows and the command service <SERVICE_NAME> stop on Unix systems. The algorithm for gracefully shutting down a service is the same as it is when shutting down an application using a script—Launcher gracefully shuts the service down, waits until it is not responsive, and then proceeds with the next application.

Shutting Down Manta Data Lineage Applications Forcefully

Manta Launcher also provides an option to forcefully shut down all Manta Data Lineage processes. This option should only be used if the graceful shutdown is not successful. The algorithm determines the process ID of the running Manta Data Lineage process and then kills it using the appropriate OS-specific command.

HTTPS Configuration of Manta Data Lineage Components

If other Manta Data Lineage applications such as Manta Flow Server, Manta Admin UI, or Keycloak are running on HTTPS, Manta Launcher requires changes in its configuration for its health checks to work properly. To change the health check URLs, follow these steps.

1. Run the <MANTA_HOME>/launcher/bin/shutdown.[sh|bat] shutdown script to shut down the Manta Data Lineage platform.

2. Open <MANTA_HOME>/launcher/manta-launcher-dir/conf/applications.json in any text editor.

3. Locate the JSON object of the application that you wish to change the configuration for. The object always contains an attribute name with the name of the application.

4. Change the protocol in the attributes healthEndpoint and url to https.

5. Run the <MANTA_HOME>/launcher/bin/startup.[sh|bat] startup script to launch the Manta Data Lineage platform again.

Optionally, if using an untrusted self-signed certificate, the following steps will also be required.

1. Run the <MANTA_HOME>/launcher/bin/shutdown.[sh|bat] shutdown script to shut down the Manta Data Lineage platform.

2. Open <MANTA_HOME>/launcher/manta-launcher-dir/conf/applications.json in any text editor.

3. Add the following line to the file: process.verification.health.endpoint.verify-hostname=false

4. Run the <MANTA_HOME>/launcher/bin/startup.[sh|bat] startup script to launch the Manta Data Lineage platform again.