Managing the Spark engine details

IBM® watsonx.data allows you to view and edit the details of a Spark engine. You can also monitor the status of the applications that are submitted in the instance.

watsonx.data on IBM Software Hub

About this task

Viewing Spark details
You can view the Spark details in list and topology views.
  1. Click the name of Spark engine (either from list or topology view). Engine information window opens.
  2. In the Details tab, you can view the following details:
    Field Description
    Display name The Spark engine name.
    Engine ID The unique identifier of the Spark instance.
    Description The description of the engine.
    Tags The tag that is specified at the time of registering an engine.
    Type The engine type.
    Instance URL The Spark URL.
    watsonx.data application endpoint The application submission endpoint. To submit an application by using API, see API Docs.
    Instance API endpoint The IBM Analytics Engine (Spark) API endpoint.
    History server endpoint The IBM Analytics Engine (Spark) history server endpoint.
    History server The history server URL, where you can view the history details of the applications that are run.
Editing Spark details
You can edit the Spark details in list and topology views.
  1. Click the name of Spark engine (either from list or topology view). Engine information window opens.
  2. In the Details tab, click Edit.
  3. In the Display name field, enter the display name for the Spark engine.
  4. In the Description field, enter the description of the engine or edit the existing description.
  5. In the Tags field, select the tags from the list or start typing to define a new tag.
  6. Click Save.Click the name of Spark engine (either from list or topology view). Engine information window opens.
Viewing the submitted Spark applications in watsonx.data
You can view the status of the submitted Spark applications in list and topology views.
  1. Click the name of the Spark engine (either from list or topology view). Engine information window opens.
  2. In Applications tab, you can view the list of all applications that are submitted to watsonx.data. The tab also displays the details such as the status of each application, the Spark version that is used for running the application, creation date, and run date.