System services

A system service runs internal processes for IBM® Spectrum Symphony software.

System services may have multiple concurrent service instances running on multiple hosts. All system services (except for Derby database) are automatically enabled by default at installation.
By default, management services that are installed with EGO include the following services:
System service Process Description
Cluster management console WEBGUI A system service that runs the cluster management console.
EGO ExecProxy ExecProxy A service that allows you to run a script with Admin credential. The ExecProxy service is not supported by Simplified WEM.
EGO service director ServiceDirector A system service that functions as a locating mechanism for other system services.
EGO web service gateway WebServiceGateway A system service that provides a standards-based web services interface for web service clients (applications) to contact EGO.
EGO repository service RS A system service that manages package deployment. Allows deployment of a service without reliance on a shared file system.
Elasticsearch client node elk-elasticsearch A system service for the Elasticsearch client node. The client node cannot hold data or become the primary node. It behaves as a smart router, and is used to forward cluster-level requests to the primary node and data-related requests (such as search) to the appropriate data nodes.

IBM Spectrum Symphony uses Elasticsearch to generate Explorer reports.
Elasticsearch primary node elk-elasticsearch-master The main Elasticsearch system service that all Elasticsearch services depend on, directly as a specified dependency or indirectly through another dependent service. The Elasticsearch primary node monitor ensures that the Elasticsearch primary node service becomes TENTATIVE when the Elasticsearch cluster is red (unavailable), and moves to READY only when the Elasticsearch cluster is yellow or green.

IBM Spectrum Symphony uses Elasticsearch to generate Explorer reports.
Elasticsearch data node elk-elasticsearch-data A system service that performs data related operations such as CRUD, search, and aggregations.

IBM Spectrum Symphony uses Elasticsearch to generate Explorer reports.
Elastic Stack management service elk-manager A system service to delete Elasticsearch indices, and to manage Elasticsearch related EGO services.

IBM Spectrum Symphony uses Elasticsearch to generate Explorer reports.
Filebeat elk-shipper A system service that harvests and forwards log files to the elk-indexer service.

The elk-shipper and elk-indexer services are not currently used by IBM Spectrum Symphony; however, they are related service sand may be used for future features or extensions to the Explorer reports.
Logstash elk-indexer A system service that parses log files and forwards them to Elasticsearch to index.

The elk-shipper and elk-indexer services are not currently used by IBM Spectrum Symphony; however, they are related services and may be used for future features or extensions to the Explorer reports.
Explorer reports service Explorer A system service used by the Explorer reporting feature.
Reporting framework data purger purger A system service used by the reporting feature.
Reporting framework loader controller plc A system service used by the reporting feature. Manages the data loaders that gather data from the system and writes the data into the database.
Reporting framework Derby database derbydb A small-footprint open source database that runs as a system service when first installed. This database is only appropriate for demo clusters. This service is only enabled if an environment variable is set prior to installation (Linux®) or during the Windows installation.
REST web server REST A system service that manages the REST web server, which hosts the RESTful APIs for EGO resource management, package deployment, and SOAM MapReduce application resource management.
Session director SD The session director acts as a liaison between the client application and the session manager. There is one session director process per cluster and it runs on the primary or a management host.
Session manager SSM Primary workload scheduler for an application. There is one session manager per application.
User space socket USSD A system service for the User Space Sockets (USS) library to support Infiniband hardware. Manages communication with the USS library.
Space daemon Not applicable A system service for an Infiniband network configured on a compute host.
There may be other services based on your entitlement configuration. By default, management services that are installed with the Standard and Advanced Editions include the following services:
System service Process IBM Spectrum Symphony Edition Description
IS Orchestrator IS Standard and Advanced Only with virtual server harvesting: The IS service connects to the IBM Spectrum Symphony primary host and to each configured VMware vCenter or Citrix pool primary host. After the connections have been established, the service identifies all the IBM Spectrum Symphony virtual machines in the virtual clusters/pools. It then periodically monitors the VMware clusters and Citrix pools to gather cluster-level and physical host-level utilization metrics (CPU and memory) and compare them to preconfigured thresholds.
Client REST server SYMREST Advanced A long-running service that connects to an IBM Spectrum Symphony client through RESTful APIs and submits application workload to the cluster.
Host factory service for cloud bursting HostFactory Advanced A long-running service that runs on the primary management host and handles resource scale-out and scale-in requests for the cluster to and from external cloud providers. Only one host factory service instance exists per cluster.
IBM Spectrum Symphony Multicluster Proxy SMCP Advanced Only with Multicluster: The SMCP service is installed on management hosts to discover hosts in the cluster and trigger actions to repurpose them. There is one Multicluster proxy per cluster.
Shuffle service MRSS Advanced Only with the MapReduce framework: MRSS is a MapReduce data transfer daemon which runs as an IBM Spectrum Symphony service on each host. The daemon optimizes local memory and disk usage to facilitate faster shuffling processes for map task output data to local and remote reduce tasks.
  • Name Node
  • Secondary Node
  • Data Node
  • NameNode
  • SecondaryNode
  • DataNode
 Advanced Only with the MapReduce framework: When high-availability is configured for the Hadoop Distributed File System (HDFS), the following HDFS daemons run as services in the MapReduce framework:
  • NameNode: Is the primary HDFS daemon. Manages the file system namespace and regulates access to clients. It executes file system operations and tracks where file data is stored across the cluster.

    For the NameNode (which is a single point of failure in the HDFS cluster), IBM Spectrum Symphony provides high availability through EGO, which runs HDFS daemons as services in the MapReduce framework. In case of a failure, EGO restarts them on the same host or on another host.

  • Secondary NameNode: Creates checkpoints of the namespace by merging the edits file into the fsimage file, which can be used if the NameNode fails. When the NameNode is started, the SecondaryNameNode service is automatically started.
  • DataNode: Stores and retrieves data blocks when told to (by clients or the NameNode), and reports back to the NameNode periodically with lists of blocks that it is storing. When the NameNode is started, the DataNode service is automatically started.
Grid synchronization agent RSA Advanced A repository server agent running on compute hosts for grid synchronization of data sets.
YARN integration service EGOYARN Advanced A system service that enables YARN integration with IBM Spectrum Symphony. YARN is a framework for job scheduling and cluster resource management.

By default, system services are registered to the ManagementServices > EGOManagementServices consumer that uses the ManagementHosts resource group. The MapReduce services, however, are registered to the ManagementServices > MapReduceConsumer consumer. You do not need to register any non-system services to consumers using the ManagementHosts resource group, although it is recommended that you do so if your service is a scheduler or controlling service instead of a regular application (management hosts are not expected to execute workload units for users, but are configured to run important services instead).

Each started system service requires one slot, assuming the system service is configured for one instance (default setting). By default, EGO allots twelve slots per management host to run system services, although only seven (or eight if derbydb is automatically enabled at installation) are actually used out-of-the-box by the installed management services previously listed. For maximum performance in case of failover, you must retain seven or eight of the ten slots on each primary host candidate for use by the system services; the others can be used by non-system services that you might register that require a management host.

Slots assigned to run system services for consumers the ManagementServices consumer branch do not have to be on the primary host (for example, any management host in the cluster can be the web server); the cluster determines where the system service instance runs at startup.

Registered system services do not require a host, only the system service instance that it runs.

Service profile

A service profile is an XML file (configured through the cluster management console) that contains the service definition for a service; when you update a system service, you are updating the system service profile containing the system service definition. Every service, whether it is installed by default (a system service) or added by an administrator, has a service profile. The XML file lets you configure properties for the service such as :

  • The maximum and minimum number of service instances required by the service
  • A service instance description, which describes how to start a service instance for this service
  • Any resources required to run the service instances for this service
  • Any dependencies this service has (such as another service being started)
Note: We do not recommend changing the XML file for any of the system services (those installed by default) unless specifically told to do so. You may need to change ports.

Service name

If you plan on running a custom service as an EGO service, the service naming convention specifies that the maximum length of the service name is 200 characters and the first character can only be an alphabet. The remaining characters can be alphabets, digits, and hyphens.

Troubleshooting system services

If a system service has the state error or unknown, check the log files to troubleshoot. The logs for the EGO service controller, the load information manager, and the VEM kernel daemon are good places to start.