IBM Support

How to enable spark history server UI in Watson Studio Local

Question & Answer


Question

How to enable spark history server UI in Watson Studio Local

Answer

  1. Edit spark history server deployment
kubectl -n dsx edit svc spark-history-server-svc

# Please edit the object below. Lines beginning with a '#' will be ignored,
# and an empty file will abort the edit. If an error occurs while saving this file will be
# reopened with the relevant failures.
#
apiVersion: v1
kind: Service
metadata:
  annotations:
    description: The server that maintains a list of historic Spark jobs.
    descriptiveName: Spark history
  creationTimestamp: 2019-02-20T21:30:07Z
  labels:
    app: dsx-ibm-dsx-prod
    chart: ibm-dsx-prod
    component: spark-history-server
    heritage: Tiller
    release: dsx
    run: spark-history-server-svc
  name: spark-history-server-svc
  namespace: dsx
  resourceVersion: "7207"
  selfLink: /api/v1/namespaces/dsx/services/spark-history-server-svc
  uid: b1db043c-3556-11e9-b1e5-00163e01b87c
spec:
  clusterIP: 10.4.136.112
  ports:
  - port: 18080
    protocol: TCP
    targetPort: 18080
  selector:
    component: spark-history-server
    run: spark-history-server-deployment-pod
  sessionAffinity: None
  type: ClusterIP
status:
  loadBalancer: {}

We need to update the spec section to become.

spec:
  clusterIP: 10.4.136.112
  ports:
  - port: 18080
    protocol: TCP
    targetPort: 18080
    nodePort: 30088
  selector:
    component: spark-history-server
    run: spark-history-server-deployment-pod
  sessionAffinity: None
  type: NodePort

You shouldn't change any other field, ie clusterIP.
In git diff view

 spec:
   clusterIP: 10.4.136.112
   ports:
   - port: 18080
     protocol: TCP
     targetPort: 18080
+    nodePort: 30088
   selector:
     component: spark-history-server
     run: spark-history-server-deployment-pod
   sessionAffinity: None
-  type: ClusterIP
+  type: NodePort
  1. Access the history server UI from the following link
http://${clusterIP}:30088/

where clusterIP is the IP we normally use to access the DSX on the cluster from browser.

Related Information

How to video

[{"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSHGWL","label":"IBM Watson Studio Local"},"Component":"","Platform":[{"code":"PF016","label":"Linux"}],"Version":"1.2.x","Edition":"","Line of Business":{"code":"LOB10","label":"Data and AI"}}]

Document Information

Modified date:
30 December 2019

UID

ibm10958075

Page Feedback