IBM Support

How to add compute/deployment node to an existing Watson Studio Local 1.2.3.x cluster?

Question & Answer


Question

In the Watson Studio Local 1.2.3.x Admin Console - Nodes, we can find an [Add node] button. Could you tell me how to add node to an existing Watson Studio Local 1.2.3.x cluster by using this feature?
image-20230703084439-1

Answer

To add new node to an existing Watson Studio Local 1.2.3.x cluster, you can follow these steps:

1. Create new Linux server (w123hs-compute-4)
2. Confirm the disk structure
[root@w123hs-compute-4 ~]# lsblk
NAME          MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
vda           252:0    0  250G  0 disk
├─vda1        252:1    0    1G  0 part /boot
└─vda2        252:2    0  249G  0 part
  ├─rhel-root 253:0    0  233G  0 lvm  /
  └─rhel-swap 253:1    0   16G  0 lvm  [SWAP]
vdb           252:16   0  250G  0 disk /ibm

[root@w123hs-compute-4 ~]# cat /etc/fstab
/dev/mapper/rhel-root                     /                       xfs     defaults        0 0
UUID=afcf9c6c-e713-4ae2-b59e-380b77d83744 /boot                   xfs     defaults        0 0
UUID=48eed125-9d6d-4cc9-b08a-06e719d0b66d /ibm                    xfs     defaults        0 0
/dev/mapper/rhel-swap                     swap                    swap    defaults        0 0
3. Confirm /etc/hosts on all nodes in the cluster has entry for the new node
[root@w123hs-compute-4 ~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
10.11.86.78  w123hs-compute-4.fyre.ibm.com w123hs-compute-4
10.11.64.71  w1235s-proxy.fyre.ibm.com w1235s-proxy
10.11.67.140 w1235s-compute-1.fyre.ibm.com w1235s-compute-1
10.11.67.169 w1235s-compute-2.fyre.ibm.com w1235s-compute-2
10.11.68.21  w1235s-compute-3.fyre.ibm.com w1235s-compute-3
10.11.67.76  w1235s-deploy-1.fyre.ibm.com w1235s-deploy-1
10.11.64.245 w1235s-mstr-1.fyre.ibm.com w1235s-mstr-1
10.11.64.245 w1235s-mstr-1.fyre.ibm.com w1235s-mstr-1
10.11.66.117 w1235s-mstr-2.fyre.ibm.com w1235s-mstr-2
10.11.66.117 w1235s-mstr-2.fyre.ibm.com w1235s-mstr-2
10.11.66.123 w1235s-mstr-3.fyre.ibm.com w1235s-mstr-3
10.11.66.123 w1235s-mstr-3.fyre.ibm.com w1235s-mstr-3
4. Confirm /etc/ntp.conf have same setting with other nodes
[root@w123hs-compute-4 ~]# cat /etc/ntp.conf
server fintpsvl1p.fyre.ibm.com iburst
[root@w123hs-compute-4 ~]# ntpq -p
     remote           refid      st t when poll reach   delay   offset  jitter

==============================================================================
 fintpsvl1p.fyre .INIT.          16 u    -   64    0    0.000    0.000   0.000
5. Enable IP forwarding on the new node
[root@w123hs-compute-4 ~]# vi /etc/sysctl.conf
[root@w123hs-compute-4 ~]# sysctl -p
net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1
net.ipv6.conf.lo.disable_ipv6 = 1
net.ipv4.ip_forward = 1
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.default.send_redirects = 0
net.ipv4.conf.all.accept_source_route = 0
net.ipv4.conf.default.accept_source_route = 0
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.all.secure_redirects = 0
net.ipv4.conf.default.secure_redirects = 0
net.ipv4.conf.all.log_martians = 1
net.ipv4.conf.default.log_martians = 1
net.ipv6.conf.all.accept_ra = 0
net.ipv6.conf.default.accept_ra = 0
net.ipv6.conf.all.accept_redirects = 0
net.ipv6.conf.default.accept_redirects = 0
net.ipv4.conf.all.rp_filter = 0
net.ipv4.conf.default.rp_filter = 0
6. Copy docker_redhat_install.sh and pre_install_check.sh to /ibm
[root@w1235s-compute-1 ibm]# scp ./docker_redhat_install.sh root@w123hs-compute-4:/ibm/
[root@w1235s-compute-1 ibm]# scp ./pre_install_check.sh root@w123hs-compute-4:/ibm/
[root@w123hs-compute-4 ibm]# ls
docker_redhat_install.sh  pre_install_check.sh
7. Install docker by using docker_redhat_install.sh
[root@w123hs-compute-4 ibm]# ./docker_redhat_install.sh /ibm
8. Confirm all prerequisite are PASS
[root@w123hs-compute-4 ibm]# ./pre_install_check.sh --type=6nodes_compute
...
Finished with WARNING, please check /tmp/preInstallCheckResult
9. Confirm that the cluster is working fine and all nodes are fine
[root@w1235s-mstr-1 ibm]# gluster pool list
UUID                                    Hostname        State
d7e4c8fc-235c-4fee-9ff6-a97c4c87907a    10.11.67.169    Connected
222f570d-aa28-4bfb-8ee5-74a918593f53    10.11.66.117    Connected
316de6fd-e50c-4861-8431-139bab100bc1    10.11.67.76     Connected
fff47faa-d0e0-4143-9cd1-c59c69766a5e    10.11.68.21     Connected
be339ca3-91d8-468a-8377-27f66ea6a820    10.11.66.123    Connected
5fdffee5-c660-48ca-babb-61160c783f35    10.11.67.140    Connected
841a2c4f-eb40-4d23-8b19-1764504e1de5    localhost       Connected
[root@w1235s-mstr-1 ibm]#
[root@w1235s-mstr-1 ibm]# kubectl get nodes
NAME                            STATUS   ROLES    AGE   VERSION
w1235s-compute-1.fyre.ibm.com   Ready    <none>   28d   v1.13.5
w1235s-compute-2.fyre.ibm.com   Ready    <none>   28d   v1.13.5
w1235s-compute-3.fyre.ibm.com   Ready    <none>   28d   v1.13.5
w1235s-deploy-1.fyre.ibm.com    Ready    <none>   28d   v1.13.5
w1235s-mstr-1.fyre.ibm.com    Ready    <none>   28d   v1.13.5
w1235s-mstr-2.fyre.ibm.com    Ready    <none>   28d   v1.13.5
w1235s-mstr-3.fyre.ibm.com    Ready    <none>   28d   v1.13.5
[root@w1235s-mstr-1 ibm]#
[root@w1235s-mstr-1 ibm]# kubectl get pod --all-namespaces|grep "0/"
dsx           dash-post-install-job-69ddl                     0/1     Completed   0          28d
dsx           dsx-influxdb-set-auth-f9nxl                     0/1     Completed   2          28d
 10. Access WSL Admin Console and then click Add Node

image-20210724163545-1

image-20210724163545-2
11. Confirm every step run without any error

image-20210724163545-3

image-20210724163545-4

 12. Confirm glusterd, docker, and kubelet services on the new node are running
[root@w123hs-compute-4 ibm]# systemctl status glusterd
● glusterd.service - GlusterFS, a clustered file-system server
   Loaded: loaded (/usr/lib/systemd/system/glusterd.service; enabled; vendor preset: disabled)
   Active: active (running) since Thu 2021-05-06 16:46:00 PDT; 5min ago
 Main PID: 11495 (glusterd)
[root@w123hs-compute-4 ibm]#
[root@w123hs-compute-4 ibm]# systemctl status docker
● docker.service - Docker Application Container Engine
   Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled; vendor preset: disabled)
   Active: active (running) since Thu 2021-05-06 07:08:55 PDT; 9h ago
     Docs: http://docs.docker.com
 Main PID: 1415 (dockerd-current)
[root@w123hs-compute-4 ibm]#
[root@w123hs-compute-4 ibm]# systemctl status kubelet
● kubelet.service
   Loaded: loaded (/etc/systemd/system/kubelet.service; enabled; vendor preset: disabled)
   Active: active (running) since Thu 2021-05-06 16:46:00 PDT; 5min ago
 Main PID: 11529 (kubelet)
 13. Confirm that new node is listed on the cluster
[root@w1235s-mstr-1 ibm]# kubectl get nodes
NAME                            STATUS   ROLES    AGE     VERSION
w1235s-compute-1.fyre.ibm.com   Ready    <none>   28d     v1.13.5
w1235s-compute-2.fyre.ibm.com   Ready    <none>   28d     v1.13.5
w1235s-compute-3.fyre.ibm.com   Ready    <none>   28d     v1.13.5
w1235s-deploy-1.fyre.ibm.com    Ready    <none>   28d     v1.13.5
w1235s-mstr-1.fyre.ibm.com    Ready    <none>   28d     v1.13.5
w1235s-mstr-2.fyre.ibm.com    Ready    <none>   28d     v1.13.5
w1235s-mstr-3.fyre.ibm.com    Ready    <none>   28d     v1.13.5
w123hs-compute-4.fyre.ibm.com   Ready    <none>   4m27s   v1.13.5
 14. Confirm that new node is listed on the Admin Console

image-20210724163545-5


[{"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSHGWL","label":"IBM Watson Studio Local"},"Component":"WSL 1.2.3.1","Platform":[{"code":"PF016","label":"Linux"}],"Version":"WSL 1.2.3.1","Edition":"None","Line of Business":{"code":"LOB10","label":"Data and AI"}}]

Document Information

More support for:
IBM Watson Studio Local

Software version:
WSL 1.2.3.1

Operating system(s):
Linux

Document number:
967583

Modified date:
02 July 2023

UID

ibm10967583