Troubleshooting
Problem
Gluster mount dsx-user-home fails with transport endpoint is not connected.
observed the following messages doing a df -h command.
root@adsx001:/root: df -h | more
df: â/ibm/ibm-data-platform/kubelet/pods/17be5108-e445-11e8-9d3c-005056835e7d/volumes/kubernetes.io~glusterfs/dsx-user-homeâ: Transport endpoint is not connected
df: â/ibm/ibm-data-platform/kubelet/pods/a9e82f11-291b-11e9-86c4-00505683c59c/volumes/kubernetes.io~glusterfs/dsx-redisâ: Transport endpoint is not connected
df: â/ibm/ibm-data-platform/kubelet/pods/5eec535c-2948-11e9-a2e5-005056835e7d/volumes/kubernetes.io~glusterfs/dsx-user-homeâ: Transport endpoint is not connected
df: â/ibm/ibm-data-platform/kubelet/pods/5f1e27d4-2948-11e9-a2e5-005056835e7d/volumes/kubernetes.io~glusterfs/dsx-user-homeâ: Transport endpoint is not connected
df: â/ibm/ibm-data-platform/kubelet/pods/c52b910c-2999-11e9-a2e5-005056835e7d/volumes/kubernetes.io~glusterfs/dsx-user-homeâ: Transport endpoint is not connected
df: â/ibm/ibm-data-platform/kubelet/pods/ae0ef331-2fd4-11e9-930c-005056835a11/volumes/kubernetes.io~glusterfs/dsx-user-homeâ: Transport endpoint is not connected
df: â/ibm/ibm-data-platform/kubelet/pods/ae9014be-2fd4-11e9-930c-005056835a11/volumes/kubernetes.io~glusterfs/dsx-user-homeâ: Transport endpoint is not connected
Filesystem Size Used Avail Use% Mounted on
Symptom
Gluster mount dsx-user-home fails with transport endpoint is not connected.
observed the following messages doing a df -h command.
root@adsx001:/root: df -h | more
df: â/ibm/ibm-data-platform/kubelet/pods/17be5108-e445-11e8-9d3c-005056835e7d/volumes/kubernetes.io~glusterfs/dsx-user-homeâ: Transport endpoint is not connected
df: â/ibm/ibm-data-platform/kubelet/pods/a9e82f11-291b-11e9-86c4-00505683c59c/volumes/kubernetes.io~glusterfs/dsx-redisâ: Transport endpoint is not connected
df: â/ibm/ibm-data-platform/kubelet/pods/5eec535c-2948-11e9-a2e5-005056835e7d/volumes/kubernetes.io~glusterfs/dsx-user-homeâ: Transport endpoint is not connected
df: â/ibm/ibm-data-platform/kubelet/pods/5f1e27d4-2948-11e9-a2e5-005056835e7d/volumes/kubernetes.io~glusterfs/dsx-user-homeâ: Transport endpoint is not connected
df: â/ibm/ibm-data-platform/kubelet/pods/c52b910c-2999-11e9-a2e5-005056835e7d/volumes/kubernetes.io~glusterfs/dsx-user-homeâ: Transport endpoint is not connected
df: â/ibm/ibm-data-platform/kubelet/pods/ae0ef331-2fd4-11e9-930c-005056835a11/volumes/kubernetes.io~glusterfs/dsx-user-homeâ: Transport endpoint is not connected
df: â/ibm/ibm-data-platform/kubelet/pods/ae9014be-2fd4-11e9-930c-005056835a11/volumes/kubernetes.io~glusterfs/dsx-user-homeâ: Transport endpoint is not connected
Filesystem Size Used Avail Use% Mounted on
Cause
Appeared to be a stale mount:
tried navigating to one of these shares.
root@a70ldsx001:/ibm/ibm-data-platform/kubelet/pods/17be5108-e445-11e8-9d3c-005056835e7d/volumes/kubernetes.io~glusterfs: ls
ls: cannot access dsx-user-home: Transport endpoint is not connected
dsx-user-home
root@a70ldsx001:/ibm/ibm-data-platform/kubelet/pods/17be5108-e445-11e8-9d3c-005056835e7d/volumes/kubernetes.io~glusterfs: er-home <
-ksh: cd: dsx-user-home: [Transport endpoint is not connected]
ls: cannot access dsx-user-home: Transport endpoint is not connected
dsx-user-home
root@a70ldsx001:/ibm/ibm-data-platform/kubelet/pods/17be5108-e445-11e8-9d3c-005056835e7d/volumes/kubernetes.io~glusterfs: er-home <
-ksh: cd: dsx-user-home: [Transport endpoint is not connected]
Environment
WSL 1.2.2.0 on Linux
Diagnosing The Problem
try navigating to one of these shares.
root@a70ldsx001:/ibm/ibm-data-platform/kubelet/pods/17be5108-e445-11e8-9d3c-005056835e7d/volumes/kubernetes.io~glusterfs: ls
ls: cannot access dsx-user-home: Transport endpoint is not connected
dsx-user-home
root@a70ldsx001:/ibm/ibm-data-platform/kubelet/pods/17be5108-e445-11e8-9d3c-005056835e7d/volumes/kubernetes.io~glusterfs: er-home <
-ksh: cd: dsx-user-home: [Transport endpoint is not connected]
ls: cannot access dsx-user-home: Transport endpoint is not connected
dsx-user-home
root@a70ldsx001:/ibm/ibm-data-platform/kubelet/pods/17be5108-e445-11e8-9d3c-005056835e7d/volumes/kubernetes.io~glusterfs: er-home <
-ksh: cd: dsx-user-home: [Transport endpoint is not connected]
Resolving The Problem
Run the following commands to resolve the issue:
1. systemctl stop kubelet
2. systemctl stop docker
3. systemtl disable docker kubelet
4. reboot
5. systemtl enable docker kubelet
6. systemctl start docker
7. docker system prune (answer y) - this may take a little while
8. systemctl start kubelet
9. Now run the df -h command to make sure the error is gone.
Document Location
Worldwide
[{"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSHGWL","label":"IBM Watson Studio Local"},"Component":"","Platform":[{"code":"PF016","label":"Linux"}],"Version":"1.2.2.0","Edition":"","Line of Business":{"code":"LOB10","label":"Data and AI"}}]
Was this topic helpful?
Document Information
Modified date:
30 December 2019
UID
ibm10875128