IBM Storage Scale container native and SELinux
IBM Storage Scale container native offers a default for Container Storage Interface (CSI) volume attachment behavior for Security-Enhanced Linux (SELinux) labels. This default is introduced beginning with IBM Storage Scale container native 5.1.7.0 release.
What is SELinux?
SELinux, or Security-Enhanced Linux, defines access controls for the applications, processes, and files on a system. It uses security policies, which are a set of rules that tell SELinux what can or cannot be accessed, to enforce the access allowed by a policy.
For more information about SELinux, see What is SELinux? in Red Hat Documentation.
How is SELinux controlled in Kubernetes?
Container processes are started with the SELinux context in the .spec.securityContext.seLinuxOptions field, while CSI-based volumes have all their files labeled to match on pod attachment during container creation.
SELinux Multi-Category Security (MCS) is used in Red Hat OpenShift. By default, containers have their SELinux level set to values annotated on the namespace. The category defaults annotated on the namespace are assigned automatically by OpenShift to limit overlap.
Red Hat OpenShift is based on Kubernetes.
Problems with SELinux relabel on volume attach
When attaching CSI-based volumes to pods in Kubernetes the container-runtime SELinux relabels all files in the volume. This is problematic for shared filesystems that support SELinux, such as the IBM Storage Scale filesystem.
Relabeling on volume attachment moves the security control to the consumer of file volumes instead of the owner of the files. In classic shared filesystem environments, such as IBM Storage Scale, access is controlled by node administrators and file owners. In Kubernetes, volume access is isolated to a namespaced volume claim and controlled using Kubernetes role-based access control (RBAC). While in classic shared filesystem environments, such as IBM Storage Scale, access is controlled by node administrators and file owners. This Kubernetes behavior makes it very difficult to maintain access controls for volumes and files that are shared outside of Kubernetes. This is because Kubernetes ignores and overwrites any SELinux security isolation set by external administrators.
In addition, relabeling all files in a volume is a non-trivial operation, which may generate a large I/O load on the backend storage system. This introduces a performance and denial of service concern. Volumes with too many files, or shared volumes that are attached concurrently, may easily swamp the storage subsystem and cause cascading pod creation timeouts.
Upstream Kubernetes limitation for SELinux relabel
The SELinux relabel issue is not unique to IBM Storage Scale. Upstream Kubernetes does not give CSI volume driver implementations control of SELinux relabel on volume attachment.
Legacy in-tree volume drivers, such as the in-tree NFS volume driver, have the ability to control SELinux relabeling. However, legacy in-tree volume drivers are deprecated.
In Red Hat OpenShift, the container-runtime that performs the SELinux relabel will skip the relabeling if the container SELinux context is set to spc_t, which is the super privileged container. It is an uncontained type and may access
any file on the system allowed by standard file permissions.
Default volume attachment behavior by IBM Storage Scale container native
To prevent Kubernetes from relabeling SELinux file labels, IBM Storage Scale container native by default will mount the filesystem with a container permissive context. This disallows the security.selinux label from being set, and all
files inside the filesystem will be considered to have the context defined on the filesystem mount. By default that context would allow all containers running as container_t SELinux type to access files on the filesystem allowed by
standard file permissions. By default, the mount context is set to system_u:object_r:container_file_t:s0.
Comparing SELinux relabel on attach and SELinux mount context
Security considerations
Setting a container permissive SELinux mount context or doing an SELinux relabel on volume attachment have similar access control within Kubernetes. Any container that can claim a volume may access its files.
However, if a process could escape a container or the host was able to be compromised, then any container_t constrained context would have access to the filesystem from a SELinux access perspective. Standard linux file permissions would
still prevent access. This means containers that run within the "restricted" SecurityContextConstraint defined by OpenShift would not be able to access files in a volume that it did not explicitly share.
If a volume is shared with Kubernetes from an external application, then SELinux relabeling breaks SELinux security isolation managed externally. SELinux relabel on attach allows a volume consumer to ignore and overwrite SELinux labels, potentially exposing the files to other external systems. Setting the SELinux mount context will only allow the external SELinux labels to be ignored within the confines of the Kubernetes cluster.
Performance considerations
Since SELinux mount context disallows SELinux relabeling, volume attachments to pods do not generate extra I/O load and will attach faster than if SELinux relabeling is done.
If SELinux relabeling occurs, it must complete within a minute or the container creation will timeout and fail. Multiple relabel operations occurring concurrently are more likely to trigger timeout and failure. This cascading failure may only be exposed due to wider outages (node, cluster, lab), or maintenance.
External access considerations
External access refers to any components outside of the IBM Storage Scale container native cluster. This includes the storage cluster that is remotely mounted to the IBM Storage Scale container native cluster.
SELinux mount context means all new files are created without SELinux labels, and are considered unlabeled_t. This would include files created by application containers using IBM Storage Scale container native volumes. Generally, access
to unlabeled_t files is only allowed by uncontained or more privileged process contexts. To access the unlabeled files, the external applications outside of Kubernetes must be given access to unlabeled_t via SELinux user
policies, the files must be labeled manually, or the mount used by the application has a valid SELinux context configured.
The behavior without mount context, which does an SELinux relabel of all files, is similar in that the SELinux context of the container should be configured to match what is desired externally, or the external application should be granted access to files created by the container SELinux context.
How to change mount context
The mount context may be set on the Filesystem kind. This applies to all applications using volumes within the filesystem. For more details, run oc explain fs.spec.seLinuxOptions command.
When changing the SELinux context, applications with ReadWriteMany volumes running on nodes with mixed SELinux contexts may experience "Permission Denied" errors. Applications running on nodes with differing SELinux contexts may not have access to each other's files during the duration of the SELinux context update.
Changing the fs.spec.seLinuxOptions field of a Filesystem kind will cause pods to restart, and the restart of the pod will cause a reboot of the node itself. Ensure that the proper maintenance window and precautions, similar to upgrade,
have been taken prior to changing this field.
spec:
seLinuxOptions:
user: <user>
role: <object>
type: <type>
level: <level>
Legacy relabel on attach behavior
Earlier behavior of relabel on attach is considered legacy and the support is limited for it. If this behavior is required, contact IBM support.