Multiprotocol export considerations

Exports for SMB and NFS protocols can be configured so that they have access to the same data in the file system. In addition, the data can be accessed directly in the file system on the cluster nodes. When configuring access to the same GPFS file system via both the NFS and SMB protocols, certain limitations apply.

These restrictions apply to the general areas of file locking (including share reservation and lock semantics), recovery (reclaim), and cross-protocol notifications.

Access Control Lists (ACLs): In IBM Storage Scale, there is a single common ACL per file or directory in the cluster file system that is used for POSIX, NFS, and SMB access. The SMB server converts each Windows ACL into an NFS4 ACLs for the corresponding file system object.

Shared access (share modes, share reservations): Share modes are feature of the SMB protocol that allows clients to announce what type of parallel access should be allowed by other clients while the file is open. NFSv4 share reservations are the equivalent of SMB share modes for the NFS protocol. There is no equivalent in NFSv3 but IBM Storage Scale allows the SMB server to propagate the share modes into the cluster file system so that NFS clients can honor share modes on commonly used files. The corresponding SMB option is gpfs:sharemodes.NFSv4 share reservations are currently not supported.

Note that disabling the SMB option gpfs:sharemodes can result in data integrity issues, as SMB application can rely on the enforcement of exclusive access to data to protect the integrity of a file's data. As the SMB file server also does sharemode checks internally, gpfs:sharemodes can safely be disabled for data that is only accessed through the SMB protocol.

The important point for POSIX and NFS applications is that file system sharemodes can result in the open() or unlink() system calls to return EACCES. Applications must be prepared to handle this situation.

Details for the interaction of SMB with the file system sharemodes:

When an SMB client requests to open a file, it must specify the allowed share modes. The share modes are specified with the FILE_SHARE_READ, FILE_SHARE_WRITE and FILE_SHARE_DELETE flags; each one of those indicate which parallel access is allowed while the SMB client has the file open.
  • If FILE_SHARE_READ is not allowed and another application requests to access the same file for reading data (through the POSIX system call open() with the O_RDONLY flag) then the system call fails with the EACCES error code.
  • If FILE_SHARE_WRITE is not allowed and another application requests to access the same file for writing data (through the POSIX system call open() with the O_WRONLY flag), then the system call fails with the EACCES error code.
  • If FILE_SHARE_DELETE is not allowed and another application requests to delete the file through the unlink() system call, the system call fails with the EACCES error code.

If another application already has the file open for reading and writing, and the specified share mode from the SMB client conflicts with the existing open, then the SMB client cannot open the file and a "sharing violation" error is returned back to the SMB client.

Share modes in the file system are only enforced if the file is actually opened for READ, EXECUTE, WRITE or APPEND access from the SMB client.

Another limitation of the share mode enforcement in the file system is that it is not possible to grant parallel FILE_SHARE_READ and FILE_SHARE_WRITE access, while not granting FILE_SHARE_DELETE access. In this case, the file system does not enforce that the FILE_SHARE_DELETE restriction and the file can still be deleted.

These limitations only apply to enforcement of sharemodes in the file system. The SMB server also performs internal sharemode checks and handles the sharemode correctly for all SMB access.

Note: The CES NFS server keeps the files accessed by NFSv3 open for a while for performance reasons. This might lead to conflicts during concurrent SMB access to these files. You can use the following command to find out whether the NFS server holds the specified file open: 
ls /proc/$(pidof gpfs.ganesha.nfsd)/fd -l | grep <file-name>

Opportunistic Locking: Oplocks are a feature of the SMB protocol that allows clients to cache files locally on the client. If the SMB server is set to propagate oplocks into the cluster file system (gpfs:leases), other clients (NFS, POSIX) can break SMB oplocks. NFS4 delegations are currently not supported.

Byte-range locks: Byte-range locks from SMB clients are propagated into the cluster file system if the SMB option "posix locking" is true. In that case, POSIX and NFS clients are made aware of those locks. Note that for Windows byte-range locks are mandatory whereas for POSIX they are advisory.

File change notifications: SMB clients can request notifications when objects change in the file system. The SMB server notifies its clients about the changes. The notifications include changes that are triggered by POSIX and NFS clients in the directory for which notifications are requested, but not in its subdirectories, if they are done on any CES node. File changes initiated on non-CES cluster nodes will not trigger a notification.

Grace period: The grace period allows NFS clients to reclaim their locks and state for a certain amount of time after a server failure. SMB clients are not aware of NFS grace periods. If you expect a lot of contention between SMB and NFS, NFSv4 reclaims might fail.

Multiprotocol access of protocol exports is only allowed between NFSV4 and SMB. That is, you cannot access the same export by using both NFSV3 and SMB protocols. The reason is that SMB clients typically request exclusive access to a file which does not work with the CES NFS server that keeps files accessed through NFSv3 open.