Topic
  • 5 replies
  • Latest Post - ‏2012-09-28T20:16:13Z by nhardt
nhardt
nhardt
26 Posts

Pinned topic applying a value with gpfs_fputattrs that was retrieved with gpfs_fgetattrs

‏2012-09-11T22:50:56Z |
Hello,

I am running into an issue using gpfs_fgetattrs and
gpfs_fputattrs. The error I am seeing looks very similar to the issue
at http://www-01.ibm.com/support/docview.wss?uid=isg1IY90477, but
that issue is listed as fixed in GPFS 2.3. The cluster in question is
on version 3.4.0-15.

It appears that the issue is occurring specifically on a file that has
a default ACL but no extended access ACL, but I'm not positive I am
reading the mmgetacl output correctly or using mmeditacl appropriately.

I have created a small test program that reads the opaque attr
structure with gpfs_fgetattrs and writes the attr back with gpfs_fputattrs,
and so far it fails consistently in this specific situation.

Here is the test program:

////////////////////////////////////////////////////////////////////////////////
#include <gpfs.h>
#include <iostream>
#include <stdlib.h>
#include <errno.h>
#include <fcntl.h>

using namespace std;

void usage()
{
cout << "this program will attempt to read the attributes "
"of a file then set those values back to the same file. takes "
"the file name as the only argument"
<< endl;
}

int main(int argc, char *argv[])
{
if (argc != 2)
{
usage();
return 1;
}
// NOTE: letting these fds leak in favor of brevity
int fd = open(argv[1], O_RDWR);
if (fd == -1)
{
cerr << "could not open " << argv[1] << endl;
return 1;
}

char buf4096;
int size;

if (gpfs_fgetattrs(
fd, GPFS_ATTRFLAG_NO_PLACEMENT, buf, sizeof(buf), &size) == -1)
{
cerr << "got " << errno << " from gpfs_fgetattrs" << endl;
return 1;
}

if (size == 0)
{
if (gpfs_fputattrs(fd, GPFS_ATTRFLAG_NO_PLACEMENT, NULL) == -1)
{
cerr << "Unable to apply extended attributes to: "
<< argv[0]
<< endl;
return 1;
}
}
else
{
if (gpfs_fputattrs(fd, GPFS_ATTRFLAG_NO_PLACEMENT, buf) == -1)
{
cerr << "Unable to apply extended attributes to: "
<< argv[0]
<< endl;
return 1;
}
}
return 0;
}
////////////////////////////////////////////////////////////////////////////////
To reproduce the issue, I followed these steps:

--
First, the working case, on file with some nfsv4 ACEs.
--

root@localhost # mmgetacl ./dir3/file0
#NFSv4 ACL
#owner:root
#group:root
special:owner@:rw-c:allow
(X)READ/LIST (X)WRITE/CREATE (-)MKDIR (X)SYNCHRONIZE (X)READ_ACL (X)READ_ATTR (-)READ_NAMED
(-)DELETE (-)DELETE_CHILD (X)CHOWN (-)EXEC/SEARCH (X)WRITE_ACL (X)WRITE_ATTR (-)WRITE_NAMED

special:group@:r---:allow
(X)READ/LIST (-)WRITE/CREATE (-)MKDIR (X)SYNCHRONIZE (X)READ_ACL (X)READ_ATTR (-)READ_NAMED
(-)DELETE (-)DELETE_CHILD (-)CHOWN (-)EXEC/SEARCH (-)WRITE_ACL (-)WRITE_ATTR (-)WRITE_NAMED

special:everyone@:r---:allow
(X)READ/LIST (-)WRITE/CREATE (-)MKDIR (X)SYNCHRONIZE (X)READ_ACL (X)READ_ATTR (-)READ_NAMED
(-)DELETE (-)DELETE_CHILD (-)CHOWN (-)EXEC/SEARCH (-)WRITE_ACL (-)WRITE_ATTR (-)WRITE_NAMED

root@localhost # ~/test_gpfs_fputattrs ./dir3/file0
root@localhost # echo $?
0

--

Next I edit the acl, and remove everything but the owner and group
info. After that, I can no longer apply the attribute structure that
gpfs is giving me.

--

root@localhost # mmeditacl ./dir3/file0
mmeditacl: Should the modified ACL be applied? (yes) or (no)
yes

root@localhost # mmgetacl ./dir3/file0
#NFSv4 ACL
#owner:root
#group:root

root@localhost # ~/test_gpfs_fputattrs ./dir3/file0
Unable to apply extended attributes to: /root/test_gpfs_fputattrs

--
So, a few questions.

First, is that a valid ACL (I've seen it occur under normal usage in
the interaction between samba and gpfs, I'm assuming it is)? Second,
can you validate this as a bug, or point out how this is misusing the
API? Third, any advice as to what I should I do about this issue?

Thanks,
Nate Hardt
Updated on 2012-09-28T20:16:13Z at 2012-09-28T20:16:13Z by nhardt
  • nhardt
    nhardt
    26 Posts

    Re: applying a value with gpfs_fputattrs that was retrieved with gpfs_fgetattrs

    ‏2012-09-12T16:50:38Z  
    Looks like there were some formatting issues with pasting the code. Please see attached.
  • LuoChen
    LuoChen
    14 Posts

    Re: applying a value with gpfs_fputattrs that was retrieved with gpfs_fgetattrs

    ‏2012-09-12T22:08:40Z  
    • nhardt
    • ‏2012-09-12T16:50:38Z
    Looks like there were some formatting issues with pasting the code. Please see attached.
    I wasn't be able to recreate your problem. There might be something else going on.
    Please change the error message for gpfs_fputattr() to distinguish the one with NULL buffer, and the one with non-empty buffer.

    Also, what version of GPFS you are using, on which OS?

    Take gpfs traces will help as well. try the following:
    mmtrace start
    reproduce the problem
    mmtrace stop

    then attach the generated trace file here, we will be able to find to root cause then.
  • nhardt
    nhardt
    26 Posts

    Re: applying a value with gpfs_fputattrs that was retrieved with gpfs_fgetattrs

    ‏2012-09-12T23:08:35Z  
    • LuoChen
    • ‏2012-09-12T22:08:40Z
    I wasn't be able to recreate your problem. There might be something else going on.
    Please change the error message for gpfs_fputattr() to distinguish the one with NULL buffer, and the one with non-empty buffer.

    Also, what version of GPFS you are using, on which OS?

    Take gpfs traces will help as well. try the following:
    mmtrace start
    reproduce the problem
    mmtrace stop

    then attach the generated trace file here, we will be able to find to root cause then.
    Hi,
    Thanks for the reply, those are good suggestions. I have updated the code to either print "Unable to apply empty attributes to: " or ""Unable to apply extended attributes to: ". I also add the value of errno (failed to update the example code).

    It looks like it is taking the case where it is getting a value back and applying that (the non-NULL case), and the errno value is 22.

    Here is the new output:
    Unable to apply extended attributes to: ./dir4/file0 errno: 22

    Attached is the mmtrace output.

    I am able to reproduce this on two gpfs versions. The newer one is gpfs 3.4.0-15 (has one patch gpfs34PTF15.843852.ifix) on rhel 6.2, kernel is 2.6.32-220.23.1.el6.x86_64.

    Since the ifix might throw off the results, I checked 3.4.0-8 stock on rhel 6.1, kernel 2.6.32-220.7.1.el6.x86_64. The attached zip file contains a trace of the working case and the failed case, as well the updated code and terminal output.

    Thanks again for taking a look, let me know what else I can do to help!

    Nate
  • dlmcnabb
    dlmcnabb
    1012 Posts

    Re: applying a value with gpfs_fputattrs that was retrieved with gpfs_fgetattrs

    ‏2012-09-13T21:39:37Z  
    • nhardt
    • ‏2012-09-12T23:08:35Z
    Hi,
    Thanks for the reply, those are good suggestions. I have updated the code to either print "Unable to apply empty attributes to: " or ""Unable to apply extended attributes to: ". I also add the value of errno (failed to update the example code).

    It looks like it is taking the case where it is getting a value back and applying that (the non-NULL case), and the errno value is 22.

    Here is the new output:
    Unable to apply extended attributes to: ./dir4/file0 errno: 22

    Attached is the mmtrace output.

    I am able to reproduce this on two gpfs versions. The newer one is gpfs 3.4.0-15 (has one patch gpfs34PTF15.843852.ifix) on rhel 6.2, kernel is 2.6.32-220.23.1.el6.x86_64.

    Since the ifix might throw off the results, I checked 3.4.0-8 stock on rhel 6.1, kernel 2.6.32-220.7.1.el6.x86_64. The attached zip file contains a trace of the working case and the failed case, as well the updated code and terminal output.

    Thanks again for taking a look, let me know what else I can do to help!

    Nate
    This was fixed by defect 844500 which shipped in 3.4.0.16 which became available Sep 5th.
  • nhardt
    nhardt
    26 Posts

    Re: applying a value with gpfs_fputattrs that was retrieved with gpfs_fgetattrs

    ‏2012-09-28T20:16:13Z  
    Thanks Dan. Confirmed fixed on gpfs.base-3.4.0-16.x86_64.