Standardizing user UID and GID numbers on AIX

If you have more than one AIX® server that uses local user accounts, you probably have some inconsistent user ID (UID) and group ID (GID) numbers across the different systems. This article explains how to standardize your UID and GID numbers across all of your AIX systems so that they are consistent.

Share:

Brian K. Smith (brian.k.smith@selecthealth.org), System administrator, Intermountain Healthcare

Brian Smith is a UNIX system administrator for Intermountain Healthcare in Salt Lake City, Utah.



04 October 2011

Also available in Chinese

Introduction

In environments that have multiple AIX systems, it is common to have inconsistent UID (User ID) and GID (Group ID) numbers across the different systems. This can be problematic for several reasons and is an issue worth addressing. This article explains what UID and GID numbers are on AIX and some of the specific problems of having the previously mentioned inconsistencies among multiple servers. Manually changing UID and GID numbers on AIX to create consistency, as well as a script that can automate the task to standardize UIDs and GIDs on a large scale, will also be addressed.


Overview of UID and GID numbers in AIX

On UNIX systems such as AIX, the operating system represents users with UID numbers and groups with GID numbers. You can easily see your UID and GID numbers on your AIX account by running the id command.

Listing 1. id output
$ id
uid=404(brian) gid=402(testgroup) groups=1(staff)

In Listing 1 above, you can see that the UID number for the brian account is 404. The user also has a primary group with a GID of 402 (testgroup) and is also a member of GID 1 (staff).

Each file stored on disk has an owner and a group, however the actual names of the user and group are not stored on disk. Instead, the UID and GID numbers that represent the user and group are stored. You can see this by running the istat command on a file.

Listing 2. istat output
$ istat testfile
Inode 131073 on device 10/5     File
Protection: rwxr-----
Owner: 404(brian)              Group: 402(testgroup)
Link count:   1         Length 291 bytes

Last updated:   Tue Apr  8 09:04:17 MDT 2008
Last modified:  Wed Feb  7 13:22:36 MST 2007
Last accessed:  Thu Sep 30 12:36:59 MDT 2010

In Listing 2 above, you can see that the owner of the file is UID 404 (brian), and the group is 402 (testgroup).

Other items on the system, such as the process table that contains the running processes, keep track of who owns the process by tracking UID and GID numbers rather than actually using the user name and group name.


The problem

If you have multiple servers, UID and GID numbers might not be consistent across servers. By default when you create a user or group in AIX, it simply assigns it the next available UID or GID number. If you have more than one server in your environment, UID and GID numbers can quickly become inconsistent between servers. What this means is that the "brian" user might have a UID of 404 on Server1, a UID of 406 on Server2, and a UID of 402 on Server3.

This can be problematic for a couple of reasons. One of the biggest reasons to standardize consistently UID and GID numbers across all servers is so that you can move to a central authentication system, such as LDAP. Central authentication systems, like LDAP, generally require that LDAP enabled users and groups have consistent UIDs and GIDs across all servers that are LDAP connected.

Even if you are not looking to utilize central authentication such as LDAP, you can still run in to problems with having inconsistent UID and GID numbers. For example, suppose you have a SAN LUN mapped to ServerA. This LUN might have thousands of files stored on it. Each file stored on the LUN has the file owner and group stored as UID and GID numbers. So if you take this LUN and unmap it from ServerA and map it to ServerB, you will have issues if the UID and GID numbers are not consistent between ServerA and ServerB. In this scenario, you could have a couple of problems. If user brian was UID 404 on ServerA, and user bob is UID 404 on ServerB, after moving the LUN user bob now owns all of user brian's files. If there is no UID 404 on ServerB, then the file does not have an owner on ServerB, and you simply see "404" as the owner when you run a ls –al command.

You might also have issues with inconsistent UID/GID numbers across servers when you are exporting NFS shares between servers.


Steps to manually change a GID and UID number

In this example, you will change a single UID for a user (brian) and a GID for a group (testgroup). The "brian" users original UID is 404, and the new UID will be 3504. The "testgroup" groups original GID is 402, and the new GID will be 5001. Changing a GID or UID number is a multi-step process.

Step 1. Stop applications and have users log off

Before changing GID or UID numbers, it is important to stop all running applications and have all users log off of the server. The process table keeps track of running processes based on UID and GID numbers. So, if users have running processes while these numbers are changed, unpredictable results occur. Also, until the file ownership is fixed in a later step, the users temporarily loses access to their files.

Step 2. Find users who have the group as their primary group

Each user on the system has a primary group defined in the /etc/passwd file. When you change the GID number in Step 3, AIX prints a warning message that it does not update the /etc/passwd file with the new GID. So before you change the GID, you first want to get a listing of all users that have the group you are changing as their primary group. There might not be any users with this as their primary group, or there might be multiple users. To find out, run the command in Listing 3 below:

Listing 3. lsuser output
# lsuser -a pgrp ALL | grep pgrp=testgroup
brian pgrp=testgroup

This command shows that there is one user (brian) on the system with testgroup as their primary group. Make a note of the users that this command displays as you will need to run a command to fix them in a later step.

Step 3. Change the GID number

Use the chgroup command to change the GID of testgroup to 5001 (originally it was GID 402).

Listing 4. chgroup output
# chgroup id=5001 testgroup
3004-719 Warning: /usr/bin/chgroup does not update /etc/passwd with the new gid.

The chgroup prints a warning message letting you know that it does not update the /etc/passwd with the new GID. This warning applies to any users that have this group as their primary group. You collected a list of these users in Step 2, and in the next step you will fix this.

Step 4. Fix users primary group

For each user that you noted in Step 2, run the following command to fix their primary group. Note that there might not be any users with the group as their primary group, or there might be multiple users.

Listing 5. chuser command to update primary group
# chuser pgrp=testgroup brian

This chuser command updates the /etc/passwd file for the brian user with the testgroup's new GID number.

Step 5. Change the UID number

Use the chuser command to change the UID for the user brian to UID 3504 (originally it was 404).

Listing 6. chuser command to change UID
# chuser id=3504 brian

At this point, the brian UID and testgroup GID have been changed on the system. You can run the id brian command to see the new UID and GID.

Listing 7. id brian output
# id brian
uid=3504(brian) gid=5001(testgroup) groups=1(staff)

You are not yet done. If you run a ls –al command in the users home directory, you quickly see a problem.

Listing 8. ls output for users home directory
# ls -al /home/brian

drwxr-xr-x  5  404    402    4096 2009-04-08 09:12 .
drwxr-xr-x 10  bin    bin     256 2007-03-01 15:06 ..
-rw-r--r--  1  404    402      10 2007-02-07 13:22 .kshrc
-rwxr-----  1  404    402     291 2007-02-07 13:22 .profile
-rw-------  1  404    402     438 2010-11-10 11:40 .sh_history

As you can see, where the user and group are normally displayed, you now only see the previous UID and GID numbers displayed (404 and 402). If the user brian logged in to the system, he would no longer be the owner of these files. This is because the system stores the UID and GID number for the owner and group on each file rather than storing a user or group name. In the next step, you will fix this.

Step 6. Fix user and group ownership for all files on the system

In this step, you fix the user and group ownership for all files on the system. This is done by running two find commands which search for all files with the previous UID and GID. For each file found that meets one of these criteria, the file is updated with the new user and group ownership.

Listing 9. find commands to fix ownership
# find / -group 402 -exec chgrp -h testgroup {} \;
# find / -user 404 -exec chown -h brian {} \;

Once these commands are completed, the user and group ownerships are corrected for all files on the system for the testgroup group and the brian user. If you run an ls –al command on the brian user's home directory, you can confirm this.


Automating the process

The previous steps took time to complete, and you only changed the UID and GID for a single user and group. If your environment has dozens of AIX servers and each server has dozens of users and groups, it becomes very obvious that manually changing all UIDs and GIDs is not practical.

I have written a Perl script that automates this process. You supply the script with two input files: a file that contains the updated UID information, and a file that contains the updated GID information.

In the UID file, you list out two columns of information per line. The first column has the new UID that you want set, and the second column has the account name. The GID file is similar. The first column has the new GID that you want set, and the second column has the group name.

Listing 10 below shows the contents of these files.

Listing 10. Contents of UID and GID files
# cat uid.txt
3500    megan
3501    todd
3502    app_user
#
# cat gid.txt
5000    app_group

If the script finds a line in the UID or GID file for a user or group that does not exist on the system, the line is simply skipped. This makes it easy to create a list of all users and groups across all systems and creates a single UID and GID file that could be used to standardize the UIDs and GIDs on any one of your systems, even if each system doesn't have the same users or groups. Also, if the script runs and detects that the current UID or GID on the system for a user or group is the same as the desired UID/GID from the input files, it simply skips these lines. This makes it possible to run the script using a master UID/GID list for the input file even on systems that have some user accounts that already have standardized UIDs and GIDs.

When the script runs, it doesn't change anything on your system; it simply gathers the user and group information from the system and displays on the screen the commands you would need to standardize the UID and GID numbers.

To run the script, you use a command similar to Listing 11 below.

Listing 11. Running the fix_gid_uid.pl script
# ./fix_gid_uid.pl --uidfile uid.txt --gidfile gid.txt
### Commands to update Groups ###

chgroup id=5000 app_group
chuser pgrp=app_group todd
chuser pgrp=app_group megan
chuser pgrp=app_group app_user
chuser pgrp=app_group app_2
chuser pgrp=app_group app_3
chuser pgrp=app_group app_4
find / -group 14 -exec chgrp -h app_group {} \;

### Commands to update Users ###

chuser id=3500 megan
find / -user 406 -exec chown -h megan {} \;
chuser id=3501 todd 
find / -user 402 -exec chown -h todd {} \;
chuser id=3502 app_user
find / -user 409 -exec chown -h app_user {} \;

The script displays on the screen all of the commands you would need to standardize the UIDs and GIDs on this system to the new UIDs and GIDs listed in the uid.txt and gid.txt input files.

Note that the output shows that six users need to have their primary group changed after the GID changes for the app_group group. This includes users that were not included in the uid.txt file. The reason for this is that once the group GID changes all users which had this group as a primary group need to have their primary group updated, even if these users are not changing their UID.

Once you have run the script and verified the output is correct, it is simple to run the commands. To do this, run the script again and redirect the output to a file. Make the output file executable, and then run it.

Listing 12. Actually making the changes on the system
# ./fix_gid_uid.pl --uidfile uid.txt --gidfile gid.txt > commands
# chmod +x commands
# ./commands

The run time depends on how many groups and users are being changed, and how many files are present on the system. The time range can vary between a minute to over an hour depending on these factors. If you are concerned with the run time, you can break the commands file into smaller sections and run them separately. If your maintenance window is almost over, you can let the current section finish. When you have another maintenance window, simply run the fix_gid_uid.pl script again. It will generate new commands that you need to run, and the UIDs and GIDs that were already fixed previously will not be listed in the output again.

Listing 13. fix_gid_uid.pl script
#!/usr/bin/perl
#  This is unsupported code.  This script is provided "as is" without warranty of
#  any kind, expressed or implied, including, but not limited to, the implied
#  warranty of merchantability or fitness for a particular purpose.
#  Use at your own risk.
#
use Getopt::Long;
use User::pwent;
use User::grent;

my ($uid_file, $gid_file);

GetOptions("uidfile=s" => \$uid_file, "gidfile=s" => \$gid_file);

if (!((defined $gid_file) || (defined $uid_file))){
  print "Specify at least one arguments:  --uidfile <filename> ";
  print "AND/OR --gidfile <filename>\n\n";
  print "Example: $0 --uidfile uid.txt --gidfile gid.txt\n\n";
  print "Format of UID and GID files should be: \n";
  print "<Desired GID/UID#>  <User/Group Name>\n\n";
  print "Example:\n";
  print "3000 user1\n";
  print "3001 user2\n\n";
  exit 1;
}

if (defined $gid_file){
  open GIDFILE, "<$gid_file" or die $!;
  while (<GIDFILE>){
    my $line = $_;
    if ($line =~ /(\S+)\s+(\S+)\s*/){
      my $gid = $1;
      my $group = $2;
      while($ent = getgrent()){
        my $ent_name = $ent->name;
        my $ent_gid = $ent->gid;
        if ($ent_gid == $gid){
          if ($ent_name ne $group){
            print "### Error, Group in $gid_file file ($group, GID: $gid)";
            print " conflicts with group on system ($ent_name, GID: $ent_gid)\n";
            print "### Exiting program, please fix and rerun\n";
            exit 2;
          }
        }
      }
      endgrent();
    }
  }
  close GIDFILE;
}

if (defined $uid_file){
  open UIDFILE, "<$uid_file" or die $!;
  while (<UIDFILE>){
    my $line = $_;
    if ($line =~ /(\S+)\s+(\S+)\s*/){
      my $uid = $1;
      my $name = $2;
      while($ent = getpwent()){
        my $ent_name = $ent->name;
        my $ent_uid = $ent->uid;
        if ($ent_uid == $uid){
          if ($ent_name ne $name){
            print "### Error, User in $uid_file file ($name, UID: $uid)";
            print "conflicts with user on system ($ent_name, UID: $ent_uid)\n";
            print "### Exiting program, please fix and rerun\n";
            exit 2;
          }
        }
      }
      endpwent();
    }
  }
  close UIDFILE;
}


if (defined $gid_file){
  print "\n### Commands to update Groups ###\n\n";

  open GIDFILE, "<$gid_file" or die $!;
  while (<GIDFILE>){
    my $line = $_;
    chomp($line);
    if ($line =~ /(\S+)\s+(\S+)\s*/){
      my $newgid = $1;
      my $group = $2;

      my $return = system("lsgroup $group >/dev/null 2>&1");
      if ($return == 0) {
        my $oldgid = `lsgroup -a id $group | cut -f 2 -d =`;
        chomp($oldgid);

        if ($oldgid != $newgid){
          print "chgroup id=$newgid $group\n";
          while($ent = getpwent()){
            my $ent_user = $ent->name;
            my $ent_gid = $ent->gid;
            if ($ent_gid == $oldgid){
              print "chuser pgrp=$group $ent_user \n";
            }
          }
          endpwent();
          print "find / -group $oldgid -exec chgrp -h $group {} \\;\n";
        }
      }
    }
  }
  close GIDFILE;
}

if (defined $uid_file){
  print "\n### Commands to update Users ###\n\n";

  open UIDFILE, "<$uid_file" or die $!;
  while (<UIDFILE>){
    my $line = $_;
    chomp($line);
    if ($line =~ /(\S+)\s+(\S+)\s*/){
      my $newuid = $1;
      my $user = $2;

      my $return = system("lsuser $user >/dev/null 2>&1");
      if ($return == 0) {
        my $olduid = `lsuser -a id $user | cut -f 2 -d =`;
        chomp($olduid);
        if ($olduid != $newuid){
          print "chuser id=$newuid $user\n";
          print "find / -user $olduid -exec chown -h $user {} \\;\n";
        }
      }
    }
  }
  close UIDFILE;
}

Conclusion

Having consistent UID and GID numbers across multiple AIX servers is considered best practice and, as demonstrated in this article, is an achievable goal. By following the process outlined throughout this article, system administrators will likely prevent future issues from occurring by standardizing UID and GID numbers.

Resources

Learn

Get products and technologies

  • Try out IBM software for free. Download a trial version, log into an online trial, work with a product in a sandbox environment, or access it through the cloud. Choose from over 100 IBM product trials.

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into AIX and Unix on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=AIX and UNIX
ArticleID=762641
ArticleTitle=Standardizing user UID and GID numbers on AIX
publish-date=10042011