User account synchronization across a HPC (PCM) cluster

Answer

Please find below explanation about how the system passwd, shadow, group, *.OS and *.merge files work.

1.How and when does .OS files are created? (passwd.OS/group.OS/Shadow.OS) and what is the relationship of .OS file (passwd.OS, group.OS, shadow.OS) to system files (passwd, shadow,group) under /etc/directory.
*.OS files (passwd.os, shadow.os, group.os) are created after PCM/HPC installed. It's sub-aggregate of passwd, shadow and group files on MN.
The *.OS files does not include the users which was defined by user manually.

2. How and when does .merge files knows that the updates has been made to /etc/passwd, /etc/groups? Is .merge files look at .OS file to see if there are updates on that file?
When updatenode command is called, the *.merge will be refreshed, it will based on /etc/passwd and /etc/*.OS to refresh. if the accounts are not in *.OS, but in passwd and shadow, this account will be added into *.merge and sync to CN.
If user account is created in passwd, shadow, group and also in *.OS (passwd.OS, shadow.OS, group.OS) then it wont sync to computenodes.

If the user is removed using userdel command, this removes entry in /etc/passwd, /etc/shadow, /etc/group.. but the entries are in /etc/passwd.OS, /etc/shadow.OS, /etc/group.OS. It will be put in .merge and synchronized. So anytime the user is removed it needs to be removed manually from *.OS files.
for example:
# sed -i '/^testuser/'d /etc/*.OS

Here is a table that shows how the synchronization works:

3. How can someone disable propagation of users (phpcadmin) from headnode to compute node:
To disable push out phpcadmin user to compute node, you can add the user info into *.OS. then run "updatenode", this user won't sync to compute node then.

Steps:
Preparation
1. Node information check
# xdsh compute000 cat /etc/passwd |grep bash | grep phpcadmin
compute000: phpcadmin:x:30495:30495::/home/phpcadmin:/bin/bash
#lsdef compute000 |grep provmethod
    provmethod=rhels6.5-x86_64-stateful-compute

2. Account (phpcadmin) info from *.merge
# grep phpcadmin /etc/*.merge
/etc/group.merge:phpcadmin:x:30495:
/etc/passwd.merge:phpcadmin:x:30495:30495::/home/phpcadmin:/bin/bash
/etc/shadow.merge:phpcadmin:$6$EvGaQT9V$ojKcYZqW1CVfAF/RyoPrVuTC0VyCod.N3B5LtCN8olBrKmug3ZQ82mjGmAyVQnFe0soRmUHo6iKggUYWZjVYP/:16308:0:99999:7:::

# cd /install/osimages/rhels6.5-x86_64-stateful-compute/cfmdir
[root@headnode cfmdir]# grep phpcadmin /etc/*.merge
/etc/group.merge:phpcadmin:x:30495:
/etc/passwd.merge:phpcadmin:x:30495:30495::/home/phpcadmin:/bin/bash
/etc/shadow.merge:phpcadmin:$6$EvGaQT9V$ojKcYZqW1CVfAF/RyoPrVuTC0VyCod.N3B5LtCN8olBrKmug3ZQ82mjGmAyVQnFe0soRmUHo6iKggUYWZjVYP/:16308:0:99999:7:::

Implementation:
3. Add account(phpcadmin) to *.OS
[root@headnode cfmdir]# echo "phpcadmin:x:30495:" >> group.OS
[root@headnode cfmdir]# echo "phpcadmin:x:30495:30495::/home/phpcadmin:/bin/bash" >> passwd.OS
[root@headnode cfmdir]# echo 'phpcadmin:$6$EvGaQT9V$ojKcYZqW1CVfAF/RyoPrVuTC0VyCod.N3B5LtCN8olBrKmug3ZQ82mjGmAyVQnFe0soRmUHo6iKggUYWZjVYP/:16308:0:99999:7:::' >> shadow.OS

4. Add account(phpcadmin) from compute nodes
[root@headnode cfmdir]# xdsh compute000 userdel phpcadmin

Verification:
Verify the account (phpcadmin) on compute nodes after node synchronization:

[root@headnode cfmdir]# updatenode compute000 -F
File synchronization has completed for nodes.
[root@headnode cfmdir]# xdsh compute000 cat /etc/passwd |grep bash | grep phpcadmin
[root@headnode cfmdir]#