Topic
2 replies Latest Post - ‏2013-01-23T17:52:33Z by SystemAdmin
SystemAdmin
SystemAdmin
2092 Posts
ACCEPTED ANSWER

Pinned topic policy rules - list and delete - how to exclude the same file

‏2013-01-23T16:30:19Z |
Hello,

I use gpfs policies and rules to delete old files. I use several "exclude" rules to exclude certain files from deletion. Before I actually do delete I want to create a list of all files that will be deleted some days in advance to inform the users.

What I don't know: How can I apply the same "exclude" rules when I create the list? I really want to use the same rules for exclusion to make 100% sure that what I send to the users is really what I'll delete later. I don't want to keep manually two different syntaxes in sync. I would welcome and hint or example.

Example - the deletion rules look like:

Rule "XcludeTXT" EXCLUDE WHERE SUBSTR(NAME,LENGTH(NAME)-3)='.txt'
Rule "XcludeSH" EXCLUDE WHERE SUBSTR(NAME,LENGTH(NAME)-2)='.sh'

RULE "OldFiles"
DELETE
WHERE PATH_NAME LIKE '%/Data10/e_____/%'
AND DAYS(CURRENT_TIMESTAMP) - DAYS(MODIFICATION_TIME) > 60

How do I create a file list which lists the files to be deleted using the same Rules for exclusion? Currently I create a list of all files and do the exclusion by name and age later with some postprocessing. I create a list with some additional data using SHOW().

My current list rule file looks like:

define(toSeconds,(($1) SECONDS(12,6)))
define(toUnixSeconds,)

RULE

LIST 'list'
SHOW(varchar(USER_ID) || ' ' || varchar( toUnixSeconds(modification_time) ) || ' '
|| varchar (file_size)
)

RULE
EXTERNAL LIST 'list'
EXEC '/scratch/script.sh'
The script 'script.sh' just dumps the file lists to individual files.

Kind regards,

Heiner Billich
Updated on 2013-01-23T17:52:33Z at 2013-01-23T17:52:33Z by SystemAdmin
  • SystemAdmin
    SystemAdmin
    2092 Posts
    ACCEPTED ANSWER

    correction

    ‏2013-01-23T16:43:06Z  in response to SystemAdmin
    Hello,

    the m4 macros in the list rules don't show up correctly in my original post. I'm sorry for that, I don't know how to fix it. But it make no difference regarding my question ...

    Sorry,

    Heiner

    I try once more:
     define(
    toSeconds,
    (($1) SECONDS(12,6)))
    define( toUnixSeconds,)

    RULE

    LIST 'list'
    SHOW(varchar(USER_ID) || ' ' || varchar( toUnixSeconds(modification_time) ) || ' '
    || varchar (file_size)
    )

    RULE
    EXTERNAL LIST 'list'
    EXEC '/scratch/2013-01-17-X02DA-file-list/script.sh'
    • SystemAdmin
      SystemAdmin
      2092 Posts
      ACCEPTED ANSWER

      Re: correction

      ‏2013-01-23T17:52:33Z  in response to SystemAdmin
      Use LIST rules with the mmapplypolicy -I defer option to create a list of files that are candidates for deletion.
      You can edit the filelist as you please. When you and your users are satisfied with the list, run a script against it to do the deletions.

      For example, your policy rule file would look like this:
      
      rule 
      'a' external list 
      'x' exec 
      '/ghome/makaplan/policies/doer.sh' opts 
      'CMD'   rule 
      'bx' list 
      'x' EXCLUDE WHERE  criteria-to-avoid-deletion-even-if-file-would-match-the-next-rule rule 
      'bi' list 
      'x'  WHERE criteria-for-deletion
      


      You would issue mmapplypolicy like this:
      
      mmapplypolicy /directory-to-explore  -P /path-to-policy-rules -I defer -f /tmp/xcaution
      


      The doer.sh script could look like this:
      
      
      
      if [ $1 = TEST ] then echo TEST OKAY. exit 0 fi   export filelist=$2 export cmd=$3   # execute cmd against input list x=
      "$*" 
      
      while read inode gen some opt path otherstuff ; 
      
      do #   echo 
      "$cmd $path"  # remove comment to see what is going on $cmd $path done  < $filelist
      


      You could view the filelist like this:
      
      ./doer.sh list /tmp/xcaution.list.x  
      "ls -ld"
      


      After you are sure that is the list of files you wish to delete, throw caution to the wind and do this!
      
      ./doer.sh list /tmp/xcaution.list.x  
      "rm -f"