Topic
4 replies Latest Post - ‏2013-03-28T19:47:07Z by SystemAdmin
SystemAdmin
SystemAdmin
1316 Posts
ACCEPTED ANSWER

Pinned topic Granularity of Watch File Interval in File Agent

‏2013-03-26T21:23:42Z |
Simple question, hopefully simple solution that I'm just missing. I have a set of 20 directories on a single server that I want to monitor with File Agent. However, there's the potential that some of these directories will only need to be polled every hour, some every 15 minutes, some as quickly as possible (which as far as I can tell is every minute).

My questions are: Do I need to create a separate config for each interval I need to watch over, or can I specify somehow multiple times for different directories in a single config? If the former, would this mean running multiple File Agent services, one for each similar time interval? Seems like that has the potential to cause a large amount of services needed.

Thanks for any input you can provide on the best way to handle this.

Jason
Updated on 2013-03-28T19:47:07Z at 2013-03-28T19:47:07Z by SystemAdmin
  • SAFlanigan
    SAFlanigan
    12 Posts
    ACCEPTED ANSWER

    Re: Granularity of Watch File Interval in File Agent

    ‏2013-03-27T13:52:49Z  in response to SystemAdmin
    You should only need to use one Connect:Direct File Agent service, it can scan all 20 directories. It should not matter that the directories all get scanned at the same interval.

    Note be sure that your C:D processes that are kicked off remove the files from the directory once they are copying, this will make File Agent more efficient reducing scan time, and remove the need to check if a file has already been processed previously.
    • SystemAdmin
      SystemAdmin
      1316 Posts
      ACCEPTED ANSWER

      Re: Granularity of Watch File Interval in File Agent

      ‏2013-03-27T15:57:15Z  in response to SAFlanigan
      While I agree it shouldn't matter, the requirements I am using C:D to support may very well mandate multiple polling periods. And if the system architects/engineers/managers/customers want that, well I can try to persuade them til I'm blue in the face but if it's what they want, it's what they expect to get.

      Thank you for the tip on the check if a file has been processed. I unfortunately cannot remove files from the scanned directories, as we are using C:D for file replication between two environments, creating one-way mirror (among other things).
      • SystemAdmin
        SystemAdmin
        1316 Posts
        ACCEPTED ANSWER

        Re: Granularity of Watch File Interval in File Agent

        ‏2013-03-28T19:29:17Z  in response to SystemAdmin
        So I've configured these 20 directories, and I am stumped by something that I'm either not doing right (the likely scenario) or a limitation in File Agent that just seems weird.

        For most of these directories, I want to copy anything put in them. Easy enough.

        However, there are a few directories where I only care about files starting with CMD, and in a couple there are static file names that I care about, but none of the other files that will be placed in those directories.

        From what I can tell, if I want to filter on these few directories, I now have to create a rule for each directory stating "matches <somedirpath>*.", and then two sets of rules for the files I want to filter on, for example one copying files with "matches <anotherdirpath>\CMD." and one that says "not matches <dirpath>\CMD.*" and have that rule do nothing.

        I discovered this by creating a test File Agent config that watches E:\data\foldera, folderb, and folderc. I created a rule that said "matches E:\data\folderb\*.cmd" and had it do the default process (basically wanting to filter on only .cmd files for copying that exist in folderb, but copy any files that were placed in foldera and folderc). This basically did nothing; anything placed in any of the folders copied.

        I then changed the rule to "not matches E:\data\folderb\*.cmd" and made it do nothing, hoping this would keep any file in folderb that wasn't a .cmd from being copied. However, this caused any file that did not match that criteria to be copied, no matter if it was placed in folderb, a or c.

        It wasn't until I created a rule set for "matches E:\data\foldera\*.*", "matches E:\data\folderb*.cmd", and "matches E:\data\folderc\*.*", and had all of them use the same process and variables, that I was able to get things to work right.

        If this is working as designed, if you're talking about a large number of directories and wanting to filter on a few, it seems you either need to come up with multiple watcher configs, or create many rules. Is this true, or have I flubbed something up?
        • SystemAdmin
          SystemAdmin
          1316 Posts
          ACCEPTED ANSWER

          Re: Granularity of Watch File Interval in File Agent

          ‏2013-03-28T19:47:07Z  in response to SystemAdmin
          Sorry for the bold, not sure how that happened.

          And I also had to have a "not matches E:\data\folderb\*.cmd" rule that didn't do anything.