Topic
  • 37 replies
  • Latest Post - ‏2013-10-01T23:47:37Z by MBARTOSH
BenKus
BenKus
1408 Posts

Pinned topic Searching a computer for a file

‏2006-11-06T09:05:21Z |
Several threads have been posted about how to search for files with specific extensions using BES.

Searching a computer for file can take a long time and it will use a fair amount of CPU and lots of disk IO. The more files, the longer it will take. This can potentially bother the end-user of the computer or cause performance concerns.

Rather than create relevance that searches for a file (which will run when the BES Client starts and when a ForceRefresh is sent), it is a much better idea to create an action that searches the computer and then writes the results to a file, which can be read with a property. The action will give you much more control about when the search runs (using the action scheduling parameters).

Attached is a .bes file that can be imported into BES 6.0+ deployments. The file includes a Task with two actions and a property to bring back the results:
  • One action will search for specific files and return the counts. For instance, if you search for "avi" and "mp3", it might return
mp3 - 502
avi - 16

  • The other action will search for files with specific extensions and return the pathname and size. For instance, if you search for ".pst", it might return:
C:\Outlook\personaldata.pst (1023 KB)
C:\Archive Data\archive.pst (134310 KB)
As always, when searching for files across a system: BE CAREFUL! Searching a file system takes a lot of resources. We continue to run across situations where users are saying that the BES Client is taking too much CPU and we find that they are constantly running searches across their systems.

Please test the attached Task/Property. It didn't go through any particular QA other than me testing it on a system or two. Let me know if it has any issues.

Ben
  • ivynash
    ivynash
    1 Post

    Re: Searching a computer for a file

    ‏2007-01-23T07:28:38Z  
    hi ben,
    I have imported the fixlet and tried to search an extention "txt", but it is showing failed and the error is :
    The action has been run 1 time, and will not be run again.
    The action failed.

    Completed: // prompt the BES Console user for extensions to report on
    Completed: action parameter query "extensions" with description "Please enter the extensions (one or more) that you wish to count (separated by ';')" with default "mp3;avi"
    Completed: // create a reg value to indicate last time started
    Completed: regset "HKEY_LOCAL_MACHINE\SOFTWARE\BigFix\EnterpriseClient" "LastSearchStarted"="{now}"
    Completed: // delete temporary file (probably doesn't exist, but might cause problems if it does)
    Completed: delete __appendfile
    Completed: // search computer and results in temp file
    Failed: appendfile {concatenation "%0d%0a" of (it & "-" & multiplicity of it as string) of unique values whose ((";" & (parameter "extensions" of action) & ";") contains (";" & it & ";")) of ( following texts of lasts "." of (it as lowercase) whose (it contains ".") of names of descendants of folder "\" of drives whose (type of it = "DRIVE_FIXED")) }
    // overwrite previous results (if they exist)
    delete "{pathname of parent folder of regapp "besclient.exe"}\searchresults.txt"
    move __appendfile "{pathname of parent folder of regapp "besclient.exe"}\searchresults.txt"
    // create a reg value to indicate last finish started
    regset "HKEY_LOCAL_MACHINE\SOFTWARE\BigFix\EnterpriseClient" "LastSearchFinish"="{now}"
  • Edj
    Edj
    34 Posts

    Re: Searching a computer for a file

    ‏2007-03-14T18:45:51Z  
    Ben,

    I have tested the Task and it works fine. I was wondering however, what lines of code would you add if you would like to return the "Pathname" and "Modification Time" along with the file size. I have tried modifying your code but I cannot get it to work.

    Thanks,

    Edj
  • BenKus
    BenKus
    1408 Posts

    Re: Searching a computer for a file

    ‏2007-03-15T02:51:46Z  
    Hey Ed,

    You would modify part of the second action... Before it was:
    ...(pathname of it & " (" & (size of it / 1000) as string & " KB)")...

    Change it to:

    ...(pathname of it & " (" & (size of it / 1000) as string & " KB) -- " & modification time of it as string)...
    BTW, please be VERY careful with this type of action... Not only is it expected to run for a long time on computers, but if it returns lots of results (like dozens of lines per computer), it will cause your BES Console to bloat its memory...

    Ben
  • Edj
    Edj
    34 Posts

    Re: Searching a computer for a file

    ‏2007-03-15T12:39:25Z  
    Thanks Ben.

    I must run this task on a selection of Database servers to locate their Database files.

    Edj
  • Edj
    Edj
    34 Posts

    Re: Searching a computer for a file

    ‏2007-03-15T15:51:29Z  
    Ben,

    The process worked. However, I do have another question. The "Search results" returns a list of the files but does not display it in regards to the Machine. Is there a way to lists the Property values by the Machine Name?

    Thanks Again,

    Edj
  • BenKus
    BenKus
    1408 Posts

    Re: Searching a computer for a file

    ‏2007-03-16T07:07:14Z  
    Hey Ed,

    I am not too sure what you mean... Are you looking at the results in an Analysis view (if so, look at the top-right corner of the analysis page for "view as list") or is it a global property?

    Ben
  • Edj
    Edj
    34 Posts

    Re: Searching a computer for a file

    ‏2007-03-16T11:58:17Z  
    Its the "Search Results" property that was imported with the "Search computer for files.bes" Task.
  • Edj
    Edj
    34 Posts

    Re: Searching a computer for a file

    ‏2007-03-20T12:37:43Z  
    I am now receiving the same error that ivynash received, when running this process on my servers. Any Ideas?

    Thanks,
    Edj
    Failed: appendfile {concatenation "%0d%0a" of (it & "-" & multiplicity of it as string) of unique values whose ((";" & (parameter "extensions" of action) & ";") contains (";" & it & ";")) of ( following texts of lasts "." of (it as lowercase) whose (it contains ".") of names of descendants of folder "\" of drives whose (type of it = "DRIVE_FIXED")) }
    // overwrite previous results (if they exist)
  • Edj
    Edj
    34 Posts

    Re: Searching a computer for a file

    ‏2007-03-21T11:39:00Z  
    Almost There!!!!!!
    One Reason that the actionscript below is failing is due to the folder object not being plural (to account for clients with multiple fixed drives). So, tweaking the actionscript below (note the ‘s’ added to folders “\” of drives whose (type of it….) should allow it to run:

    // prompt the BES Console user for extensions to report on
    action parameter query "extensions" with description "Please enter the extensions (one or more) that you wish to count (separated by ';')" with default "mp3;avi"
    // create a reg value to indicate last time started
    regset "HKEY_LOCAL_MACHINE\SOFTWARE\BigFix\EnterpriseClient" "LastSearchStarted"="{now}"
    // delete temporary file (probably doesn't exist, but might cause problems if it does)
    delete __appendfile
    // search computer and results in temp file
    appendfile {concatenation "%0d%0a" of (pathname of it & " (" & (size of it / 1000) as string & " KB)" & ""& modification time of it as string & "" & pathname of it as string) of (descendants whose (pathname of it as lowercase ends with "." & (parameter "extension" of action) ) of folders "\" of drives whose (type of it = "DRIVE_FIXED")) } // overwrite previous results (if they exist)
    delete "{pathname of parent folder of regapp "besclient.exe"}\searchresults.txt"
    move __appendfile "{pathname of parent folder of regapp "besclient.exe"}\searchresults.txt"
    // create a reg value to indicate last finish started
    regset "HKEY_LOCAL_MACHINE\SOFTWARE\BigFix\EnterpriseClient" "LastSearchFinish"="{now}"

    That said, in testing the above relevance, another issue appeared which may be more problematic. The above relevance did not find all instances of files with a certain extension on the drives during testing. It found some, but not all. This may be due to a limitation of one of the inspectors. There may be other ways to search the drive for certain types of files that would work (like using dir /s/b instead of descendants). I am not quite sure how to fit this within the action script. I will take a look but if someone comes across a working solution before I do, Please post it.

    Notice that I added "Modified Time" and "Pathname" as a varible that I would like returned other than that it is the same action script as the original.
    Thanks

    And another Thank You to Aram for helping me with this last post.

    Edj
  • BenKus
    BenKus
    1408 Posts

    Re: Searching a computer for a file

    ‏2007-03-22T00:45:56Z  
    Hey Ed,

    I think the "descendants of folder" inspector aborts if it hits a file that it doesn't have access to... We can check this out a bit to see if we spot the issue...

    Ben
  • SystemAdmin
    SystemAdmin
    2808 Posts

    Re: Searching a computer for a file

    ‏2007-03-27T21:59:14Z  
    I would be interested to know if "descendants of folder" does abort when it hits a file it doesn't have access to. I tried before to search for a set of executables and the analysis was failing on ~20% of the machines even when I activated the analysis on a Saturday when no one was in the office. I ran the same search on just the "C:\Program Files" folder and didn't receive any errors.
  • BenKus
    BenKus
    1408 Posts

    Re: Searching a computer for a file

    ‏2007-03-28T00:33:45Z  
    Hi Edj and Hexadecimal,

    We found that the "decendants of folder" inspector will stop recursing if it hits a file with characters in the filename that we don't recognize (such as high or low ascii characters). I believe if the SYSTEM account doesn't have access to a file it will also stop. We filed a bug about this, but in the meantime we will need to rewrite this "search the computer" action to use something like the "dir" command.

    I will see what we can whip up...

    Ben
  • dankudos91
    dankudos91
    2 Posts

    Re: Searching a computer for a file

    ‏2007-05-11T19:03:26Z  
    I just registered on these forums today after finding this thread. Thank you!

    I understand the issues others are seeing but for our environment this worked perfectly in testing today. We'll run a larger test tonight but I expect this will give us the information we need in an easy to understand way thanks to adding an Analysis and Web reports.

    Thanks again and expect to see me back to steal more good ideas!

    Dan
  • frankc91
    frankc91
    7 Posts

    Re: Searching a computer for a file

    ‏2007-05-15T00:19:10Z  
    Expanding on thread above, does anyone know of a simple and easy way to append to an existing text file via a fixlet or task?

    i.e., add an entry to a Windows "\drivers\etc\Hosts" file with something like:

    10.X.Y.Z xyz.test.com

    Thanks!
  • jessewk
    jessewk
    536 Posts

    Re: Searching a computer for a file

    ‏2007-05-15T00:24:26Z  
    delete __createfile
    createfile until end_create
    {(concatenation of lines of file (pathname of system folder & "\drivers\etc\Hosts")) & "%0d%0a" & "10.X.Y.Z xyz.test.com"}
    end_create
    delete "{pathname of system folder & "\drivers\etc\Hosts"}"
    move __createfile "{pathname of system folder & "\drivers\etc\Hosts"}"

    Note: completely untested.
  • BenKus
    BenKus
    1408 Posts

    Re: Searching a computer for a file

    ‏2007-05-15T00:34:04Z  
    Hey Frank / Jesse,

    I posted another way to do this in a new topic:
    http://forum.bigfix.com/viewtopic.php?pid=3667#p3667

    My method is different than Jesse's but either should work...

    Ben
  • dankudos91
    dankudos91
    2 Posts

    Re: Searching a computer for a file

    ‏2007-05-17T14:11:32Z  
    I returned to my client today and am looking at the results from an overnight search of about 150 machines. We did error out on about 40 machines but the results we have in hand are great. Any plans to revise this action soon or should I investigate other methods for querying with 100% accuracy?

    Thank you for the fine start with this action. The level of support offered on these forums certainly has improved my opinion of BigFix overall. Thank you for your excellent help!

    Dan
  • SystemAdmin
    SystemAdmin
    2808 Posts

    Re: Searching a computer for a file

    ‏2009-04-23T19:23:34Z  
    What version of BES was this bug an issue?

    We found that the "decendants of folder" inspector will stop recursing if it hits a file with characters in the filename that we don't recognize (such as high or low ascii characters). I believe if the SYSTEM account doesn't have access to a file it will also stop.

    Has it been fixed? Anyway to workaround without using Dir?

    I need to find if there is a file present on a machine from a list of files.
  • BenKus
    BenKus
    1408 Posts

    Re: Searching a computer for a file

    ‏2009-04-24T00:49:12Z  
    Hey Lewis,

    This was Bug 14393 that was fixed in BigFix 7.0 Agents...

    Here is a comment from one of our senior developers that discusses what you mentioned with various characters in filenames:

    "Characters in unicode filenames that are not representable in MBCS strings are
    translated to question marks by the MS FindFirst/FindNext API. We consider
    question marks to be wildcards and reject the file/folder, terminating the
    descendants iterator loop. I've changed the descendants inspector to skip these
    files/folders and continue with the rest of the loop."

    Ben
  • amerriam91
    amerriam91
    4 Posts

    Re: Searching a computer for a file

    ‏2009-05-14T12:56:55Z  
    I have a large list of files that I need to search for. Is there an easy way to change this from searching for file extensions to searching for full file names located on the C: drive? I can't seem to get the syntax quite correct.
  • jessewk
    jessewk
    536 Posts

    Re: Searching a computer for a file

    ‏2009-05-14T17:59:22Z  
    If you are using Brolly's technique, I think all you need to do is change "c:\*.MP3" to "c:*BigFix*" . That would return any files with "BigFix" in the name.

    Again, please be aware of the performance implications of walking every directory on the drive.
  • RachelleAnne91
    RachelleAnne91
    1 Post

    Re: Searching a computer for a file

    ‏2009-07-27T22:43:33Z  
    "c:\*.MP3" to "c:*BigFix*" . Thanks for this. It works.

    Great job.
    [color=#DEDFDF_Pret immobilier_[/color]|http://www.simulationpretimmobilier.net]
  • Shlomi91
    Shlomi91
    30 Posts

    Re: Searching a computer for a file

    ‏2009-08-31T15:21:24Z  
    Hi,

    sorry for double-posting, but my original post ( http://forum.bigfix.com/viewtopic.php?pid=15080#p15080) is related to the bug that was mentioned here.
    i am using BES 7.2.4, and i am trying to get a specific folder size on about 120 servers worldwide, using this query:
    sum of ((sum of sizes of descendants of it) /1024/1024) of folders of folder "e:\home"
    

    I am getting only partial results (70 GB instead of over 200 GB), and this thread shed some light on this (the server is in Ukraine, many files in Cyrillic, and file names appear as question marks, and it is possible that SYSTEM doesnt have access to all folders).
    i tried running a dos dir /s command with output to a file (dir /-C /s e:\home | find /I "File(s)" > e:\home\size.txt) , and extract the line with "File(s)" in it, but it gives one line with "File(s)" for each folder (over 6000 folders, in this case), so my options are as follows:

    1. get the BES Client query way to work
    2. using the dos DIR command, return only the last line
    3. using the dos DIR command, return only the line with the biggest integer

    help anyone!

    thanks,

    Shlomi
  • BenKus
    BenKus
    1408 Posts

    Re: Searching a computer for a file

    ‏2009-09-01T05:36:22Z  
    Hey Shlomi,

    It seems like option #2 would work well best, don't you think?

    Ben