• 1 reply
  • Latest Post - ‏2013-07-02T09:28:17Z by wvh
1 Post

Pinned topic Permission issue for seed of Search Collection

‏2013-06-10T18:39:57Z | converting crawler-error denied permission


I am new to Data Explorer.  I have installed the software (Data Explorer 8.2-2) on Centos 6.3 as root, and using the embedded web server.

I am following an example from IBM's big data bootcamp.  I am creating a search collection for IOD-sessions.  I have copied the file "session.xls" to my VM with Data Explorer installed, which is the seed for the crawler.  When I go to the Overview tab and click "Test it" by the file URL under Seeds, it gives me this error "[CRAWLER_FS_OPERATION]The URL could not be fetched due to a system error that prevented the crawler from performing a filesystem operation: Permission denied. "

I have changed the permissions of the file to be read & writeable to all and tested with this, but it resulted in the same error.

Please let me know if I should submit a service request directly to IBM.  Also, can  you point me to where relevant log files are located?


  • wvh
    2 Posts

    Re: Permission issue for seed of Search Collection


    The file that you're trying to open has to be accessible by the crawler, which is probably running as the user and group that is associated with the embedded web server. (You specified that user/group when you configured the embedded web server.) The entire path to your file has to be readable (and accessible) by that user/group. If you don't want to open up access to the location where your copy of session.xls is stored, you could copy it to the examples/data  subdirectory of your Data Explorer Engine installation and set its ownership to the same user/group as the other files/directories in that location. After modifying your seed to point to that file, your crawl should succeed.

    The Data Explorer Engine documentation includes a number of tutorials that can help you get started using Data Explorer Engine. The one called "Using Metadata for Searching, Sorting, and Filtering" is a starting point for many of them. You can get an overview of available tutorials by selecting the Help -> Tutorials menu item in the Data Explorer Engine administration tool.

    The easiest way to examine log files in Data Explorer Engine is to enable debugging in the project that you're working with, and then use the Management -> Debug Sessions -> Log menu item to explore the log files for your most recent session. You can use the search box on the Engine administration tool's Help menu to search the Data Explorer Engine documentation for "log file" or "debugging" for more details.