Best practices for scanning an S3 object storage system

Use best practices for scanning S3-compliant object storage systems.

It is recommended to check the log files in the following directories after each scan:

/opt/ibm/metaocean/data/connections/s3/<connection_name>/debug/<scan_timestamp>/scanner.debug indicates whether the scan was successful or not.

/opt/ibm/metaocean/data/connections/s3/<connection_name>/error/<scan_timestamp>/scanner.error contains a list of all the messages that are not delivered to IBM Spectrum® Discover.

/opt/ibm/metaocean/data/connections/s3/<connection_name>/data/<scan_timestamp>/ contains a subfolder with the scanned data source name. There is a stats folder inside this folder that contains information about the number of objects in the data source or the number of objects or scanned files.

You can also compare the total size of the bucket that is reported in IBM Spectrum Discover with the total size of the S3 object store at source (if it is available).