TEM Audit Trail Cleaner

This page has not been liked. Updated 2/5/15 1:32 PM by VinhDaoTags: None

This page contains information about the TEM Audit Trail Cleaner for Windows and Linux servers.

 

For Windows Servers

Download - version 3.0.10.89 (2.36 MB) 

 ***  Note:  The current version of the tool does not remove some data from the database as intended (specifically in the area of blob fields.  However, this fact does not cause any problems for the data that it does remove or with the database itself.  Development is working on updating this tool to be more comprehensive.  ***    

The TEM Audit Trail Cleaner application is a command line utility that will allow you to remove previous versions and deleted versions of custom Fixlets, Properties, Analyses and Actions from the IBM Endpoint Manager database. The associated QuestionResults and FixletResults are also removed along with the definitions. The data that will be removed is no longer being used by any of the IBM Endpoint Manager components (Consoles, Web Reports, Server, Clients) but is retained in the database to serve as an audit trail of your deployment. Running this tool will delete that audit trail.

The tool will physically remove the data from the database, reducing the database size and potentially improving performance of the database. This may help to improve insertion speeds into the database for FillDB and load times for the console. .

NOTE: Support recommends that you create a database backup before running this tool and save the backup in a secure location. This database backup will preserve your audit trail while allowing the production database to be smaller.

NOTE: Support recommends disabling any re-indexing/maintenance jobs in the database until the audit trail cleaner has finished. Once the audit trail cleaner finishes re-enable these re-indexing/maintenance jobs. If the audit trail cleaner tool is running while the re-indexing/maintenance jobs are running database contention may result causing the FillDB service to stop and reports received from clients will not make it into the database until the FillDB service is restarted.

NOTE: Currently researching the requirements for running this tool when the deployment has additional DSA servers.

NOTE: Currently the audit trail cleaner tool does not delete hidden expired/orphaned actions from the database. To clean these out, take the following steps in this KB article.

NOTE:  We are no longer recommending performing a database shrink operation after running this tool to reclaim freed disk space from the database unless you are running into disk space issues.  Running a database shrink creates fragments in database file pages which may lead to inefficiencies during the creation of new database records.

The tool can count and delete the following sets of data: 

  • Older Versions of Custom Authored Content - Every edit to Fixlets/Tasks/Baselines/Analyses creates a new version, the older versions can be deleted.
  • Older Versions of Actions - Any time you stop or start and Action a new version is created, the older versions can be deleted.
  • Deleted Custom Authored Content (all versions) - When you delete a Fixlet/Task/Besline/Analysis using the console, the data is marked as deleted in the database and preserved. The deleted content, including all of the previous versions, and the corresponding client reports can be deleted .
  • Deleted Actions (all versions) - When you delete an action using the console, the data is marked as deleted in the database and preserved. The deleted actions, including all of the previous versions , and the corresponding client reports can be deleted.
  • Useless Action Results - Older versions of IBM Endpoint Manager (before BigFix 7.2.4.60) could cause clients to report ActionResults which were not used in any way but would use up space in the database. These useless ActionResults can be deleted.

 

Additional Information

Synchronizing BES Consoles - The console maintains a local cache of the database that will become unsynchronized when data is removed with this tool. To prevent this from happening, the tool sets a flag the database to force all consoles to re-load the cache on the next started up.

 

Command Line Arguments

Optional Arguments:

-h : Displays this help message
-? : Displays this help message
-help : Displays help message for installing this tool as a service.
-f : Runs in silent mode.  No standard output and script runs automatically to completion.
-l : Sends standard out to logfile created in the directory of this application with name BESAuditTrailCleaner.log
-L <Folder> : Specify a folder to create logs in. Do not include a file name.
-d <BES DSN> : Used to specify a DSN to connect to the BES Database with.  Defaults to bes_bfenterprise if not specified
-u <username> : Used to specify a username for SQL authentication to the database.
-p <password> : Used to specify a password for SQL authentication to the database.
-c : Do NOT count rows prior to deletion.
-S : Remove All Audit Trail data.
-v : Remove Older Versions of Custom Authored Content.
-V : Remove Older Versions of Actions.
-R : Remove Older Versions of Relay.dat
-o : Remove Deleted Custom Authored Content.
-O : Remove Deleted Actions.
-t : Remove Useless Action Results.
-g : Remove Hidden Manual Computer Group Actions.
-G <Days> : Hidden Manual Computer Group Actions over this many days will be deleted. (Default 180)
-s : Synchronize BES Consoles
-b <batch size> : Performs database deletion operations in batches of this size. The tool will pause between batches until the user presses a key (unless run in silent mode with -f).
-M : Removes Deleted Mailbox Files.
-D : Specify a date in MM-DD-YYYY format to have old data removed prior to this date.

 

Sample calls:

BESAuditTrailCleaner.exe -h

BESAuditTrailCleaner.exe -l

BESAuditTrailCleaner.exe -l -L "C:\Temp"

BESAuditTrailCleaner.exe -d database_dsn -u database_user -p database_password

BESAuditTrailCleaner.exe -S 

BESAuditTrailCleaner.exe -S -b 1000000 -f 

 

 

 

For Linux Servers

Download - version 3.0.11.8 (5.29 MB)

The downloaded package includes the three files: BESAuditTrailCleaner.shBESAuditTrailCleaner binary, and BESAuditTrailCleaner_linux_help.txt.

To use the tool, you just need to:

  1. Extract the tar file (BESAuditTrailCleaner_v3.0.11.#.tar) on the server system.
  2. Edit the file BESAuditTrailCleaner.sh to update the DB2LIBPATH and DB2INSTANCE variables (if needed).
  3. Run the tool using the command the command ./BESAuditTrailCleaner.sh  with the proper options. 
NOTE: Make sure you run the BESAuditTrailCleaner.sh, instead of the BESAuditTrailCleaner binary, because the environment has to be properly set to allow the tool to run.

 

To display the usage of the tool your can use the option "-h" or look at the BESAuditTrailCleaner_linux_help.txt.

 

 This tool is used to clear out historical data from the IEM Server database that is stored to serve as an audit trail. This audit trail will grow slowly over the lifetime of a BES deployment. The audit trail contains deleted and previous VERSIONS of Fixlets, Tasks, Actions and Analyses.  The audit trail is not used by BES in any way and can be deleted to reduce the database size. BigFix recommends you create a historic archive of the current database and save it to a secure location before running this tool to preserve the audit trail, thus removing it from the product database but not completely deleting the history.

NOTE: Support recommends that you create a database backup before running this tool and save the backup in a secure location. This database backup will preserve your audit trail while allowing the production database to be smaller.

NOTE: Support recommends disabling any re-indexing/maintenance jobs in the database until the audit trail cleaner has finished. Once the audit trail cleaner finishes re-enable these re-indexing/maintenance jobs. If the audit trail cleaner tool is running while the re-indexing/maintenance jobs are running database contention may result causing the FillDB service to stop and reports received from clients will not make it into the database until the FillDB service is restarted.

NOTE: Currently researching the requirements for running this tool when the deployment has additional DSA servers.

NOTE:  We are no longer recommending performing a database shrink operation after running this tool to reclaim freed disk space from the database unless you are running into disk space issues.  Running a database shrink creates fragments in database file pages which may lead to inefficiencies during the creation of new database records.

The tool can count and delete the following sets of data:

  • Older Versions of Custom Authored Content - Every edit to Fixlets/Tasks/Baselines/Analyses creates a new version, the older VERSIONS can be deleted.
  • Older Versions of Actions - Any time you stop or start and Action a new version is created, the older VERSIONS can be deleted.
  • Older Versions of relay.dat - Any time you install or uninstall a new relay a new version is created, the older VERSIONS can be deleted.
  • Deleted Custom Authored Content (all VERSIONS) - When you delete a Fixlet/Task/Besline/Analysis using the BES Console, the data is marked as deleted in the database and preserved.  The deleted content, including all of the previous VERSIONS, and the corresponding client reports can be deleted
  • Deleted Actions (all VERSIONS) - When you delete an action using the BES Console, the data is marked as deleted in the database and preserved.  The deleted actions, including all of the previous VERSIONS, and the corresponding client reports can be deleted.
  • Useless Action Results - Older VERSIONS of BigFix ( before 7.2.4.60 ) could cause BES Clients to report ActionResults which were not used in any way but would use up space in the database.  These useless ActionResults can be deleted.
  • Hidden Manual Computer Group Actions - Manual Computer Groups create hidden actions that add and remove computers to groups and the actions can build up over time.  This option deletes actions after an expiration period (default 180 days) from when they are created.  If BES Clients do not run the actions prior to deletion, the BES Client may behave inconsistently with how it appears in the BES Console.  Any BES Consoles left running must be restarted when this tool is run, you will receive error messages when using computer groups until the BES Console is restarted.
  • Older Version of Mailbox Files - Deleted Mailbox Files are stored in a table in the database and can be removed.

 

 
Additional Information:

Batched deletion - Deleting large sets of data will lead to the SQL transaction log growing quickly, the log will temporarly be larger than the data being removed until the database is shrunk.  Batched deletion will remove results in sets and pause between each set, giving you time to shrink and reclaim disk space as you go. 

Synchronizing BES Consoles - The BES Console maintains a local cache of the database that will become unsynchronized when data is removed with this tool. To prevent this from happening, the tool sets a flag the database to force all BES Consoles to re-load the cache on the next started up.

 

Command Line Arguments

Optional Arguments:

-h : Displays this help message
-? : Displays this help message
-f : Runs in silent mode.  No standard output and script runs automatically to completion.
-l : Sends standard out to logfile created in the directory of this application with name BESAuditTrailCleaner.log
-L <Folder> : Specify a folder to create logs in. Do not include a file name.
-u <username> : Used to specify a username for SQL authentication to the database.
-p <password> : Used to specify a password for SQL authentication to the database.

-H <hostname> : Used to specify a hostname for SQL authentication to the database.

-d <port> : Used to specify a port for SQL authentication to the database. 

-C <db name> : Used to specify the database name.
-c : Do NOT count rows prior to deletion.
-S : Remove All Audit Trail data.
-v : Remove Older Versions of Custom Authored Content.
-V : Remove Older Versions of Actions.
-R : Remove Older Versions of Relay.dat
-o : Remove Deleted Custom Authored Content.
-O : Remove Deleted Actions.
-g : Remove Hidden Manual Computer Group Actions.

-M : Removes Deleted Mailbox Files.
-G <Days> : Hidden Manual Computer Group Actions over this many days will be deleted. (Default 180)
-s : Synchronize BES Consoles
-b <batch size> : Performs database deletion operations in batches of this size. The tool will pause between batches until the user presses a key (unless run in silent mode with -f).

 

Sample calls:

./BESAuditTrailCleaner -h
./BESAuditTrailCleaner -l
./BESAuditTrailCleaner -l -L "/tmp"
./BESAuditTrailCleaner -u database_user -p database_password -H database_hostname -d database_port -C database_name
./BESAuditTrailCleaner -S
./BESAuditTrailCleaner -S -b 1000000 -f