Exploring archives

Managing archive collections

Archive collections enhance discoverability so you can quickly locate relevant archives through collection names and descriptions rather than searching through individual files. You can create named collections, add multiple archive files to each collection, and manage these collections through the interface. Each archive file can belong to multiple collections, and deleting a collection does not affect the archive files themselves.

Adding an archive collection:

1. From the navigation menu, click Archives . Select the Archive Collections tab to see a table that shows the available archive files and collections.
2. Click New Collection to open the collection creation dialog.
3. Enter a name for the collection in the Name field.
4. Select one or more Archive File from the list to include them in the collection
5. Enter an optional description in the Description field to provide more context about the collection's purpose.
6. Click Create to select archive files to include in the collection.

   The new collection appears in the table.

After you create collections, you can also perform the following actions:

• To view files within a collection, click the expand icon next to the collection name in the collections table to display all archive files in an expandable row interface.
• To search for collections, enter search terms in the Search collections field to filter collections by name or description.
• From the action menu at the end of the collection row, click the Edit icon, modify the collection name, description, or included archive files, and click Save.
• To remove archive files from a collection:
  • Open the collection in edit mode, select the files to remove, click Remove, and save the collection.

    OR

  • Click the expand icon next to the collection name, all files that are included in the collection appear in the table. Click the remove icon at the end of the row.
• To delete a collection, from the action menu at the end of the row, click the Delete icon next to the collection name and confirm the deletion when prompted. Deleting a collection removes only the collection grouping; the archive files remain intact and accessible.
• To browse individual archive files in a collection, click the file name in the expanded collection view.

Viewing archive summaries and log history

You can view a summary of an archive, including source and target information, and details about who created the archive and when.

From the summary page, you can also review the log history table to see how many times the archive job was run and view associated log files.

To view an archive summary:

1. From the navigation menu, click Archives . Select the Archive files tab to view all available archive files.
2. Click the name of an archive to see a summary of its contents, including source and target information, and details about who created the archive and when.
3. In the log history table, review the list of archive job runs for this archive.
4. To export the IBM Optim Archive logs associated with creating this archive, click Export.

Filtering archive files

The Archive files page supports advanced filtering capabilities that help you quickly locate specific archive files within large collections. You can filter archive files by multiple criteria to narrow the results and find relevant archives more efficiently. Filtering works for both newly created archives and registered InfoSphere Optim archives.

To filter archive files:

1. From the navigation menu, click Archives . Select the Archive files tab to view all available archive files.
2. Click the Filter icon to open the filter panel.
3. Select one or more filter criteria from the available options:
   • File type - Filter by the type of archive file. InfoSphere Optim archives are identified with the file type InfoSphere Optim archive.
   • Source - Filter by the source. For flow-generated archives, this shows the flow name from which the archive was created. For registered InfoSphere Optim archives, this shows the associated storage profile or connection name.
   • Source type - Filter by the type of source system. Registered InfoSphere Optim archives show Storage profile as the source type.
   • Created by - Filter by the user who created or registered the archive.
   • Created at - Filter by a date range when the archive was created or registered.
4. Apply the selected filters to narrow the list of archive files.
5. Review the filtered list of archive files that match the specified criteria.
6. To modify filters, adjust the filter criteria in the filter panel and reapply.
7. To remove individual filters, clear the specific filter option.
8. To clear all filters, click Clear all or close the filter panel.

Multiple filters can be applied simultaneously to further refine the results. Active filters are displayed in the filter panel and can be reviewed or modified at any time. Filtering works alongside existing search and sorting functions to provide a more efficient archive browsing experience.

Browsing archived data

You can browse and view data from within your archives by using either the Archive Query Builder for visual filtering or the SQL Query Editor for custom SQL queries. The Archive Query Builder provides a visual interface for filtering and exploring archived table data, supporting the creation of complex queries without requiring SQL. For users who prefer SQL syntax, the SQL Query Editor provides direct SQL-based querying capabilities.

Querying archived data with SQL

The SQL Query Editor provides direct execution of custom SQL queries on archived data at the collection level, archive file level, and table level. You can write and run SQL queries with syntax highlighting to explore, filter, and analyze archived data without restoring it to the source database.

To query archived data by using SQL:

1. Navigate to the required level:
   • Collection level: From the navigation menu, click Archives . Select the Archive Collections tab, select a collection, and click the Query or SQL Editor tab.
   • Archive file level: Select the Archive files tab, select an archive file, open the file details view, and click the Query or SQL Editor tab.
   • Table level: From the list of tables in an archive, select a table name to open the table details, then click the Data Query tab.
2. In the SQL editor panel, write or paste your SQL query.
   • For table-level queries, the editor prepopulates with context-aware schema and table information. See example:

     SELECT * FROM ${SchemaName}.${Tablename} WHERE age > 25

   • For collection-level queries, see example:

     -- Collection-level query
     SELECT
         o.${Column1},
         o.${Column2},
         o.${Column3},
         o.${Column4},
         o.${Column5},
         d.${Column6},
         d.${Column7},
         d.${Column8}
     FROM ${SchemaName}.${TableName1} AS o
     LEFT JOIN ${SchemaName}.${TableName2} AS d
     ON CAST(o.${JoinColumn1} AS STRING) = d.${JoinColumn2}
     LIMIT 10;

   • For file-level queries, see example:

     -- File-level query
     SELECT
         s.${Column1},
         s.${Column2},
         s.${Column3},
         d.${Column4},
         d.${Column5},
         d.${Column6}
     FROM ${SchemaName}.${TableName1} AS s
     CROSS JOIN ${SchemaName}.${TableName2} AS d
     WHERE CAST(d.${Column6} AS INT) > ${Value};

3. Click Run all to execute the query.
4. View the results in the paginated data table. By default, 50 rows are displayed per page. Optionally, use the search functionality in the results table to filter displayed data.

The SQL Query Editor provides flexible, on-demand data access through familiar SQL syntax while maintaining data security and archive integrity. You can validate data, perform ad hoc analysis, and verify archive contents without the time-consuming process of restoring data to production databases.

Defining archive retention policies

Configure retention policies to define how long archive files are retained and to support compliance with business, legal, and regulatory requirements.

When an archive expires based on its retention policy, you are notified through the notifications menu . Expired archive files remain accessible. You can remove the expired policy and assign a new one.

The following scenarios apply when you manage a retention policy:

No retention policy set

You can set a new retention policy for the archive file.

Retention policy set, not expired

You can update the policy to a new date or remove it. Removing an active policy ends its protection period. The archive can then be modified or deleted, and you can assign a new retention policy.

Retention policy set, expired

Remove the expired policy before you can assign a new one. An inline notification displays the following message: Retention policy expired. The retention policy expired on [Date]. Remove the current policy to set a new one.

To configure the retention policy for an archive:

1. From the navigation menu, click Archives . Select the Archive files tab to view all available archive files.
2. Hover over the row for the archive for which you want to set the retention policy.
3. Click the overflow menu (⋮) icon on the right side of the archive row, then choose the retention policy action that is displayed for this archive file:
   • Click Set retention policy if no retention policy has been configured.
   • Click Update retention policy if a retention policy exists and has not expired.
   • Click Remove policy to remove an existing retention policy.
4. In the Retention Policy dialog, choose the type of retention policy you want to configure:
   • Define a specific Expiration Date
   • Define a retention period relative to the archive file creation date
5. Provide the required retention policy values.
6. Review the retention policy configuration, then click Save to apply the retention policy.

To remove a retention policy:

1. From the navigation menu, click Archives . Select the Archive files tab.
2. Hover over the archive row, then click the overflow menu (⋮) icon and select Remove policy.
3. In the confirmation dialog, review the message and click Remove retention policy to confirm.
   Important: This action cannot be undone. If the policy is active, removing it ends its protection period and allows the archive to be modified or deleted.

Restoring an archive

You can configure a new flow to restore the archived data to a selected target connection. This process works for both newly created archives and registered InfoSphere Optim archives.

To restore an archive:

1. From the navigation menu, click Archives . Select the Archive files tab to view all available archive files.
2. Hover over the row for the archive file you want to restore. Click the action menu at the end of the row and select Restore.

   The flow builder tool opens with the selected archive, already added to the flow builder canvas as the source connection node.

3. Click Nodes and then complete the flow by dragging other nodes onto the canvas. Add an access definition node and optionally, subsetting and masking nodes. Complete the flow by adding a target connection node. Click Save.
4. When you are ready to run the restore job, click Run flow.

For more information about using the flow builder to restore an archive, see Restoring data from an archive file.

Deleting source data from production

The Delete Source Data feature enables you to permanently remove archived data from the production database after a successful archive operation. This feature helps reduce database storage costs, comply with data retention policies, and implement effective data lifecycle management strategies.

When you select the Delete source data option, the system automatically creates a deletion operation based on the original archive configuration. The deletion process uses the same source connection and filtering criteria from the original archive to identify and remove the archived data from the production database.

The feature supports both first-time deletions and reexecution of existing deletion operations. For first-time deletions, the system creates a new deletion workflow and stores its identifier for future reference. For subsequent deletion attempts on job failure, the system reuses the existing workflow without creating duplicates.

To delete source data from production:

1. From the navigation menu, click Archives . Select the Archive files tab to view all available archive files.
2. Locate the archive file for which you want to delete source data in the Archives table.
3. Click the overflow menu (⋮) icon on the right side of the archive row.
4. Select Delete source data from the options menu.

   The Delete Source Data modal opens, displaying the archive name and a warning message about the irreversible nature of the operation.

5. Choose one of the following deletion modes:
   • Leave the checkbox unchecked to delete with validation (recommended for safer execution).
   • Check Delete source data without validation to delete without validation (faster processing).
6. Click Delete to confirm and run the deletion operation.

   The system creates a deletion job based on the original archive configuration.

7. Monitor the deletion job status in the Jobs page. For more information about viewing job details and logs, see Viewing job details, logs, and process reports.
8. Once the status shows FINISHED, verify the deletion completion by reviewing job logs and database storage metrics.

The Delete source data option is automatically disabled when a deletion operation is already in progress or has been completed, preventing accidental duplicate deletions. You can track the deletion progress through standard job status indicators that are displayed directly in the Archives table.