Creating a sweep for retention date changes

You can configure and run a retention update sweep job in the Administration Console for Content Platform Engine to update retention for unexpired Automation Document Processing documents so that the Clean utility can complete successfully.

About this task

The CPDS Cleanup process fails when it tries to delete Automation Document Processing documents in Content Platform Engine and the document's retention period has not yet expired. While this is the intended behavior, the unexpired retention period prevents the cleanup endpoint from successfully completing in an authoring environment.

When you try to delete a project, you might see an error like the following example returned by the CPDS Clean REST API.
{"errors":[{"action":"Ask your system administrator to review the error and check the Content Project Deployment 
server error log files.","errorId":500,"explanation":
"The server reported an unspecified error."}],"status":{"code":500,"message":"Internal Runtime Server Error::com.filenet.api.exception.EngineRuntimeException 
FNRCC0011 Content Engine could not delete or move content because the retention period for the item has not expired. 
Current time: 1/5/21 10:31 PM; Retention time: 2/3/22 10:07 PM.","messageId":"FNRDD0501E"}}
Return Code=500

The retention date of the documents is set by the Doc Processing event handler according to the retention that was set in the Document Processing Designer.

To solve this problem, you can use the Administration Console for Content Platform Engine to create a sweep job. You set up the sweep to update unexpired retention settings. This change enables the Clean process to complete.

Procedure

To create the sweep job in the Administration Console for Content Platform Engine:

  1. Log in with a credential that allows you to create sweep jobs.
    This is most likely an administrator login.
  2. Find the deployed project that you want to clean:
    1. In the navigation pane, expand the object store and click Search.
    2. Create a New Object Store Search.
    3. For Class, choose Deployed Project.
    4. In the list of returned projects, choose the one that you want to clean.
  3. Determine the list of classes that were created when this project was deployed.

    Multiple versions of one project can be deployed. Gather all the classes across all project versions. You must modify the retention dates for all documents across all of these classes.

    Use the SQL View tab to enter an SQL statement that can find the classes, for example:
    SELECT This, * FROM ClassDefinition cd WHERE EXISTS 
    (SELECT v.* FROM DbaDeployedProjectVersion v WHERE v.DbaDeployedProject = OBJECT({<Deployed Project GUID>}) 
    AND v.DbaDeployedArtifactVersions AS a SATISFIES (a.DbaDeployedClassDefinition = cd.This))

    The query returns a list of document classes.

  4. Optional: Check for documents in the class with retention date set.
    You can use the results of this query to help verify the results of the later query.

    Go to New object store search > Simple View, select your class from the Class drop-down list, and click Run.

    You can use the Properties filter to show only classes with the retention date set.

  5. Make sure that the retention sweep job can modify the retention date property on each document of the class.
    This step must be done only once and then applies to all subsequently created retention sweep jobs.
    Retention update sweeps are supported in two modes: RetentionReductionAllow and RetentionReductionPrevent, controlled by the AllowRetentionReduction property.
    • When the mode is set to RetentionReductionAllow, the sweep updates the retention on all affected items, even if doing so results in the retention period being reduced.
    • When the mode is set to RetentionReductionPrevent, the sweep does not update the retention on any objects for which the update would result in reducing the retention period.
    The AllowRetentionReduction property is read-only, which means that the only way to change the value is to update the default value that is given in its property definition. You can change the default value by modifying the PropertyDefaultBoolean property on the Property Definition for the AllowRetentionReduction value.
    1. To get to the PropertyDefinition, navigate to OS > Data Design > Classes > Other Classes > Sweep > Sweep Job > Retention Update Job. There you can view the Class Definition for the Retention Update Job. The Class Definition contains a list of Property Definitions. Look for the PropertyDefinition of the AllowRetentionReduction property in order to change its default value from False to True.
      Important: You must set this property value back when you're done.
    2. Select the Allow Retention Reduction Definition to get to a Property Definition screen. On the new screen, look for the field Property Default Boolean. Change its value from False to True. Remember to reset this back when you are done.
  6. Create a sweep job for each gathered class:
    1. In the navigation pane, go to OS > Sweep Management > Job sweeps > Retention Update Jobs.
    2. Right-click to create a new Retention Update Job.
      Complete the wizard screens.

      For the first screen, give the job a display name to indicate which project and which class this job is for. Change the Sweep Mode from the Preview Only Counters default value to Normal so that when this job runs, it applies the changes to the retention date.

    3. Click Next to go to the Define Sweep Targets screen.

      For the Target class field, type in the display name of the class for which to update retention date. If you have multiple entries with the same display name, because the display name is not unique, you might want to change the display name of the class. Check the Include subclass enabled flag.

      You can also check Record failures to review errors in the Sweep Result page instead of the FileNet® P8 error log.

    4. Set the new retention date to be earlier than today.
    5. Click Finish.
  7. Verify and run the sweep job that you have just created:
    1. Make sure that the owner of the sweep job has permission to modify the retention date on the Object Store.

      Retention-update sweeps run with the access rights of the security principal that is specified as the owner of the sweep.

      Access the sweep job that is just created and check who is listed as the Owner. You can now update the owner.

      Make sure that the Owner has permission to Modify retention on the whole object store.

    2. Perform the last check of the sweep job before you run it.
      On the General page, make sure that the following fields are set properly:
      • Include subclasses is checked.
      • Record failures is checked.
      • Target class specifies the correct class.
      • Retention date is set to the expected value.
      • Sweep Mode is set to Normal.
    3. Run the job by clicking Enable, saving your change, and refreshing.

      The sweep start and end date are updated when the job runs and it also reports the number of elements updated.

      Note: After the job runs to completion, it cannot be rerun. If you want to run it again, you must create a new job. You might need to wait for 15 minutes to schedule a new job.
  8. Check the output of the sweep job by using the query that is shown in step 3.
  9. After updating retention dates on all documents across all classes of the project, delete all sweep jobs that were created in step 7.
  10. Reset the value for the PropertyDefinition of the AllowRetentionReduction property from True to False.
  11. Rerun the CPDS Clean project process.