Modifying the data retention policy for logging services

Logging service data is persisted on disk. Over time, unmanaged data growth fills up your disk space. To keep data size under control, retention policies are defined. The logging service curator component cleans up log data based on your retention policy.

In addition to the automatic removal of log data, you can manually remove log indices. For more information, see Manually removing log indices

Before you begin, consider the following tips:

curator runs on a daily schedule. As a result, you might retain one extra day of data.
When the data index is removed, a new data index is re-created when new data is sent to it.
Increased retention periods require more disk space and other compute resources. You might experience stability issues when you define longer retention periods without adjusting your resources.

View log indexes and the default retention policies for a logging instance in the following table.

Table 1. Logging service indexes
Log Index	Description	Default Retention Period (day)
`logstash-*`	Container logs	`1`
`audit-*`	ICP audit logs	`1`
`compliance-*`	ICP Vulnerability Advisor data	`90`
`secconfig1-*`	ICP Vulnerability Advisor data	`90`
`vulnerabilityscan-*`	ICP Vulnerability Advisor data	`90`
`.monitoring`	Elastic monitoring data	`7`
`.monitoring-alerts`	Elastic monitoring alert	`7`
`.watcher-history`	Elastic watcher	`7`

Complete the following steps to customize the data retention policy.

Note: If you follow the procedure to update the chart values, add the same lines that you include in the values-override.yaml file to your config.yaml file. This method allows the installer to reapply the settings changes during upgrade and rollback operations. Upgrade resets the chart to the chart defaults, overridden by the values set in your config.yaml.

Extract the existing logging service (Helm release) parameters.
- Extract Helm parameters by running the following command:
```
helm get values logging_service_release_name --tls > values-old.yaml
```
  For example:
```
helm get values logging --tls > values-old.yaml
```
  Note: logging is the Helm release name of the logging service installed by the IBM Cloud Private installer.
- Optionally, apply prior adjustments. All Kubernetes resource manifest adjustments that are made by using the kubectl command are overridden with values that are defined in chart parameters. Replica count, JVM heap size, or container memory limits are a few examples. If prior Kubernetes resource manifests were adjusted, make sure that you apply the same adjustments to values-old.yaml.
Prepare chart parameters.
- Create a values-override.yaml file to include the following curator settings.
  
  Note:
  - app refers to the container log.
  - You can set the unit value to values other than days.
  - Avoid setting a retention policy to less than one day.
```
curator:
 # in this example, container log retention period is set to 2 days
 app:
   count: 2
   unit: days
```
    For detailed parameter information, see the Helm chart readme file.
Download the chart.
- Identify chart version.
  
  Logging chart versions vary based on the installed IBM Cloud Private version. You can use IBM Cloud Private management console to find chart versions in the service catalog. The logging chart can be identified by the name ibm-icplogging under the mgmt-repo repository. You can also select SOURCE & TAR FILES from the IBM Cloud Private management console to find a local link to a chart.
- Download the chart .tar file.
  
  Run the following command by using the local link found in Step 3:
```
 curl -k https://<master ip>:8443/mgmt-repo/requiredAssets/ibm-icplogging-x.y.z.tgz > ibm-icplogging-x.y.z.tgz
```

Upgrade the Helm chart. Run the following command. Replace x.y.z with the version found in Step 3.

 helm upgrade logging_service_release_name ibm-icplogging-x.y.z.tgz -f values-old.yaml -f values-override.yaml --force --timeout 600 --tls

The logging service becomes available in approximately 5 - 10 minutes. You can also check Helm upgrade status by using the following command:
```
 helm history --tls logging_service_release_name
```

Modifying the data retention policy for the Vulnerability Advisor (VA) data

You can modify the data retention period by updating the retention policy. Complete the following steps to edit the data retention policy for the VA data:

Edit the logging-elk-elasticsearch-curator-config retention policy by running the following command:
```
kubectl edit cm logging-elk-elasticsearch-curator-config -nkube-system
```

Update the unit_count parameter in the policy. Your data retention policy for the VA might resemble the following content:

    action: delete_indices
    description: "Delete vulnerability advisor indices that are older than 90 days. Cron schedule: 30 23 * * *"
     options:
      timeout_override:
      continue_if_exception: True
      ignore_empty_list: True
      disable_action: False
    filters:
    - filtertype: pattern
      kind: regex
      value: '^(compliance-|secconfig1-|vulnerabilityscan-).*$'
    - filtertype: age
      source: name
      direction: older
      timestring: '%Y.%m.%d'
      unit: days
      unit_count: 90

Your data retention policy for the VA data is updated.