Overview
The Data Retention Regulator is a software utility designed to manage and regulate the lifespan of data within Aeron Cluster and Aeron Archive environments. It is available to Aeron Premium subscribers. Its primary function is to enable disk space management by controlling how long data is retained.
Key Features
How it works
The tool works by listing recordings in the archive and applying a Data Retention Policy to determine the extent of data retention for each recording. This policy can be a simple, single-action policy or a composite of multiple policies forming a comprehensive strategy. The tool includes built-in policies for Aeron Cluster and Cluster Standby nodes that understand node recovery plans and ensure data necessary for recovery is retained.
The regulator then applies the instructions to the recording, which may involve detaching segments, deleting segments, or purging the entire recording. If the policy doesn’t apply to the recording, the regulator will keep it as is. If it fails to process a recording, for example, due to an active replay, the tool will continue to process other recordings and, when rerun, re-attempt any failed steps.
Policy examples
RetainEnoughDataForRecovery Policy
Keeps a specified number of snapshots and associated logs for node recovery.
RetainLengthPolicy
Ensures a minimum amount of data (e.g., 10 GB) is kept per recording.
MatchingStreamsPolicy
Applies retention rules to recordings matching certain channel URIs or stream IDs.
MatchingLogRecordingPolicy
Targets the Consensus Module’s log recording for retention or deletion.
RetainMaxOfPolicy
Combines multiple policies to retain the maximum data dictated by any of the policies.
DeleteRecordingsAfterPeriodPolicy
Deletes recordings that have been stopped for a specified period.
DeleteSegmentsAfterPeriodPolicy
Removes old segment files not modified within a set timeframe.
CascadePolicy
Applies a sequence of policies, executing the first one that covers the recording.
Aeron Data Retention Regulator Datasheet
For a detailed overview of the Aeron Data Retention Regulator, download the datasheet:
- Key features
- How it works
- Policy examples
- Usage scenarios
