We complete the actual data purge by deleting entire indices once all records have expired. For example, an index will rollover either when the average shard size breaches a threshold or when the age of the index exceeds the data retention period for the account/event type. We will wait until all documents in the index have expired, then delete the entire index.
Sometimes, the index doesn't rollover itself. In those cases, you can follow the instructions below to roll it over.
1. Use the rollover index API call:
http://<host:9080>/v1/admin/cluster/{cluster}/index/{index}/rollover
2. Complete the request with headers:
curl -XPOST http://<host:9080>/v1/admin/cluster/{cluster}/index/{index}/rollover -H"Authorization: Basic <Base64 encoded ad.accountmanager.key.controller from events-service-api-store.properties>" -H"Content-Type: application/json" -H"Accept: application/json"
3. Once the command is successfully executed, verify it using the output of the following:
curl http://localhost:9200/_cat/indices?v
4. Later, once the retention period setting comes into effect and all documents in the old index become expired, the entire index will be deleted.
