fix(common-services): add connector.log retention to curator - #64
Open
gnanirahulnutakki wants to merge 1 commit into
Open
fix(common-services): add connector.log retention to curator#64gnanirahulnutakki wants to merge 1 commit into
gnanirahulnutakki wants to merge 1 commit into
Conversation
connector.log is shipped to Elasticsearch by the fid-exporter sidecar (fid chart source 'connector' -> index connector.log) but has no delete_indices action in curator.logs, so its daily indices are never reclaimed while every other shipped log is pruned at 7 days. Observed on the BSWH clusters before cleanup: bswh-use1 90 indices 55.8 GB oldest 2026-05-13 (78% of index data) bswh-use2 348 indices 167.6 GB oldest 2025-08-28 (85% of index data) Adds connector.log to curator.logs so it inherits the same 7-day delete_indices action as the other 24 prefixes.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
connector.logis shipped to Elasticsearch by the fid-exporter sidecar — the fid chart defines the source incharts/fid/values.yaml:…but
curator.logsin this chart has no entry for it. Every other prefix the exporter produces is pruned at 7 days;connector.logis the one exception, so its daily indices accumulate indefinitely.It is also the highest-volume index in the stack — one document per connector event, across every sync agent — so it becomes the dominant consumer of Elasticsearch disk within a few months.
Evidence
Measured on the BSWH clusters (before a manual cleanup):
connector.logindicesbswh-use12026-05-13bswh-use22025-08-28On
bswh-use2that is ~12 months of unpruned data. Curator was running nightly and healthy the whole time — it simply had no action matching this prefix, so every run loggedSkipping action "delete_indices" due to empty listfor the prefixes that had already aged out and never touchedconnector.log.Change
One entry added to
curator.logs. It inherits the chart's existing defaults (action: delete_indices,unit: days,unit_count: 7,direction: older), so it behaves exactly like the other 24 prefixes.Verification
Rendered
templates/elasticcurator/configmap.yamlbefore and after and diffed the result:connector.log, removed noneunit_count: 7action_file.ymlparses as valid YAML in both rendersThe only other diff is the positional renumbering of actions after index 11, which is inherent to
range $indexin the template — curator treats these as opaque action IDs executed in order, so the renumbering is behaviourally inert.Rollout note
This is a values-only change with no chart version bump, to avoid colliding with the in-flight
common-services/2.0.3release (#63). It needs to be picked up by the next published chart version.Consuming tenants pin the dependency explicitly, so they will each need a bump before this reaches a cluster — e.g.
radiantlogic-saas/bswh-use1is oncommon-services 2.0.1andradiantlogic-saas/bswh-use2on2.0.0, both inapp/charts/common-services/Chart.yaml.Curator runs once per cluster in the
-svcnamespace, so this single change covers every-svcnamespace that consumes the chart. The fid chart ships its own curator, but it defaults toenabled: falseand is not deployed on these clusters, so no corresponding change is needed there.