episodic: scheduled retention/GC - #61
Merged
Merged
Conversation
A config-driven background scheduler that prunes the episodic event log by
policy — the composition-root counterpart to the kv drivers' per-record TTL
sweepers. Opt-in per namespace; episodic only (other blocks reject a
retention block).
Policy (per namespace):
retention:
max_age_seconds: N # delete events older than N
max_items: N # and/or keep only the newest N by cursor
action: hard|soft # remove (default) or tombstone
interval_seconds: N # sweep cadence; 0 = default 300s
The scheduler (internal/retention) runs one ticker goroutine per policy,
driving the existing episodic Expire primitive in bounded batches: age via
BeforeTime, count via a reverse-limit-1 max-cursor probe + BeforeCursor.
It lives above the service layer on raw storage namespaces, so with
tenant_isolation on it discovers and prunes every tenant's namespace via a
new optional NamespaceLister driver capability (memory: map keys; postgres:
SELECT DISTINCT namespace) matched back to the config namespace with a new
auth.UnqualifyNamespace helper. Pruned events increment
memsidecar.eviction.total{block="episodic",cause="retention"}.
Lifecycle: constructed in main after the registries, Started, and Close
registered so LIFO shutdown stops sweeps before drivers close.
- internal/retention: Scheduler + PoliciesFromConfig + white-box tests
(age, count-keeps-newest, soft-tombstone, tenant fan-out, helpers)
- config: RetentionConfig + Enabled() + per-namespace validation (+tests)
- episodic memory/postgres: Namespaces() capability
- auth.UnqualifyNamespace; obs.EvictionRetention cause
- docs: episodic scheduled-retention section + config reference + example.yaml
Verified: unit suite green; full retention flow (age, count, tenant fan-out)
run against a real Postgres episodic driver; go build/vet, go test -race
./..., golangci-lint ./... all clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
7 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #60 (redis driver).
Config-driven background scheduler that prunes the episodic event log by policy — the composition-root counterpart to the kv TTL sweepers. Opt-in per namespace; episodic-only.
internal/retention.Scheduler: one ticker goroutine per policy, driving the existing episodicExpirein bounded batches (age → BeforeTime; count → reverse-limit-1 max-cursor probe + BeforeCursor)tenant_isolationon, fans out to every tenant's namespace via a new optionalNamespaceListerdriver capability (memory map keys; pgSELECT DISTINCT namespace) matched back with newauth.UnqualifyNamespaceobs.EvictionRetentioncause onmemsidecar.eviction.total; lifecycle wired for LIFO shutdownRetentionConfig+ validation (+tests); white-box scheduler tests; docsVerified: unit suite + full flow (age, count, tenant fan-out) against a real Postgres episodic driver.
🤖 Generated with Claude Code