TT-17238: recover cache for secret stores

[vladzabolotnyi]

Description

Related Issue

Motivation and Context

Test Coverage For This Change

Screenshots (if appropriate)

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Refactoring or add test (improvements in base code or adds test coverage to functionality)
• Documentation updates or improvements.

Checklist

• I have reviewed the guidelines for contributing to this repository.
• Make sure you are requesting to pull a topic/feature/bugfix branch (right side). If PRing from your fork, don't come from your master!
• Make sure you are making a pull request against our master branch (left side). Also, it would be best if you started your change off our latest master.
• My change requires a change to the documentation.
  • I have manually updated the README(s)/documentation accordingly.
  • If you've changed APIs, describe what needs to be updated in the documentation.
• I have updated the documentation accordingly.
• Modules and vendor dependencies have been updated; run go mod tidy && go mod vendor
• When updating library version must provide reason/explanation for this update.
• I have added tests to cover my changes.
• All new and existing tests passed.
• Check your code additions will not fail linting checks:
  • gofmt -s -w .
  • go vet ./...

Ticket Details

TT-17238

Status	Closed
Summary	Implement core caching and secret store wrapper

Generated at: 2026-08-14 06:41:10

[github-actions]

CLA Assistant Lite bot:
Thank you for your submission, we really appreciate it. Like many open-source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution. You can sign the CLA by just posting a Pull Request Comment same as the below format.

---

I have read the CLA Document and I hereby sign the CLA

---

1 out of 2 committers have signed the CLA.
✅ @imogenkraak
❌ @vladzabolotnyi
_{You can retrigger this bot by commenting recheck in this Pull Request}

[probelabs]

This PR introduces a configurable, in-memory caching layer for secret stores to improve performance and reduce the load on backend providers like Vault or Consul.

Files Changed Analysis

The change is substantial, with 13 files modified, adding over 2000 lines and removing ~120. The bulk of the additions are in new files that implement the caching logic and its comprehensive test suite:

• kv/internal/cache/cache.go: The core TTL-based, thread-safe cache implementation.
• kv/internal/cache/cache_test.go: Extensive unit and concurrency tests for the new cache.
• kv/context.go: Introduces a context-based mechanism to bypass the cache for force-refreshes.
• kv/internal/store/cache_bypass_test.go: Dedicated tests for the cache bypass functionality.

The caching logic is integrated into the existing architecture by modifying key files:

• kv/internal/store/store.go: The SecretStore is enhanced to act as a caching decorator, wrapping the underlying providers.
• kv/config.go: A new CacheConfig struct is added to allow users to configure caching behavior (TTLs, refresh windows, etc.).
• kv/registry/registry.go & from_config.go: These are updated to plumb the new cache configuration down to the SecretStore during initialization.
• Various test files (store_test.go, from_config_test.go, integration_test.go) are updated to validate the new caching behavior.

Architecture & Impact Assessment

• What this PR accomplishes
  This PR adds a caching layer to the secret resolution process, aiming to decrease latency for secret lookups and reduce the request volume sent to external secret management systems.

• Key technical changes introduced

1. In-Memory Cache: A new cache.Cache component (kv/internal/cache/cache.go) provides TTL-based caching for secret values and specific provider errors.
2. Stale-While-Revalidate: The SecretStore now implements a stale-while-revalidate strategy. It returns a cached (stale) value immediately while triggering a non-blocking background refresh if the value is nearing its expiry. This ensures low latency for callers.
3. Negative Caching: Caches KeyNotFoundError and StoreUnavailableError with separate, configurable TTLs. This prevents hammering a backend for a key that doesn't exist or when the service is temporarily down.
4. Cache Bypass: A new kv.WithCacheBypass(ctx) function allows callers to explicitly skip the cache and fetch a fresh value from the provider. This is critical for use cases like immediate secret rotation.
5. Deduplication: The implementation uses two separate singleflight.Groups to deduplicate requests: one for initial fetches (sf) and another for background refreshes (sfRefresh), preventing redundant calls to the provider under concurrent load.

• Affected system components

  • Secret Resolution Flow: The core logic of SecretStore.Get is fundamentally changed. All consumers resolving secrets through the kv.Registry will now interact with the cache, unless the provider implements the Standaloner interface.
  • Configuration: A new kv.cache section is introduced in the configuration, which operators will need to manage.
  • Provider Interaction: The frequency of calls to kv.Provider implementations (like Vault and Consul) will be significantly reduced when caching is enabled.

• Visualization of the Get flow:

sequenceDiagram
    participant C as Caller
    participant S as SecretStore
    participant Cache as cache.Cache
    participant P as Provider

    C->>S: Get(ctx, path)
    S->>Cache: Get(path)

    alt Cache Hit (and not near expiry)
        Cache-->>S: value, found=true, needsRefresh=false
        S-->>C: value
    else Cache Hit (near expiry)
        Cache-->>S: value, found=true, needsRefresh=true
        S-->>C: value (stale value returned immediately)
        S->>S: triggerBackgroundRefreshOnce(path)
        Note right of S: singleflight deduplicates background fetches
    else Cache Miss or Bypass
        Cache-->>S: "", found=false
        S->>P: Get(ctx, path)
        Note right of S: singleflight deduplicates concurrent requests
        P-->>S: newValue, err
        S->>Cache: Set(path, newValue, err)
        S-->>C: newValue, err
    end

Loading

Scope Discovery & Context Expansion

This is a foundational change to a core infrastructure component. The impact extends beyond the kv package to any service or module that relies on it for secret management.

• Operational Impact: Operators gain new knobs for performance tuning but must understand the implications of TTL settings. Secret rotation procedures may need to be updated to use kv.WithCacheBypass to ensure new secrets are fetched immediately.
• Reliability: The negative caching feature can improve system resilience during temporary provider outages. However, misconfigured TTLs could delay recovery visibility.
• Further Exploration: To fully grasp the impact, it would be beneficial to investigate:
  1. Consumers: Where is resolver.NewResolver used throughout the codebase to identify all systems that will now use cached secrets?
  2. Configuration: Are there example or default configuration files that should be updated with sensible defaults for the new kv.cache settings?
  3. Standalone Providers: Check for implementations of the Standaloner interface, as these providers will bypass the new caching wrapper entirely.

Metadata

• Review Effort: 4 / 5
• Primary Label: feature

---

Powered by Visor from Probelabs

Last updated: 2026-08-14T06:44:15.680Z | Triggered by: pr_updated | Commit: 3daf5f6

💡 TIP: You can chat with Visor using /visor ask <your question>

[probelabs]

Security Issues (1)

Severity	Location	Issue
🟡 Warning	kv/internal/cache/cache.go:23-32	The in-memory cache for secrets is unbounded. If a large number of unique secret keys are requested, this can lead to uncontrolled memory growth, potentially causing the application to run out of memory and crash. This could be exploited as a denial-of-service vector. 💡 Suggestion Replace the simple map-based cache with a bounded cache that has a configurable size limit. An LRU (Least Recently Used) eviction policy is typically a good choice for this kind of cache. This will protect the service from running out of memory when dealing with a large and unpredictable number of secret keys.

Security Issues (1)

Severity	Location	Issue
🟡 Warning	kv/internal/cache/cache.go:23-32	The in-memory cache for secrets is unbounded. If a large number of unique secret keys are requested, this can lead to uncontrolled memory growth, potentially causing the application to run out of memory and crash. This could be exploited as a denial-of-service vector. 💡 Suggestion Replace the simple map-based cache with a bounded cache that has a configurable size limit. An LRU (Least Recently Used) eviction policy is typically a good choice for this kind of cache. This will protect the service from running out of memory when dealing with a large and unpredictable number of secret keys.

\n\n ### Architecture Issues (1)

Severity	Location	Issue
🟠 Error	kv/internal/store/store.go:141-143	The background cache refresh logic (`doBackgroundRefresh`) only updates the cache on success (`if err == nil`). If a secret is deleted from the backend, the provider will return a `KeyNotFoundError`. This error is ignored by the background refresh, causing the store to continue serving a stale value until the original TTL expires. This could lead to the application using a revoked secret for an extended period. 💡 Suggestion The background refresh should update the cache with any cacheable result, including `KeyNotFoundError`. The `cache.Set` method already contains the logic to determine which errors are cacheable (e.g., `KeyNotFoundError`, `StoreUnavailableError`) and which are not (e.g., transient or context errors). The logic in `doBackgroundRefresh` should be consistent with the foreground fetch path and delegate this decision to `cache.Set` by passing the error through.

Performance Issues (1)

Severity	Location	Issue
🟡 Warning	kv/internal/cache/cache.go:193-219	The cache cleanup mechanism iterates over all entries to find and remove expired items. This O(N) scan holds a read lock for the duration of the iteration, which could lead to lock contention and CPU spikes if the number of cached keys becomes very large. While acceptable for a moderate number of keys, it does not scale well to hundreds of thousands or millions of entries. 💡 Suggestion For improved scalability with a very large number of keys, consider a data structure that allows for more efficient eviction of expired items without a full scan. Examples include using a min-heap to track expiration times or a time-bucketed hash wheel. However, for the expected use case of caching a few hundred to a few thousand secrets, the current implementation is a reasonable and simple trade-off.

---

Powered by Visor from Probelabs

Last updated: 2026-08-14T06:42:22.886Z | Triggered by: pr_updated | Commit: 3daf5f6

💡 TIP: You can chat with Visor using /visor ask <your question>

[github-actions]

🚨 Jira Linter Failed

Commit: 3daf5f6
Failed at: 2026-08-14 06:41:11 UTC

The Jira linter failed to validate your PR. Please check the error details below:

🔍 Click to view error details

failed to validate Jira issue: jira ticket TT-17238 has status 'Closed' but must be one of: In Dev, In Code Review, Ready For Dev, Dod Check, Merge, In Design Review

Next Steps

• Ensure your branch name contains a valid Jira ticket ID (e.g., ABC-123)
• Verify your PR title matches the branch's Jira ticket ID
• Check that the Jira ticket exists and is accessible

---

This comment will be automatically deleted once the linter passes.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
96.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud