TT-18001: Add caching with invalidation logic

[vladzabolotnyi]

Description

Ticket: https://tyktech.atlassian.net/browse/TT-18001

Related Issue

Motivation and Context

Test Coverage For This Change

Screenshots (if appropriate)

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Refactoring or add test (improvements in base code or adds test coverage to functionality)
• Documentation updates or improvements.

Checklist

• I have reviewed the guidelines for contributing to this repository.
• Make sure you are requesting to pull a topic/feature/bugfix branch (right side). If PRing from your fork, don't come from your master!
• Make sure you are making a pull request against our master branch (left side). Also, it would be best if you started your change off our latest master.
• My change requires a change to the documentation.
  • I have manually updated the README(s)/documentation accordingly.
  • If you've changed APIs, describe what needs to be updated in the documentation.
• I have updated the documentation accordingly.
• Modules and vendor dependencies have been updated; run go mod tidy && go mod vendor
• When updating library version must provide reason/explanation for this update.
• I have added tests to cover my changes.
• All new and existing tests passed.
• Check your code additions will not fail linting checks:
  • gofmt -s -w .
  • go vet ./...

Ticket Details

TT-18001

Status	Open
Summary	Add caching layer with invalidation logic

Generated at: 2026-08-14 08:06:22

[github-actions]

CLA Assistant Lite bot:
Thank you for your submission, we really appreciate it. Like many open-source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution. You can sign the CLA by just posting a Pull Request Comment same as the below format.

---

I have read the CLA Document and I hereby sign the CLA

---

1 out of 2 committers have signed the CLA.
✅ @imogenkraak
❌ @vladzabolotnyi
_{You can retrigger this bot by commenting recheck in this Pull Request}

[github-actions]

🚨 Jira Linter Failed

Commit: 3daf5f6
Failed at: 2026-08-14 08:06:23 UTC

The Jira linter failed to validate your PR. Please check the error details below:

🔍 Click to view error details

failed to validate Jira issue: jira ticket TT-18001 has status 'Open' but must be one of: Merge, In Design Review, In Dev, In Code Review, Ready For Dev, Dod Check

Next Steps

• Ensure your branch name contains a valid Jira ticket ID (e.g., ABC-123)
• Verify your PR title matches the branch's Jira ticket ID
• Check that the Jira ticket exists and is accessible

---

This comment will be automatically deleted once the linter passes.

[probelabs]

This PR introduces a comprehensive, in-memory caching layer for the kv (key-value) secret management package. The goal is to reduce latency and decrease the load on backend secret providers like Vault or Consul by serving frequently accessed secrets from memory.

Files Changed Analysis

• Total Files Changed: 13
• Additions/Deletions: +2009 / -118
• Key Observations: The change is dominated by new code, primarily for the caching logic (kv/internal/cache/) and its extensive tests. The SecretStore component is significantly refactored to integrate caching, and existing tests are updated to be cache-aware. Configuration is plumbed through the registry to the SecretStore instances.

Architecture & Impact Assessment

What this PR accomplishes

This PR adds a configurable in-memory caching layer to the secret retrieval process. It aims to improve performance and resilience by reducing direct calls to external secret backends.

Key technical changes introduced

1. Configurable Caching: A new cache section in the kv configuration (kv/config.go) allows enabling/disabling the cache and setting TTLs.
2. Stale-While-Revalidate: Implements a proactive cache refresh mechanism (RefreshBeforeExpiry) where expired items are still served while a background task fetches the new value. This prevents latency spikes on cache misses.
3. Negative Caching: Caches errors like "key not found" and transient provider failures for configurable durations (NegativeTTLNotFound, NegativeTTLTransient). This prevents hammering a failing or misconfigured backend.
4. Cache Invalidation: A context-based mechanism (kv.WithCacheBypass) is introduced to force a fetch from the backend, bypassing the cache and re-populating the entry with a fresh value.
5. Core Implementation: The new kv/internal/cache/cache.go contains the thread-safe caching logic, including a background goroutine for cleaning up expired entries.
6. Integration: The SecretStore (kv/internal/store/store.go) is refactored to wrap the underlying provider with this caching logic, sitting alongside the existing singleflight deduplication.

Affected system components

• kv.Config: The main configuration schema is updated, requiring documentation for operators.
• kv.SecretStore: Its internal logic is fundamentally changed. All secret lookups now go through the caching layer first.
• kv.Registry: The store initialization process is modified to construct stores with the new cache configuration.

System Flow Diagram

sequenceDiagram
    participant C as Caller
    participant SS as SecretStore
    participant CA as cache.Cache
    participant P as Provider

    C->>SS: Get(ctx, "key")
    SS->>CA: Get("key")
    alt Cache Hit (valid)
        CA-->>SS: value, found=true, needsRefresh=false
        SS-->>C: returns value
    else Cache Hit (stale)
        CA-->>SS: value, found=true, needsRefresh=true
        SS-->>C: returns stale value
        note right of SS: Triggers non-blocking background refresh
        SS->>P: Get(bg_ctx, "key")
        P-->>SS: new_value
        SS->>CA: Set("key", new_value)
    else Cache Miss / Bypass
        CA-->>SS: "", found=false
        note right of SS: singleflight deduplicates provider calls
        SS->>P: Get(ctx, "key")
        P-->>SS: value, err
        SS->>CA: Set("key", value, err)
        SS-->>C: returns value, err
    end

Loading

Scope Discovery & Context Expansion

This change impacts the entire secret resolution pipeline. The introduction of caching alters the performance and data consistency characteristics of the system.

• Consumers: Any service using this storage library will need to be aware of the new kv.cache configuration options. The default behavior (cache disabled) remains the same, but enabling it requires understanding the TTL and stale data implications.
• Error Handling: Negative caching means that KeyNotFoundError will be cached. Application-level retry logic might be affected, as the error will persist for the duration of the negative TTL without hitting the backend again.
• Operations: Operators will need to be able to tune cache TTLs based on secret rotation policies and performance requirements. The cache bypass mechanism provides a crucial tool for forcing immediate updates.

Metadata

• Review Effort: 4 / 5
• Primary Label: feature

---

Powered by Visor from Probelabs

Last updated: 2026-08-14T08:08:19.320Z | Triggered by: pr_opened | Commit: 3daf5f6

💡 TIP: You can chat with Visor using /visor ask <your question>

[probelabs]

Security Issues (1)

Severity	Location	Issue
🟡 Warning	kv/internal/cache/cache.go:32	The in-memory cache for secrets is unbounded, which can lead to a denial-of-service (DoS) attack through memory exhaustion. If an attacker can influence the secret paths being resolved, they could force the application to cache an unlimited number of entries, causing the process to run out of memory. 💡 Suggestion Implement a limit on the cache size and an eviction policy. A simple approach is to add a `maxSize` configuration option. When the cache exceeds this size, evict entries based on a policy like Least Recently Used (LRU). Consider using a library like `hashicorp/golang-lru` for a robust implementation.

Security Issues (1)

Severity	Location	Issue
🟡 Warning	kv/internal/cache/cache.go:32	The in-memory cache for secrets is unbounded, which can lead to a denial-of-service (DoS) attack through memory exhaustion. If an attacker can influence the secret paths being resolved, they could force the application to cache an unlimited number of entries, causing the process to run out of memory. 💡 Suggestion Implement a limit on the cache size and an eviction policy. A simple approach is to add a `maxSize` configuration option. When the cache exceeds this size, evict entries based on a policy like Least Recently Used (LRU). Consider using a library like `hashicorp/golang-lru` for a robust implementation.

\n\n ### Architecture Issues (1)

Severity	Location	Issue
🟡 Warning	kv/internal/cache/cache.go:168-171	The cache cleanup interval is dynamically set to the minimum configured TTL. This can lead to infrequent cleanups and potential memory bloat if all configured TTLs are very long (e.g., hours), as expired items will linger in memory until the next tick. A fixed, more frequent cleanup interval would provide more predictable memory reclamation. 💡 Suggestion Consider decoupling the cleanup interval from the item TTLs. A fixed, reasonably frequent interval (e.g., every minute) could be used instead to ensure timely cleanup of expired entries regardless of TTL length. This would make memory usage more predictable.

Performance Issues (1)

Severity	Location	Issue
🟡 Warning	kv/internal/cache/cache.go:197-201	The `cleanup` function performs a full scan of the cache map to find expired entries. This O(N) operation, where N is the number of cached items, can become a performance bottleneck for very large caches. It holds a read lock during the scan, which can block write operations and lead to contention. 💡 Suggestion For improved scalability with a very large number of keys, consider an alternative eviction strategy. One common approach is to shard the cache into multiple independent maps, each with its own lock and cleanup routine, to reduce the scope of locks and parallelize the work. Another option is a probabilistic approach, where a small random sample of keys is checked for expiration during each cleanup cycle, which is much faster than a full scan but less precise.

---

Powered by Visor from Probelabs

Last updated: 2026-08-14T08:07:55.634Z | Triggered by: pr_opened | Commit: 3daf5f6

💡 TIP: You can chat with Visor using /visor ask <your question>

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
96.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud