Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion instructions/cache-usage.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ var entryOptions = new MemoryCacheEntryOptions()
.SetSize(1);
```

In the code: `Program.cs` registers the cache with `SizeLimit` taken from the `TextServices:CacheMaxEntries` setting (default 20; read directly from configuration because options binding isn't available at that point in startup). `TextCache` sets `.SetSize(1)` on every entry — both `Text` and `AutoComplete` objects share the same slot budget.
In the code: `Program.cs` registers the cache with `SizeLimit` taken from the `TextServices:CacheMaxEntries` setting (default 20; read directly from configuration because options binding isn't available at that point in startup). The config key is looked up via `nameof(SearchApiOptions.CacheMaxEntries)` rather than a string literal, so the `SearchApiOptions` property — otherwise unreferenced in code, since this path bypasses options binding — stays in lockstep with the setting it documents. `TextCache` sets `.SetSize(1)` on every entry — both `Text` and `AutoComplete` objects share the same slot budget.

2. **Add an absolute expiration floor.** ✅ **Implemented.** Sliding expiration alone means a popular text stays cached forever. Adding an absolute cap forces periodic refresh and bounds LOH lifetime. `TextCache` applies `.SetAbsoluteExpiration()` from the `TextServices:CacheAbsoluteExpirationHours` setting (default 4 hours) alongside the sliding expiration.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,11 @@ public class SearchApiOptions
public int CacheSlidingExpirationMinutes { get; set; } = 30;

/// <summary>
/// Absolute expiration cap for cached objects (hours).
/// Absolute expiration cap for cached objects (hours). Fractional values are allowed
/// (e.g. 0.25 for 15 minutes), for deployments that need a tighter cap than whole hours.
/// Prevents popular texts from staying in the LOH indefinitely.
/// </summary>
public int CacheAbsoluteExpirationHours { get; set; } = 4;
public double CacheAbsoluteExpirationHours { get; set; } = 4;

/// <summary>
/// Maximum number of Text objects to hold in the memory cache simultaneously.
Expand Down
3 changes: 2 additions & 1 deletion src/TextServices.Search.Api/Program.cs
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,8 @@
// ---- Cache ------------------------------------------------------------------

builder.Services.AddMemoryCache(opts =>
opts.SizeLimit = builder.Configuration.GetSection("TextServices").GetValue<int?>("CacheMaxEntries") ?? 20);
opts.SizeLimit = builder.Configuration.GetSection("TextServices")
.GetValue<int?>(nameof(SearchApiOptions.CacheMaxEntries)) ?? 20);
builder.Services.AddSingleton(new AsyncKeyedLocker<string>());
builder.Services.AddSingleton<ITextCache, TextCache>();

Expand Down
Loading