Skip to content

Optimize anti-slop audits with persistent JSON pattern caching - #24

Merged
REX-codebase merged 1 commit into
mainfrom
jules/audit-json-cache-js1-be796adf-21be-4608-9ec7-ff615b98421f
Jul 18, 2026
Merged

REX-codebase merged 1 commit into
mainfrom
jules/audit-json-cache-js1-be796adf-21be-4608-9ec7-ff615b98421f

Conversation

@google-labs-jules

Copy link
Copy Markdown
Contributor

Why This Change is Necessary

During multi-step Monte Carlo Tree Search (MCTS) simulations, the anti-slop audit tool is invoked repeatedly via shell execution. Historically, each invocation read and parsed a large CSV file (ai-slop-banned.csv) containing banned patterns. This process added roughly 100ms of parsing and object-allocation overhead per call. For simulations requiring hundreds of steps, this cumulative latency became a major bottleneck.

Because we need to preserve the standalone CLI execution model and avoid heavy additions like background daemons or shared memory, a file-system-based persistent JSON cache is the ideal lightweight solution to eliminate redundant parsing.


What This PR Does

We have implemented a persistent caching mechanism for the parsed patterns inside the audit utility:

  1. Integrated Cache-Check in Utility:
    Updated loadBannedPatterns in /app/skills/utils/anti-slop-checker.js to look for a cached representation (anti-slop-cache.json) in the operating system's temporary directory (os.tmpdir()).

  2. Implemented Smart Invalidation via Timestamps:
    To guarantee that any manual updates to the raw CSV list are instantly respected, the cache mechanism checks the mtimeMs (last modified time) of the source CSV file and compares it to the timestamp embedded within the JSON cache.

    • Cache Hit: If the cache is present and up-to-date, the system parses the lightweight JSON file directly—bypassing CPU-heavy string/regex parsing and CSV mapping.
    • Cache Miss / Stale Cache: If the CSV has been updated or the cache is missing, the system falls back to parsing the CSV and asynchronously generates/writes a fresh JSON cache.
  3. Graceful Error Handling (Guardrails):
    File system writes to the temp directory are wrapped in defensive try/catch blocks. If a permission issue or unexpected I/O error occurs, the cache fails silently and the tool proceeds with raw CSV processing. This guarantees that audit logic remains highly resilient.


Key Technical Decisions

  • Use of os.tmpdir(): Ensures cross-platform compatibility and guarantees that cached files stay within standard, disposable system boundaries (staying well under our 10MB limit).
  • Out-of-Process Independence: This approach requires zero background processes or shared memory, which aligns perfectly with our standalone execution constraints.

Verification & Testing

  • Performance: Verified a substantial reduction in repeated execution latency on the CLI, removing the ~100ms bottleneck on consecutive runs.
  • Correctness: Ran validation checks to ensure identical match results between raw CSV parsing and cached JSON parsing.
  • Cache Invalidation: Modified the source CSV and confirmed that the cache was automatically invalidated and successfully regenerated on the subsequent audit run.

@REX-codebase
REX-codebase merged commit 1884009 into main Jul 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant