Repository navigation
Optimize anti-slop audits with persistent JSON pattern caching - #24
Merged
REX-codebase merged 1 commit intoJul 18, 2026
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why This Change is Necessary
During multi-step Monte Carlo Tree Search (MCTS) simulations, the anti-slop audit tool is invoked repeatedly via shell execution. Historically, each invocation read and parsed a large CSV file (
ai-slop-banned.csv) containing banned patterns. This process added roughly 100ms of parsing and object-allocation overhead per call. For simulations requiring hundreds of steps, this cumulative latency became a major bottleneck.Because we need to preserve the standalone CLI execution model and avoid heavy additions like background daemons or shared memory, a file-system-based persistent JSON cache is the ideal lightweight solution to eliminate redundant parsing.
What This PR Does
We have implemented a persistent caching mechanism for the parsed patterns inside the audit utility:
Integrated Cache-Check in Utility:
Updated
loadBannedPatternsin/app/skills/utils/anti-slop-checker.jsto look for a cached representation (anti-slop-cache.json) in the operating system's temporary directory (os.tmpdir()).Implemented Smart Invalidation via Timestamps:
To guarantee that any manual updates to the raw CSV list are instantly respected, the cache mechanism checks the
mtimeMs(last modified time) of the source CSV file and compares it to the timestamp embedded within the JSON cache.Graceful Error Handling (Guardrails):
File system writes to the temp directory are wrapped in defensive try/catch blocks. If a permission issue or unexpected I/O error occurs, the cache fails silently and the tool proceeds with raw CSV processing. This guarantees that audit logic remains highly resilient.
Key Technical Decisions
os.tmpdir(): Ensures cross-platform compatibility and guarantees that cached files stay within standard, disposable system boundaries (staying well under our 10MB limit).Verification & Testing