Closed
Conversation
* Initial plan * Fix DataFilter to not filter UUIDs and MongoDB ObjectIds Co-authored-by: neSpecc <3684889+neSpecc@users.noreply.github.com> * Address code review feedback: improve UUID regex and test coverage Co-authored-by: neSpecc <3684889+neSpecc@users.noreply.github.com> * Fix tests to use values that would actually fail without UUID/ObjectId detection Co-authored-by: neSpecc <3684889+neSpecc@users.noreply.github.com> * upd version --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: neSpecc <3684889+neSpecc@users.noreply.github.com> Co-authored-by: Peter Savchenko <specc.dev@gmail.com>
* fix(grouper): filter oldPassword and newPassword in event payload * chore: add more keys and update tests * chore: lint fix * fix(tests): rename sessionId to requestId in data-filter tests for clarity
* Limit path depth and add cache cleanup Prevent excessive memory usage and reduce GC pressure by limiting traversal path depth in data-filter (cap depth logic at 20). In grouper worker, add a periodic cache cleanup interval (every 5 minutes) started on worker start and cleared on finish to avoid unbounded cache growth. Free large references after delta computation by nulling event payloads to allow garbage collection. Also tighten memoization for findSimilarEvent (max reduced from 200 to 50 and ttl set to 600s) to further limit memory retained by caches. * Prevent deep-path allocations; tune timeouts & tests Limit path growth in data filter to avoid creating new arrays past 20 levels (reduces excessive allocations for deeply nested objects). Import TimeMs and replace magic numbers: set MEMOIZATION_TTL to 600_000, use TimeMs.MINUTE for cache cleanup interval, and apply MEMOIZATION_TTL to the memoize decorator. Clear large delta references by setting them to undefined to aid GC. Add a test that verifies filtering works on objects nested >20 levels without causing excessive memory allocations. * Use named constants for traversal and cache interval Introduce MAX_TRAVERSAL_DEPTH in data-filter.ts and replace the hardcoded depth check (20) to prevent excessive memory allocations from deep object nesting. Add CACHE_CLEANUP_INTERVAL_MINUTES in index.ts and use it for the cache cleanup setInterval instead of the literal 5, improving readability and making these tuning values easier to adjust. * Move eslint ignore to MEMOIZATION_TTL Remove the unnecessary eslint-disable-next-line on the memoize import and apply the no-unused-vars ignore to the MEMOIZATION_TTL constant instead. This ensures the linter suppression targets the unused constant (decorators not counted) rather than the import, improving clarity. * Suppress no-unused-vars for memoize import Add an explanatory comment and an `/* eslint-disable-next-line no-unused-vars */` directive before the `memoize` import in workers/grouper/src/index.ts. This prevents ESLint from flagging the import as unused since decorators (which rely on the import) are not recognized as usages by the linter.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Auto-generated prod update suggestion