fix: offload delete member and try to handle member identities conflicts better (CM-1054)#3974
fix: offload delete member and try to handle member identities conflicts better (CM-1054)#3974
Conversation
…cts better Signed-off-by: Uroš Marolt <uros@marolt.me>
There was a problem hiding this comment.
Pull request overview
This PR improves data-sink member creation/update behavior when identities conflict, and offloads orphan-member cleanup to the entity-merging Temporal worker. It also aligns git activities’ username with the identity key used for member lookups, and includes a DB migration that drops unused indexes/constraints and adds a covering index for org segment aggregates.
Changes:
- Add a Temporal workflow (
deleteOrphanMember) and workflow ID enum for asynchronously deleting orphan members. - Adjust member identity insertion to optionally fail on conflict and return inserted row count; add conflict-resolution logic during member creation (including optional auto-merge).
- Update git activity payloads to use author/committer email as
username, plus a DB migration for index cleanup/optimization.
Reviewed changes
Copilot reviewed 9 out of 10 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| services/libs/types/src/enums/temporal.ts | Adds workflow ID constant for orphan-member deletion. |
| services/libs/data-access-layer/src/old/apps/data_sink_worker/repo/member.repo.ts | Updates insertIdentities to accept conflict behavior and return inserted count. |
| services/libs/data-access-layer/src/members/identities.ts | Adds helper to find a member by a verified identity. |
| services/apps/git_integration/src/crowdgit/services/commit/commit_service.py | Ensures git activities use email-based username to match identity keys. |
| services/apps/entity_merging_worker/src/workflows/all.ts | Adds deleteOrphanMember workflow that calls deleteMember. |
| services/apps/entity_merging_worker/src/workflows.ts | Re-exports the new workflow. |
| services/apps/data_sink_worker/src/service/member.service.ts | Implements identity-conflict handling, auto-merge logic, and schedules orphan deletion via Temporal. |
| services/apps/data_sink_worker/src/service/activity.service.ts | Normalizes activity.username to the member identity value; refactors merge logic via mergeIfAllowed. |
| backend/src/database/migrations/V1774609007__data-sink-worker-optimizations.sql | Drops unused activityRelations indexes/constraint; adds index on organizationSegmentsAgg. |
| backend/src/database/migrations/U1774609007__data-sink-worker-optimizations.sql | Empty undo migration placeholder. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
services/apps/git_integration/src/crowdgit/services/commit/commit_service.py
Show resolved
Hide resolved
Signed-off-by: Uroš Marolt <uros@marolt.me>
Signed-off-by: Uroš Marolt <uros@marolt.me>
services/libs/data-access-layer/src/old/apps/data_sink_worker/repo/member.repo.ts
Show resolved
Hide resolved
Signed-off-by: Uroš Marolt <uros@marolt.me>
Signed-off-by: Uroš Marolt <uros@marolt.me>
Signed-off-by: Uroš Marolt <uros@marolt.me>
Signed-off-by: Uroš Marolt <uros@marolt.me>
Signed-off-by: Uroš Marolt <uros@marolt.me>
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 2 potential issues.
Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.
backend/src/database/migrations/V1774609007__data-sink-worker-optimizations.sql
Show resolved
Hide resolved
Signed-off-by: Uroš Marolt <uros@marolt.me>

Note
Medium Risk
Touches member identity uniqueness handling, automated member merges, and asynchronous deletions via Temporal, which can affect data integrity if edge cases are missed. Includes concurrent index DDL changes that must run safely in production.
Overview
Improves data-sink member/activity upserts by reducing orphan members and handling
memberIdentitiesuniqueness conflicts more safely.Git commit ingestion now uses author/committer email as
username(instead of display name), and the data-sink worker overridesactivity.usernameto match the member’s verified platform username identity when they differ, preventing missed lookups.Member creation now deduplicates incoming identities, makes identity inserts tolerant/observable (returns inserted count), and adds a conflict-resolution path that can reuse an existing member, auto-merge conflicting members when allowed (
memberNoMergerespected), move any inserted identities over, and schedule deletion of the orphan member via a new Temporal workflowdeleteOrphanMember. A migration also drops unusedactivityRelationsindexes/constraint and adds an index onorganizationSegmentsAggfor faster lookups.Written by Cursor Bugbot for commit d864eec. This will update automatically on new commits. Configure here.