feat(ops): prune superseded dataset releases through a manual workflow - #265
Merged
Merged
Conversation
Every ingest creates a new release while activation and promotion only move pointers, so superseded releases accumulate. Add a manual prune-releases workflow (dry run by default, production environment, shares the update-dataset concurrency group) driving a Node script that: - pages through maintenance:previewInactiveReleases, - never touches unsealed or recent releases, - always finishes releases already marked pruning, - retains the newest verified declared release per distro as a rollback target and prunes legacy or failed-manifest releases that can never be activated again, - deletes in bounded confirmed batches via deleteInactiveReleaseBatch, - then sweeps orphaned content blobs. previewInactiveReleases now paginates with a cursor and reports sealed, pruning, manifestBasis, manifestVerified, and manifestError so callers can judge rollback eligibility. Covered by ops tests, a Convex integration test, and runbook/architecture/changelog updates.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
Sorry @amanthanvi, you've used your own review budget of 250,000 diff characters for the last 7 days.
You can request another review in 1 hour and 13 minutes by commenting @sourcery-ai review. Upgrade to get a review now.
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Advanced Run ID: Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Reviewer's GuideThis PR adds a manual, production-gated, dry-run-by-default release-pruning workflow. It paginates inactive-release discovery, applies conservative age/sealing/manifest-based rollback retention, deletes selected releases in bounded Convex batches, and performs orphan-blob cleanup while coordinating with dataset updates; associated tests and operational documentation cover the policy and workflow wiring. Sequence diagram for the manual release pruning workflowsequenceDiagram
participant Operator
participant GitHubActions
participant PruneScript
participant Convex
Operator->>GitHubActions: workflow_dispatch
GitHubActions->>PruneScript: pnpm convex:prune-releases
PruneScript->>Convex: maintenance:previewInactiveReleases(cursor, limit)
Convex-->>PruneScript: inactive releases, continueCursor
loop Until isDone
PruneScript->>Convex: maintenance:previewInactiveReleases(cursor, limit)
Convex-->>PruneScript: next inactive page
end
PruneScript->>PruneScript: selectPrunable
alt apply is false
PruneScript->>Convex: cleanupOrphanContentBlobsBatch(dryRun=true)
else apply is true
loop Selected releases
PruneScript->>Convex: maintenance:deleteInactiveReleaseBatch(maxDocs=200)
Convex-->>PruneScript: deletion progress
end
PruneScript->>Convex: cleanupOrphanContentBlobsBatch(dryRun=false)
end
Flow diagram for inactive release selection policyflowchart TD
A[Inactive release preview] --> B{Already pruning?}
B -- Yes --> C[Prune and finish]
B -- No --> D{Sealed?}
D -- No --> E[Skip unsealed upload]
D -- Yes --> F{Minimum age reached?}
F -- No --> G[Skip recent release]
F -- Yes --> H{Declared and manifest verified?}
H -- No --> C
H -- Yes --> I{Within keep_per_distro rollback retention?}
I -- Yes --> J[Retain rollback candidate]
I -- No --> C
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
 |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
- Stop handing out new deletions after the first failure and surface the Convex error text (redacting deploy keys) instead of a bare exit code. - Retry each batch with backoff; batch limits start below the Convex maxima and halve on failure so oversized transactions degrade instead of stalling. - Apply the age floor only to sealed declared releases, which are the only ones activation could still accept; legacy and failed manifests are pruned regardless of age. Prefer the server-assigned creation time over the client-supplied ingestedAt. - Require keep_per_distro >= 1 and min_age_hours >= 1; make the whole-table orphan sweep opt-in for previews; add an opt-in age floor for abandoned unsealed drafts (sample or interrupted ingests). - Delete stored payload files with pruned page contents and with orphaned blobs when no other blob shares the file, so _storage no longer grows monotonically. - Document the manual /ingest/activate rollback interlock gap.
- Treat an empty BETTERMAN_PRUNE_SWEEP_ORPHANS as unset so the workflow's 'auto' default actually sweeps orphans on apply; reject other values. - Keep the reduced batch size after a failed attempt instead of re-escalating on every batch. - Report uploadComplete from the preview and never prune an unsealed release whose declared upload is complete; it is still activatable. - Reject unparseable release times instead of sorting NaN. - Redact project-scoped deploy keys and arbitrary token characters. - Log createdAt in the plan and each release as it finishes. - Propagate a thrown null from the concurrency helper.
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
prune-releasesworkflow (dry run by default,productionenvironment, shares theupdate-datasetconcurrency group) that removes superseded inactive dataset releases in bounded, retried Convex batches and then sweeps orphaned content blobs and their stored files.deleteInactiveReleaseBatch(which already refuses active pointers and markspruningbefore its first delete): always finish releases already markedpruning; never touch unsealed releases whose declared upload is complete (still activatable); prune incomplete unsealed drafts only behind an opt-inunsealed_min_age_hours; prune sealed legacy or failed-manifest releases regardless of age (activation rejects them); applymin_age_hours(default 24, min 1) to sealed declared releases; retain the newestkeep_per_distro(default 1, min 1) verified releases per distro as rollback targets.maintenance:previewInactiveReleasesnow paginates with a cursor and reportscreatedAt(server time),sealed,pruning,manifestBasis,manifestVerified,manifestError, anduploadComplete.deleteInactiveReleaseBatchandcleanupOrphanContentBlobsBatchnow delete the_storagefiles behind pruned page contents and orphaned blobs (only when no other blob shares the file)./ingest/*paths are not covered by the concurrency group.Motivation
Every ingest creates a new release while activation and promotion only move pointers, so superseded releases accumulate. After the September 14 re-ingest that unblocked deploys, the legacy September 1 and 3 releases for all seven distros are still stored and can never be re-activated (
LEGACY_ALIAS_EXPECTATION_UNKNOWN).Validation
pnpm ops:test: 37 passed (17 for the prune script, including a workflow-wiring contract that checks provenance precedes deploy-key exposure).pnpm convex:test: 121 passed (paginated preview, storage-file deletion for page contents and shared/sole blob files);pnpm convex:typecheckclean.scripts/check-convex-public-api.mjsandscripts/check-osv-exceptions.mjspass; workflow YAML parses.Rollout
Merging deploys the updated internal functions only. Pruning runs only on manual dispatch: first with defaults to preview, then
-f apply=true.