Skip to content

Feature/workflow - #55

Open
FSEPIC wants to merge 5 commits into
mainfrom
feature/workflow
Open

FSEPIC wants to merge 5 commits into
mainfrom
feature/workflow

Conversation

@FSEPIC

@FSEPIC FSEPIC commented Sep 21, 2026

Copy link
Copy Markdown
Collaborator

Summary

Add reusable workflow variables, explicit Prefect deployment routing, and database-backed publication snapshots. Workflow edits remain separate from published definitions, while execution and historical callbacks retain their selected versions.

Changes

  • Add Text, JSON, and encrypted Secret variables to both workflow editors, with validation and context-aware reference suggestions.
  • Keep saved secrets write-only, restrict secret references to sensitive action fields, and redact resolved values from action results and errors.
  • Introduce a Prefect deployment registry synchronized through deployment events and periodic reconciliation.
  • Require explicit deployment bindings for publication, execution, and scheduling; remove the global PREFECT_DEPLOYMENT_ID fallback.
  • Add deployment selection to the visual editor, including pool/queue information and unavailable-target warnings.
  • Store published definitions in WorkflowRevision, with a current-publication pointer, atomic version allocation, and transaction locking.
  • Mark definition edits as unpublished changes without overwriting published snapshots.
  • Persist schedule synchronization status, version, timestamps, and errors. Retain local changes on synchronization failure and support retries for active and paused plans.
  • Add a legacy manifest importer with read-only preflight, validation, atomic import, and idempotency checks. Adapt secret rotation to database revisions.
  • Strengthen step identity, ownership, duplicate-ID, and ordering validation.
  • Improve editor sidebar resizing and distinguish publication success from schedule synchronization failures.
  • Add upgrade documentation, regression tests, isolated test settings, and a real-service smoke test.
  • Keep personal VS Code debugging files untracked while retaining shared Prefect start/stop scripts.

Upgrade and compatibility

  • Apply workflow migrations 0011 through 0014.
  • Explicitly bind workflows that previously relied on the global deployment fallback before upgrading.
  • Import legacy publication snapshots during a maintenance window using the original encryption keys. Preserve database and source-file backups until verification is complete.
  • Restart the updated services and run sync_prefect_schedules after migration.
  • Remove existing schedules, including paused plans, before changing a workflow's deployment.
  • The legacy Prefect-to-workflow synchronization endpoint now returns HTTP 410.
  • JSON export remains available; manifest_ref and manifest_filename are logical identifiers, and the deprecated manifest_path is empty.

See docs/workflow-deployments.zh-CN.md and backend/workflows/README.md for rollout instructions.

Validation

  • 81 backend workflow tests passed using an isolated PostgreSQL test database.
  • Django migration consistency check passed.
  • Git diff checks passed.
  • Frontend tests/build and the real-service integration smoke test were not run during this verification.

Persist variable declarations across save, clone and import/export. Add write-only secret variables with encrypted storage, safe credential references, redaction and key rotation.

Add collapsible variable cards, configured-secret badges, and a resizable sidebar with adaptive font sizes. Fix workflow editor key warnings and the model class placement required by migrations.
Register compatible Prefect deployments automatically, validate workflow bindings, protect schedule rebinding, and remove the environment deployment fallback.

Add two named Prefect worker debug terminals to Full Stack with reusable second-deployment preparation and focused regression tests.
- Add WorkflowRevision, publication pointers, and schema migration
- Separate drafts from published snapshots and strengthen locking and validation
- Persist schedule synchronization status and support retries
- Add legacy snapshot import and adapt secret rotation to database revisions
- Remove legacy flows placeholders and add upgrade docs and regression tests

Validation: 81 tests passed; migration consistency check passed
@FSEPIC
FSEPIC requested a review from alexchen16 as a code owner September 21, 2026 16:32
Comment thread backend/workflows/tests/smoke_without_flows.py Dismissed
Comment thread backend/workflows/tests/smoke_without_flows.py Dismissed
Comment thread backend/workflows/tests/test_deployment_transactions.py Dismissed
Comment thread backend/workflows/tests/test_deployment_transactions.py Dismissed
Comment thread backend/workflows/event_consumer.py Dismissed
Comment thread backend/workflows/prefect_dispatcher.py Dismissed
Comment thread backend/workflows/tests/smoke_without_flows.py Dismissed
Comment thread backend/workflows/tests/test_secret_variable_runtime.py Dismissed
Comment thread backend/workflows/tests/test_secret_variables.py Dismissed
Comment thread backend/workflows/tests/test_secret_variables.py Dismissed
Comment thread backend/workflows/tests/settings.py Dismissed

@alexchen16 alexchen16 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P0] GitHub 当前 mergeable=false,技术上不能直接合并到 main
影响:GitHub 当前 PR metadata 显示 mergeable=false。无论代码质量如何,现状都必须先 rebase/merge main 并解决冲突,再重新跑完整测试。
证据 / 关键代码:
PR #55
base: main
head: feature/workflow
mergeable: false
changed_files: 66
+4798 / -734
建议:先更新 main 到该分支并解决冲突;冲突解决后重新审查重叠文件,特别是 workflows/*、frontend workflow editor,以及与 #52/#53 可能重叠的文件。随后重新跑 backend、frontend、migration、smoke test

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants