Skip to content

Feature/workflow - #50

Merged
FSEPIC merged 2 commits into
mainfrom
feature/workflow
Sep 9, 2026
Merged

FSEPIC merged 2 commits into
mainfrom
feature/workflow

Conversation

@FSEPIC

@FSEPIC FSEPIC commented Sep 6, 2026

Copy link
Copy Markdown
Collaborator

This PR isolates workflow execution in Prefect and adds node-level progress updates through Worker → Prefect Server → Django → frontend, removing the worker’s dependency on Django ORM and the application database.

  • Independent execution runtime: Move actions, conditions and secret handling into workflows/prefect/, remove legacy execution paths, and make new workflows Prefect-only.
  • Stable execution history: Bind executions to immutable published versions and preserve step metadata when drafts change. Isolate schedules sharing the same Prefect deployment.
  • Controlled API actions: Add an API Call editor with Basic/Bearer authentication, headers, query parameters and JSON templates. Administrators manage allowed hostnames, IPs and CIDRs through System Settings.
  • Protected credentials: Encrypt sensitive action configuration and redact execution output. Automatically provision restricted worker access through Prefect Secret, eliminating manual PREFECT_WORKER_TOKEN setup.
  • Live progress updates: Consume custom progress snapshots and native Prefect state events, persist progress in Django, and notify the frontend through PostgreSQL notifications and SSE. Support checkpoint replay and stale-event filtering.
  • Automatic consumer startup: Start the background consumer on Django’s first request. Use PostgreSQL advisory locking to coordinate multiple processes and allow standby consumers to take over.

Deployment requires the new database migrations, updated backend/worker dependencies, and the new Prefect flow entry point. Django and workers must share the workflow encryption keys. Existing local workflows need republishing; legacy send_webhook nodes should migrate to API Call with approved targets configured.

Validated locally with 124 backend tests, real Prefect-to-frontend progress delivery, and HTTP-triggered consumer startup. The local test files are not included in these two commits.

Make new workflows Prefect-only and separate worker dependencies.
Bind execution history to published versions and immutable step snapshots.
Use local list queries with serialized five-second frontend polling.
Add administrator-managed API target allowlisting and runtime policy checks.
@FSEPIC
FSEPIC requested a review from alexchen16 as a code owner September 6, 2026 14:54
Comment thread backend/workflows/event_consumer.py Dismissed
Comment thread backend/workflows/prefect/actions/disable_user.py Dismissed
Comment thread backend/workflows/prefect/actions/enable_user.py Dismissed
Comment thread backend/workflows/prefect/actions/hash_lookup.py Dismissed
Comment thread backend/workflows/prefect/actions/ip_lookup.py Dismissed
Comment thread backend/workflows/models.py Dismissed
Comment thread backend/workflows/prefect/actions/api_call.py Dismissed
Comment thread backend/workflows/management/commands/consume_prefect_events.py Dismissed
Comment thread backend/workflows/publisher.py Dismissed
Comment thread backend/workflows/runtime.py Dismissed
Comment thread backend/workflows/prefect/actions/update_ticket.py Dismissed

@alexchen16 alexchen16 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@FSEPIC
FSEPIC merged commit 99d13a0 into main Sep 9, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants