Skip to content

docs: align FlowForge architecture with the durable runtime - #16

Merged
mkarson1997 merged 2 commits into
mainfrom
docs/recruiter-architecture
Sep 9, 2026
Merged

mkarson1997 merged 2 commits into
mainfrom
docs/recruiter-architecture

Conversation

@mkarson1997

@mkarson1997 mkarson1997 commented Sep 9, 2026 •

Copy link
Copy Markdown
Owner

What changed

  • replace the stale architecture note that described PostgreSQL durability as a future milestone
  • document the current in-process DAG + PostgreSQL state + durable queue + worker architecture
  • add explicit invariants for scheduling, retries, idempotency, leases, fencing and at-least-once delivery
  • add recruiter-facing engineering proof points and security/delivery controls to the README

Why

FlowForge's implementation has moved beyond the old architecture document. The repository should describe what is actually implemented today and expose the strongest distributed-systems evidence clearly.

No runtime behavior changes.

Summary by Sourcery

Align the project documentation with the implemented durable runtime and make its distributed-systems guarantees and engineering controls explicit.

Enhancements:

  • Document the current in-process DAG, PostgreSQL state, durable queue, and worker architecture, including scheduling, retry, idempotency, lease, fencing, and delivery invariants.
  • Clarify durable work distribution, at-least-once delivery semantics, observability boundaries, and intentional non-claims.
  • Surface engineering proof points and security and delivery controls in the README.

Documentation:

  • Align the architecture documentation with the implemented durable runtime and its failure guarantees.

Copilot AI lite review requested due to automatic review settings September 9, 2026 10:21
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry @mkarson1997, you've used your own review budget of 250,000 diff characters for the last 7 days.

You can request another review in 2 days and 1 hour by commenting @sourcery-ai review. Upgrade to get a review now.

@sonarqubecloud

sonarqubecloud Bot commented Sep 9, 2026

Copy link
Copy Markdown

@sourcery-ai

sourcery-ai Bot commented Sep 9, 2026

Copy link
Copy Markdown

Reviewer's Guide

Documentation is aligned with the implemented FlowForge durable runtime: the architecture now explains the in-process DAG and PostgreSQL-backed worker system with explicit invariants and delivery guarantees, while the README highlights engineering proof points and CI/security controls. No runtime behavior is changed.

Sequence diagram for durable work claiming and recovery

sequenceDiagram
    participant Worker as DurableWorker
    participant Queue as DurableWorkQueue
    participant DB as PostgreSQL
    participant Handler as TaskHandler

    Worker->>Queue: register
    Worker->>Queue: heartbeat
    Worker->>Queue: claim work
    Queue->>DB: acquire expiring lease
    Queue-->>Worker: leased work
    Worker->>Handler: execute task
    alt task succeeds
        Worker->>Queue: complete work
        Queue->>DB: record terminal state
    else worker disappears
        Queue->>DB: reclaim expired lease
        Queue->>DB: retry with row locking
    end
Loading

File-Level Changes

Change Details Files
Replaces the outdated architecture description with a current durable-runtime model and explicit distributed-systems guarantees.
  • Adds a system-view diagram covering DAG validation, layered execution, state stores, durable queue, workers, handlers, and observability.
  • Documents scheduling, failure boundaries, retries, cancellation, idempotency, durable state, leases, fencing, crash recovery, delivery semantics, and observability invariants.
  • Describes PostgreSQL-backed state and queue behavior, including locked retry transitions, dead-lettering, and lease reclamation.
  • Removes future-milestone language and states deliberate non-claims such as exactly-once external effects and forcibly terminating arbitrary JavaScript.
docs/architecture.md
Adds recruiter-facing engineering evidence and clarifies security and CI delivery controls in the project README.
  • Adds proof points for workflow semantics, failure handling, durable state, worker coordination, concurrency, idempotency, delivery, observability, verification, and supply-chain security.
  • Updates architecture-documentation wording to emphasize current durable-runtime semantics.
  • Adds security and delivery controls covering PostgreSQL-backed CI verification, lifecycle-script restrictions, aggregate CI gating, CodeQL, immutable action pinning, and telemetry data boundaries.
README.md

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@mkarson1997
mkarson1997 merged commit 775ec3b into main Sep 9, 2026
7 checks passed
@mkarson1997
mkarson1997 deleted the docs/recruiter-architecture branch September 9, 2026 11:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants