Skip to content

Refactor template execution contract around dotflow start --workflow factory #22

Description

@FernandoCelmer

Summary

Align every cloud platform template on a single execution contract built around dotflow start --workflow <module>.workflow:main. Previously, templates used three different entrypoints (python -m <module>.workflow, handler.handler, and a Python -c one-liner) with a mix of buildtime and runtime scheduling, leading to silent bugs in several combinations of cookiecutter options.

Motivation

The current templates have three compatibility and correctness issues:

  1. Dead cookiecutter optionsexecution_mode is prompted but not used anywhere in the generated project (was dropped when workflow.py became a factory).
  2. Double scheduling — picking scheduler=cron together with a cloud that schedules externally (lambda-scheduled, ecs-scheduled, cloud-run-scheduled, alibaba-fc-scheduled, github-actions) ends up with both AWS EventBridge/Cloud Scheduler AND SchedulerCron running in parallel.
  3. Mixed entrypoint stylesworkflow.py's main() returns a DotFlow factory, but container-style Dockerfiles invoked python -m <module>.workflow (which does nothing without if __name__), and Lambda handlers called main() without .start() (also no-op under the factory pattern).

Proposed changes

1. Single execution contract

All cloud entrypoints converge on the factory pattern consumed by dotflow start --workflow <module>.workflow:main:

  • 13 container-style clouds (docker, cloud-run, ecs, ecs-scheduled, cloud-run-scheduled, kubernetes, heroku, fly-io, railway, render, digital-ocean, azure-container, github-actions): Dockerfile/Procfile/heroku.yml/fly.toml/railway.json/dotflow.yml all use dotflow start --workflow {{MODULE_NAME}}.workflow:main --mode {{EXECUTION_MODE}}.
  • 7 Lambda-compatible clouds (lambda, lambda-scheduled, lambda-s3-trigger, lambda-sqs-trigger, lambda-api-trigger, alibaba-fc, alibaba-fc-scheduled): keep the platform-required handler.py entrypoint, but main() itself calls workflow.start(mode=..., resume=...) internally; the handler just calls main().
  • dotflow-cloud (new platform): managed runtime that consumes the same factory — only a README is shipped.

2. Execution_mode wired through

  • Hook post_gen_project.py now exposes EXECUTION_MODE as a placeholder alongside PROJECT_NAME, MODULE_NAME, etc.
  • Every CMD and deploy config appends --mode {{EXECUTION_MODE}}.
  • Lambda/alibaba main() calls workflow.start(mode="{{ cookiecutter.execution_mode }}").

3. Scheduler override for externally-scheduled clouds

Jinja set at the top of workflow.py and pyproject.toml:

{%- set scheduled_clouds = ["dotflow-cloud", "lambda-scheduled", "ecs-scheduled", "cloud-run-scheduled", "alibaba-fc-scheduled", "github-actions"] -%}
{%- set scheduler = "none" if cookiecutter.cloud in scheduled_clouds else cookiecutter.scheduler -%}

Forces scheduler="none" for any cloud that already provides scheduling, eliminating the double-scheduling footgun.

4. Checkpoint resume for Lambda/alibaba

When cookiecutter.checkpoint == "yes", Lambda/alibaba main() calls workflow.start(..., resume=True) so the persistent workflow_id actually triggers recovery.

5. New dotflow-cloud platform

Added to cookiecutter.json choices and registered in cloud/registry.json with a README-only footprint. The managed Dotflow Cloud runtime invokes dotflow start --workflow <module>.workflow:main.

6. Dependency pin bump

pyproject.toml generated dependency updated from dotflow==0.15.0.dev7 to dotflow==1.0.0.dev2.

Known limitation (out of scope)

dotflow start CLI is missing --resume flag, so checkpoint=yes combined with a CLI-based cloud silently does nothing. Tracked upstream in dotflow-io/dotflow#267; once shipped, the container-style templates can start emitting --resume when checkpoint is enabled.

Impact

  • No more dead config: execution_mode is honored by every runtime.
  • No more double schedulers when user picks cron + a scheduled cloud.
  • Uniform mental model: "main() is the factory, dotflow start --workflow is how it runs" — works identically for local runs, containers, and the managed Dotflow Cloud runtime.
  • Lambda/alibaba remain Lambda-compatible (still use handler/RIC contract).

Files affected

  • cookiecutter.json, hooks/post_gen_project.py
  • {{cookiecutter.project_name}}/pyproject.toml, README.md, {{cookiecutter.module_name}}/workflow.py
  • cloud/registry.json
  • 12 container-style Dockerfiles, 7 Lambda/alibaba Dockerfiles + handler.pys
  • cloud/heroku/Procfile, cloud/heroku/heroku.yml, cloud/fly-io/fly.toml, cloud/railway/railway.json, cloud/github-actions/.github/workflows/dotflow.yml
  • 14 README.mds updated to reflect new entrypoint semantics
  • New cloud/dotflow-cloud/README.md

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions