Skip to content

Release v3.0.0 / v3.0.1 — state root, singular command_response, WEC rehydrate honesty - #63

Merged
drawal1 merged 5 commits into
radiantlogicinc:mainfrom
dharrawal:main
Aug 12, 2026
Merged

Release v3.0.0 / v3.0.1 — state root, singular command_response, WEC rehydrate honesty#63
drawal1 merged 5 commits into
radiantlogicinc:mainfrom
dharrawal:main

Conversation

@dharrawal

@dharrawal dharrawal commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Summary

Brings dharrawal/fastworkflow main (5 commits ahead of radiantlogicinc/fastworkflow main) up to v3.0.1.

Note for reviewers on commit history: Two of the five commits (135cdf1, 5eea8f9) are the original sqlite/Python>=3.13 work that already landed on upstream via squash-merge as #61 (092ea6a, v2.31.0). GitHub will list them as commits on this PR, but the net tree diff vs current upstream main is essentially the v3.0.0 breaking cutover plus the v3.0.1 cold-rehydrate fix (with a small residual utterance-cache ordering fix that corrects a regression introduced during the sqlite streaming refactor).

Version Commit What it does
(already on upstream as #61) 135cdf1 / 5eea8f9 speedict → sqlite3, requires-python >=3.13,<3.15
v3.0.0 64a82ea (+ merge a145cf5) State root relocation, command_response collapse, process_message removal, schema bump
v3.0.1 e8f885e CommandOutput.command_parameters: Any so WEC serialize→rehydrate does not ValidationError

Closes / implements beads: fix-i0e (command_response collapse), fix-4od (schema_version 2→3), fix-fjh (command_parameters honesty). Sqlite epic fix-lvl already shipped via #61.


Breaking changes (v3.0.0) — review focus

1. CommandOutput.command_responsescommand_response (fix-i0e)

  • Field is now singular: command_response: CommandResponse.
  • Passing the legacy keyword command_responses=[...] raises ValueError with a migration hint (the old forward-compat shim that mapped singular→list is reversed and then removed).
  • Call sites across examples, CME commands, FastAPI turns, agent/planner paths, and tests were updated.
  • Turn-level multi-command history remains on TurnOutput / the turn accumulator — this change is per-command wire/data-model only (design of record: docs/turn_result_design_final.md).

2. WorkflowExecutionContext.process_message() removed

  • Deprecated process_message() is gone; callers must use process_turn() (or the internal _execute_message() path).
  • Docs / FastAPI embedding guidance updated accordingly (bind_app_workflow once → process_turn per request).

3. State directory: SPEEDDICT_FOLDERNAMEFASTWORKFLOW_STATE_ROOT

  • New module fastworkflow/state_paths.py centralizes path construction.
  • Default root: ~/.local/state/fastworkflow, namespaced under workflows/<id>/.
  • Env example drops SPEEDDICT_FOLDERNAME; documents optional FASTWORKFLOW_STATE_ROOT.
  • Related cleanup: drop dead LLM_RESPONSE_GEN; relax run's hard-fail on missing LITELLM_API_KEY_SYNDATA_GEN when Bedrock/proxy is in use.

4. Session-state schema_version 2 → 3 (fix-4od)

  • Bump with no migrator: older blobs are refused rather than best-effort migrated (same posture as prior schema refusals). Operators must not expect silent upgrade of suspended/checkpoint blobs written by pre-v3.0 binaries.

Bug fix (v3.0.1) — fix-fjh

CommandOutput.command_parameters was declared str but CommandExecutor assigns a typed Pydantic params model. After model_dump(mode="json") the value is a dict, so serialize_stateapply_serialized_state / turn-accumulator model_validate raised ValidationError on WEC cold rehydrate.

  • Declared as Any (typed model in memory; dict in records), matching design note A10.
  • Regression coverage in tests/test_turn_and_cme_continuation.py.
  • Package version → 3.0.1.

Other net-diff items worth a look

  • Utterance-cache empty short-circuit (cache_matching.py): restore “empty cache → return None before DistilBERT embed”. The sqlite streaming refactor had inverted that order (cold-start cost + behavioral regression vs RocksDB / first sqlite port).
  • Small residual edits under kvstore.py / sqlite cache tests vs the feat: v2.31.0 — replace speedict with sqlite3 and require Python 3.13+ #61 squash tip.
  • Tracker export: .beads/issues.jsonl / .beads/interactions.jsonl.
  • Doc touch-ups: README, FastAPI spec, turn-result design/checklist, integrate-chat-agent reference, articles.

Files / surface area (high level)

  • New: fastworkflow/state_paths.py, tests/test_state_paths.py
  • Core API: fastworkflow/__init__.py (CommandOutput), workflow_execution_context.py, session_state_store.py, turn.py, workflow_agent.py, command_executor.py, run_fastapi_mcp/*, run/__main__.py
  • Mass call-site update: examples + test workflows (command_response=), CME _commands
  • Version: pyproject.toml 2.31.03.0.1

Compare: main...dharrawal:main


Test plan

  • Review breaking-change call sites: any remaining command_responses= or process_message( in public API / examples / docs
  • Confirm FASTWORKFLOW_STATE_ROOT / state_paths.py layout and that SPEEDDICT_FOLDERNAME is fully gone from runtime + examples
  • Confirm schema_version 3 refusal behavior for pre-v3 blobs (no silent migrator)
  • Confirm v3.0.1: typed command_parameters survive serialize_stateapply_serialized_state (test added in test_turn_and_cme_continuation.py)
  • Confirm empty utterance-cache path does not embed before short-circuit
  • Run full pytest suite on a machine with trained example ___command_info artifacts present (do not retrain/wipe bundled examples); suite is ~24 min and memory-heavy — do not run two suites concurrently
  • Smoke: fastworkflow run / FastAPI MCP turn against an example workflow after cold start and after a checkpoint rehydrate

Migration notes for downstream consumers

  1. Replace CommandOutput(command_responses=[...]) with CommandOutput(command_response=...).
  2. Replace wec.process_message(...) with wec.process_turn(...).
  3. Point state dirs at FASTWORKFLOW_STATE_ROOT (or accept the new default under ~/.local/state/fastworkflow); do not expect old SPEEDDICT_FOLDERNAME / RocksDB .rdb trees to be reused.
  4. Treat pre-v3.0 session/checkpoint blobs as non-migrated (schema 3 refusal).
  5. Python remains >=3.13,<3.15 as established in v2.31.0 / feat: v2.31.0 — replace speedict with sqlite3 and require Python 3.13+ #61.

Made with Cursor

Summary by Sourcery

Relocate persistent workflow state to a new FASTWORKFLOW_STATE_ROOT layout, collapse CommandOutput to a singular command_response shape across core and FastAPI surfaces, and finalize the v3.0+ turn/result and storage contracts with schema and documentation updates.

New Features:

  • Introduce a centralized state_paths module and workflow-scoped directories so conversations, session state, checkpoints, and function caches are rooted under FASTWORKFLOW_STATE_ROOT per workflow.
  • Expose per-workflow state namespace selection via FASTWORKFLOW_WORKFLOW_ID to keep durable data stable across workflow renames or relocations.

Bug Fixes:

  • Allow CommandOutput.command_parameters to hold typed models or dicts so serialized turn state can be restored without validation errors.
  • Restore the utterance cache behavior so an empty cache short-circuits without running the embedding model, avoiding unnecessary compute and failures in minimal predictor setups.

Enhancements:

  • Standardize CommandOutput on a singular command_response field, rejecting the legacy command_responses list and updating helpers, routing, and MCP/FastAPI consumers accordingly.
  • Remove the deprecated WorkflowExecutionContext.process_message entry point in favor of process_turn and the internal _execute_message path, simplifying the execution API.
  • Bump session-state SCHEMA_VERSION to 3 and explicitly refuse older blobs rather than attempting implicit migration.
  • Make FastAPI turn responses consistently return the TurnOutput projection without a top-level command_responses list, directing clients to answer and command_outputs[*].command_response.
  • Improve env-file validation and error reporting for train and run entry points, and relax hard failure when LITELLM_API_KEY_SYNDATA_GEN is absent in Bedrock/proxy setups.
  • Refine workflow_agent handling of command_parameters to accept live Pydantic instances and restored dicts, preserving response text when parameters round-trip through storage.
  • Add a KV store has_entries helper and use it to streamline cache_matching’s empty-cache and iteration logic.
  • Isolate test runs from developer state by pointing FASTWORKFLOW_STATE_ROOT at per-test temp directories and cleaning up legacy workflow-context folders.

Documentation:

  • Update README, FastAPI spec, agent integration docs, articles, and example snippets to reflect the singular command_response shape, the new state-root layout, and the removal of process_message.
  • Clarify wire-level migration guidance for v3.0 turn responses, including the move away from CommandOutput-shaped top-level payloads and command_responses lists.

Tests:

  • Update unit and integration tests, example workflows, and FastAPI contract checks to assert the new command_response shape and absence of top-level command_responses.
  • Add coverage for state_paths behavior, schema_version 3 serialization, typed command_parameters cold rehydrate, and utterance cache empty short-circuit behavior.

drawal1 and others added 5 commits August 8, 2026 18:34
…+ (fix-lvl)

Drop the abandoned speedict/RocksDB dependency in favor of stdlib sqlite3
(WAL, JSON values, float32 BLOB utterance cache). Unblocks Python 3.13
installs, removes pickle-on-disk from first-party stores, and fixes
process-exclusive LOCK failures under concurrent writers. requires-python
is now >=3.13,<3.15; pre-existing .rdb / cache.db RocksDB dirs are abandoned.

Co-authored-by: Cursor <cursoragent@cursor.com>
Stream utterance_cache rows instead of fetchall, prefer context managers
at call sites, rely on sqlite3.connect(timeout=) instead of interpolating
PRAGMA busy_timeout, and harden concurrency/empty-embedding tests.

Co-authored-by: Cursor <cursoragent@cursor.com>
…ess_message removal

Breaking release bundling the remaining v3.0 cutovers:

- Replace SPEEDDICT_FOLDERNAME with FASTWORKFLOW_STATE_ROOT (default
  ~/.local/state/fastworkflow), namespaced per workflow under
  workflows/<id>/; centralize path construction in state_paths.py.
- Collapse CommandOutput.command_responses to singular command_response
  (fix-i0e); legacy list keyword raises ValueError.
- Remove deprecated WorkflowExecutionContext.process_message(); use
  process_turn() / _execute_message().
- Bump session-state schema_version 2→3 with no migrator (fix-4od).
- Drop dead LLM_RESPONSE_GEN; relax run's SYNDATA_GEN hard-fail.
- Restore empty utterance-cache short-circuit before DistilBERT embed
  (regression from 5eea8f9 streaming refactor).

Co-authored-by: Cursor <cursoragent@cursor.com>
Brings v3.0.0 (state root, command_response collapse, process_message
removal) and the post-radiantlogicinc#61 sqlite review fixes onto main.

Co-authored-by: Cursor <cursoragent@cursor.com>
…ehydrate (fix-fjh)

Declare command_parameters as Any so typed params models dump to dict and
round-trip through serialize_state → apply_serialized_state without ValidationError.

Co-authored-by: Cursor <cursoragent@cursor.com>

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry @dharrawal, your pull request is larger than the review limit of 150000 diff characters

@sourcery-ai

sourcery-ai Bot commented Aug 12, 2026

Copy link
Copy Markdown

Reviewer's Guide

Implements the v3.0.0 breaking API/storage changes (singular command_response, removal of process_message, new FASTWORKFLOW_STATE_ROOT-based state layout with schema_version=3) plus the v3.0.1 bugfix for command_parameters rehydration, and updates tests/docs/examples accordingly.

Sequence diagram for turn execution with singular command_response

sequenceDiagram
    actor User
    participant FastAPI as FastAPI_turn_endpoint
    participant WEC as WorkflowExecutionContext
    participant Executor as CommandExecutor

    User->>FastAPI: POST /invoke_agent (message)
    FastAPI->>WEC: process_turn(message)
    WEC->>WEC: _execute_message(message)
    WEC->>Executor: perform_action(workflow, message, command_parameters)
    Executor-->>WEC: CommandOutput
    note right of Executor: CommandOutput.command_response
    WEC->>WEC: _build_turn_result(CommandOutput)
    WEC-->>FastAPI: TurnOutput
    note right of WEC: TurnOutput.answer
    note right of WEC: TurnOutput.command_outputs[*].command_response
    FastAPI-->>User: HTTP 200 {answer, command_outputs, success}
Loading

File-Level Changes

Change Details Files
Collapse CommandOutput from a list of command_responses to a singular command_response and remove legacy shims/wire aliases.
  • Replace CommandOutput.command_responses: list[CommandResponse] with command_response: CommandResponse and adjust success/ask_user helpers and MCP conversion.
  • Reject constructor keyword command_responses=[...] via a model_validator raising ValueError with migration guidance.
  • Update WorkflowExecutionContext, TurnResult/TurnOutput, workflow_agent, command_executor, chat_session, CLI printing, FastAPI MCP turns, and all tests/examples/articles to use command_response.
  • Remove FastAPI top-level command_responses field from turn responses; clients must read answer and command_outputs[*].command_response.
fastworkflow/__init__.py
fastworkflow/workflow_execution_context.py
fastworkflow/turn.py
fastworkflow/workflow_agent.py
fastworkflow/command_executor.py
fastworkflow/chat_session.py
fastworkflow/run/__main__.py
fastworkflow/run_fastapi_mcp/turns.py
fastworkflow/run_fastapi_mcp/__main__.py
fastworkflow/docs/integrate-chat-agent/reference.md
docs/fastworkflow_fastapi_spec.md
docs/turn_result_design_final.md
tests/test_turn_result_capture.py
tests/test_fastapi_turn_output_contract.py
tests/test_execution_context_agent.py
tests/test_turn_and_cme_continuation.py
tests/**/*_command*.py
fastworkflow/examples/**/*_commands/*.py
fastworkflow-article-*.md
Remove WorkflowExecutionContext.process_message() from the public API and route internal call sites through _execute_message/process_turn.
  • Delete process_message method and DeprecationWarning path from WorkflowExecutionContext, keeping _execute_message as the shared dispatcher.
  • Update docs and README to recommend process_turn() for embedders and show TurnOutput-based integration examples.
  • Change internal call sites (agent tests, chat_session back-compat shims, FastAPI runtime comments/tests) to call _execute_message or process_turn as appropriate.
  • Adjust tests that asserted on process_message behavior to instead validate absence of the attribute and use _execute_message in white-box paths.
fastworkflow/workflow_execution_context.py
fastworkflow/chat_session.py
fastworkflow/run_fastapi_mcp/turns.py
fastworkflow/run_fastapi_mcp/__main__.py
README.md
docs/fastworkflow_fastapi_spec.md
tests/test_execution_context_agent.py
tests/test_turn_result_capture.py
tests/test_session_state_serialization.py
Introduce FASTWORKFLOW_STATE_ROOT and centralized state_paths, replacing SPEEDDICT_FOLDERNAME and namespacing all persistent state per workflow, with schema_version bumped to 3 and tests/docs aligned.
  • Add fastworkflow/state_paths.py to compute an absolute state_root, per-workflow namespace, and subdirectories for conversations, session_state, checkpoints, and function_cache (partitioned by commands fingerprint).
  • Switch FastAPI MCP session manager, checkpoint store, and helpers to use state_paths.* and a workflow_path pinned at startup; update dump_all_conversations and readiness comments accordingly.
  • Update session_state_store.get_session_state_store to default to a state_root-based channel_session_state and bump SCHEMA_VERSION from 2 to 3, adjusting tests to assert schema_version==3 and to gather checkpoint files under workflows/*/checkpoints.
  • Change kvstore-based enablecache paths and Workflow.get_cachedb_folderpath to use function_cache under workflow_state_dir instead of SPEEDDICT_FOLDERNAME; update related tests.
  • Update CLI/env validation (run, train, memory_soak, fastapi_hermetic, manager_shutdown_matrix) and README/run_fastapi_mcp docs to drop SPEEDDICT_FOLDERNAME, describe FASTWORKFLOW_STATE_ROOT/FASTWORKFLOW_WORKFLOW_ID, and adjust error messages and guidance.
  • Add pytest fixtures that isolate FASTWORKFLOW_STATE_ROOT per test (conftest.py and specific tests), and migrate existing tests from SPEEDDICT_FOLDERNAME overrides to FASTWORKFLOW_STATE_ROOT/temp dirs.
fastworkflow/state_paths.py
fastworkflow/session_state_store.py
fastworkflow/run_fastapi_mcp/utils.py
fastworkflow/run_fastapi_mcp/checkpoint.py
fastworkflow/run_fastapi_mcp/__main__.py
fastworkflow/kvstore.py
fastworkflow/workflow.py
fastworkflow/run/__main__.py
fastworkflow/train/__main__.py
tests/conftest.py
tests/test_state_paths.py
tests/test_checkpoint_integration.py
tests/soak/memory_soak.py
tests/fastapi_hermetic.py
tests/test_fastapi_service.py
tests/test_fastapi_turn_output_contract.py
tests/test_fastapi_turns_async.py
tests/test_fastapi_streaming_lifecycle.py
tests/test_fastapi_topology_b.py
tests/test_fastapi_session_leases.py
tests/test_fastapi_memory_bounds.py
tests/test_manager_shutdown_matrix.py
tests/test_enablecache_kvstore.py
tests/test_direct_action_validation.py
tests/test_session_state_serialization.py
tests/test_command_context_serialization_spike.py
tests/test_carry_forward_cache_content.py
tests/test_escalation_floor_symmetry.py
tests/test_workflow_training.py
tests/test_train_modern_stack.py
tests/test_checkpoint_integration.py
README.md
fastworkflow/run_fastapi_mcp/README.md
docs/fastworkflow_fastapi_spec.md
Fix CommandOutput.command_parameters type honesty so WEC serialize/apply_serialized_state can cold-rehydrate typed params without ValidationError (v3.0.1).
  • Change CommandOutput.command_parameters type from str to Any with documentation that in-memory it is a typed Pydantic params instance and on-wire it is a dict produced by model_dump(mode="json").
  • Teach workflow_agent._execute_workflow_query to accept params as model instance or dict, falling back safely when neither.
  • Add regression test ensuring a turn output with typed params survives serialize_state/apply_serialized_state round-trip and yields dict on wire and dict on restore.
  • Ensure turn accumulator serialization uses the new Any type without failing model_validate.
fastworkflow/__init__.py
fastworkflow/workflow_agent.py
tests/test_turn_and_cme_continuation.py
Restore utterance-cache empty short-circuit behavior so embedding is skipped when the cache is empty and add coverage around kvstore entries.
  • Add UtteranceCacheStore.has_entries() helper to kvstore.sqlite store.
  • Change cache_matching.cache_match to check has_entries() before computing embeddings and to drop the saw_any sentinel logic.
  • Add a test that asserts cache_match returns None and does not call get_embedding when the cache DB has no rows, plus tests that exercise skipping None/empty embeddings.
fastworkflow/kvstore.py
fastworkflow/cache_matching.py
tests/test_cache_matching_sqlite.py
Improve CLI/env validation and remove dead LLM_RESPONSE_GEN configuration while bumping package version to 3.0.1.
  • Update fastworkflow/run/main.py and fastworkflow/train/main.py to validate env file existence and non-empty env_vars, soften LITELLM_API_KEY_SYNDATA_GEN requirement in run (warn instead of hard-fail), and adjust error messages.
  • Remove LLM_RESPONSE_GEN and LITELLM_API_KEY_RESPONSE_GEN from README, env examples, CLI fetch_example, and password docs; replace with planner key where appropriate.
  • Update documentation with a concise list of 3.0.0 breaking changes and Python version notes, and adjust troubleshooting guidance around state-root.
  • Bump pyproject.toml version from 2.31.0 to 3.0.1.
fastworkflow/run/__main__.py
fastworkflow/train/__main__.py
fastworkflow/cli.py
README.md
fastworkflow/examples/fastworkflow.env
fastworkflow/examples/fastworkflow.passwords.env
pyproject.toml

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@drawal1
drawal1 merged commit caf2e45 into radiantlogicinc:main Aug 12, 2026
1 of 2 checks passed

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CVE automation (post-merge of #63)

Multi-scanner refresh completed against main at v3.0.1.

Tool Result
Syft / Trivy / Grype / osv-scanner Ran on poetry.lock (141 packages)
Dockle / Dive Ran on proxy image python:3.13-slim-bookworm
Snyk / Docker Scout Skipped (no SNYK_TOKEN / Docker Hub login)

Easy pyproject.toml fixes: none. Only unfixed PyPI advisory is diskcache==5.6.3 (CVE-2025-69872 / GHSA-w8v5-vhqr-4h9v); no patched release on PyPI. OpenVEX not_affected re-verified (document version 4).

Follow-up PR with HTML reports: #64

  • security/reports/cve-report.html
  • security/reports/human-review.html (all current items recommended IGNORE with rationale)

View PR

Open in Web View Automation 

Sent by Cursor Automation: Untitled

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants