Skip to content

feat: v3.1.0 — lazy conversation labeling, off process shutdown (#65) - #66

Merged
drawal1 merged 3 commits into
radiantlogicinc:mainfrom
dharrawal:main
Aug 13, 2026
Merged

feat: v3.1.0 — lazy conversation labeling, off process shutdown (#65)#66
drawal1 merged 3 commits into
radiantlogicinc:mainfrom
dharrawal:main

Conversation

@dharrawal

@dharrawal dharrawal commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Resolves #65, plus ten further findings from an adversarial review of it. Full findings and resolutions are in bd epic fix-dzs (11 children, all closed).

Summary

Process shutdown no longer calls LLM_CONVERSATION_STORE. Conversations are labeled when they are actually used as a chat instead.

The harm wasn't a missing title. finalize_conversations_on_shutdown ran up to max_live_sessions (50) sequential, untimed LLM calls, and it ran before stop_all_chat_sessions writes checkpoints — so under a 30s termination grace period the process was killed generating titles and lost everything each channel had accumulated since its last eviction. Shutdown labeling was also already LRU roulette: in the soak that motivated #65, the ~92% of conversations evicted before SIGTERM were never labeled at all.

  • Shutdown makes unsaved turns durable through the same save_conversation_incremental chokepoint the turn path uses, and leaves blank placeholders. The active_conversation_id == 0 fallback is kept deliberately — it is the only path that records turns for a channel whose persist was skipped by a raising work_fn.
  • Lazy fill on the first completed chat turn (after DONE, so the call is outside the drain, off the turn's critical path, and outside the 409 window), on activate when still blank, and on rotate. Refreshed at turn counts 1, 4, 16, 64, … so a long thread costs O(log n) calls; labeling once and never again would give a 40-turn thread the title of its first exchange.
  • Bounded at the LLM client, not by the caller's await. timeout and num_retries=1 go to dspy.LM. asyncio.wait_for cancels the wait while the thread runs on, and CPython then joins default-executor threads for up to THREAD_JOIN_TIMEOUT (300s) at exit. New LLM_CONVERSATION_STORE_TIMEOUT_SECONDS (default 12), resolved process-environment-first so a container override is reachable, validated and logged at startup.
  • "conversation summary" now identifies its turn on the deterministic and action paths. Both hardcoded a constant ("assistant_mode_command", "process_action command"), and since generate_topic_and_summary reads only that field, /initialize conversations were asking an LLM to title a fixed string.
  • Blank topic is a sound "no title yet" sentinel. An empty generated topic collided with every other unlabeled conversation and stored as ' 1', counting as labeled forever.
  • _ensure_unique_topic excludes the record being written. A rewrite with an unchanged topic oscillated between 'T' and 'T 1'.
  • New get_conversation_window. The restore paths kept 20 turns but read every turn, so at 450 KB payloads restoring a long conversation was tens of MB resident.
  • /new_conversation persists before labeling. In the reserved-but-unwritten state it spent a generation and then raised, making the rotate that would have cleaned the state up the one unavailable operation.
  • Streaming EOF no longer waits on the label (new on_done on run_owned_turn). invoke_agent, the MCP-exposed tool, is the streaming endpoint, and fastapi-mcp reads the whole stream as the tool result.

Three of #65's specific designs were wrong

Corrected here rather than implemented as written:

  1. preview rested on "conversation summary" being meaningful; it was a constant literal on exactly the paths the motivating workload uses. Deferred and closed with the research recorded — a blank topic turns out to be harmless anyway (get_conversation_by_topic has no production caller; activate takes a conversation_id).
  2. The per-call timeout could not bound the process, for the THREAD_JOIN_TIMEOUT reason above.
  3. The proposed call site ("beside the incremental persist, under runtime.lock") would have put the LLM call back inside the 30s drain it was written to protect, added its latency to the user's response, and widened the 409 window.

Test plan

  • Full suite: 1788 passed, 15 skipped, 0 failed, 0 errors (48:56)
  • Trained example artifacts intact — 225 files before and after, zero deletions, no .pth or threshold.json touched (the fix-0hb class of incident did not recur)
  • 37 new tests across 5 new files, integration-only against real stores, runtimes, registry and lifespan closures; no LLM called
  • Four fixes verified by falsification — reverting each makes its own test fail and no other: the shutdown call count, the windowed read (20 turn reads vs 100, identical turns returned), persist-then-label (500 with ValueError), and streaming EOF
  • run_owned_turn labeling now covered at all (it had no test; only _run_turn's did)
  • New env var documented in both READMEs and the example env file, with its default and its relationship to the shutdown drain

Reviewer notes

docs/ gets no new design doc: this was a review of an existing public issue rather than a new design, so the bd epic and #65 are the design record (agreed with the maintainer). Two pre-existing Sourcery style warnings in conversation_store.py and utils.py are untouched. The suite is 1803 tests, not the ~495 our docs claim — that doc rot is not fixed here.

Made with Cursor

Summary by Sourcery

Implement lazy, client-bounded conversation labeling and remove LLM usage from shutdown, improving reliability, performance, and operational control for FastWorkflow conversation management.

New Features:

  • Introduce lazy conversation topic/summary labeling triggered by completed chat turns, conversation activation, and rotation, with geometric refresh scheduling.
  • Add a windowed conversation read API that returns only the newest turns for restoration, reducing memory usage for long threads.

Bug Fixes:

  • Stop process shutdown from calling the conversation-store LLM and instead persist unsaved turns via the incremental save path to avoid data loss under termination grace periods.
  • Ensure blank or whitespace-only topics are treated as "no successful title yet" and never uniquified or stored as misleading numeric placeholders.
  • Fix conversation topic uniqueness so a conversation does not collide with itself when its topic is rewritten, preventing oscillating suffixed titles.
  • Ensure conversation labeling for streaming turns occurs off the streaming critical path and does not delay EOF or hang clients.
  • Persist conversations before labeling during rotation so previously unsaved turns are durably recorded and rotation cannot fail on missing records.

Enhancements:

  • Bound topic/summary LLM calls at the client via a configurable timeout and capped retries, with validation and startup logging of the effective settings.
  • Refine conversation turn summaries to carry concise, informative identifiers for deterministic and direct-action paths, suitable for topic generation and query refinement.
  • Add per-channel serialization for topic generation to avoid duplicate LLM calls while allowing concurrent work on the same channel.
  • Expose conversation label state (topic and durable turn count) and centralize labeling logic in shared helpers used by turns, activation, and rotation.

Documentation:

  • Document the new LLM_CONVERSATION_STORE_TIMEOUT_SECONDS configuration and the lazy labeling behavior, including shutdown semantics and refresh cadence in README files.

Tests:

  • Add comprehensive integration test suites covering lazy labeling triggers and schedules, topic uniqueness and blank-topic handling, topic generation bounds and failure behavior, turn summary content, shutdown behavior, windowed conversation reads, and streaming lifecycle interactions.

…dzs)

Adversarial review of radiantlogicinc#65 and its eleven findings. Shutdown no longer calls
LLM_CONVERSATION_STORE; conversations are labeled when they are used as a chat.

The harm was not a missing title. finalize_conversations_on_shutdown ran up to
max_live_sessions (50) sequential, untimed LLM calls, and it ran BEFORE
stop_all_chat_sessions writes checkpoints — so under a 30s termination grace
period the process was killed generating titles and lost everything each channel
had accumulated since its last eviction. Shutdown labeling was also already LRU
roulette: the ~92% of conversations evicted before SIGTERM were never labeled at
all.

- Shutdown makes unsaved turns durable through the same save_conversation_
  incremental chokepoint the turn path uses, and leaves blank placeholders. The
  active_conversation_id == 0 fallback is kept: it is the only path that records
  turns for a channel whose persist was skipped by a raising work_fn.
- Labels fill on the first completed chat turn (after DONE, so the LLM call is
  outside the drain, off the turn's critical path and outside the 409 window), on
  activate when still blank, and on rotate. Refreshed at turn counts 1, 4, 16,
  64, … so a long thread costs O(log n) calls instead of one or n; labeling once
  and never again would give a 40-turn thread the title of its first exchange.
- Generation is bounded at the LLM client, not by the caller's await: timeout and
  num_retries=1 go to dspy.LM. asyncio.wait_for would cancel the wait while the
  thread ran on, and CPython then joins default-executor threads for up to
  THREAD_JOIN_TIMEOUT (300s) at exit. New LLM_CONVERSATION_STORE_TIMEOUT_SECONDS
  (default 12), resolved process-environment-first so a container override is
  reachable, validated and logged at startup.
- "conversation summary" now identifies its turn on the deterministic and action
  paths. Both hardcoded a constant ("assistant_mode_command",
  "process_action command"), and since generate_topic_and_summary reads only that
  field, /initialize conversations were asking an LLM to title a fixed string.
  Bounded and newline-collapsed; parameters stay in conversation_traces.
- Blank topic is now a sound "no title yet" sentinel. An empty generated topic
  collided with every other unlabeled conversation and stored as ' 1', which
  counted as labeled forever.
- ConversationStore._ensure_unique_topic excludes the record being written; a
  rewrite with an unchanged topic oscillated between 'T' and 'T 1'.
- New get_conversation_window: the restore paths kept 20 turns but READ every
  turn, so at 450 KB payloads restoring a long conversation was tens of MB
  resident.
- /new_conversation persists before labeling. In the reserved-but-unwritten state
  it spent a generation and then raised out of update_conversation_topic_summary,
  making the rotate that would have cleaned the state up the one unavailable
  operation.
- Streaming EOF no longer waits on the label (new on_done on run_owned_turn).
  invoke_agent, the MCP-exposed tool, IS the streaming endpoint, and fastapi-mcp
  reads the whole stream as the tool result.

Three of radiantlogicinc#65's specific designs were wrong and are corrected here: preview rested
on that constant-literal field, its per-call timeout could not bound the process,
and its call site would have put the LLM call back inside the drain it was written
to protect. Full findings and resolutions in bd epic fix-dzs (11 children).

Verified: full suite 1788 passed, 15 skipped, 0 failed (48:56); trained example
artifacts byte-identical apart from regenerable build caches. Four fixes verified
by falsification — reverting each makes its own test fail and no other.

Co-authored-by: Cursor <cursoragent@cursor.com>

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry @dharrawal, your pull request is larger than the review limit of 150000 diff characters

@sourcery-ai

sourcery-ai Bot commented Aug 13, 2026

Copy link
Copy Markdown

Reviewer's Guide

Implements lazy, bounded conversation labeling and removes LLM work from shutdown, adds windowed conversation reads, tightens topic uniqueness/blank handling, and wires labeling into turn and streaming lifecycles with new configuration and tests.

Sequence diagram for lazy conversation labeling after a completed turn

sequenceDiagram
    actor User
    participant FastAPIRoute as invoke_agent
    participant Turns as run_owned_turn
    participant Utils as save_conversation_incremental
    participant Label as _label_conversation_after_turn
    participant LabelHelper as try_ensure_topic_and_summary
    participant StoreHelper as ensure_topic_and_summary
    participant Store as ConversationStore
    participant LLM as generate_topic_and_summary

    User->>FastAPIRoute: POST /invoke_agent (chat turn)
    FastAPIRoute->>Turns: run_owned_turn(runtime, turn_registry, execn, work, session_manager, on_done)
    Turns->>Utils: save_conversation_incremental(runtime, extract_turns_from_history, logger)
    Utils-->>Turns: turns_appended
    Turns->>Turns: execn.exec_state = DONE
    Turns->>Label: _label_conversation_after_turn(runtime, execn, turns_appended)
    Label->>LabelHelper: try_ensure_topic_and_summary(runtime, generate_topic_and_summary, turns_appended)
    LabelHelper->>StoreHelper: ensure_topic_and_summary(runtime, generate_topic_and_summary, turns_appended)
    StoreHelper->>Store: get_conversation_label_state(conversation_id)
    Store-->>StoreHelper: (stored_topic, durable_turns)
    StoreHelper->>StoreHelper: _label_is_due(stored_topic, durable_turns, turns_appended)
    alt label due
        StoreHelper->>Store: get_conversation_summaries(conversation_id)
        Store-->>StoreHelper: turns
        StoreHelper->>LLM: generate_topic_and_summary(turns)
        LLM-->>StoreHelper: (topic, summary)
        StoreHelper->>Store: update_conversation_topic_summary(conversation_id, topic, summary)
    else label not due or blank topic
        StoreHelper-->>LabelHelper: None
    end
    LabelHelper-->>Label: (topic, summary) or None
    Label-->>Turns: labeling complete
    Turns-->>FastAPIRoute: turn finished (response already sent via on_done)
Loading

Sequence diagram for activation with windowed restore and lazy labeling

sequenceDiagram
    actor User
    participant FastAPIRoute as activate_conversation
    participant Runtime as ChannelRuntime
    participant Store as ConversationStore
    participant Utils as restore_history_from_turns
    participant LabelHelper as try_ensure_topic_and_summary
    participant StoreHelper as ensure_topic_and_summary
    participant LLM as generate_topic_and_summary

    User->>FastAPIRoute: POST /activate_conversation
    FastAPIRoute->>Runtime: get_session(channel_id)
    Runtime-->>FastAPIRoute: runtime
    FastAPIRoute->>Store: get_conversation_window(conv_id, MAX_CONVERSATION_TURNS_IN_MEMORY)
    Store-->>FastAPIRoute: conv (turns window)
    FastAPIRoute->>Utils: restore_history_from_turns(conv["turns"])
    Utils-->>FastAPIRoute: restored_history
    FastAPIRoute->>Runtime: set execution_context._conversation_history, durable_turn_count

    FastAPIRoute->>LabelHelper: try_ensure_topic_and_summary(runtime, generate_topic_and_summary)
    LabelHelper->>StoreHelper: ensure_topic_and_summary(runtime, generate_topic_and_summary, turns_appended=0)
    StoreHelper->>Store: get_conversation_label_state(conversation_id)
    Store-->>StoreHelper: (stored_topic, durable_turns)
    StoreHelper->>StoreHelper: _label_is_due(stored_topic, durable_turns, 0)
    alt blank topic and durable_turns > 0
        StoreHelper->>Store: get_conversation_summaries(conversation_id)
        Store-->>StoreHelper: turns
        StoreHelper->>LLM: generate_topic_and_summary(turns)
        LLM-->>StoreHelper: (topic, summary)
        StoreHelper->>Store: update_conversation_topic_summary(conversation_id, topic, summary)
    else already labeled or no turns
        StoreHelper-->>LabelHelper: None
    end

    LabelHelper-->>FastAPIRoute: (topic, summary) or None
    FastAPIRoute-->>User: {status: "ok"}
Loading

File-Level Changes

Change Details Files
Introduce windowed conversation reads and lightweight label state access in ConversationStore to avoid deserializing entire long conversations and support lazy labeling decisions.
  • Add _read_turn_window to read only the last N turns by key range instead of scanning all turns.
  • Add get_conversation_window to return a conversation record with only its newest window of turns.
  • Add get_conversation_label_state to fetch topic and durable turn count in one record read.
  • Update cold restore and activate paths to use get_conversation_window rather than get_conversation + slicing.
fastworkflow/run_fastapi_mcp/conversation_store.py
fastworkflow/run_fastapi_mcp/utils.py
fastworkflow/run_fastapi_mcp/__main__.py
tests/test_conversation_window_reads.py
Refine conversation topic generation configuration and bounding at the LLM client, with environment-driven timeout, fixed retry count, and startup validation/logging.
  • Add TOPIC_GENERATION_TIMEOUT_ENV_VAR, DEFAULT_TOPIC_GENERATION_TIMEOUT_SECONDS, TOPIC_GENERATION_MAX_RETRIES constants and resolver helpers.
  • Pass timeout and num_retries into get_lm for LLM_CONVERSATION_STORE calls.
  • Resolve and log topic generation timeout and attempt count at startup, warn if the worst case exceeds SHUTDOWN_DRAIN_SECONDS.
  • Document LLM_CONVERSATION_STORE_TIMEOUT_SECONDS in READMEs and example env.
fastworkflow/run_fastapi_mcp/conversation_store.py
fastworkflow/run_fastapi_mcp/__main__.py
fastworkflow/run_fastapi_mcp/utils.py
README.md
fastworkflow/run_fastapi_mcp/README.md
fastworkflow/examples/fastworkflow.env
tests/test_conversation_topic_generation_bounds.py
Change shutdown behavior to persist unsaved conversation turns via the incremental save path while removing all topic/summary generation from shutdown.
  • Replace finalize_conversations_on_shutdown to call save_conversation_incremental and leave topics/summaries blank.
  • Use SHUTDOWN_DRAIN_SECONDS constant for wait_for_active_turns_to_complete instead of hardcoded 30.
  • Add test to assert shutdown persists unsaved turns without triggering topic generation and to ensure no LLM calls at shutdown.
fastworkflow/run_fastapi_mcp/__main__.py
fastworkflow/run_fastapi_mcp/utils.py
tests/test_manager_shutdown_matrix.py
tests/soak/memory_soak.py
Fix topic uniqueness and blank-topic handling so conversations don’t self-collide, retain good titles, and treat blank topics as a retryable sentinel.
  • Extend _ensure_unique_topic to take exclude_conversation_id and ignore that record in collision checks.
  • Return "" immediately for blank candidate topics to avoid suffixing them and to skip DB scans.
  • Update save_conversation, update_conversation, and update_conversation_topic_summary to use exclusion and to preserve existing topics when generation returns blank.
  • Add integration tests covering uniqueness, self-collision, explicit-id saves, blank topics as sentinel, and logging behavior.
fastworkflow/run_fastapi_mcp/conversation_store.py
tests/test_conversation_topic_labeling.py
Introduce lazy conversation labeling with a geometric refresh schedule, per-channel label locks, and shared helpers for opportunistic and forced labeling triggers.
  • Add CONVERSATION_LABEL_GROWTH_FACTOR and milestone computation helpers to decide when a refresh is due.
  • Implement ensure_topic_and_summary and try_ensure_topic_and_summary helpers that run generation in an executor, gate on topic/turn state, and update the store.
  • Add label_lock to ChannelRuntime to serialize topic generation per writer without blocking runtime.lock.
  • Wire lazy labeling into _run_turn and run_owned_turn via _label_conversation_after_turn, using turns_appended and labelability gates.
  • Add comprehensive tests for lazy labeling behavior, triggers (chat turn, activation, rotate, streaming), failure retry semantics, concurrency, and refresh schedule.
fastworkflow/run_fastapi_mcp/utils.py
fastworkflow/run_fastapi_mcp/turns.py
fastworkflow/run_fastapi_mcp/conversation_store.py
fastworkflow/workflow_execution_context.py
fastworkflow/run_fastapi_mcp/__main__.py
tests/test_conversation_lazy_labeling.py
tests/test_conversation_topic_generation_bounds.py
Adjust turn summary content for deterministic and process_action paths so stored "conversation summary" is informative, bounded, and preserves full traces separately.
  • Change WorkflowExecutionContext._process_message to set conversation summary to a compact "user message -> response" string instead of a fixed constant.
  • Change _process_action to set conversation summary to "command_name -> response" string with length bounds and newline collapsing.
  • Verify via tests that summaries identify the command, stay single-line and bounded, and that conversation_traces still hold the full payload record.
fastworkflow/workflow_execution_context.py
tests/test_conversation_turn_summary_content.py
Ensure /new_conversation persists turns before labeling, runs generation off the event loop, keeps rotate strict on generation failure, and relies on shared labeling helpers.
  • Call save_conversation_incremental before topic generation in new_conversation to guarantee a durable record and clear reserved-but-unwritten states.
  • Use ensure_topic_and_summary with force=True to generate and write labels from full conversation turns in a worker thread without asyncio.wait_for.
  • On generation or label-write failure, return HTTP 500 that explains that the conversation was not rotated, its turns are durable, and includes timeout configuration in the message.
  • Add tests for rotate behavior on failure, persistence of previously unsaved turns, off-loop behavior, and that the endpoint waits for generation to finish.
fastworkflow/run_fastapi_mcp/__main__.py
fastworkflow/run_fastapi_mcp/utils.py
tests/test_conversation_topic_generation_bounds.py
tests/test_conversation_lazy_labeling.py
Update streaming turn lifecycle run_owned_turn to support an on_done callback, label conversations after streaming, and avoid EOF waiting on label generation.
  • Extend run_owned_turn to accept on_done, call it after execn.done_event is set and before trimming sessions and labeling.
  • Add _streaming_turn helper and tests that verify streaming turns label conversations, EOF precedes label completion, on_done sees execn.error, and failing on_done doesn’t skip labeling.
  • Modify invoke_agent_stream owned_turn implementation to use on_done to flush end-of-stream (error/event) before labeling and to always send a sentinel even on failures.
fastworkflow/run_fastapi_mcp/__main__.py
fastworkflow/run_fastapi_mcp/turns.py
tests/test_conversation_lazy_labeling.py
Bump framework version to 3.1.0 and align docs with new env knob and shutdown behavior.
  • Update pyproject.toml version to 3.1.0.
  • Document new LLM_CONVERSATION_STORE_TIMEOUT_SECONDS and lazy labeling/shutdown semantics in top-level and FastAPI README.
  • Note test suite size and leave existing doc rot unchanged by design.
pyproject.toml
README.md
fastworkflow/run_fastapi_mcp/README.md

Assessment against linked issues

Issue Objective Addressed Explanation
#65 Remove LLM topic/summary generation from process shutdown and instead generate topics/summaries lazily when conversations are actually used as chats (first completed chat turn, POST /activate_conversation when topic is blank, keep generation on POST /new_conversation), with no LLM calls on GET /conversations or /perform_action.
#65 Extend GET /conversations to return a cheap preview (e.g., truncated first-turn conversation summary) so pickers are usable when topic is blank, without invoking the LLM. The PR explicitly notes that the proposed preview design was incorrect and defers implementing it. No preview field is added to ConversationSummary or GET /conversations; the list endpoint remains unchanged aside from behavior around topics, and no non-LLM preview is introduced.

Possibly linked issues


Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

drawal1 and others added 2 commits August 13, 2026 18:29
test_invoke_command_wraps_error had been skipping on a hasattr() guard for
CommandExecutor._invoke_command_metadata_extraction_workflow, so it asserted
nothing instead of failing when the implementation moved. Two things had gone
stale, not one: the CME hop is now a perform_action call with command_name
"wildcard", and ChatSession no longer takes workflow_folderpath/workflow_id_str,
so the session the test built could not have been constructed either.

Rewritten against the production caller. workflow_execution_context.py:1175
passes `self` to invoke_command, so the test drives a real
WorkflowExecutionContext rather than a ChatSession — the parameter is annotated
ChatSession but only cme_workflow, get_active_workflow() and
cme_workflow._context are touched, and WEC supplies all three. The active
workflow is pushed as production pushes it, because a WEC's
get_active_workflow() reads only the context-local stack and deliberately does
not fall back to the bound app workflow.

perform_action is the stub seam because that IS the CME hop now; the stubbed
registry cannot serve it (DummyCRD has no class for "wildcard", so
perform_action would raise ValueError over the RuntimeError under test). Only
the CME hop routes through perform_action — invoke_command instantiates the
resolved command's generator directly — so stubbing it does not hide the call
being tested.

Verified discriminating rather than assumed: with a non-raising generator in the
same wiring, invoke_command returns normally with the response text and
command_name set, so the RuntimeError is attributable to the faulty generator
reached through invoke_command. Swap the generator and the test fails with DID
NOT RAISE. This was the only hasattr-guarded skip of its kind in the suite.

Co-authored-by: Cursor <cursoragent@cursor.com>
Six verified-wrong claims in the agent-facing docs, found while releasing v3.1.0.
No version bump: documentation only.

- Suite size and duration. AGENTS.md said ~24 minutes and the skills said 495
  collected; it is 1803 tests and 48:56, measured 2026-08-13 at v3.1.0. An agent
  budgeting against ~24 minutes concludes a healthy run has hung. Also updated in
  two helper scripts whose sanity thresholds keyed off 495.
- The full-run baseline the validation skill asked for. Its row said "unverified —
  run it and record it"; recorded as 1788 passed / 15 skipped / 0 failed / 0
  errors, with the honest caveat that this was ONE run and so does not meet that
  skill's own two-consecutive-runs idempotence bar, plus the checksum finding that
  10 regenerable build-cache JSONs are rewritten by the build tests.
- The 0.5s autouse sleep. Documented as costing ~4 min of dead time with the
  underlying thread-lifecycle issue unfixed; conftest.py:151-169 has since been
  changed to join only when a ChatWorker is actually alive.
- `bd close --reason`. Held across THIRTEEN skill files as silently failing to
  persist. Retested on bd 1.1.2: twelve consecutive closes with --reason all
  landed, verified by reading close_reason back out of the JSONL. Corrected in
  every one of those files rather than only the owning skill — a library that
  contradicts itself is worse than one uniformly stale. The 2026-06-11 observation
  is kept, attributed to the older bd, because the symptom may return if Dolt
  wedges; the verify-the-JSONL half of that rule is left as permanent.
- The publication boundary. change-control section 7 said `.claude/skills/` was
  untracked-but-not-gitignored, making `git add -A` a publication risk. It is now
  a mix: 14 skill dirs plus README are deliberately tracked and public, while the
  four team-private ones are gitignored by exact path (.gitignore:219-237), as are
  the tau2 plan, RSI harness report and Forge spec. So -A can no longer publish
  them. Kept the stage-by-explicit-path habit, with its real remaining rationale.
- Recorded what a stale-old_value chain in .beads/interactions.jsonl means, since
  an audit found 225 of them: it is the flakiness fingerprint, it is harmless
  (old_value is a breadcrumb nothing reads; every status was correct), and that
  log must not be "repaired" because it is the only evidence those episodes left.

The four gitignored team-private skills carry the same --reason correction
locally; they are excluded from this commit by .gitignore, as intended.

Co-authored-by: Cursor <cursoragent@cursor.com>
@drawal1
drawal1 merged commit 28c39fe into radiantlogicinc:main Aug 13, 2026
1 of 2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Defer conversation topic/summary generation off process shutdown

2 participants