Skip to content
This repository was archived by the owner on Jul 13, 2026. It is now read-only.
This repository was archived by the owner on Jul 13, 2026. It is now read-only.

Increase automator observability and validation clarity #5

Description

@bma-d

Summary

Increase observability across the automator so orchestration failures, malformed inputs, and recovery paths are easier to diagnose. This came out of Discord discussion around the current Python internals, integration boundaries, and how hard it can be to reason about failures when data flows through loosely typed dictionaries. Thanks @alexeyv for the thoughtful feedback.

Problem

The automator currently relies heavily on untyped dictionary-style payloads moving between LLM output, files on disk, and orchestration code. That was practical for the original port, but it makes failures harder to inspect when the project grows beyond solo usage.

The highest-value first step is observability: make the automator explain what it saw, what it accepted or rejected, where state changed, and what recovery action is possible.

Goals

  • Add clearer structured logging around orchestration steps, story/epic/session state transitions, and policy decisions.
  • Validate external boundary data from LLM output and disk before it enters core orchestration paths.
  • Emit specific, actionable error messages for invalid keys, missing fields, wrong value types, and illegal state transitions.
  • Make recovery/debugging easier for both humans and agents by preserving enough context without leaking sensitive data.
  • Lay groundwork for future typed domain objects such as Epic, Story, Session, and OrchestrationPolicy, without requiring a full rewrite in this issue.

Non-goals

  • Full OO/domain-model rewrite.
  • Large behavior changes to orchestration semantics.
  • New persistence format unless required for compatibility-safe diagnostics.

Suggested approach

  1. Identify the main trust boundaries: LLM responses, file reads, CLI/config inputs, and persisted session/story data.
  2. Add lightweight schema/validator functions or typed models at those boundaries.
  3. Introduce a consistent error type or error-report format with field path, expected value, actual value, and suggested recovery.
  4. Add structured log events for lifecycle milestones and state transitions.
  5. Add tests for invalid payloads and failure diagnostics so errors stay specific over time.

Acceptance criteria

  • Invalid external payloads fail with clear, field-specific errors before deep orchestration code runs.
  • Key orchestration stages emit structured diagnostics suitable for debugging a failed run.
  • State transition failures identify current state, attempted transition, and allowed transitions.
  • Tests cover representative malformed LLM/file inputs and assert useful error messages.
  • Existing successful automator workflows continue to run unchanged.

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions