implementation.md

Implementation Log

<<<<<<< Updated upstream

Route Splits: sandboxes-api, ai-providers-api, experimentation-api

Completed Steps

1. Split sandboxes-api/routes.py into sandboxes.py (lifecycle), execution.py (execute/logs), files.py (read/write/tree/cleanup) — committed as 3a8b723
2. Split ai-providers-api/routes.py into providers.py (CRUD + default), models.py (model discovery), api_keys.py (API key update) — committed as c654ee3
3. Split experimentation-api/routes.py into kpis.py, experiments.py, metrics.py — committed as b967ef9

Deviations from Plan

• ai-providers-api: model_store_provider and ai_provider_store_provider were both declared in providers.py (rather than routes.py) so all sub-modules share the exact same StoreProvider objects via import, avoiding the Python function-default capture problem with Depends().
• ai-providers-api: Updated test patch path from lintel.ai_providers_api.routes.httpx.AsyncClient to lintel.ai_providers_api.models.httpx.AsyncClient since httpx now lives in models.py.

Files Created

• packages/sandboxes-api/src/lintel/sandboxes_api/sandboxes.py
• packages/sandboxes-api/src/lintel/sandboxes_api/execution.py
• packages/sandboxes-api/src/lintel/sandboxes_api/files.py
• packages/ai-providers-api/src/lintel/ai_providers_api/providers.py
• packages/ai-providers-api/src/lintel/ai_providers_api/models.py
• packages/ai-providers-api/src/lintel/ai_providers_api/api_keys.py
• packages/experimentation-api/src/lintel/experimentation_api/kpis.py
• packages/experimentation-api/src/lintel/experimentation_api/experiments.py
• packages/experimentation-api/src/lintel/experimentation_api/metrics.py

Test Results

• sandboxes-api: 13 passed
• ai-providers-api: 24 passed
• experimentation-api: 8 passed =======

Refactor: Split chat-api routes.py (commit 8bad589)

Completed Steps

1. Extracted ChatStore to store.py
2. Extracted Pydantic models to models.py
3. Extracted ChatService to service.py
4. Extracted SSE streaming endpoints to streaming.py
5. Rewrote routes.py to CRUD handlers + re-exports
6. Mounted streaming_router in routers.py

Deviations from Plan

• ChatService extracted to service.py (not in the original spec) to avoid circular imports between streaming.py and routes.py
• Fixed _stage_names_for_workflow import to use lintel.pipelines_api._helpers (original used the wrong module lintel.pipelines_api.routes)

Files Created

• packages/chat-api/src/lintel/chat_api/store.py — ChatStore
• packages/chat-api/src/lintel/chat_api/models.py — request/response models
• packages/chat-api/src/lintel/chat_api/service.py — ChatService
• packages/chat-api/src/lintel/chat_api/streaming.py — SSE streaming_router

Files Modified

• packages/chat-api/src/lintel/chat_api/routes.py — CRUD handlers + re-exports
• packages/app/src/lintel/api/routers.py — mounts chat_streaming_router

Test Results

• 61 tests passing, 0 failures

---

Phase 6: Split WorkflowExecutor into smaller files

Completed Steps

1. Read full workflow_executor.py (1,063 lines) to map all method dependencies.
2. Created _executor_lifecycle.py with stage lifecycle helpers as standalone async functions.
3. Created _executor_artifacts.py with work item / approval / policy helpers as standalone async functions.
4. Rewrote workflow_executor.py to delegate to extracted functions; added thin shim methods for backwards compatibility.
5. Verified 276 tests pass (13 pre-existing test_setup_workspace failures unchanged). — committed as a98ebed

Deviations from Plan

• The plan specified method names that did not exist in the source file. The extraction was done on the actual methods present, grouped by logical concern.
• _executor_artifacts.py received work item / approval / policy methods rather than artifact I/O.
• _executor_lifecycle.py received stage lifecycle and pipeline status methods.
• A _dict_to_stage_local helper was duplicated in both extracted modules to avoid circular imports; the canonical _dict_to_stage stays in workflow_executor.py.
• Thin shim wrappers were added to WorkflowExecutor so any callers using self._mark_stage_completed(...) style continue to work.

Files Created

• /Users/bamdad/projects/lintel/packages/workflows/src/lintel/workflows/_executor_lifecycle.py — stage lifecycle helpers
• /Users/bamdad/projects/lintel/packages/workflows/src/lintel/workflows/_executor_artifacts.py — work item / approval / policy helpers

Files Modified

• /Users/bamdad/projects/lintel/packages/workflows/src/lintel/workflows/workflow_executor.py — reduced to core class with shim methods

Test Results

• 276 tests passing; 13 pre-existing failures in test_setup_workspace.py

Stashed changes

---

Phase 5: Extract 6 Large/Complex Packages

Completed Steps

1. Fixed StoreProvider.__class_getitem__ to support generic subscript StoreProvider[T] notation
2. Added test_sandboxes.py to sandboxes-api/tests/ (was missing), updating import to lintel.sandboxes_api.routes
3. Added client fixture to chat-api/tests/conftest.py for test_chat_project_selection.py and test_chat_retry.py
4. Ran ruff auto-fix (59 import-sort issues) across all 6 packages
5. Committed all 6 packages + app wiring — committed as 0fd4a71

Deviations from Plan

• All 6 packages were already scaffolded (routes, stores, domain logic, tests) when work began — the main gaps were StoreProvider subscript support, the missing sandboxes test file, and the missing chat conftest fixture.

Files Created

• packages/compliance-api/ — ComplianceStore, routes, seed, 14 tests
• packages/experimentation-api/ — uses ComplianceStore, routes, 9 tests
• packages/automations-api/ — InMemoryAutomationStore, routes, scheduler, hooks, 48 tests
• packages/sandboxes-api/ — SandboxStore, routes, 11 tests
• packages/pipelines-api/ — InMemoryPipelineStore, routes, delivery_loop, 26+ tests
• packages/chat-api/ — ChatStore, routes, chat_router, 110+ tests

Files Modified

• packages/api-support/src/lintel/api_support/provider.py — added __class_getitem__ for generic subscript support
• packages/app/src/lintel/api/app.py — wired all 6 new packages via StoreProvider.override()
• packages/app/pyproject.toml — added 6 new dependencies
• packages/app/tests/conftest.py — updated stale imports to new package paths
• pyproject.toml — added 6 new test/src paths

Test Results

• 218 tests passing across all 6 new packages
• 388 app tests passing, 59 skipped — no regressions

---

Phase 4: Extract Medium CRUD Batch (8 packages)

Completed Steps

1. Created lintel-boards package with TagStore, BoardStore, routes — committed as cbceef8
2. Created lintel-triggers-api package with InMemoryTriggerStore, routes — committed as ea9494c
3. Created lintel-artifacts-api package with CodeArtifactStore, TestResultStore, routes — committed as 01d5e60
4. Created lintel-projects-api package with ProjectStore, routes — committed as 6e90009
5. Created lintel-work-items-api package with WorkItemStore, routes — committed as 55684fd
6. Created lintel-skills-api package with InMemorySkillStore, routes, and moved domain/skills/ — committed as 30ea09e
7. Created lintel-agent-definitions-api package with AgentDefinitionStore, routes — committed as 62bc05b
8. Created lintel-mcp-servers-api package with InMemoryMCPServerStore, routes — committed as 5dffa70
9. Wired all 8 packages into app, updated pyproject files, fixed stale imports — committed as 8f2d9ac

Deviations from Plan

• Fixed pre-existing stale imports in workflow tests (test_policy.py → lintel.policies_api.store, test_setup_workspace.py → lintel.variables_api.store)
• Added lintel-skills-api as a dependency to lintel-workflows (workflows nodes import discover_test_command)

Files Created (new packages)

• packages/boards/ — TagStore, BoardStore, routes, 11 tests
• packages/triggers-api/ — InMemoryTriggerStore, routes, 6 tests
• packages/artifacts-api/ — CodeArtifactStore, TestResultStore, routes, 9 tests
• packages/projects-api/ — ProjectStore, routes, 10 tests
• packages/work-items-api/ — WorkItemStore, routes, 8 tests
• packages/skills-api/ — InMemorySkillStore, routes, domain skills, 29 tests
• packages/agent-definitions-api/ — AgentDefinitionStore, routes, 13 tests
• packages/mcp-servers-api/ — InMemoryMCPServerStore, routes, 8 tests

Test Results

• All 8 new packages: 100 tests passing
• packages/app/tests/: 388 passed, 59 skipped
• packages/workflows/tests/: 276 passed (1 pre-existing failure in test_setup_workspace)

Phase 1: Protocol + Docker backend execute_stream (Tasks 1 & 2)

Completed Steps

1. Added execute_stream method to SandboxManager protocol with docstring and yield "" stub — committed as 1b53c55
2. Created packages/contracts/tests/test_execute_stream_protocol.py with conformance tests — committed as 1b53c55
3. Added AsyncIterator to TYPE_CHECKING import block in docker_backend.py — committed as 9531e6a
4. Implemented execute_stream in DockerSandboxManager using low-level exec_create/exec_start(demux=True) — committed as 9531e6a
5. Created packages/infrastructure/tests/sandbox/test_docker_streaming.py with 8 tests — committed as 9531e6a

Deviations from Plan

• None

Files Created

• /Users/bamdad/projects/lintel/packages/contracts/tests/test_execute_stream_protocol.py — protocol conformance tests
• /Users/bamdad/projects/lintel/packages/infrastructure/tests/sandbox/test_docker_streaming.py — Docker streaming tests

Files Modified

• /Users/bamdad/projects/lintel/packages/contracts/src/lintel/contracts/protocols.py:200-218 — added execute_stream method to SandboxManager
• /Users/bamdad/projects/lintel/packages/infrastructure/src/lintel/infrastructure/sandbox/docker_backend.py:16-17,203-268 — added AsyncIterator import and execute_stream implementation

Test Results

• contracts: 106 tests passing
• infrastructure: 372 passed, 7 skipped

Notes for Reviewer

• execute_stream returns _stream() (inner async generator) eagerly — setup (exec_create, exec_start) happens synchronously at call time; only iteration is lazy. This matches the plan's design note.
• Per-chunk timeout uses asyncio.wait_for wrapping asyncio.to_thread(_next_chunk). On timeout, raises SandboxTimeoutError.
• The protocol stub uses yield "" (like stream_model does) rather than ... because the method is an async generator in the protocol definition.

---

Phase 3: Extract Simple CRUD Batch (8 packages)

Completed Steps

1. Extracted lintel-teams — committed as 2c6ffe9
2. Extracted lintel-policies-api — committed as 4191d32
3. Extracted lintel-notifications-api — committed as a3c62e5
4. Extracted lintel-environments-api — committed as a1a542a
5. Extracted lintel-variables-api — committed as 25944f7
6. Extracted lintel-credentials-api — committed as c907e0a
7. Extracted lintel-audit-api — committed as deaf3a6
8. Extracted lintel-approval-requests-api — committed as 9642174

Deviations from Plan

• lintel-credentials-api depends on lintel-persistence (not just domain) because it imports Credential, CredentialType from lintel.persistence.types and CredentialStored, CredentialRevoked from lintel.persistence.events.
• lintel-audit-api routes.py does not import dispatch_event — the original audit.py had no event dispatching.
• approval_requests.py originally used a request-based store getter pattern — converted to StoreProvider pattern to match all other extracted packages.

Files Created

• packages/teams/ — lintel-teams package
• packages/policies-api/ — lintel-policies-api package
• packages/notifications-api/ — lintel-notifications-api package
• packages/environments-api/ — lintel-environments-api package
• packages/variables-api/ — lintel-variables-api package
• packages/credentials-api/ — lintel-credentials-api package
• packages/audit-api/ — lintel-audit-api package
• packages/approval-requests-api/ — lintel-approval-requests-api package

Test Results

• All 8 extracted packages: all tests passing (teams: 5, policies: 5, notifications: 5, environments: 8, variables: 6, credentials: 11, audit: 5, approval-requests: 6)
• App test suite: 491 passed, 67 skipped — no regressions

---

Import Migration: lintel.infrastructure.* → new package paths

Completed Steps

1. Updated all lintel.infrastructure.* imports in consumer packages to use new extracted package paths.

Files Modified

• packages/app/src/lintel/api/app.py — 10 top-level imports + 4 inline imports updated
• packages/app/src/lintel/api/deps.py — 5 imports updated
• packages/app/src/lintel/api/routes/admin.py — 3 imports updated (1 top-level, 2 inline)
• packages/app/src/lintel/api/routes/events.py — 2 imports updated (1 top-level, 1 inline)
• packages/app/src/lintel/api/routes/repositories.py — 1 import updated
• packages/app/src/lintel/api/routes/workflows.py — 1 import updated
• packages/app/src/lintel/api/routes/threads.py — 1 import updated
• packages/app/src/lintel/api/routes/debug.py — 1 inline import updated
• packages/app/src/lintel/api/domain/chat_router.py — 1 import updated
• packages/app/src/lintel/api/domain/scheduler_loop.py — 1 import updated
• packages/app/tests/conftest.py — 2 imports updated
• packages/app/tests/api/test_admin_projections.py — 1 import updated
• packages/workflows/src/lintel/workflows/nodes/close.py — 1 inline import updated
• packages/workflows/tests/workflows/test_close_node.py — 3 @patch paths updated
• packages/models/tests/models/test_claude_code_streaming.py — 3 @patch paths updated
• packages/sandbox/tests/sandbox/test_docker_streaming.py — 1 @patch path updated
• tests/e2e/test_claude_code_sandbox.py — 1 import updated
• tests/integration/test_event_store.py — 1 import updated
• tests/integration/test_pii_pipeline.py — 3 imports updated
• tests/integration/test_postgres_chat_store.py — 1 import updated
• tests/integration/test_pii_vault.py — 1 import updated
• tests/integration/test_full_pipeline.py — 3 imports updated
• tests/integration/test_workflow_lifecycle.py — 3 imports updated
• tests/integration/sandbox/conftest.py — 1 inline import updated

Deviations from Plan

• None. Files inside packages/infrastructure/ and packages/domain/ were left untouched as specified.
• lintel.infrastructure.mcp references kept as-is (MCP stays in infrastructure per the task spec).