feat: check_models external readiness check

[mikeknep]

📋 Summary

Introduces a new check_models CLI command and DataDesigner interface method for checking external readiness of models and tools without triggering a full workload (preview or create). This is the "external deps" analogue to the existing validate functionality (internal coherence).

🔗 Related Issue

N/A, direct to PR

🔄 Changes

• Lifts the checks run by DatasetBuilder to a standalone readiness.py engine module. Used by both the builder and the end user-facing interfaces
• Creates a flags.py engine module where the async engine flag is centralized. This cleans up some duplication of the env var magic string / constant that was floating around in a few places.

🧪 Testing

• make test passes
• Unit tests added/updated
• E2E tests added/updated (if applicable)

✅ Checklist

• Follows commit message conventions
• Commits are signed off (DCO)
• Architecture docs updated (if applicable)

[greptile-apps]

Greptile Summary

This PR introduces a check_models command (CLI + DataDesigner method) that exposes a standalone external-readiness check — probing every model alias and MCP tool alias in a configuration without starting a workload. The implementation lifts the pre-flight health-check logic out of DatasetBuilder into a new shared readiness.py engine module and centralises the DATA_DESIGNER_ASYNC_ENGINE flag in flags.py, so both the builder startup path and the new public method share one code path.

• engine/readiness.py: New module hosting run_readiness_check, _run_model_health_check, and _run_mcp_tool_health_check. Called by both DatasetBuilder.build/build_preview (unchanged behavior) and the new DataDesigner.check_models method.
• engine/flags.py: Centralises the async-engine boolean so the engine, interface, and readiness module share one source of truth and tests can reliably monkeypatch a single location.
• CLI: New check-models command wired into the lazy-dispatch app alongside the existing validate command, with matching error-handling in GenerationController.run_check_models.

Confidence Score: 5/5

Safe to merge — the refactor faithfully preserves existing workload-startup behavior while adding a well-tested standalone check path.

The readiness logic moved into its own module is functionally identical to the removed DatasetBuilder methods: same async/sync dispatch, same MCP column-type filter via column_type_is_model_generated, same 180-second timeout. Flags centralisation is a clean single-write change. The new check_models interface method follows the same shape and ResourceProvider construction as validate. Test coverage is thorough across async dispatch, timeout/cancel, ordering guarantees, MCP dedup/sort, and all error propagation paths.

No files require special attention.

Important Files Changed

Filename	Overview
packages/data-designer-engine/src/data_designer/engine/readiness.py	New module hosting the shared readiness-check logic lifted from DatasetBuilder; logic is functionally equivalent to what was removed, with correct async/sync dispatch and MCP column-type filter.
packages/data-designer-engine/src/data_designer/engine/flags.py	New module centralising DATA_DESIGNER_ASYNC_ENGINE; clean single-source-of-truth, evaluated at import time, intended to be monkeypatched in tests.
packages/data-designer-engine/src/data_designer/engine/dataset_builders/dataset_builder.py	Removes the two run*_if_needed instance methods, delegates to run_readiness_check, and switches all flag reads to flags.DATA_DESIGNER_ASYNC_ENGINE. Behavior is unchanged.
packages/data-designer/src/data_designer/interface/data_designer.py	Adds check_models method following the same shape as validate(); delegates to run_readiness_check after constructing a ResourceProvider.
packages/data-designer/src/data_designer/cli/controllers/generation_controller.py	Adds run_check_models with typed-error handling that surfaces the class name for DataDesignerError subclasses, and a generic fallback for other exceptions.
packages/data-designer/src/data_designer/cli/commands/check_models.py	New CLI command delegating to GenerationController.run_check_models; mirrors the validate command structure.
packages/data-designer-engine/tests/engine/test_readiness.py	Comprehensive test coverage: model alias collection, MCP alias deduplication/sorting, ordering guarantee, async dispatch, timeout/cancel path, and column-type coverage.

Sequence Diagram

sequenceDiagram
    participant User
    participant RC as GenerationController
    participant DD as DataDesigner
    participant RD as readiness.run_readiness_check
    participant MR as ModelRegistry
    participant MCP as MCPRegistry

    Note over User,MCP: New check-models path
    User->>RC: check-models config.yaml
    RC->>DD: check_models(config_builder)
    DD->>RD: run_readiness_check(columns, resource_provider)
    RD->>MR: run_health_check(model_aliases)
    MR-->>RD: ok or typed error
    RD->>MCP: run_health_check(tool_aliases)
    MCP-->>RD: ok or RuntimeError
    RD-->>DD: None
    DD-->>RC: None
    RC-->>User: All models and tools responded successfully

    Note over User,MCP: Existing workload startup (unchanged)
    User->>RC: create config.yaml
    RC->>DD: create(config_builder)
    DD->>RD: run_readiness_check(columns, resource_provider)
    RD->>MR: run_health_check
    RD->>MCP: run_health_check
    RD-->>DD: None
    DD->>DD: proceed with workload

Loading

_{Reviews (1): Last reviewed commit: "Lint fix" | Re-trigger Greptile}