DataCoach

Databricks-native data readiness and EDA tool: deterministic metrics, a 0..100 readiness score, ML/mapping assessment, and report artifacts for Unity Catalog tables. No LLM calls; same input -> same output.

Status

Databricks MVP (not production-ready yet): PySpark engine, Unity Catalog reads, Databricks notebook UI, UC Volume artifacts. Local file support remains for development and tests. The EDA agent MVP is implemented as the deterministic datacoach_report Databricks Workflow task; downstream agents remain in research-driven planning.

Full, canonical status: docs/STATUS.md.

Why

Built at LSports to answer a recurring question on real-time sports data feeds: is this data trustworthy enough to act on? It is intentionally generic — the same pipeline runs on any dataset, from a local CSV to a full Unity Catalog table. See docs/PRODUCT.md.

Quickstart

Standalone (CSV / Parquet, no Databricks)

Docker (recommended — no host Java; start Docker Desktop first):

docker build -t data-coach:standalone .
scripts/data-coach analyze examples/datasets/data_coach_real_sample.csv -o out/real/

Local venv (needs Java 17):

python3 -m venv .venv && source .venv/bin/activate
pip install -e ".[standalone]"
data-coach analyze examples/datasets/data_coach_real_sample.csv -o out/real/

Full workflow (infer → flatten → analyze → export → viz), memory tips, and test script: docs/STANDALONE.md.

Development

python3 -m venv .venv && source .venv/bin/activate   # Python >= 3.10
pip install -e ".[dev,local]"
data-coach --help
pytest -q -m 'not integration'

Full CLI reference (UC tables, flattening, model-ready export, examples) is in docs/CLI.md. Local Docker setup is in docs/LOCAL_DOCKER.md.

Databricks notebook (self-service)

Set a table name, click Run All, and get the report inline — no CLI required. See docs/databricks_notebook_usage.md.

Architecture

load -> analyze (8 analyzers) -> score -> report. The full diagram (including both read paths and the planned agent layer), module responsibilities, and invariants are in docs/ARCHITECTURE.md.

Documentation

• Status & scope: docs/STATUS.md
• Product overview: docs/PRODUCT.md
• Architecture: docs/ARCHITECTURE.md
• CLI reference: docs/CLI.md
• Report schema: docs/REPORT_SCHEMA.md
• Scoring: docs/SCORING.md
• Role config: docs/ROLE_CONFIG.md
• Flatten spec: docs/FLATTEN_SPEC.md
• Model-ready export: docs/MODEL_READY_EXPORT.md
• Execution backends: docs/EXECUTION_BACKENDS.md
• Standalone (CSV/Parquet): docs/STANDALONE.md
• Databricks job: docs/DATABRICKS_JOB.md
• Visual dashboard: docs/VISUAL_DASHBOARD.md
• Agent MVP plan: docs/DATABRICKS_AGENT_MVP.md
• Agent runbook: docs/AGENT_RUNBOOK.md
• Planning workflow: .cursor/rules/planning-workflow.mdc

License

Internal / proprietary — TBD by LSports.