Rename CLI from ds-update to fc-data

.github/workflows/publish.yml

@@ -0,0 +1,24 @@
+name: Publish to PyPI
+
+on:
+  release:
+    types: [published]
+
+jobs:
+  publish:
+    runs-on: ubuntu-latest
+    environment: pypi
+    permissions:
+      id-token: write
+    steps:
+      - name: Check out
+        uses: actions/checkout@v4
+
+      - name: Set up the environment
+        uses: ./.github/actions/setup-python-env
+
+      - name: Build package
+        run: uv run python -m build
+
+      - name: Publish to PyPI
+        uses: pypa/gh-action-pypi-publish@release/v1

AGENTS.md

@@ -1,6 +1,6 @@
 # Repository Guidelines
 
-This guide helps contributors work effectively in the datasmith repository.
+This guide helps contributors work effectively in the fc-data repository.
 
 ## Project Structure & Module Organization
 - Source: `src/datasmith/` — current modules: `agents/`, `docker/`, `github/`, `publish/`, `resolution/`, `runners/`, `update/`, `utils/`, plus helper modules like `filters.py` and `preflight.py`.
@@ -17,7 +17,7 @@ This guide helps contributors work effectively in the datasmith repository.
 - `uv run python -m pytest --doctest-modules` — run doctests with the standard test suite.
 - `uvx tox -q` — run the tox matrix (py39–py312) as defined in `tox.ini`.
 - To run commands using the same environment variables as the user, use `uv run <command>`.
-- Pipeline entrypoint: `ds-update --help` and `ds-update --start-date ... --end-date ...`.
+- Pipeline entrypoint: `fc-data --help` and `fc-data --start-date ... --end-date ...`.
 
 ## Coding Style & Naming Conventions
 - Python 3.9–3.12. 4‑space indentation, type hints required (mypy strict; see `pyproject.toml`).

CLAUDE.md

@@ -4,7 +4,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
 
 ## What is this project?
 
-DataSmith is the Python toolchain for building the **FormulaCode** dataset — a benchmark of 67+ repositories with 964+ performance-improving commits, designed to evaluate LLMs' ability to optimize real-world codebases. It scores LLMs relative to the human-authored speedup using ASV (Airspeed Velocity) benchmarks rather than binary pass/fail tests.
+fc-data is the Python toolchain for building the **FormulaCode** dataset — a benchmark of 67+ repositories with 964+ performance-improving commits, designed to evaluate LLMs' ability to optimize real-world codebases. It scores LLMs relative to the human-authored speedup using ASV (Airspeed Velocity) benchmarks rather than binary pass/fail tests.
 
 ## Common commands
 
@@ -24,11 +24,11 @@ uv run pre-commit run -a                     # Run all pre-commit hooks
 # Dataset verification (iterative Docker build debugging)
 python dataset/verify.py --task dataset/formulacode_verified/<owner_repo>/<sha>
 
-# Pipeline update (monthly, the primary entrypoint for DataSmith)
-ds-update --start-date YYYY-MM-DD --end-date YYYY-MM-DD          # Run all 6 stages
-ds-update --start-date 2026-01-01 --end-date 2026-01-31 --stage 4  # Run a single stage
-ds-update --start-date 2026-01-01 --end-date 2026-01-31 --resume   # Resume from last completed
-ds-update --help                                                    # See all options
+# Pipeline update (monthly, the primary entrypoint for fc-data)
+fc-data --start-date YYYY-MM-DD --end-date YYYY-MM-DD          # Run all 6 stages
+fc-data --start-date 2026-01-01 --end-date 2026-01-31 --stage 4  # Run a single stage
+fc-data --start-date 2026-01-01 --end-date 2026-01-31 --resume   # Resume from last completed
+fc-data --help                                                    # See all options
 ```
 
 ## Architecture
@@ -47,7 +47,7 @@ ds-update --help                                                    # See all op
 | `benchmark/` | ASV benchmark collection |
 | `collation/` | Data aggregation |
 
-### Pipeline stages (`ds-update`)
+### Pipeline stages (`fc-data`)
 
 1. **scrape_repos** — Fetch repository metadata from GitHub
 2. **scrape_commits** — Scrape merged PR commits and patches
@@ -73,7 +73,7 @@ Each task lives in `dataset/formulacode_verified/<owner_repo>/<sha>/` with a mul
 
 ## Supabase (local)
 
-DataSmith uses a **local Supabase** instance for all persistent state. Connection details live in `tokens.env`:
+fc-data uses a **local Supabase** instance for all persistent state. Connection details live in `tokens.env`:
 
 - `SUPABASE_URL=http://127.0.0.1:54321` (PostgREST API)
 - `SUPABASE_KEY=sb_secret_...` (service-role key)

docs/getting-started/installation.md

@@ -33,7 +33,7 @@ This creates a virtual environment with `uv`, installs all dependencies, and set
 
 ## 3. Configure `tokens.env`
 
-DataSmith reads all configuration from a `tokens.env` file in the repo root. The `Settings` class (powered by `pydantic-settings`) loads it automatically — no manual `source` or `export` needed.
+fc-data reads all configuration from a `tokens.env` file in the repo root. The `Settings` class (powered by `pydantic-settings`) loads it automatically — no manual `source` or `export` needed.
 
 Create the file:
 
@@ -55,7 +55,7 @@ SUPABASE_KEY=<paste service-role key here>
 
 # === GitHub (required) ===
 # One or more GitHub personal access tokens, comma-separated.
-# DataSmith rotates tokens automatically when one hits the rate limit.
+# fc-data rotates tokens automatically when one hits the rate limit.
 # Create tokens at https://github.com/settings/tokens with `repo` scope.
 GH_TOKENS=github_pat_xxx
 ```
@@ -98,7 +98,7 @@ See [Configuration](../guide/configuration.md) for a complete reference of all e
 
 ## 4. Set up Supabase
 
-DataSmith uses a local Supabase instance for all persistent state (no cloud account needed).
+fc-data uses a local Supabase instance for all persistent state (no cloud account needed).
 
 ### Start the instance
 
@@ -129,7 +129,7 @@ This prints connection details. Copy the **service_role key** (not the anon key)
 
 ### Apply migrations
 
-DataSmith's schema is defined in numbered SQL migrations:
+fc-data's schema is defined in numbered SQL migrations:
 
 ```bash
 npx supabase migration up --local
@@ -187,7 +187,7 @@ make test     # pytest
 You're ready to run the pipeline:
 
 ```bash
-ds-update --start-date 2026-03-01 --end-date 2026-04-01
+fc-data --start-date 2026-03-01 --end-date 2026-04-01
 ```
 
 See the **[Pipeline guide](../guide/pipeline.md)** for the full CLI reference and stage descriptions.

docs/getting-started/quickstart.md

@@ -1,6 +1,6 @@
 # Quickstart
 
-This guide walks through common DataSmith operations with code examples.
+This guide walks through common fc-data operations with code examples.
 
 ## Working with pull requests
 

docs/guide/configuration.md

@@ -1,6 +1,6 @@
 # Configuration
 
-DataSmith is configured primarily through a `tokens.env` file in the repository root. The `Settings` class (powered by `pydantic-settings`) loads these automatically.
+fc-data is configured primarily through a `tokens.env` file in the repository root. The `Settings` class (powered by `pydantic-settings`) loads these automatically.
 
 ## Environment variables
 

docs/guide/docker-images.md

@@ -1,6 +1,6 @@
 # Docker Images
 
-DataSmith uses a three-tier Docker image hierarchy to build reproducible environments for each pull request.
+fc-data uses a three-tier Docker image hierarchy to build reproducible environments for each pull request.
 
 ## Image hierarchy
 

docs/guide/pipeline.md

@@ -1,22 +1,22 @@
-# Pipeline (`ds-update`)
+# Pipeline (`fc-data`)
 
-`ds-update` is the primary command for running DataSmith. It discovers performance-improving commits from GitHub, classifies them with LLM agents, resolves dependencies, synthesizes Docker build contexts, and publishes verified images.
+`fc-data` is the primary command for running fc-data. It discovers performance-improving commits from GitHub, classifies them with LLM agents, resolves dependencies, synthesizes Docker build contexts, and publishes verified images.
 
 ## Quick reference
 
 ```bash
 # Run all 7 stages for a date range
-ds-update --start-date 2026-02-01 --end-date 2026-03-01
+fc-data --start-date 2026-02-01 --end-date 2026-03-01
 
 # Resume from where you left off (skips completed stages)
-ds-update --start-date 2026-02-01 --end-date 2026-03-01 --resume
+fc-data --start-date 2026-02-01 --end-date 2026-03-01 --resume
 
 # Run only specific stages
-ds-update --start-date 2026-02-01 --end-date 2026-03-01 --stage 3
-ds-update --start-date 2026-02-01 --end-date 2026-03-01 --stage 5 --stage 6
+fc-data --start-date 2026-02-01 --end-date 2026-03-01 --stage 3
+fc-data --start-date 2026-02-01 --end-date 2026-03-01 --stage 5 --stage 6
 
 # Preview what would run without executing
-ds-update --start-date 2026-02-01 --end-date 2026-03-01 --dry-run
+fc-data --start-date 2026-02-01 --end-date 2026-03-01 --dry-run
 ```
 
 ## CLI flags
@@ -53,10 +53,10 @@ For each repository found, the runner fetches and stores metadata including desc
 
 ```bash
 # Discover repos with at least 1000 stars
-ds-update --start-date 2026-02-01 --end-date 2026-03-01 --stage 1 --min-stars 1000
+fc-data --start-date 2026-02-01 --end-date 2026-03-01 --stage 1 --min-stars 1000
 
 # Include repos from an offline dataset
-ds-update --start-date 2026-02-01 --end-date 2026-03-01 --stage 1 --offline-source data.parquet
+fc-data --start-date 2026-02-01 --end-date 2026-03-01 --stage 1 --offline-source data.parquet
 ```
 
 **Writes to:** `repositories` table
@@ -76,7 +76,7 @@ For every repository in the `repositories` table, scrapes all merged pull reques
 If `--offline-source` is provided, also bulk-imports PR records from the Parquet file (useful for seeding the database with historical data).
 
 ```bash
-ds-update --start-date 2026-02-01 --end-date 2026-03-01 --stage 2
+fc-data --start-date 2026-02-01 --end-date 2026-03-01 --stage 2
 ```
 
 **Writes to:** `pull_requests` table (one row per PR)
@@ -95,10 +95,10 @@ Only PRs that pass a symbolic pre-filter (`is_performance_commit_symbolic = True
 
 ```bash
 # Classify all unclassified PRs
-ds-update --start-date 2026-02-01 --end-date 2026-03-01 --stage 3
+fc-data --start-date 2026-02-01 --end-date 2026-03-01 --stage 3
 
 # Re-classify all PRs (including already-classified ones)
-ds-update --start-date 2026-02-01 --end-date 2026-03-01 --stage 3 --force
+fc-data --start-date 2026-02-01 --end-date 2026-03-01 --stage 3 --force
 ```
 
 **Updates:** `pull_requests` table (sets `is_performance_commit`, `classification`)
@@ -118,7 +118,7 @@ For each performance-classified PR, checks out the repository at the merge commi
 The resolved dependency set (`env_payload`) and the Python version used are stored in the `packages` table. These are consumed by stage 6 to build `docker_build_env.sh` — the shell script that installs dependencies inside the Docker image.
 
 ```bash
-ds-update --start-date 2026-02-01 --end-date 2026-03-01 --stage 4
+fc-data --start-date 2026-02-01 --end-date 2026-03-01 --stage 4
 ```
 
 **Writes to:** `packages` table (`env_payload`, `python_version`, `can_install`)
@@ -139,13 +139,13 @@ Builds a rich, deconstructed problem context for each PR. This context is what t
 
 ```bash
 # Render problem contexts
-ds-update --start-date 2026-02-01 --end-date 2026-03-01 --stage 5
+fc-data --start-date 2026-02-01 --end-date 2026-03-01 --stage 5
 
 # Limit to 3 PRs per repo (useful for testing)
-ds-update --start-date 2026-02-01 --end-date 2026-03-01 --stage 5 --tasks-per-repo 3
+fc-data --start-date 2026-02-01 --end-date 2026-03-01 --stage 5 --tasks-per-repo 3
 
 # Re-render already-processed PRs
-ds-update --start-date 2026-02-01 --end-date 2026-03-01 --stage 5 --force
+fc-data --start-date 2026-02-01 --end-date 2026-03-01 --stage 5 --force
 ```
 
 **Writes to:** `candidate_prs` table
@@ -169,19 +169,19 @@ Each attempt (success or failure) is logged to the `error_logs` table with the a
 
 ```bash
 # Use the default auto-detected agent
-ds-update --start-date 2026-02-01 --end-date 2026-03-01 --stage 6
+fc-data --start-date 2026-02-01 --end-date 2026-03-01 --stage 6
 
 # Force a specific agent
-ds-update --start-date 2026-02-01 --end-date 2026-03-01 --stage 6 --agent claude
+fc-data --start-date 2026-02-01 --end-date 2026-03-01 --stage 6 --agent claude
 
 # Skip LLM generation entirely — only use cached/similar scripts
-ds-update --start-date 2026-02-01 --end-date 2026-03-01 --stage 6 --agent none
+fc-data --start-date 2026-02-01 --end-date 2026-03-01 --stage 6 --agent none
 
 # Limit concurrency and tasks per repo (controls cost)
-ds-update --start-date 2026-02-01 --end-date 2026-03-01 --stage 6 --n-concurrent 2 --tasks-per-repo 5
+fc-data --start-date 2026-02-01 --end-date 2026-03-01 --stage 6 --n-concurrent 2 --tasks-per-repo 5
 
 # Re-synthesize already-completed PRs
-ds-update --start-date 2026-02-01 --end-date 2026-03-01 --stage 6 --force
+fc-data --start-date 2026-02-01 --end-date 2026-03-01 --stage 6 --force
 ```
 
 **Writes to:** `candidate_containers` table (on success), `error_logs` table (every attempt)
@@ -207,7 +207,7 @@ The publish pipeline:
 6. **Mark published** — Sets `published_at` timestamp on each published PR row
 
 ```bash
-ds-update --start-date 2026-02-01 --end-date 2026-03-01 --stage 7
+fc-data --start-date 2026-02-01 --end-date 2026-03-01 --stage 7
 ```
 
 **Reads from:** `pull_requests`, `packages`, `candidate_containers`
@@ -221,13 +221,13 @@ A monthly update usually looks like this:
 
 ```bash
 # 1. Run the full pipeline
-ds-update --start-date 2026-03-01 --end-date 2026-04-01
+fc-data --start-date 2026-03-01 --end-date 2026-04-01
 
 # 2. If it gets interrupted, resume where it left off
-ds-update --start-date 2026-03-01 --end-date 2026-04-01 --resume
+fc-data --start-date 2026-03-01 --end-date 2026-04-01 --resume
 
 # 3. After fixing a synthesis issue, re-run just stages 6-7
-ds-update --start-date 2026-03-01 --end-date 2026-04-01 --stage 6 --stage 7 --force
+fc-data --start-date 2026-03-01 --end-date 2026-04-01 --stage 6 --stage 7 --force
 ```
 
 ## Monitoring progress

docs/guide/publishing.md

@@ -1,6 +1,6 @@
 # Publishing
 
-DataSmith publishes verified tasks to DockerHub (Docker images) and HuggingFace (Parquet datasets).
+fc-data publishes verified tasks to DockerHub (Docker images) and HuggingFace (Parquet datasets).
 
 ## Publishing workflow
 
@@ -49,7 +49,7 @@ rows = sb.table("pull_requests") \
 
 ## Pipeline integration
 
-Publishing is the final stage of the `ds-update` pipeline (stage 7). The `publish_pipeline()` function orchestrates:
+Publishing is the final stage of the `fc-data` pipeline (stage 7). The `publish_pipeline()` function orchestrates:
 
 1. Query DB for unpublished, verified PRs
 2. Push Docker images to DockerHub

docs/index.md

@@ -1,11 +1,11 @@
-# 🔧 DataSmith
+# 🔧 fc-data
 
 **Python toolchain for building the [FormulaCode](https://github.com/formula-code) benchmark.**
 
 [![Build status](https://img.shields.io/github/actions/workflow/status/formula-code/datasmith/main.yml?branch=main)](https://github.com/formula-code/datasmith/actions/workflows/main.yml?query=branch%3Amain)
 [![License](https://img.shields.io/github/license/formula-code/datasmith)](https://img.shields.io/github/license/formula-code/datasmith)
 
-DataSmith is a package for automatically building and maintaining FormulaCode tasks. It is engineered to support any repository-level, verification-by-execution based coding benchmark that heavily uses Docker and GitHub.
+fc-data is a package for automatically building and maintaining FormulaCode tasks. It is engineered to support any repository-level, verification-by-execution based coding benchmark that heavily uses Docker and GitHub.
 
 FormulaCode is a benchmark of **67+ repositories** with **964+ performance-improving commits**, designed to evaluate LLMs' ability to optimize real-world codebases. It scores optimizers relative to the human-authored speedup using ASV (Airspeed Velocity) benchmarks — providing a dense performance signal instead of binary pass/fail.
 
@@ -20,24 +20,24 @@ graph LR
 
     A[GitHub]
     A2[Supabase]
-    B["DataSmith<br/>(This repository)"]
+    B["fc-data<br/>(This repository)"]
     C[DockerHub]
     D[HuggingFace]
 ```
 
 ## Get started
 
-Most interaction with DataSmith is through a single command:
+Most interaction with fc-data is through a single command:
 
 ```bash
-ds-update --start-date 2026-03-01 --end-date 2026-04-01
+fc-data --start-date 2026-03-01 --end-date 2026-04-01
 ```
 
 This runs all 7 pipeline stages: repo discovery, PR scraping, LLM classification, dependency resolution, problem rendering, Docker synthesis, and publishing. See the **[Pipeline guide](guide/pipeline.md)** for the full CLI reference.
 
 ## Key features
 
-- **Single-command pipeline** — `ds-update` runs all stages with `--resume`, `--stage`, and `--dry-run` support
+- **Single-command pipeline** — `fc-data` runs all stages with `--resume`, `--stage`, and `--dry-run` support
 - **GitHub scraping** — Async `httpx` client with automatic token rotation across multiple `GH_TOKENS`
 - **LLM classification** — DSPy-based agents classify PRs by performance category and difficulty
 - **Docker synthesis** — Automatically generate Docker build contexts using coding agents (Claude, Codex, Gemini)
@@ -47,6 +47,6 @@ This runs all 7 pipeline stages: repo discovery, PR scraping, LLM classification
 ## Quick links
 
 - [Installation](getting-started/installation.md) — Set up your development environment
-- [Pipeline guide (`ds-update`)](guide/pipeline.md) — **The primary entrypoint** — full CLI reference and stage descriptions
+- [Pipeline guide (`fc-data`)](guide/pipeline.md) — **The primary entrypoint** — full CLI reference and stage descriptions
 - [Quickstart](getting-started/quickstart.md) — Python API examples
 - [Configuration](guide/configuration.md) — `tokens.env` and environment variables

docs/stage6-failure-analysis.md

@@ -1,7 +1,7 @@
 # Stage 6 (synthesize_images) Failure Analysis
 
 **Date**: 2026-04-04
-**Pipeline invocation**: `ds-update --start-date 2017-01-01 --end-date 2026-03-01 --stage 6 --n-concurrent 4 --tasks-per-repo 1`
+**Pipeline invocation**: `fc-data --start-date 2017-01-01 --end-date 2026-03-01 --stage 6 --n-concurrent 4 --tasks-per-repo 1`
 
 ## High-level numbers
 

mkdocs.yml

@@ -1,4 +1,4 @@
-site_name: 🔧 DataSmith
+site_name: 🔧 fc-data
 site_url: https://formula-code.github.io/datasmith/
 site_description: Python toolchain for building the FormulaCode benchmark
 repo_url: https://github.com/formula-code/datasmith
@@ -81,7 +81,7 @@ nav:
   - Home: index.md
   - Getting Started:
       - Installation: getting-started/installation.md
-      - Pipeline (ds-update): guide/pipeline.md
+      - Pipeline (fc-data): guide/pipeline.md
       - Configuration: guide/configuration.md
   - User Guide:
       - Docker Images: guide/docker-images.md