diff --git a/quantecon/.gitignore b/quantecon/.gitignore index de862353f..e7f27f041 100644 --- a/quantecon/.gitignore +++ b/quantecon/.gitignore @@ -6,3 +6,4 @@ !.gitignore !README.md !VERSION.yml +!UPSTREAM-PRS.yml diff --git a/quantecon/README.md b/quantecon/README.md index c721163c8..7aa1b965f 100644 --- a/quantecon/README.md +++ b/quantecon/README.md @@ -1,20 +1,37 @@ # QuantEcon fork of `mystmd` — maintenance guide -This fork lets QuantEcon develop and use new `mystmd` features before they land in `jupyter-book/mystmd`. Features are developed on feature branches, squash-merged into this fork's `main`, and the same feature branches are kept alive so they can be opened as upstream PRs whenever the upstream team is ready to review them. +This fork lets QuantEcon develop and use new `mystmd` features before they land in `jupyter-book/mystmd`. Features are developed on short-lived feature branches off this fork's `main`, squash-merged in, and the feature branch is deleted. Upstream PRs are prepared later by **cherry-picking** one or more squash commits from `main` onto a fresh branch off `upstream/main` — bundling related features into a cohesive upstream story when that makes sense. -> **About this folder.** `quantecon/` doubles as a local scratch space for planning docs, demo books, and experiments. Everything except [`README.md`](README.md) and [`VERSION.yml`](VERSION.yml) is gitignored — feel free to drop PLAN docs, demo `myst.yml` projects, etc. here without worrying about accidental commits. To track something new intentionally, add it to the allow-list in [.gitignore](.gitignore). +> **About this folder.** `quantecon/` doubles as a local scratch space for planning docs, demo books, and experiments. By default everything is gitignored; only the files in [`.gitignore`](.gitignore)'s allow-list are tracked — currently [`README.md`](README.md), [`VERSION.yml`](VERSION.yml), [`UPSTREAM-PRS.yml`](UPSTREAM-PRS.yml), and [`.gitignore`](.gitignore) itself. Feel free to drop PLAN docs, demo `myst.yml` projects, etc. here without worrying about accidental commits. To track something new intentionally, add it to the allow-list. -## Build identifier +## The two tracker files -[`VERSION.yml`](VERSION.yml) records which QuantEcon-specific features are merged into this fork's `main`, identified by a `qe-vN` tag that's also a git tag on the corresponding squash-merge commit. It's a diagnostic/traceability artifact, not a release version — lecture builds can cat the file to log what fork state they're using. +Two tracked YAML files in `quantecon/` record orthogonal facts. Keep them in sync — cross-reference is by squash-commit SHA. -When landing a new feature: +| File | Question it answers | +|---|---| +| [`VERSION.yml`](VERSION.yml) | *What QuantEcon squash commits are in our `main` right now?* Diagnostic / traceability — lecture builds cat this to log the fork state they're using. | +| [`UPSTREAM-PRS.yml`](UPSTREAM-PRS.yml) | *How do we plan to ship those squash commits upstream?* Bundles related squashes into logical upstream PR candidates, records dependency order for cherry-pick, tracks upstream PR / merge status. | + +### Maintaining `VERSION.yml` + +Every time a feature PR lands on `main`, append a row to `merged_features` with its squash `merge_sha`. The `tag` field stays null until the next `qe-v` checkpoint. + +Tags are cut at meaningful checkpoints, **not per-PR** — typically when a batch of features is ready for downstream dogfooding. To cut a tag: + +1. Pick the `main` commit at the head of the batch +2. Tag it: `git tag qe-v -m "qe-v: "` then `git push origin qe-v` +3. Set `tag: qe-v` on each newly-included feature in `merged_features` and bump `qe_version` + +### Maintaining `UPSTREAM-PRS.yml` + +Update this whenever a feature lands on `main` or its upstream plan changes: -1. Squash-merge the feature PR into `main` -2. Tag the resulting commit: `git tag qe-v -m "qe-v: feature/ merged via #"` then `git push origin qe-v` -3. Append the feature to `merged_features` in `VERSION.yml` and bump `qe_version` +- New feature with no obvious bundle → add as a standalone candidate (`status: pending`, `commits: []`). +- Feature extends an existing candidate → append its sha to that candidate's `commits` list (e.g. a follow-up Copilot-fix PR that lands on `main` after the original feature). +- Feature deserves its own upstream story but depends on another → new candidate with `depends_on: []`. -When upstream merges one of our features, update the entry's `upstream` block rather than deleting it — `VERSION.yml` doubles as an upstreaming tracker. +Status transitions: `planned` → `pending` (all commits landed) → `open` (upstream PR exists) → `merged` (upstream merged it). On `merged`, also run the post-merge sync workflow below. ## How it works — the key idea @@ -22,32 +39,39 @@ When upstream merges one of our features, update the entry's `upstream` block ra jupyter-book/mystmd:main ───── (sync periodically via merge) │ ▼ -QuantEcon/mystmd:main ◄────────────────────────────────────┐ - │ │ - │ feature/ (branched from upstream/main) │ - │ │ │ - │ │ PR against QuantEcon/mystmd:main │ - │ ├──── squash-merge ────────────────────────► │ - │ │ │ - │ │ (branch kept alive after merge, - │ │ used later for upstream PR against - │ ▼ jupyter-book/mystmd:main) - │ feature/ (preserved) +QuantEcon/mystmd:main ◄──────────────────────────────────────┐ + │ │ + │ feature/ (branched from origin/main) │ + │ │ │ + │ │ PR against QuantEcon/mystmd:main │ + │ ├──── squash-merge ──────────────────────────► │ + │ │ │ + │ ▼ (branch deleted after merge) + │ (gone — the main-line squash commit is the artifact) + │ + │ …later, when ready to upstream: + │ + │ upstream/ (fresh branch off upstream/main) + │ │ cherry-pick [ …] + │ │ + │ │ PR against jupyter-book/mystmd:main + │ ▼ + │ upstream review & merge ``` -**Feature branches serve two purposes:** - -1. **Local integration.** Each branch is squash-merged into `main` once it's ready, so projects can install from `main` and immediately use the feature. -2. **Upstream PR artifact.** The branch is *not deleted* after squash-merge. When upstream is ready to review, the original branch (with its granular commit history) is pushed and opened as a PR against `jupyter-book/mystmd:main`. +**Why this works:** -This means you get a clean integration `main` for day-to-day use *and* preserved per-commit history for upstream review — without maintaining a separate integration branch. +1. **Local integration.** Each feature is squash-merged into `main` once it's ready, so projects can install from `main` and immediately use the feature. Feature branches are throwaway scaffolding; the squash commit is the canonical artifact. +2. **Upstream PR composition.** When upstream is ready, we cherry-pick one or more squash commits onto a fresh branch from `upstream/main` and open the PR. The cherry-pick lets us bundle related features ("book mode + section scope") as a cohesive upstream story, or split them apart, depending on what the upstream maintainers want to review. +3. **No long-lived branches.** Feature dependencies (PR #28 building on PR #22) just work — branch off `main`, get the prior features for free. No parallel rebases against `upstream/main`. ## Branching model | Branch | Purpose | |---|---| -| `main` | `upstream/main` **plus** all squash-merged features. Synced from upstream periodically via merge (see below). Projects install from here. | -| `feature/` | One branch per logical patch. **Branched from `upstream/main`** (not `main`), kept rebased on `upstream/main`. Opened as a PR against `QuantEcon/mystmd:main` for local merge, then preserved for the eventual upstream PR. | +| `main` | `upstream/main` **plus** all squash-merged QuantEcon features. Synced from upstream periodically via merge (see below). Projects install from here. | +| `feature/` | One short-lived branch per logical patch. **Branched from `origin/main`** (the fork's `main`). Opened as a PR against `QuantEcon/mystmd:main`, squash-merged, then deleted. | +| `upstream/` | Short-lived branch prepared at upstream-PR time. **Branched from `upstream/main`**. One or more squash commits from `main` are cherry-picked onto it, then it's opened as a PR against `jupyter-book/mystmd:main`. Deleted once that PR resolves. | ## One-time setup @@ -66,18 +90,16 @@ upstream https://github.com/jupyter-book/mystmd.git ### Develop a new feature -> **Important:** branch from `upstream/main`, **not** from `main`. The feature branch must stay rebaseable onto `upstream/main` so it remains a clean upstream PR candidate. - ```bash -git fetch upstream -git checkout -b feature/ upstream/main +git fetch origin +git checkout -b feature/ origin/main # make your changes, commit -git push origin feature/ +git push -u origin feature/ ``` Open a PR on GitHub: **base: `QuantEcon/mystmd:main`**, **compare: `QuantEcon/mystmd:feature/`**. -Review locally, address feedback, then **squash-merge** through the GitHub UI. Do **not** delete the branch after merging — it is the upstream PR artifact. +Review locally, address feedback, then **squash-merge** through the GitHub UI. Delete the branch after merging — the squash commit on `main` is the canonical artifact, and the branch is no longer needed. (GitHub offers a "Delete branch" button right after the merge.) ### Sync `main` with upstream @@ -94,41 +116,57 @@ git push origin main > **If `main` is branch-protected and the sync has to go through a PR**, choose **"Create a merge commit"** when merging — *never* "Squash and merge". A real merge commit preserves the ancestry so `git merge upstream/main` works cleanly next time. -### Keep a feature branch current with upstream +### Keep a feature branch current with `main` -If `upstream/main` moves and you need to refresh a still-open feature branch (e.g., to address feedback or prepare for upstreaming): +If `main` moves while a feature PR is in review (e.g. another feature lands first), rebase onto the new `main`: ```bash -git fetch upstream +git fetch origin git checkout feature/ -git rebase upstream/main +git rebase origin/main +# resolve any conflicts git push --force-with-lease origin feature/ ``` -If that feature has already been squash-merged into our `main`, the rebased branch simply replays the same commits onto a newer base — upstream PR readiness is preserved. +This is the only rebase you need during normal development — the cherry-pick model means we never rebase a feature branch onto `upstream/main` itself. -## Opening the upstream PR +## Opening an upstream PR -When the upstream team is ready to review a feature: +When the upstream team is ready to review one or more of our features: -1. Make sure the feature branch is rebased onto current `upstream/main` (see above). -2. Push the branch (likely already pushed). -3. Open a PR on GitHub: **base: `jupyter-book/mystmd:main`**, **compare: `QuantEcon/mystmd:feature/`**. +1. Look up the candidate in [`UPSTREAM-PRS.yml`](UPSTREAM-PRS.yml) — its `commits` block lists the squash SHAs to cherry-pick, in dependency order. (If no candidate exists yet for what you're shipping, add or adjust one first.) +2. Create a fresh branch off `upstream/main`: + ```bash + git fetch upstream + git checkout -b upstream/ upstream/main + ``` +3. Cherry-pick the squash commits in dependency order: + ```bash + git cherry-pick [ …] + # resolve conflicts if upstream has drifted + ``` +4. Push and open the upstream PR: + ```bash + git push -u origin upstream/ + ``` + Open a PR on GitHub: **base: `jupyter-book/mystmd:main`**, **compare: `QuantEcon/mystmd:upstream/`**. + +**Bundling vs. splitting.** Whether to cherry-pick one squash commit or several into the same upstream PR is a per-attempt judgment call: -The PR shows the granular per-commit history, which reviewers prefer. The squash commit on QuantEcon's `main` is *not* what upstream sees — that's a local-integration artifact. +- *Bundle* when the features form one coherent story upstream maintainers will review together (e.g. "book mode + section-scoped numbering" — the second extends the first; reviewing them apart wastes everyone's time). +- *Split* when the features are independent. Two upstream PRs, two cherry-pick branches. + +If the cherry-picked commits should appear as one upstream commit (cleaner review), `git rebase -i upstream/main` to fixup before pushing. ### When upstream merges the feature Once the upstream PR is merged into `jupyter-book/mystmd:main`: -1. Sync our `main` with upstream (instructions above) — upstream's version now lands. -2. Delete the local feature branch: - ```bash - git branch -d feature/ - git push origin --delete feature/ - ``` +1. **Sync our `main` with upstream** (instructions above) — upstream's version of the change now lands in our `main`. +2. **Update [`UPSTREAM-PRS.yml`](UPSTREAM-PRS.yml)** — set the candidate's `status: merged` and fill in `upstream.pr` and `upstream.merged_sha`. +3. **Delete the `upstream/` branch** if it's still around. -The squash commit that lived on our `main` is now redundant with the upstream merge. Git's merge machinery handles this correctly (the changes are already in the tree), so no manual cleanup is needed. +The original squash commit on our `main` is now redundant with the upstream merge. Git's merge machinery handles this correctly (the changes are already in the tree), so no manual cleanup is needed in source files. ## Resolving merge conflicts between features @@ -136,14 +174,12 @@ If two feature branches touch the same lines, squash-merge them in dependency or ```bash git checkout feature/ -git rebase main +git rebase origin/main # resolve conflicts, git add, git rebase --continue git push --force-with-lease origin feature/ ``` -Then continue with the normal PR review and squash-merge. - -> The rebased `feature/` is still upstream-PR-ready — when the time comes to upstream it, rebase it back onto `upstream/main` (which will pull in `feature/` if that has already been upstreamed, or stage the upstream PR after `feature/`'s). +Then continue with the normal PR review and squash-merge. When eventually upstreaming, the cherry-pick order on the `upstream/` branch is the same dependency order. ## Installing the QuantEcon build in GitHub Actions @@ -164,25 +200,24 @@ This is a standard monorepo — clone, build, install globally. There is no publ bun install bun run build npm install -g /tmp/qe-mystmd/packages/mystmd - -- name: Verify - run: myst --version ``` -Pin to a specific commit if you need reproducibility: +Pin to a specific tag if you need reproducibility: ```bash git clone https://github.com/QuantEcon/mystmd.git /tmp/qe-mystmd cd /tmp/qe-mystmd -git checkout +git checkout qe-v # or a specific commit sha bun install && bun run build npm install -g /tmp/qe-mystmd/packages/mystmd ``` -## Active feature branches +## In-flight feature branches + +Feature branches are short-lived: open, review, squash-merge, delete. If any remain on `origin`: ```bash git branch -r | grep '^ origin/feature/' ``` -Each is squash-merged into `main` once ready, and preserved for eventual upstream PR. +…they're either work in progress or stale leftovers that can be deleted. Check the corresponding PR state before deleting. diff --git a/quantecon/UPSTREAM-PRS.yml b/quantecon/UPSTREAM-PRS.yml new file mode 100644 index 000000000..822ba8e3c --- /dev/null +++ b/quantecon/UPSTREAM-PRS.yml @@ -0,0 +1,90 @@ +# QuantEcon mystmd → upstream PR tracker +# +# Each entry is a logical *upstream PR candidate* — a coherent feature +# that we plan to eventually open as a PR against jupyter-book/mystmd:main. +# A candidate may span multiple `main` squash commits (e.g. the original +# feature PR + later follow-up fixes, or several related features we want +# to bundle into one cohesive upstream review). +# +# `commits` is the ordered list of `main` squash SHAs to cherry-pick onto +# a fresh `upstream/` branch off `upstream/main`. Order matters — +# list them in dependency order so cherry-pick replays cleanly. +# +# Relationship to VERSION.yml: each `merge_sha` in VERSION.yml's +# `merged_features` should appear in some candidate's `commits` here. +# VERSION.yml is "what's in our main"; this file is "how we plan to ship +# it upstream." +# +# Status lifecycle: +# planned — candidate identified, commits may not all exist on main yet +# pending — all commits landed on main; upstream PR not yet opened +# open — upstream PR open, under review +# merged — upstream merged it (sync our main, then delete this entry's +# `upstream/` branch if still around) + +upstream_candidates: + + - id: myst-to-ipynb + title: CommonMark ipynb export + image attachment embedding + description: | + Adds ipynb export with CommonMark output and inline image + attachment embedding. Independent of book-mode work. + status: pending + commits: + - sha: a045d57d + local_pr: 16 + title: "feat(ipynb): CommonMark ipynb export + image attachment embedding" + depends_on: [] + upstream: + pr: null + branch: null # set when the upstream/ branch is pushed + merged_sha: null + notes: | + Self-contained; can ship upstream any time. + + - id: book-mode-with-section-scope + title: Book-style numbering (with LaTeX-style section scope) + description: | + Bundles the original book-mode work (#22) with the section-scope + extension (#28). The two form one coherent story upstream — #28 + builds on #22's auto-prefix machinery (`shouldAutoPrefix`, + `AUTO_PREFIX_KINDS`, the `numbering.book` flag) and is a small + targeted extension of it. Shipping them as one upstream PR avoids + the awkward two-step where reviewers see the section-scope hook + points before the machinery exists. + status: pending + commits: + - sha: 032957c2 + local_pr: 22 + title: "Book-style numbering: format, label, section-tagged TOC, auto-prefix" + - sha: 80bb9ecf + local_pr: 28 + title: "feat(book): section-level scope for proof:* / figure / equation auto-prefix" + depends_on: [] + upstream: + pr: null + branch: null + merged_sha: null + notes: | + Could be split into two upstream PRs if maintainers prefer smaller + reviews; default plan is one bundled PR. + + - id: book-parts + title: Book parts dividers + description: | + `section: parts` ParentEntry: emits a Roman-numbered divider folder + ("Part I — Theory") and wraps chapter groups. Chapter counter + continues across part boundaries (LaTeX `book` default). + status: planned # PR #26 still open against our main + commits: [] # populated once #26 squash-merges into main + depends_on: + - book-mode-with-section-scope + upstream: + pr: null + branch: null + merged_sha: null + notes: | + Depends on book-mode being upstream first (or bundled into the same + upstream PR). Could alternatively fold into `book-mode-with-section-scope` + to ship the complete book UX as one upstream PR — decide at upstream + time based on maintainer preference. diff --git a/quantecon/VERSION.yml b/quantecon/VERSION.yml index 72df520e9..18104ba8c 100644 --- a/quantecon/VERSION.yml +++ b/quantecon/VERSION.yml @@ -1,16 +1,28 @@ # QuantEcon mystmd build identifier # -# This file is the canonical diagnostic record of which QuantEcon-specific -# features are in this fork's `main`. The `qe_version` field corresponds to -# a git tag of the same name on the commit that landed the latest feature. +# This file records *which QuantEcon-specific squash commits are in our +# `main`*, identified by `qe-vN` git tags on chosen checkpoint commits. +# It is the diagnostic / traceability artifact — lecture builds can cat +# it to log what fork state they're using. # -# Update this file as part of the "land a feature" workflow: -# 1. Squash-merge the feature PR into main -# 2. Tag the resulting commit with the next `qe-vN` -# 3. Append the feature to `merged_features` and bump `qe_version` +# Upstream status is tracked separately in `UPSTREAM-PRS.yml`. That file +# plans how these squash commits get bundled (or split) into upstream +# PRs, and records the upstream PR / merge status. Cross-reference is +# by commit SHA: each `merge_sha` here appears in some upstream +# candidate's `commits` list there. # -# When upstream merges one of our features, update its `upstream` block -# rather than deleting the entry — the history is useful. +# Per-PR landing appends the feature to `merged_features` with its squash +# `merge_sha` and `tag: null`. The `tag:` field is filled in later when a +# `qe-vN` tag is cut. +# +# Tags are cut at meaningful checkpoints (typically when a batch of +# features is ready for downstream dogfooding), **not per-PR**. When +# cutting a tag: +# 1. Pick the `main` commit at the head of the batch +# 2. Tag it `qe-v` and push the tag +# 3. Set `tag: qe-v` on each previously-untagged entry in +# `merged_features` that is included in the batch, then bump +# `qe_version` to `qe-v` qe_version: qe-v2 upstream_base: 1.9.0 @@ -22,10 +34,6 @@ merged_features: local_pr: 16 merge_sha: a045d57d tag: qe-v1 - upstream: - status: pending # pending | open | merged - pr: null - merged_sha: null - id: 2 name: book-numbering @@ -33,7 +41,10 @@ merged_features: local_pr: 22 merge_sha: 032957c2 tag: qe-v2 - upstream: - status: pending - pr: null - merged_sha: null + + - id: 3 + name: book-proof-scope + description: Section-level scope for proof:* / figure / equation auto-prefix + local_pr: 28 + merge_sha: 80bb9ecf + tag: null # cut at next qe-v checkpoint