From 37136163eef753124c31fa2f2ad0910aa2bda028 Mon Sep 17 00:00:00 2001 From: Matt McKay Date: Thu, 7 May 2026 08:48:55 +1000 Subject: [PATCH 1/7] chore: add quantecon/ fork maintenance tooling MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Add quantecon/build.sh — rebuilds the `quantecon` integration branch by merging all active feature branches on top of `main`, then patches the version string with a `-qe` suffix so `myst --version` identifies the QuantEcon build. - Add quantecon/features.txt — registry of active carry-patches. - Add quantecon/README.md — documents the branching model, sync workflow, and conflict resolution strategy. - Add warning banner to README.md identifying this as the QuantEcon fork. --- README.md | 4 + quantecon/README.md | 115 +++++++++++++++++++++++++ quantecon/build.sh | 185 +++++++++++++++++++++++++++++++++++++++++ quantecon/features.txt | 12 +++ 4 files changed, 316 insertions(+) create mode 100644 quantecon/README.md create mode 100644 quantecon/build.sh create mode 100644 quantecon/features.txt diff --git a/README.md b/README.md index 2ab67674a4..93e582a270 100644 --- a/README.md +++ b/README.md @@ -1,3 +1,7 @@ +> [!WARNING] +> **This is the [QuantEcon](https://quantecon.org) fork of `mystmd`**, maintained for testing and development of features pending upstream review. +> It is **not** intended for general use. For the official project, see [jupyter-book/mystmd](https://github.com/jupyter-book/mystmd). + # MyST Markdown Command Line Interface, `mystmd` [![MIT License](https://img.shields.io/badge/license-MIT-blue.svg)](https://github.com/jupyter-book/mystmd/blob/main/LICENSE) diff --git a/quantecon/README.md b/quantecon/README.md new file mode 100644 index 0000000000..a6dca931df --- /dev/null +++ b/quantecon/README.md @@ -0,0 +1,115 @@ +# QuantEcon fork of `mystmd` — maintenance guide + +This fork exists to carry QuantEcon-specific patches while upstream review is in progress. The goal is to upstream every patch once the `jupyter-book/mystmd` team has capacity to review, then retire those patches from this fork. + +## Branching model + +| Branch | Purpose | +|---|---| +| `main` | Mirrors `jupyter-book/mystmd:main` exactly. Fast-forward only — **no direct commits**. | +| `feature/` | One branch per logical patch. Branched from `main`, kept rebased on `main`, and used as the source for upstream PRs. | +| `quantecon` | Throwaway integration branch. Rebuilt by `build.sh` as `main` + all active feature branches. **Never commit here directly.** | + +## One-time setup + +### 1. Add the upstream remote + +```bash +git remote add upstream https://github.com/jupyter-book/mystmd.git +``` + +### 2. Disable release workflows on the fork + +The upstream `release.yml` and `publish-github-release.yml` workflows will trigger on pushes to `main`. To prevent accidental npm/PyPI publishes, disable them via the GitHub UI: + +> **GitHub → QuantEcon/mystmd → Actions → (select workflow) → Disable workflow** + +Do this for `Release` and `Publish GitHub Release`. + +## Regular workflow + +### Sync `main` with upstream + +```bash +git fetch upstream +git checkout main +git merge --ff-only upstream/main +git push origin main +``` + +Then rebuild the integration branch — see below. + +### Add a new feature branch + +```bash +git checkout main +git checkout -b feature/ +# ... make changes, commit ... +git push origin feature/ +``` + +Add the branch name to `quantecon/features.txt`, then rebuild. + +### Rebuild the `quantecon` branch + +```bash +./quantecon/build.sh # dry run (local only) +./quantecon/build.sh --push # build and push to origin +``` + +The script: +1. Fast-forwards local `main` from `origin/main`. +2. Resets `quantecon` to `main`. +3. Merges each branch in `features.txt` in order (merge commits, not squash). +4. Patches `packages/mystmd/src/version.ts` to append `-qe` to the version string, so `myst --version` identifies the QuantEcon build. +5. Optionally pushes to `origin/quantecon`. + +### Rebase a feature branch after upstream moves + +```bash +git checkout feature/ +git rebase main +git push --force-with-lease origin feature/ +./quantecon/build.sh --push +``` + +## Resolving merge conflicts + +Conflicts are **always fixed on the feature branch**, never on the `quantecon` branch (which is throwaway). + +**Feature vs. `main`** (upstream changed code the feature touches): +```bash +git checkout feature/ +git rebase main # resolve conflicts here +git push --force-with-lease origin feature/ +./quantecon/build.sh --push +``` + +**Feature A vs. Feature B** (two patches touch the same lines): +```bash +# Rebase the later branch on top of the earlier one +git checkout feature/ +git rebase feature/ +git push --force-with-lease origin feature/ +# Ensure features.txt lists feature/ before feature/ +./quantecon/build.sh --push +``` + +## When upstream merges a patch + +1. Sync `main` with upstream (it now contains the patch). +2. Remove the branch from `features.txt`. +3. Delete the feature branch locally and on origin. +4. Rebuild the `quantecon` branch. + +```bash +git fetch upstream +git checkout main && git merge --ff-only upstream/main && git push origin main +# edit features.txt to remove the merged branch +git branch -d feature/ && git push origin --delete feature/ +./quantecon/build.sh --push +``` + +## Active patches + +See [features.txt](features.txt) for the current list of carry-patches and links to their upstream PRs. diff --git a/quantecon/build.sh b/quantecon/build.sh new file mode 100644 index 0000000000..0a16cd73bd --- /dev/null +++ b/quantecon/build.sh @@ -0,0 +1,185 @@ +#!/usr/bin/env bash +# quantecon/build.sh +# +# Rebuilds the `quantecon` integration branch by merging all active feature +# branches (listed in quantecon/features.txt) on top of current `main`. +# +# Usage: +# ./quantecon/build.sh [--push] +# +# Options: +# --push Push the resulting branch to origin after a successful build. +# +# Conflict resolution: +# This script aborts cleanly on any merge conflict. Conflicts must be +# resolved on the feature branch itself (never on `quantecon`): +# +# Feature vs. main: +# git checkout && git rebase main +# # resolve conflicts, git add, git rebase --continue +# git push --force-with-lease origin +# ./quantecon/build.sh [--push] +# +# Feature A vs. feature B: +# git checkout && git rebase +# # resolve conflicts, update features.txt so branch-A appears before branch-B +# git push --force-with-lease origin +# ./quantecon/build.sh [--push] + +set -euo pipefail + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +FEATURES_FILE="$SCRIPT_DIR/features.txt" +INTEGRATION_BRANCH="quantecon" +BASE_BRANCH="main" +PUSH=false + +# --------------------------------------------------------------------------- +# Parse args +# --------------------------------------------------------------------------- +for arg in "$@"; do + case "$arg" in + --push) PUSH=true ;; + *) echo "Unknown argument: $arg" >&2; exit 1 ;; + esac +done + +# --------------------------------------------------------------------------- +# Preflight checks +# --------------------------------------------------------------------------- +if ! git rev-parse --git-dir > /dev/null 2>&1; then + echo "ERROR: not inside a git repository." >&2 + exit 1 +fi + +if [[ -n "$(git status --porcelain)" ]]; then + echo "ERROR: working tree is dirty. Commit or stash changes before running." >&2 + exit 1 +fi + +# Read feature branches (strip comments and blank lines) +mapfile -t FEATURES < <(grep -v '^\s*#' "$FEATURES_FILE" | grep -v '^\s*$' | awk '{print $1}') + +if [[ ${#FEATURES[@]} -eq 0 ]]; then + echo "No feature branches listed in $FEATURES_FILE. Nothing to do." + exit 0 +fi + +echo "==> Feature branches to merge:" +for f in "${FEATURES[@]}"; do + echo " $f" +done + +# --------------------------------------------------------------------------- +# Ensure local main is up to date +# --------------------------------------------------------------------------- +echo "" +echo "==> Switching to $BASE_BRANCH and fast-forwarding..." +git checkout "$BASE_BRANCH" +git merge --ff-only origin/"$BASE_BRANCH" 2>/dev/null || { + echo "WARN: could not fast-forward $BASE_BRANCH from origin/$BASE_BRANCH." \ + "Proceeding with local state." +} + +BASE_SHA=$(git rev-parse "$BASE_BRANCH") +echo " Base commit: $BASE_SHA" + +# --------------------------------------------------------------------------- +# Verify all feature branches exist (locally or on origin) +# --------------------------------------------------------------------------- +echo "" +echo "==> Verifying feature branches exist..." +for branch in "${FEATURES[@]}"; do + if git rev-parse --verify "$branch" > /dev/null 2>&1; then + echo " $branch (local)" + elif git rev-parse --verify "origin/$branch" > /dev/null 2>&1; then + echo " $branch (origin)" + else + echo "ERROR: branch '$branch' not found locally or on origin." >&2 + exit 1 + fi +done + +# --------------------------------------------------------------------------- +# Rebuild the integration branch +# --------------------------------------------------------------------------- +echo "" +echo "==> Resetting '$INTEGRATION_BRANCH' to '$BASE_BRANCH'..." +git checkout -B "$INTEGRATION_BRANCH" "$BASE_BRANCH" + +MERGED=() +for branch in "${FEATURES[@]}"; do + echo "" + echo "==> Merging $branch..." + if git merge --no-ff "$branch" -m "chore: merge $branch into $INTEGRATION_BRANCH"; then + MERGED+=("$branch") + else + echo "" >&2 + echo "ERROR: merge conflict when merging '$branch'." >&2 + echo "" >&2 + echo "Conflicting files:" >&2 + git diff --name-only --diff-filter=U >&2 + echo "" >&2 + echo "Aborting. The '$INTEGRATION_BRANCH' branch has been reset; no changes were pushed." >&2 + git merge --abort + git checkout "$BASE_BRANCH" + git branch -D "$INTEGRATION_BRANCH" + echo "" >&2 + echo "How to fix:" >&2 + if [[ ${#MERGED[@]} -gt 0 ]]; then + echo " '$branch' conflicts with one of: ${MERGED[*]}" >&2 + echo " Rebase '$branch' on top of the conflicting branch:" >&2 + echo " git checkout $branch && git rebase " >&2 + echo " Then update features.txt so the earlier branch appears first." >&2 + else + echo " '$branch' conflicts with $BASE_BRANCH." >&2 + echo " Rebase it: git checkout $branch && git rebase $BASE_BRANCH" >&2 + fi + echo " Resolve conflicts, push the feature branch, then re-run this script." >&2 + exit 1 + fi +done + +# --------------------------------------------------------------------------- +# Patch version string with -qe suffix +# --------------------------------------------------------------------------- +VERSION_FILE="packages/mystmd/src/version.ts" +echo "" +echo "==> Patching version in $VERSION_FILE with '-qe' suffix..." + +CURRENT_VERSION=$(node -p "require('./packages/mystmd/package.json').version") +QE_VERSION="${CURRENT_VERSION}-qe" + +# Replace the version string in version.ts (matches: const version = 'X.Y.Z';) +sed -i.bak "s/const version = '[^']*'/const version = '${QE_VERSION}'/" "$VERSION_FILE" +rm -f "${VERSION_FILE}.bak" + +git add "$VERSION_FILE" +git commit -m "chore: set version to ${QE_VERSION} for QuantEcon build" +echo " Version set to ${QE_VERSION}" + +# --------------------------------------------------------------------------- +# Summary +# --------------------------------------------------------------------------- +echo "" +echo "==> Build successful." +echo " Branch '$INTEGRATION_BRANCH' contains $BASE_BRANCH + ${#MERGED[@]} feature branch(es):" +for f in "${MERGED[@]}"; do + echo " $f" +done +echo " Version: ${QE_VERSION}" + +# --------------------------------------------------------------------------- +# Optionally push +# --------------------------------------------------------------------------- +if $PUSH; then + echo "" + echo "==> Pushing '$INTEGRATION_BRANCH' to origin (force-with-lease)..." + git push --force-with-lease origin "$INTEGRATION_BRANCH" + echo " Done." +else + echo "" + echo "Tip: run with --push to push to origin/$INTEGRATION_BRANCH." +fi + +git checkout "$BASE_BRANCH" diff --git a/quantecon/features.txt b/quantecon/features.txt new file mode 100644 index 0000000000..31e051c4e5 --- /dev/null +++ b/quantecon/features.txt @@ -0,0 +1,12 @@ +# Active QuantEcon feature branches +# One branch name per line. Lines starting with # are ignored. +# Order matters: branches are merged in sequence. +# If two branches conflict with each other, rebase the later one on top +# of the earlier one and update both entries accordingly. +# +# Format: +# [# optional description / upstream PR link] +# +# Example: +# feature/xref-improvements # https://github.com/jupyter-book/mystmd/pull/1 + From d86e0809b53231d4b8193caa6606df4eea2a4df0 Mon Sep 17 00:00:00 2001 From: Matt McKay Date: Thu, 7 May 2026 08:56:11 +1000 Subject: [PATCH 2/7] chore: disable release and docs workflows on QuantEcon fork --- .github/workflows/docs.yml | 3 ++- .github/workflows/publish-github-release.yml | 8 ++------ .github/workflows/release.yml | 13 ++++--------- 3 files changed, 8 insertions(+), 16 deletions(-) diff --git a/.github/workflows/docs.yml b/.github/workflows/docs.yml index e4954478b3..d0e9f05ad8 100644 --- a/.github/workflows/docs.yml +++ b/.github/workflows/docs.yml @@ -1,6 +1,7 @@ name: Deploy Docs +# QuantEcon fork: manual dispatch disabled. Only triggers from release.yml +# which is itself disabled on this fork. on: - workflow_dispatch: workflow_run: workflows: ["Release"] types: diff --git a/.github/workflows/publish-github-release.yml b/.github/workflows/publish-github-release.yml index ef53d49eb6..ebd6d74e54 100644 --- a/.github/workflows/publish-github-release.yml +++ b/.github/workflows/publish-github-release.yml @@ -1,17 +1,13 @@ name: Publish GitHub Release +# QuantEcon fork: manual dispatch disabled. Only callable from release.yml +# which is itself disabled on this fork. on: workflow_call: inputs: tag_name: required: true type: string - workflow_dispatch: - inputs: - tag_name: - description: 'Tag name for the release (e.g. v1.2.3)' - required: true - type: string jobs: publish-release: diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index aa36100f01..6e36e2c189 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -1,17 +1,12 @@ name: Release +# QuantEcon fork: auto-release disabled. This workflow is intentionally +# restricted to a branch that does not exist on this fork. Sync with +# upstream (jupyter-book/mystmd) will restore the original trigger on main. on: push: branches: - - main - workflow_dispatch: - inputs: - forcePublishPyPI: - description: | - Force publish to PyPI, even if the NPM release has already occurred. - type: boolean - default: false - required: false + - _upstream-release-only concurrency: ${{ github.workflow }}-${{ github.ref }} permissions: contents: write From cd85f460420217ed2790ac68256d82a26eec30ae Mon Sep 17 00:00:00 2001 From: Matt McKay Date: Thu, 7 May 2026 09:10:21 +1000 Subject: [PATCH 3/7] chore: update sync docs to use git merge (not --ff-only) for fork main --- quantecon/README.md | 6 ++++-- quantecon/build.sh | 9 ++++++--- 2 files changed, 10 insertions(+), 5 deletions(-) diff --git a/quantecon/README.md b/quantecon/README.md index a6dca931df..a9c7604e40 100644 --- a/quantecon/README.md +++ b/quantecon/README.md @@ -33,10 +33,12 @@ Do this for `Release` and `Publish GitHub Release`. ```bash git fetch upstream git checkout main -git merge --ff-only upstream/main +git merge upstream/main # not --ff-only: main has fork-specific commits git push origin main ``` +> **Conflict notes:** `main` carries fork-specific files (`quantecon/`, README banner, disabled workflow `on:` blocks) that upstream will never have. Conflicts are rare but can occur if upstream edits the workflow files. When they do, keep your version of the `on:` block and take upstream's version of the rest. + Then rebuild the integration branch — see below. ### Add a new feature branch @@ -104,7 +106,7 @@ git push --force-with-lease origin feature/ ```bash git fetch upstream -git checkout main && git merge --ff-only upstream/main && git push origin main +git checkout main && git merge upstream/main && git push origin main # edit features.txt to remove the merged branch git branch -d feature/ && git push origin --delete feature/ ./quantecon/build.sh --push diff --git a/quantecon/build.sh b/quantecon/build.sh index 0a16cd73bd..8e5983a054 100644 --- a/quantecon/build.sh +++ b/quantecon/build.sh @@ -74,10 +74,13 @@ done # Ensure local main is up to date # --------------------------------------------------------------------------- echo "" -echo "==> Switching to $BASE_BRANCH and fast-forwarding..." +echo "==> Switching to $BASE_BRANCH and syncing with origin..." git checkout "$BASE_BRANCH" -git merge --ff-only origin/"$BASE_BRANCH" 2>/dev/null || { - echo "WARN: could not fast-forward $BASE_BRANCH from origin/$BASE_BRANCH." \ +# NOTE: main has fork-specific commits (quantecon/ folder, workflow overrides) +# so --ff-only is not used. Use 'git merge upstream/main' manually to sync +# with upstream; resolve conflicts in the on: blocks of disabled workflows. +git merge --no-edit origin/"$BASE_BRANCH" 2>/dev/null || { + echo "WARN: could not merge $BASE_BRANCH from origin/$BASE_BRANCH." \ "Proceeding with local state." } From 7782156b69b5b6d3dc9be1b92249c494da6b75c4 Mon Sep 17 00:00:00 2001 From: Matt McKay Date: Thu, 7 May 2026 09:15:43 +1000 Subject: [PATCH 4/7] docs: rewrite quantecon/README with clearer workflow explanation --- quantecon/README.md | 118 +++++++++++++++++++++++++++++--------------- quantecon/build.sh | 11 ++--- 2 files changed, 84 insertions(+), 45 deletions(-) diff --git a/quantecon/README.md b/quantecon/README.md index a9c7604e40..605771bed8 100644 --- a/quantecon/README.md +++ b/quantecon/README.md @@ -1,72 +1,108 @@ # QuantEcon fork of `mystmd` — maintenance guide -This fork exists to carry QuantEcon-specific patches while upstream review is in progress. The goal is to upstream every patch once the `jupyter-book/mystmd` team has capacity to review, then retire those patches from this fork. +This fork lets QuantEcon develop and use new `mystmd` features before they have been reviewed and merged by the upstream `jupyter-book/mystmd` team. The goal is to make it as easy as possible to open upstream PRs from feature branches, while also being able to run a combined build that includes all in-progress features. + +## How it works — the key idea + +``` +jupyter-book/mystmd:main + │ (sync periodically) + ▼ +QuantEcon/mystmd:main ← pure mirror, never commit here directly + │ + ├── feature/foo ← your work; PR open against main (upstream) + ├── feature/bar ← another patch; PR open against main (upstream) + │ + ▼ (built by build.sh) +QuantEcon/mystmd:quantecon ← combined build: main + all feature branches +``` + +**Feature branches serve two purposes simultaneously:** + +1. They are the source of upstream PRs — targeting `main`, showing only the diff for that one feature. GitHub doesn't know or care that `build.sh` also merges them elsewhere. +2. They are consumed by `build.sh`, which merges all of them onto a throwaway `quantecon` branch that QuantEcon actually builds from. + +This means **you never need to choose** between "make it easy to upstream" and "make it available to use now". The feature branch does both. When you update a feature branch (rebase on latest `main`, address reviewer feedback), the upstream PR updates automatically and re-running `build.sh` gives you an updated `quantecon` build. ## Branching model | Branch | Purpose | |---|---| -| `main` | Mirrors `jupyter-book/mystmd:main` exactly. Fast-forward only — **no direct commits**. | -| `feature/` | One branch per logical patch. Branched from `main`, kept rebased on `main`, and used as the source for upstream PRs. | -| `quantecon` | Throwaway integration branch. Rebuilt by `build.sh` as `main` + all active feature branches. **Never commit here directly.** | +| `main` | Mirrors `jupyter-book/mystmd:main` exactly. Synced via the GitHub "Sync fork" button or `git merge upstream/main`. **No direct commits.** | +| `feature/` | One branch per logical patch. Branched from `main`, kept rebased on `main`. This is the branch you open the upstream PR from. | +| `quantecon` | Throwaway combined build. Rebuilt by `build.sh` as `main` + all active feature branches. **Never commit here directly — it is always discarded and regenerated.** | ## One-time setup -### 1. Add the upstream remote - ```bash git remote add upstream https://github.com/jupyter-book/mystmd.git ``` -### 2. Disable release workflows on the fork - -The upstream `release.yml` and `publish-github-release.yml` workflows will trigger on pushes to `main`. To prevent accidental npm/PyPI publishes, disable them via the GitHub UI: +Verify your remotes look like this: -> **GitHub → QuantEcon/mystmd → Actions → (select workflow) → Disable workflow** - -Do this for `Release` and `Publish GitHub Release`. +``` +origin https://github.com/QuantEcon/mystmd.git (fetch/push) +upstream https://github.com/jupyter-book/mystmd.git (fetch/push) +``` ## Regular workflow ### Sync `main` with upstream +Either use the **"Sync fork"** button on the GitHub web UI (simplest), or locally: + ```bash git fetch upstream git checkout main -git merge upstream/main # not --ff-only: main has fork-specific commits +git merge upstream/main git push origin main ``` -> **Conflict notes:** `main` carries fork-specific files (`quantecon/`, README banner, disabled workflow `on:` blocks) that upstream will never have. Conflicts are rare but can occur if upstream edits the workflow files. When they do, keep your version of the `on:` block and take upstream's version of the rest. - -Then rebuild the integration branch — see below. +After syncing, rebuild the `quantecon` branch (see below) so it includes the latest upstream changes. -### Add a new feature branch +### Develop a new feature ```bash git checkout main git checkout -b feature/ -# ... make changes, commit ... +# make your changes and commit them git push origin feature/ ``` -Add the branch name to `quantecon/features.txt`, then rebuild. +Then: +1. Add the branch name to `quantecon/features.txt` +2. Open a PR on GitHub: **base: `jupyter-book/mystmd:main`**, **compare: `QuantEcon/mystmd:feature/`** +3. Rebuild the `quantecon` branch so the feature is available immediately ### Rebuild the `quantecon` branch ```bash -./quantecon/build.sh # dry run (local only) -./quantecon/build.sh --push # build and push to origin +./quantecon/build.sh # local only (safe, no push) +./quantecon/build.sh --push # build and push to origin/quantecon ``` The script: -1. Fast-forwards local `main` from `origin/main`. -2. Resets `quantecon` to `main`. -3. Merges each branch in `features.txt` in order (merge commits, not squash). -4. Patches `packages/mystmd/src/version.ts` to append `-qe` to the version string, so `myst --version` identifies the QuantEcon build. -5. Optionally pushes to `origin/quantecon`. +1. Syncs local `main` from `origin/main` +2. Resets `quantecon` to `main` (discarding any previous build) +3. Merges each branch listed in `features.txt` in order (merge commits, not squash) +4. Patches `packages/mystmd/src/version.ts` to append `-qe` to the version (e.g. `1.9.0-qe`), so `myst --version` identifies the QuantEcon build +5. Optionally pushes to `origin/quantecon` + +Run this after every upstream sync, after updating any feature branch, or after adding/removing a feature branch from `features.txt`. + +### Update a feature branch (e.g. to address upstream reviewer feedback) + +```bash +git checkout feature/ +# make changes, amend or add commits +git push --force-with-lease origin feature/ +# the upstream PR updates automatically +./quantecon/build.sh --push # rebuild quantecon with the updated branch +``` -### Rebase a feature branch after upstream moves +### Keep a feature branch current with upstream + +If `main` has moved since you branched: ```bash git checkout feature/ @@ -77,37 +113,41 @@ git push --force-with-lease origin feature/ ## Resolving merge conflicts -Conflicts are **always fixed on the feature branch**, never on the `quantecon` branch (which is throwaway). +Conflicts are **always fixed on the feature branch**, never on `quantecon` (which is throwaway and rebuilt from scratch each time). + +`build.sh` will abort cleanly and tell you which branch caused the conflict. -**Feature vs. `main`** (upstream changed code the feature touches): +**Feature conflicts with `main`** (upstream changed code the feature touches): ```bash git checkout feature/ -git rebase main # resolve conflicts here +git rebase main # resolve conflicts here, then: git add . && git rebase --continue git push --force-with-lease origin feature/ ./quantecon/build.sh --push ``` -**Feature A vs. Feature B** (two patches touch the same lines): +**Feature A conflicts with feature B** (two patches touch the same lines): ```bash -# Rebase the later branch on top of the earlier one +# Rebase the later branch on top of the earlier one to establish ordering git checkout feature/ git rebase feature/ git push --force-with-lease origin feature/ -# Ensure features.txt lists feature/ before feature/ +# Make sure features.txt lists feature/ before feature/ ./quantecon/build.sh --push ``` -## When upstream merges a patch +## When upstream merges a feature + +Once the upstream PR is merged into `jupyter-book/mystmd:main`: -1. Sync `main` with upstream (it now contains the patch). -2. Remove the branch from `features.txt`. -3. Delete the feature branch locally and on origin. -4. Rebuild the `quantecon` branch. +1. Sync `main` with upstream — it now contains the feature +2. Remove the branch from `features.txt` +3. Delete the feature branch +4. Rebuild `quantecon` — the feature is now in `main` itself, so nothing is lost ```bash git fetch upstream git checkout main && git merge upstream/main && git push origin main -# edit features.txt to remove the merged branch +# edit features.txt to remove the branch git branch -d feature/ && git push origin --delete feature/ ./quantecon/build.sh --push ``` diff --git a/quantecon/build.sh b/quantecon/build.sh index 8e5983a054..b0ea4cb31e 100644 --- a/quantecon/build.sh +++ b/quantecon/build.sh @@ -74,15 +74,14 @@ done # Ensure local main is up to date # --------------------------------------------------------------------------- echo "" -echo "==> Switching to $BASE_BRANCH and syncing with origin..." +echo "==> Switching to $BASE_BRANCH and fast-forwarding from origin..." git checkout "$BASE_BRANCH" -# NOTE: main has fork-specific commits (quantecon/ folder, workflow overrides) -# so --ff-only is not used. Use 'git merge upstream/main' manually to sync -# with upstream; resolve conflicts in the on: blocks of disabled workflows. -git merge --no-edit origin/"$BASE_BRANCH" 2>/dev/null || { - echo "WARN: could not merge $BASE_BRANCH from origin/$BASE_BRANCH." \ +# main should be a pure upstream mirror — ff-only guards against accidental commits +git merge --ff-only origin/"$BASE_BRANCH" 2>/dev/null || { + echo "WARN: could not fast-forward $BASE_BRANCH from origin/$BASE_BRANCH." \ "Proceeding with local state." } +} BASE_SHA=$(git rev-parse "$BASE_BRANCH") echo " Base commit: $BASE_SHA" From 8a8af58055c239a25f3c9a39aa194aa9a2c6802a Mon Sep 17 00:00:00 2001 From: Matt McKay Date: Thu, 7 May 2026 09:37:57 +1000 Subject: [PATCH 5/7] chore: revert README banner and workflow changes from PR Only the quantecon/ folder belongs on main. Workflow disabling is handled via the GitHub UI; README banner is unnecessary given the fork name. --- .github/workflows/docs.yml | 3 +-- .github/workflows/publish-github-release.yml | 8 ++++++-- .github/workflows/release.yml | 13 +++++++++---- README.md | 4 ---- 4 files changed, 16 insertions(+), 12 deletions(-) diff --git a/.github/workflows/docs.yml b/.github/workflows/docs.yml index d0e9f05ad8..e4954478b3 100644 --- a/.github/workflows/docs.yml +++ b/.github/workflows/docs.yml @@ -1,7 +1,6 @@ name: Deploy Docs -# QuantEcon fork: manual dispatch disabled. Only triggers from release.yml -# which is itself disabled on this fork. on: + workflow_dispatch: workflow_run: workflows: ["Release"] types: diff --git a/.github/workflows/publish-github-release.yml b/.github/workflows/publish-github-release.yml index ebd6d74e54..ef53d49eb6 100644 --- a/.github/workflows/publish-github-release.yml +++ b/.github/workflows/publish-github-release.yml @@ -1,13 +1,17 @@ name: Publish GitHub Release -# QuantEcon fork: manual dispatch disabled. Only callable from release.yml -# which is itself disabled on this fork. on: workflow_call: inputs: tag_name: required: true type: string + workflow_dispatch: + inputs: + tag_name: + description: 'Tag name for the release (e.g. v1.2.3)' + required: true + type: string jobs: publish-release: diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 6e36e2c189..aa36100f01 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -1,12 +1,17 @@ name: Release -# QuantEcon fork: auto-release disabled. This workflow is intentionally -# restricted to a branch that does not exist on this fork. Sync with -# upstream (jupyter-book/mystmd) will restore the original trigger on main. on: push: branches: - - _upstream-release-only + - main + workflow_dispatch: + inputs: + forcePublishPyPI: + description: | + Force publish to PyPI, even if the NPM release has already occurred. + type: boolean + default: false + required: false concurrency: ${{ github.workflow }}-${{ github.ref }} permissions: contents: write diff --git a/README.md b/README.md index 93e582a270..2ab67674a4 100644 --- a/README.md +++ b/README.md @@ -1,7 +1,3 @@ -> [!WARNING] -> **This is the [QuantEcon](https://quantecon.org) fork of `mystmd`**, maintained for testing and development of features pending upstream review. -> It is **not** intended for general use. For the official project, see [jupyter-book/mystmd](https://github.com/jupyter-book/mystmd). - # MyST Markdown Command Line Interface, `mystmd` [![MIT License](https://img.shields.io/badge/license-MIT-blue.svg)](https://github.com/jupyter-book/mystmd/blob/main/LICENSE) From e0f991959337e9c1484f3b6c583b39a23c70e5f6 Mon Sep 17 00:00:00 2001 From: Matt McKay Date: Thu, 7 May 2026 09:43:37 +1000 Subject: [PATCH 6/7] fix: address copilot review comments on build.sh - Use awk for features.txt parsing (POSIX-safe, replaces non-portable \s grep) - Add git fetch origin before ff-only merge so refs are current - Create local tracking branch for origin-only feature branches before merging - Fail hard (not warn) if main cannot be fast-forwarded - Patch packages/mystmd/package.json version instead of gitignored version.ts; copy:version propagates the version at build time - Update README to reflect correct version patching mechanism --- quantecon/README.md | 2 +- quantecon/build.sh | 42 +++++++++++++++++++++++++++--------------- 2 files changed, 28 insertions(+), 16 deletions(-) diff --git a/quantecon/README.md b/quantecon/README.md index 605771bed8..cc58dcf588 100644 --- a/quantecon/README.md +++ b/quantecon/README.md @@ -85,7 +85,7 @@ The script: 1. Syncs local `main` from `origin/main` 2. Resets `quantecon` to `main` (discarding any previous build) 3. Merges each branch listed in `features.txt` in order (merge commits, not squash) -4. Patches `packages/mystmd/src/version.ts` to append `-qe` to the version (e.g. `1.9.0-qe`), so `myst --version` identifies the QuantEcon build +4. Patches `packages/mystmd/package.json` version to append `-qe` (e.g. `1.9.0-qe`). The `copy:version` build step propagates this to `version.ts` at build time, so `myst --version` identifies the QuantEcon build. 5. Optionally pushes to `origin/quantecon` Run this after every upstream sync, after updating any feature branch, or after adding/removing a feature branch from `features.txt`. diff --git a/quantecon/build.sh b/quantecon/build.sh index b0ea4cb31e..7ebdb3a9d0 100644 --- a/quantecon/build.sh +++ b/quantecon/build.sh @@ -57,8 +57,8 @@ if [[ -n "$(git status --porcelain)" ]]; then exit 1 fi -# Read feature branches (strip comments and blank lines) -mapfile -t FEATURES < <(grep -v '^\s*#' "$FEATURES_FILE" | grep -v '^\s*$' | awk '{print $1}') +# Read feature branches (strip comments and blank lines, extract first token) +mapfile -t FEATURES < <(awk '/^[[:space:]]*#/{next} /^[[:space:]]*$/{next} {print $1}' "$FEATURES_FILE") if [[ ${#FEATURES[@]} -eq 0 ]]; then echo "No feature branches listed in $FEATURES_FILE. Nothing to do." @@ -73,15 +73,19 @@ done # --------------------------------------------------------------------------- # Ensure local main is up to date # --------------------------------------------------------------------------- +echo "" +echo "==> Fetching origin..." +git fetch origin + echo "" echo "==> Switching to $BASE_BRANCH and fast-forwarding from origin..." git checkout "$BASE_BRANCH" # main should be a pure upstream mirror — ff-only guards against accidental commits -git merge --ff-only origin/"$BASE_BRANCH" 2>/dev/null || { - echo "WARN: could not fast-forward $BASE_BRANCH from origin/$BASE_BRANCH." \ - "Proceeding with local state." -} -} +if ! git merge --ff-only origin/"$BASE_BRANCH"; then + echo "ERROR: could not fast-forward $BASE_BRANCH from origin/$BASE_BRANCH." \ + "Sync main with upstream first (git merge upstream/main && git push origin main)." >&2 + exit 1 +fi BASE_SHA=$(git rev-parse "$BASE_BRANCH") echo " Base commit: $BASE_SHA" @@ -95,7 +99,9 @@ for branch in "${FEATURES[@]}"; do if git rev-parse --verify "$branch" > /dev/null 2>&1; then echo " $branch (local)" elif git rev-parse --verify "origin/$branch" > /dev/null 2>&1; then - echo " $branch (origin)" + # Create a local tracking branch so 'git merge ' works + git branch --track "$branch" "origin/$branch" 2>/dev/null || git branch -f "$branch" "origin/$branch" + echo " $branch (fetched from origin)" else echo "ERROR: branch '$branch' not found locally or on origin." >&2 exit 1 @@ -145,18 +151,24 @@ done # --------------------------------------------------------------------------- # Patch version string with -qe suffix # --------------------------------------------------------------------------- -VERSION_FILE="packages/mystmd/src/version.ts" +# version.ts is generated/gitignored — patch packages/mystmd/package.json instead. +# The copy:version build step will propagate the version to version.ts at build time. +PACKAGE_JSON="packages/mystmd/package.json" echo "" -echo "==> Patching version in $VERSION_FILE with '-qe' suffix..." +echo "==> Patching version in $PACKAGE_JSON with '-qe' suffix..." -CURRENT_VERSION=$(node -p "require('./packages/mystmd/package.json').version") +CURRENT_VERSION=$(node -p "require('./$PACKAGE_JSON').version") QE_VERSION="${CURRENT_VERSION}-qe" -# Replace the version string in version.ts (matches: const version = 'X.Y.Z';) -sed -i.bak "s/const version = '[^']*'/const version = '${QE_VERSION}'/" "$VERSION_FILE" -rm -f "${VERSION_FILE}.bak" +# Use node to patch the JSON safely (avoids sed quoting issues with JSON) +node -e " + const fs = require('fs'); + const pkg = JSON.parse(fs.readFileSync('$PACKAGE_JSON', 'utf8')); + pkg.version = '$QE_VERSION'; + fs.writeFileSync('$PACKAGE_JSON', JSON.stringify(pkg, null, 2) + '\n'); +" -git add "$VERSION_FILE" +git add "$PACKAGE_JSON" git commit -m "chore: set version to ${QE_VERSION} for QuantEcon build" echo " Version set to ${QE_VERSION}" From 935442f975a463491dc22c3e0e9240c287c51c80 Mon Sep 17 00:00:00 2001 From: Matt McKay Date: Thu, 7 May 2026 10:05:20 +1000 Subject: [PATCH 7/7] docs: branch features from upstream/main to keep PR diffs clean - Update diagram and branching model table - New 'Develop a new feature' section uses 'git checkout -b ... upstream/main' - Add migration section for existing branches that include quantecon/ commits (git rebase --onto upstream/main main) - Update keep-current and conflict-resolution sections to use upstream/main - Explain why this works (git merges changes, not full trees) --- quantecon/README.md | 58 +++++++++++++++++++++++++++++---------------- 1 file changed, 38 insertions(+), 20 deletions(-) diff --git a/quantecon/README.md b/quantecon/README.md index cc58dcf588..ea3305f6fc 100644 --- a/quantecon/README.md +++ b/quantecon/README.md @@ -5,31 +5,30 @@ This fork lets QuantEcon develop and use new `mystmd` features before they have ## How it works — the key idea ``` -jupyter-book/mystmd:main - │ (sync periodically) - ▼ -QuantEcon/mystmd:main ← pure mirror, never commit here directly - │ - ├── feature/foo ← your work; PR open against main (upstream) - ├── feature/bar ← another patch; PR open against main (upstream) - │ - ▼ (built by build.sh) +jupyter-book/mystmd:main ─────┐ + │ (sync periodically) │ (feature branches start here) + ▼ │ +QuantEcon/mystmd:main │ ← mirror + quantecon/ tooling + │ │ + │ feature/foo ◄─────┤ ← branched from upstream/main; PR target = upstream main + │ feature/bar ◄─────┘ ← branched from upstream/main; PR target = upstream main + ▼ (built by build.sh, which merges feature branches into quantecon) QuantEcon/mystmd:quantecon ← combined build: main + all feature branches ``` **Feature branches serve two purposes simultaneously:** -1. They are the source of upstream PRs — targeting `main`, showing only the diff for that one feature. GitHub doesn't know or care that `build.sh` also merges them elsewhere. -2. They are consumed by `build.sh`, which merges all of them onto a throwaway `quantecon` branch that QuantEcon actually builds from. +1. They are the source of upstream PRs — targeting `jupyter-book/mystmd:main`, showing only the diff for that one feature. They are branched from `upstream/main` so they don't carry the `quantecon/` folder. +2. They are consumed by `build.sh`, which merges them onto the throwaway `quantecon` branch (which inherits the `quantecon/` folder from `main`). -This means **you never need to choose** between "make it easy to upstream" and "make it available to use now". The feature branch does both. When you update a feature branch (rebase on latest `main`, address reviewer feedback), the upstream PR updates automatically and re-running `build.sh` gives you an updated `quantecon` build. +This means **you never need to choose** between "make it easy to upstream" and "make it available to use now". The feature branch does both. ## Branching model | Branch | Purpose | |---|---| | `main` | Mirrors `jupyter-book/mystmd:main` exactly. Synced via the GitHub "Sync fork" button or `git merge upstream/main`. **No direct commits.** | -| `feature/` | One branch per logical patch. Branched from `main`, kept rebased on `main`. This is the branch you open the upstream PR from. | +| `feature/` | One branch per logical patch. **Branched from `upstream/main`** (not `main`), kept rebased on `upstream/main`. This is the branch you open the upstream PR from. | | `quantecon` | Throwaway combined build. Rebuilt by `build.sh` as `main` + all active feature branches. **Never commit here directly — it is always discarded and regenerated.** | ## One-time setup @@ -62,18 +61,35 @@ After syncing, rebuild the `quantecon` branch (see below) so it includes the lat ### Develop a new feature +> **Important:** branch from `upstream/main`, **not** from `main`. This keeps the `quantecon/` folder out of your feature branch, so the upstream PR diff shows only your actual changes. + ```bash -git checkout main -git checkout -b feature/ +git fetch upstream +git checkout -b feature/ upstream/main # make your changes and commit them git push origin feature/ ``` Then: -1. Add the branch name to `quantecon/features.txt` +1. Add the branch name to `quantecon/features.txt` (on `main`, then commit and push) 2. Open a PR on GitHub: **base: `jupyter-book/mystmd:main`**, **compare: `QuantEcon/mystmd:feature/`** 3. Rebuild the `quantecon` branch so the feature is available immediately +> **Why this works:** `build.sh` runs from `main` (which has the `quantecon/` folder), checks out the `quantecon` branch, and merges your feature branch into it. Because git merges *changes* (not full trees), the feature branch doesn't need to contain `quantecon/` — the folder comes from `main`, and your feature's changes are layered on top. + +### Migrating an existing feature branch to use upstream/main as base + +If you already have a feature branch that was created from `main` (and therefore includes the `quantecon/` folder commits), rebase it onto `upstream/main`: + +```bash +git fetch upstream +git checkout feature/ +git rebase --onto upstream/main main +git push --force-with-lease origin feature/ +``` + +This replays only your feature's commits on top of `upstream/main`, dropping the `quantecon/` folder commits from the branch's history. The upstream PR diff will then show only your actual changes. + ### Rebuild the `quantecon` branch ```bash @@ -102,11 +118,12 @@ git push --force-with-lease origin feature/ ### Keep a feature branch current with upstream -If `main` has moved since you branched: +If `upstream/main` has moved since you branched: ```bash +git fetch upstream git checkout feature/ -git rebase main +git rebase upstream/main git push --force-with-lease origin feature/ ./quantecon/build.sh --push ``` @@ -117,10 +134,11 @@ Conflicts are **always fixed on the feature branch**, never on `quantecon` (whic `build.sh` will abort cleanly and tell you which branch caused the conflict. -**Feature conflicts with `main`** (upstream changed code the feature touches): +**Feature conflicts with upstream** (upstream changed code the feature touches): ```bash +git fetch upstream git checkout feature/ -git rebase main # resolve conflicts here, then: git add . && git rebase --continue +git rebase upstream/main # resolve conflicts here, then: git add . && git rebase --continue git push --force-with-lease origin feature/ ./quantecon/build.sh --push ```