From cae40bf576a027097e5be78f8eb968cb03330b5b Mon Sep 17 00:00:00 2001 From: Matt McKay Date: Fri, 5 Jun 2026 16:40:37 +1000 Subject: [PATCH 1/2] docs: add hands-on CLI tutorial walkthrough MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add docs/user/cli-tutorial.md — a 10-minute walkthrough of `qestyle` on a real lecture (the test-action-style-guide markov_chains_jax.md, seeded with 42 catalogued violations): dry-run, read the report, apply fixes, inspect the diff, and measure recall against the catalog. Includes a copy-paste demo cheat-sheet for live screen-shares, and links to the pre-executed examples/ output as a no-API-key fallback. Wire it into the MyST TOC and cross-link from the CLI reference and README. Co-Authored-By: Claude Opus 4.8 --- README.md | 1 + docs/myst.yml | 1 + docs/user/cli-tutorial.md | 310 ++++++++++++++++++++++++++++++++++++++ docs/user/cli.md | 5 + 4 files changed, 317 insertions(+) create mode 100644 docs/user/cli-tutorial.md diff --git a/README.md b/README.md index 725b113..4a0e291 100644 --- a/README.md +++ b/README.md @@ -40,6 +40,7 @@ Full documentation is available at the [docs site](https://quantecon.github.io/a - [Configuration](docs/user/configuration.md) — Action inputs, LLM model, rule types - [Rules Reference](docs/user/rules-reference.md) — All 49 rules by category and type - [CLI Guide](docs/user/cli.md) — Local `qestyle` command usage +- [CLI Tutorial](docs/user/cli-tutorial.md) — Hands-on walkthrough on a real lecture - [GitHub App Setup](docs/user/github-app-setup.md) — Production setup with GitHub Apps ### Developer Guide diff --git a/docs/myst.yml b/docs/myst.yml index dd5d4ef..845affa 100644 --- a/docs/myst.yml +++ b/docs/myst.yml @@ -12,6 +12,7 @@ project: - file: user/configuration.md - file: user/rules-reference.md - file: user/cli.md + - file: user/cli-tutorial.md - file: user/github-app-setup.md - title: Developer Guide children: diff --git a/docs/user/cli-tutorial.md b/docs/user/cli-tutorial.md new file mode 100644 index 0000000..6ee0132 --- /dev/null +++ b/docs/user/cli-tutorial.md @@ -0,0 +1,310 @@ +--- +title: CLI Tutorial +--- + +# Tutorial: Reviewing a lecture with `qestyle` + +This is a hands-on walkthrough of the local CLI. In about 10 minutes you'll +run a real review on a QuantEcon lecture, read the report, apply fixes, and +inspect the changes with `git diff`. + +We use the [`test-action-style-guide`](https://github.com/QuantEcon/test-action-style-guide) +repository as our material. Its `markov_chains_jax.md` is a realistic QuantEcon +lecture (adapted from the intro Markov chains lecture) seeded with **42 +catalogued style violations** across all 8 categories. Because every violation +is documented, you can *measure* what `qestyle` catches — which makes it ideal +for a demo. + +```{tip} +This page doubles as a **live-demo script**. The exact command sequence is +collected at the end under [Demo cheat-sheet](#demo-cheat-sheet) for copy-paste +during a screen-share. +``` + +```{note} +**Short on time, or no API key handy?** Pre-executed output for this exact run +is committed in the test repo under +[`examples/`](https://github.com/QuantEcon/test-action-style-guide/tree/main/examples) — +the full report, the applied-changes diff, and a precision/recall analysis vs the +42-violation catalog (27/30 rules, 90%). You can show those directly instead of +running live. +``` + +## Before you start + +You need `qestyle` installed and an Anthropic API key exported. If you haven't +done this, see the [CLI reference → Installation](cli.md#installation): + +```bash +uv tool install git+https://github.com/QuantEcon/action-style-guide.git +export ANTHROPIC_API_KEY='your-key-here' + +qestyle --version # qestyle 0.7.2 +``` + +```{note} +Each category is one Claude call with extended thinking (temperature `1.0`), so +output is **not deterministic** — the exact violations and counts shift slightly +run to run. The numbers in this tutorial are illustrative. +``` + +## 1. Get the demo lecture + +Clone the test repository and move into it: + +```bash +git clone https://github.com/QuantEcon/test-action-style-guide.git +cd test-action-style-guide + +ls lectures/ +# markov_chains_jax.md ← realistic lecture, 42 seeded violations +# markov_chains_jax.annotated.md ← the violation catalog (line-by-line) +# test-lecture-violations.md ← smaller violation set +# test-lecture-clean.md ← should pass all checks +``` + +The companion [`markov_chains_jax.annotated.md`](https://github.com/QuantEcon/test-action-style-guide/blob/main/lectures/markov_chains_jax.annotated.md) +lists every intentional violation with its rule ID and line number. Keep it open +in a second pane during the demo so you can point at "it found W6, M1, C3…". + +## 2. Preview without touching the file (`--dry-run`) + +Always start with `--dry-run`. It runs the full review and writes a report, but +**never modifies the lecture**. Let's check the `writing` category first: + +```bash +qestyle lectures/markov_chains_jax.md --dry-run -c writing +``` + +The console output looks like: + +```text +📋 qestyle v0.7.2 + Lecture: markov_chains_jax.md + Categories: writing + Mode: dry-run (report only, no changes) + +✅ Review complete — 9 issue(s) found + 🔧 6 fix(es) available (run without --dry-run to apply) + 📝 3 style suggestion(s) for human review + 📄 Report: lectures/qestyle(writing)-markov_chains_jax.md +``` + +Two numbers matter here: + +- **fixes available** — *rule-type* violations (mechanical, unambiguous). These + get applied automatically when you drop `--dry-run`. +- **style suggestions** — *style-type* items that need human judgment. These are + **never** auto-applied; they only ever appear in the report. + +## 3. Read the report + +Open the report written next to the lecture: + +```bash +open lectures/'qestyle(writing)-markov_chains_jax.md' # or your editor +``` + +It has a fixed structure — **style suggestions first** (they need your +attention), then the rule violations: + +````markdown +# Style Guide Report: markov_chains_jax.md + +- **Date:** 2026-06-05 14:32 +- **Version:** qestyle v0.7.2 +- **Issues found:** 9 +- **Mode:** dry-run (no changes applied) + +## 📝 Style Suggestions (3) + +> **Action required:** These suggestions require human review and judgment. + +### 1. qe-writing-002 — Prefer clear, concise sentences +**Location:** Overview +**Description:** Opening sentence is wordy and could be tighter. +... + +--- + +## 🔧 Rule Violations (6) + +> **Fixable:** These violations can be auto-fixed (run without `--dry-run`). + +### 1. qe-writing-004 — Avoid unnecessary capitalization +**Location:** "The Transition Matrix is" +**Current text:** + +``` +The Transition Matrix is +``` + +**Suggested fix:** + +``` +The transition matrix is +``` + +**Explanation:** "Transition Matrix" is a common noun mid-sentence... +```` + +This split — *mechanical fixes* vs *human-judgment suggestions* — is the core +idea to land in the demo. The CLI only ever automates the unambiguous half. + +## 4. Apply the fixes and inspect the diff + +Now run the same command **without** `--dry-run`. This is the money shot for a +live demo: + +```bash +qestyle lectures/markov_chains_jax.md -c writing +``` + +```text +📋 qestyle v0.7.2 + Lecture: markov_chains_jax.md + Categories: writing + Mode: fix (rule violations will be applied) + +✅ Review complete — 9 issue(s) found + 🔧 Applied 6 fix(es) to markov_chains_jax.md + Restore original: git checkout markov_chains_jax.md + 📝 3 style suggestion(s) for human review + 📄 Report: lectures/qestyle(writing)-markov_chains_jax.md +``` + +Because the lecture lives in a git repo, the changes are reviewable with a plain +`git diff`: + +```bash +git diff lectures/markov_chains_jax.md +``` + +```diff +-The Transition Matrix is ++The transition matrix is +``` + +Every applied change is mechanical and visible in the diff — nothing is hidden, +and nothing requiring judgment was touched. + +```{important} +If the lecture has **uncommitted changes**, `qestyle` warns you and asks to +confirm before modifying it — commit or stash first so the diff stays clean. +Start every demo from a clean working tree. +``` + +## 5. Undo + +Roll the file back to its committed state at any time: + +```bash +git checkout -- lectures/markov_chains_jax.md +``` + +## 6. Try other categories + +Each category targets a different rule family. A couple of nice ones to show: + +```bash +qestyle lectures/markov_chains_jax.md --dry-run -c math +``` + +```diff +# qe-math-003 — use bmatrix, not pmatrix +-P = \begin{pmatrix} ++P = \begin{bmatrix} + 1 - \alpha & \alpha \\ + \beta & 1 - \beta +-\end{pmatrix} ++\end{bmatrix} +``` + +```bash +qestyle lectures/markov_chains_jax.md --dry-run -c code +``` + +```diff +# qe-code-004 — prefer qe.Timer() over manual time.time() +-start_time = time.time() +-result = power_iteration(P) +-elapsed = time.time() - start_time ++with qe.Timer(): ++ result = power_iteration(P) +``` + +You can pass several at once: `-c math,code,writing`. + +## 7. Run the full review + +With no `-c` flag, `qestyle` runs **every** category in sequence. This is the +most thorough pass — and the slowest, since it's one Claude call per category: + +```bash +qestyle lectures/markov_chains_jax.md +# Report: lectures/qestyle(all)-markov_chains_jax.md +``` + +## 8. Measure recall against the catalog (the punchline) + +This is what makes the test repo special. After a run, compare the report's rule +IDs against [`markov_chains_jax.annotated.md`](https://github.com/QuantEcon/test-action-style-guide/blob/main/lectures/markov_chains_jax.annotated.md): + +- **True positive** — a catalogued violation that `qestyle` flagged +- **False negative** — catalogued but missed +- **False positive** — flagged but not a real issue (the catalog includes + deliberate *control* cases, e.g. a correct `{cite:t}` citation, to catch these) + +For one rule you can even check recall deterministically. The repo ships a +ground-truth finder for the "one sentence per paragraph" rule: + +```bash +python scripts/find_multisentence.py lectures/markov_chains_jax.md +``` + +This parses the file structurally and lists every multi-sentence paragraph — +the exact set `qe-writing-001` should catch. It's the closest thing to an +objective score, and it's the bridge to the project's larger claim: **most style +violations are mechanical, and a checker catches them reliably.** + +(demo-cheat-sheet)= +## Demo cheat-sheet + +The full sequence, ready to paste during a screen-share. Start from a clean +working tree. + +```bash +# Setup (once) +uv tool install git+https://github.com/QuantEcon/action-style-guide.git +export ANTHROPIC_API_KEY='your-key-here' +git clone https://github.com/QuantEcon/test-action-style-guide.git +cd test-action-style-guide + +# 1. Preview — no changes to the file +qestyle lectures/markov_chains_jax.md --dry-run -c writing + +# 2. Show the report (style suggestions vs auto-fixes) +open lectures/'qestyle(writing)-markov_chains_jax.md' + +# 3. Apply fixes for real +qestyle lectures/markov_chains_jax.md -c writing + +# 4. The money shot — every change is a reviewable diff +git diff lectures/markov_chains_jax.md + +# 5. Undo +git checkout -- lectures/markov_chains_jax.md + +# (optional) other categories / full run +qestyle lectures/markov_chains_jax.md --dry-run -c math +qestyle lectures/markov_chains_jax.md # all categories + +# Reset everything between takes +git checkout -- lectures/ +``` + +## Where to next + +- [CLI reference](cli.md) — every flag, option, and exit behavior +- [Rules reference](rules-reference.md) — all 49 rules by category and type +- [Configuration](configuration.md) — model, temperature, rule selection diff --git a/docs/user/cli.md b/docs/user/cli.md index 83bcf73..bcc3b9a 100644 --- a/docs/user/cli.md +++ b/docs/user/cli.md @@ -6,6 +6,11 @@ title: Local CLI The `qestyle` command runs the **exact same** review engine locally — same prompts, rules, and fix logic as the GitHub Action. +```{tip} +New to `qestyle`? Start with the [CLI Tutorial](cli-tutorial.md) — a 10-minute +hands-on walkthrough on a real lecture. This page is the full flag reference. +``` + ## Installation ### With `pip` From e9ea891995c11f67e9f3876b1479b10e4cd6517a Mon Sep 17 00:00:00 2001 From: Matt McKay Date: Fri, 5 Jun 2026 17:01:25 +1000 Subject: [PATCH 2/2] docs: address Copilot review; fold CLI sample output into docs - Portable editor commands (note macOS `open` / Linux `xdg-open`) - Clarify recall wording: '27 of the 30 rules seeded in the catalog' - Show the printed report path as absolute (CLI resolves the path) - Add companion page cli-tutorial-output.md (console output + diff + recall), folded from the test repo so the CLI sample lives with the CLI docs; point the tutorial's fallback note at it. Keeps test-action-style-guide for the Action demo. Co-Authored-By: Claude Opus 4.8 --- docs/myst.yml | 1 + docs/user/cli-tutorial-output.md | 117 +++++++++++++++++++++++++++++++ docs/user/cli-tutorial.md | 20 +++--- 3 files changed, 128 insertions(+), 10 deletions(-) create mode 100644 docs/user/cli-tutorial-output.md diff --git a/docs/myst.yml b/docs/myst.yml index 845affa..b4a0a51 100644 --- a/docs/myst.yml +++ b/docs/myst.yml @@ -13,6 +13,7 @@ project: - file: user/rules-reference.md - file: user/cli.md - file: user/cli-tutorial.md + - file: user/cli-tutorial-output.md - file: user/github-app-setup.md - title: Developer Guide children: diff --git a/docs/user/cli-tutorial-output.md b/docs/user/cli-tutorial-output.md new file mode 100644 index 0000000..d3a9e5d --- /dev/null +++ b/docs/user/cli-tutorial-output.md @@ -0,0 +1,117 @@ +--- +title: Sample CLI output +--- + +# Sample CLI output + +Real captured output from running `qestyle` on the test lecture +[`markov_chains_jax.md`](https://github.com/QuantEcon/test-action-style-guide/blob/main/lectures/markov_chains_jax.md) +— a realistic QuantEcon lecture seeded with **42 catalogued style violations**. Use +this page to show what a run produces **without a live API call** (the companion to +the [CLI Tutorial](cli-tutorial.md)). + +- **Version:** qestyle v0.7.2 · **Date:** 2026-06-05 · **Categories:** all 8 + +```{note} +Output is **not deterministic** (temperature 1.0 + extended thinking) — a fresh run +will differ slightly. This is one representative snapshot. +``` + +## The run + +```bash +qestyle lectures/markov_chains_jax.md +``` + +```text +✅ Review complete — 86 issue(s) found + 🔧 Applied 47 fix(es) to markov_chains_jax.md + Restore original: git checkout markov_chains_jax.md + 📝 38 style suggestion(s) for human review + 📄 Report: …/lectures/qestyle(all)-markov_chains_jax.md +``` + +| Metric | Count | +|--------|-------| +| Total issues reported | **86** | +| Applied fixes (auto, rule-type) | **47** | +| Style suggestions (human review) | **38** | +| Warnings (apply/empty issues) | **3** | + +## What changed + +A `git diff` of the applied fixes — every change is mechanical and reviewable: + +```diff +-Markov chains are one of the most useful classes of stochastic processes in economics and finance. They provide a framework for modeling systems that transition between states over time, where the future state depends only on the current state. ++Markov chains are one of the most useful classes of stochastic processes in economics and finance. ++ ++They provide a framework for modeling systems that transition between states over time, where the future state depends only on the current state. + + ```{code-cell} ipython3 ++--- ++tags: [hide-output] ++--- + !pip install quantecon + +-## Definitions and Setup ++## Definitions and setup + +-### Stochastic Matrices ++### Stochastic matrices + +-In other words, knowing the Current State is enough to determine probabilities for future states. ++In other words, knowing the current state is enough to determine probabilities for future states. + +-The Transition Matrix is ++The transition matrix is +``` + +These map to real rules: one-sentence-per-paragraph (`qe-writing-001`), the +`hide-output` tag on `pip install` (`qe-code-003`), and heading/word +de-capitalization (`qe-writing-006`, `qe-writing-004`). + +## Precision / recall vs the catalog + +Every seeded violation is documented in +[`markov_chains_jax.annotated.md`](https://github.com/QuantEcon/test-action-style-guide/blob/main/lectures/markov_chains_jax.annotated.md), +so recall is measurable. + +**Rule-level recall: 27 of the 30 rules seeded in the catalog (90%)** — i.e. the run +flagged at least one violation for 27 of the 30 distinct ground-truth rules (up from +26/30 at v0.7.0). Note this is the *seeded catalog* denominator, not the full ~50-rule +registry. + +### 3 rules missed + +| Rule | Violation | Note | +|------|-----------|------| +| `qe-writing-008` | double space before "and" | possibly normalised by an earlier writing fix | +| `qe-fig-008` | line width not set to `lw=2` | **persistent miss** (also missed at v0.7.0) | +| `qe-link-001` | bare URL / generic "here" link text | **persistent miss** — `qe-link-002` fixes may transform the URLs first | + +`qe-fig-008` and `qe-link-001` are missed across both versions — the two rules most +worth a prompt-engineering pass. + +### Beyond the seeded set + +Seven additional rules fired (e.g. `qe-writing-003`, `qe-math-009`, `qe-jax-003`) — +**either genuine extra finds or false positives; they need a human spot-check.** Two +are already flagged low-quality by the run's own warnings (`qe-writing-007`: +suggestion identical to source; `qe-fig-002`: missing fix). + +```{tip} +Rule-level recall (which rules fired) is measured here. **Instance-level precision is +not** — scoring it means reading each of the 86 entries against the catalog, which is +exactly the validation the annotated ground-truth file enables, and a good live +exercise. +``` + +## Reproduce + +```bash +# in the test-action-style-guide repo, with qestyle installed + ANTHROPIC_API_KEY set +qestyle lectures/markov_chains_jax.md # all categories, applies fixes +git diff lectures/markov_chains_jax.md # inspect the applied changes +git checkout -- lectures/markov_chains_jax.md # reset +``` diff --git a/docs/user/cli-tutorial.md b/docs/user/cli-tutorial.md index 6ee0132..286b533 100644 --- a/docs/user/cli-tutorial.md +++ b/docs/user/cli-tutorial.md @@ -22,12 +22,11 @@ during a screen-share. ``` ```{note} -**Short on time, or no API key handy?** Pre-executed output for this exact run -is committed in the test repo under -[`examples/`](https://github.com/QuantEcon/test-action-style-guide/tree/main/examples) — -the full report, the applied-changes diff, and a precision/recall analysis vs the -42-violation catalog (27/30 rules, 90%). You can show those directly instead of -running live. +**Short on time, or no API key handy?** A pre-baked run of this exact tutorial — +the console output, the applied-changes diff, and a precision/recall analysis — is +on the companion page [Sample CLI output](cli-tutorial-output.md). That run found +86 issues and reached **rule-level recall of 27 of the 30 rules seeded in the test +catalog (90%)**. Walk through that page instead of running live. ``` ## Before you start @@ -87,7 +86,7 @@ The console output looks like: ✅ Review complete — 9 issue(s) found 🔧 6 fix(es) available (run without --dry-run to apply) 📝 3 style suggestion(s) for human review - 📄 Report: lectures/qestyle(writing)-markov_chains_jax.md + 📄 Report: …/lectures/qestyle(writing)-markov_chains_jax.md ``` Two numbers matter here: @@ -102,7 +101,8 @@ Two numbers matter here: Open the report written next to the lecture: ```bash -open lectures/'qestyle(writing)-markov_chains_jax.md' # or your editor +# macOS `open`; on Linux use `xdg-open`, or just open it in your editor +open lectures/'qestyle(writing)-markov_chains_jax.md' ``` It has a fixed structure — **style suggestions first** (they need your @@ -170,7 +170,7 @@ qestyle lectures/markov_chains_jax.md -c writing 🔧 Applied 6 fix(es) to markov_chains_jax.md Restore original: git checkout markov_chains_jax.md 📝 3 style suggestion(s) for human review - 📄 Report: lectures/qestyle(writing)-markov_chains_jax.md + 📄 Report: …/lectures/qestyle(writing)-markov_chains_jax.md ``` Because the lecture lives in a git repo, the changes are reviewable with a plain @@ -284,7 +284,7 @@ cd test-action-style-guide qestyle lectures/markov_chains_jax.md --dry-run -c writing # 2. Show the report (style suggestions vs auto-fixes) -open lectures/'qestyle(writing)-markov_chains_jax.md' +open lectures/'qestyle(writing)-markov_chains_jax.md' # macOS; use xdg-open / your editor elsewhere # 3. Apply fixes for real qestyle lectures/markov_chains_jax.md -c writing