From c22dc85f9b2970190627bb2c680a60e560ccc7dc Mon Sep 17 00:00:00 2001 From: GitHub Copilot Date: Fri, 24 Apr 2026 20:05:05 +0200 Subject: [PATCH 1/2] =?UTF-8?q?=F0=9F=92=AC=20clarify=20hidden-asset=20rec?= =?UTF-8?q?overy=20guidance?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Align the scaffold skills, their references, the validator, and repo docs on the fallback path for incomplete installed copies so dotfiles and other required assets are restored from upstream before generation continues. --- AGENTS.md | 74 +++++++++---------- CHANGELOG.md | 9 ++- README.md | 4 +- scripts/validate-skill-templates.ps1 | 2 + skills/dotnet-new-app-slnx/SKILL.md | 4 + skills/dotnet-new-app-slnx/references/app.md | 2 + skills/dotnet-new-lib-slnx/SKILL.md | 3 + .../dotnet-new-lib-slnx/references/library.md | 2 + 8 files changed, 60 insertions(+), 40 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 793fa6b..fd6626d 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -17,16 +17,16 @@ When running evals or testing skills, create all workspaces in a temp location: **Why:** Eval artifacts — branches, commits, local git config — leak into the real repo history and are painful to clean up. The skill source lives in a git repo; eval output does not belong here. -## Per-Skill Evals - -Every repo-managed skill must include its own `evals/evals.json` file at `skills//evals/evals.json`. - -- Treat this as a required artifact for every first-party skill in this repo -- Eval entries may include an optional `files` array of skill-relative fixture paths such as `evals/files/example.md` -- When `files` is present, keep the paths relative to `skills//` and stage those fixtures into the temp eval workspace for both `with_skill` and `without_skill` runs -- Run evals **per skill**, not as one shared repo-level eval file -- Run evals from a temp workspace such as `$env:TEMP/-workspace/`, never from inside this repository -- When creating or modifying a repo-managed skill, run both `with_skill` and `without_skill` comparison executions from that temp workspace before the work is considered complete +## Per-Skill Evals + +Every repo-managed skill must include its own `evals/evals.json` file at `skills//evals/evals.json`. + +- Treat this as a required artifact for every first-party skill in this repo +- Eval entries may include an optional `files` array of skill-relative fixture paths such as `evals/files/example.md` +- When `files` is present, keep the paths relative to `skills//` and stage those fixtures into the temp eval workspace for both `with_skill` and `without_skill` runs +- Run evals **per skill**, not as one shared repo-level eval file +- Run evals from a temp workspace such as `$env:TEMP/-workspace/`, never from inside this repository +- When creating or modifying a repo-managed skill, run the full per-skill test from that temp workspace before the work is considered complete. Full test means both `with_skill` and `without_skill` comparison executions, grading both runs, aggregating `benchmark.json`, and opening the review viewer. A reasoning-only smoke test does not count as full test. - For a brand-new skill, the baseline is `without_skill`; for an existing skill, use either `without_skill` or the previous/original skill version as the baseline, matching the `skill-creator` benchmark flow - Generate the human-review artifacts too: aggregate the comparison into `benchmark.json` and launch `eval-viewer/generate_review.py` from the installed Anthropic `skill-creator` copy (typically under `~/.agents/skills/skill-creator/` or `~/.claude/skills/skill-creator/`) so the user can inspect `Outputs` and `Benchmark` before sign-off - Deterministic scaffold/template skills must keep local deterministic validators as well; evals supplement validators, they do not replace them @@ -82,22 +82,22 @@ After changing any repo-managed skill, sync the touched files across the repo co Every skill follows this layout: ``` -skills// -├── SKILL.md # Required — the skill definition (loaded by Claude) -├── FORMS.md # Optional — structured form fields for parameter collection -├── assets/ # Optional — file templates, fonts, icons used in output -│ └── / # Group by variant when a skill supports multiple (e.g. library/, app/) -├── scripts/ # Optional — executable code (Python, Bash, etc.) -├── references/ # Optional — detailed reference docs the agent consults during generation -└── evals/ # Required for repo-managed skills — per-skill eval prompts and expectations - └── files/ # Optional — input fixtures referenced by evals/evals.json files[] -``` +skills// +├── SKILL.md # Required — the skill definition (loaded by Claude) +├── FORMS.md # Optional — structured form fields for parameter collection +├── assets/ # Optional — file templates, fonts, icons used in output +│ └── / # Group by variant when a skill supports multiple (e.g. library/, app/) +├── scripts/ # Optional — executable code (Python, Bash, etc.) +├── references/ # Optional — detailed reference docs the agent consults during generation +└── evals/ # Required for repo-managed skills — per-skill eval prompts and expectations + └── files/ # Optional — input fixtures referenced by evals/evals.json files[] +``` - `SKILL.md` is the entry point — it contains the workflow, conventions, and step-by-step instructions -- `assets/` holds file templates, fonts, icons, and other static content used in output (the agent reads and substitutes placeholders) -- `references/` holds detailed specs that `SKILL.md` references but are too long to inline -- `evals/` holds the per-skill `evals.json` definitions used to verify that the skill still works after changes -- `evals/files/` holds optional skill-local fixture inputs referenced by `evals/evals.json` when a benchmark needs attached source material +- `assets/` holds file templates, fonts, icons, and other static content used in output (the agent reads and substitutes placeholders) +- `references/` holds detailed specs that `SKILL.md` references but are too long to inline +- `evals/` holds the per-skill `evals.json` definitions used to verify that the skill still works after changes +- `evals/files/` holds optional skill-local fixture inputs referenced by `evals/evals.json` when a benchmark needs attached source material ## Template Files Are Literal @@ -145,19 +145,19 @@ Native input widgets are a **host/runtime feature**, not a guaranteed model capa - Do not switch interaction styles mid-collection unless the host explicitly upgrades from plain text to native controls - Favor consistency and low-friction UX over conversational variety during parameter collection -`FORMS.md` defines each field with: -- **type** — `text`, `single-choice`, or `multi-choice` -- **prompt** — the question to ask -- **choices** — options for choice types -- **default** — pre-filled value (mark as Recommended) -- **required** — whether the field is mandatory - -Presentation rules (enforced in every `FORMS.md`): -- Ask one field at a time — never bundle multiple questions -- Use selectable choices for `single-choice` and `multi-choice` fields — not free text -- When a default exists, present it first and append "(Recommended)" -- For `text` fields with a computed default, offer the computed value as a selectable choice alongside free text -- After all fields are collected, present a summary and ask for confirmation +`FORMS.md` defines each field with: +- **type** — `text`, `single-choice`, or `multi-choice` +- **prompt** — the question to ask +- **choices** — options for choice types +- **default** — pre-filled value (mark as Recommended) +- **required** — whether the field is mandatory + +Presentation rules (enforced in every `FORMS.md`): +- Ask one field at a time — never bundle multiple questions +- Use selectable choices for `single-choice` and `multi-choice` fields — not free text +- When a default exists, present it first and append "(Recommended)" +- For `text` fields with a computed default, offer the computed value as a selectable choice alongside free text +- After all fields are collected, present a summary and ask for confirmation This applies to all skills that collect user input, not just scaffolding skills. diff --git a/CHANGELOG.md b/CHANGELOG.md index 1f6b4d6..8f917c9 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,11 +8,16 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), ## [0.3.4] - 2026-04-24 -This is a patch release that tightens the generated .gitignore templates for the .NET app and library scaffolds, adding standard Visual Studio, Rider, build-output, test, publish, and workspace exclusions so generated solutions start with cleaner defaults. +This is a patch release that hardens the .NET scaffold guidance around hidden asset recovery, making incomplete `npx skills add` installs easier to diagnose and repair before generation continues. The validator and repo docs now reinforce the same upstream-verification and manual-restore rule so local cache or packaging mismatches do not get mistaken for template defects. ### Changed -- Expanded the shared `.gitignore` templates used by `dotnet-new-app-slnx` and `dotnet-new-lib-slnx` to ignore standard .NET build outputs, IDE state, test artifacts, publish output, and local bot workspace files. +- Clarified `dotnet-new-app-slnx` and `dotnet-new-lib-slnx` so missing required or dot-prefixed files in an installed skill copy are treated as a local install mismatch first, with explicit upstream verification and manual restoration before continuing, +- Aligned the app and library variant references, `AGENTS.md`, and `README.md` with the same hidden-asset recovery rule so incomplete `npx skills add` copies get repaired consistently across the repo. + +### Fixed + +- Tightened `scripts/validate-skill-templates.ps1` so it now asserts the hidden-asset recovery wording in both scaffold references, keeping the validator synchronized with the documented install-fallback behavior. ## [0.3.3] - 2026-03-25 diff --git a/README.md b/README.md index 11fec42..11bdac7 100644 --- a/README.md +++ b/README.md @@ -12,7 +12,7 @@ One repo-wide convention matters especially for scaffolding skills: prefer dynam Another repo rule is intentionally strict: every repo-managed skill ships with its own `evals/evals.json`, and those evals are run per skill from a temp workspace instead of from inside this repository. -Another part of that workflow is now mandatory too: when a repo-managed skill is created or modified, the author must run both `with_skill` and `without_skill` comparison executions from a temp workspace, aggregate the results into `benchmark.json`, and open `eval-viewer/generate_review.py` from the installed Anthropic `skill-creator` copy, typically under `~/.agents/skills/skill-creator/` or `~/.claude/skills/skill-creator/`, so a human can review both the `Outputs` and `Benchmark` views before sign-off. For new skills the baseline is `without_skill`; for existing skills it can be `without_skill` or the previous/original skill version, matching the `skill-creator` benchmark flow. When the available runner supports sub-agents or equivalent background tasks, the measured benchmark should fan out paired executor runs in parallel and parallelize independent grading work too, rather than running evals serially by habit. +Another part of that workflow is now mandatory too: when a repo-managed skill is created or modified, the author must run the full per-skill test from a temp workspace. Full test means both `with_skill` and `without_skill` comparison executions, grading both runs, aggregating the results into `benchmark.json`, and opening `eval-viewer/generate_review.py` from the installed Anthropic `skill-creator` copy, typically under `~/.agents/skills/skill-creator/` or `~/.claude/skills/skill-creator/`, so a human can review both the `Outputs` and `Benchmark` views before sign-off. For new skills the baseline is `without_skill`; for existing skills it can be `without_skill` or the previous/original skill version, matching the `skill-creator` benchmark flow. A reasoning-only smoke test does not count. When the available runner supports sub-agents or equivalent background tasks, the measured benchmark should fan out paired executor runs in parallel and parallelize independent grading work too, rather than running evals serially by habit. One more consistency rule matters for form-driven skills: native input fields are treated as a host feature, not something a model can rely on. Skills in this repo must stay usable with or without UI widgets, and must fall back to the same deterministic one-field-at-a-time flow when the host only supports plain chat. @@ -26,6 +26,8 @@ Install any skill directly from this repository with a single command: npx skills add https://github.com/codebeltnet/agentic --skill ``` +If an install path ever drops required files or dot-prefixed paths from a skill tree, treat that as an incomplete copy, verify the upstream repository contents, and manually restore the missing entries directly from the repository source tree before using the skill. + For example: ```bash diff --git a/scripts/validate-skill-templates.ps1 b/scripts/validate-skill-templates.ps1 index 1add7f1..4e2d790 100644 --- a/scripts/validate-skill-templates.ps1 +++ b/scripts/validate-skill-templates.ps1 @@ -585,6 +585,8 @@ Add-ValidationResult -Results $results -Name 'Shared .bot assets are tracked and Assert-Contains -Name 'lib shared .gitignore' -Content $libIgnore -Needle '!.bot/README.md' Assert-Contains -Name 'app .bot README' -Content $appBot -Needle '# .bot Workspace' Assert-Contains -Name 'lib .bot README' -Content $libBot -Needle '# .bot Workspace' + Assert-Contains -Name 'app shared copy guidance' -Content (Get-FileText -RepoRoot $repoRoot -RelativePath 'skills/dotnet-new-app-slnx/references/app.md' -GitRef $Ref) -Needle 'manually restore the missing files directly from the repository source tree' + Assert-Contains -Name 'lib shared copy guidance' -Content (Get-FileText -RepoRoot $repoRoot -RelativePath 'skills/dotnet-new-lib-slnx/references/library.md' -GitRef $Ref) -Needle 'restore the missing files manually from the upstream repository contents' } Add-ValidationResult -Results $results -Name 'Library skill documents PROJECT_NAME and DOCFX target framework' -Action { diff --git a/skills/dotnet-new-app-slnx/SKILL.md b/skills/dotnet-new-app-slnx/SKILL.md index 77d3007..8cafea2 100644 --- a/skills/dotnet-new-app-slnx/SKILL.md +++ b/skills/dotnet-new-app-slnx/SKILL.md @@ -141,6 +141,8 @@ Copy every file from `assets/shared/` to the project root, preserving directory Do this as a recursive, dotfile-aware copy. Hidden folders and files under `assets/shared/` are part of the scaffold and must not be skipped. In particular, copy `assets/shared/.bot/README.md` as a real file in the generated repo; do not replace it with a synthetic `.gitkeep` or placeholder note. +If the installed skill copy omits any required file or dot-prefixed path during a `npx skills add` copy, do not treat that as success or as a template defect. Verify the generated tree against the upstream repository contents, then manually restore the missing files directly from the repository source tree, preserving the same relative paths, before continuing. That includes hidden assets such as `.bot/`, `.github/`, and any other `.`-prefixed directories under `assets/shared/`. + Do not selectively copy only "key" shared files. The intended output includes the complete shared asset inventory, including `.gitignore`, `.gitattributes`, `AGENTS.md`, `CHANGELOG.md`, `.github/`, and `.bot/`, in addition to the build and package-management files. Exception: update `testenvironments.json` with the derived `{UBUNTU_TESTRUNNER_TAG}` instead of leaving a hardcoded runner image tag in place. Keep the `WSL-Ubuntu` entry and use the Docker major-tag convention documented for the shared Ubuntu test runner images. @@ -179,6 +181,7 @@ After generating, verify: - [ ] `.slnx` references all generated src/ and test/ projects - [ ] The generated solution filename is `{SOLUTION_NAME}.slnx` with the original user-facing casing preserved - [ ] Every file from `assets/shared/` exists in the generated repo with the same relative path, including dotfiles and dotfolders such as `.gitignore`, `.gitattributes`, `.bot/README.md`, and `.github/*` +- [ ] If a `npx skills add` install omitted any dot-prefixed files, manually restore them from the same repository commit before generation continues - [ ] `Directory.Packages.props` lists all `` packages used in the solution (including host-type-specific packages) - [ ] `Directory.Packages.props` contains concrete version numbers with no unresolved `*_VERSION` placeholders - [ ] No generated `.csproj` file or `Directory.Build.props` contains ad-hoc inline `Version=` attributes for packages that are supposed to be centrally managed by `Directory.Packages.props` @@ -191,6 +194,7 @@ After generating, verify: - [ ] `.bot/` folder exists and is listed in `.gitignore` - [ ] `.bot/README.md` exists in the generated repo and came from the shared asset template - [ ] `testenvironments.json` uses the major-tag `codebeltnet/ubuntu-testrunner:{major}` convention for the selected target framework +- [ ] If a `npx skills add` install omitted any required files, verify the upstream repository contents and manually restore the missing files directly from the repository source tree before generation continues - [ ] Correct hosting pattern files generated (`Program.cs` only for Minimal, `Program.cs` + `Startup.cs` for Startup) - [ ] `Web API` is the default `web_variant` when the user asked for a generic `Web` app - [ ] `Empty Web` uses the `Web` suffix, `Web API` uses `Api`, `MVC` uses `Mvc`, and `Web App / Razor` uses `WebApp` diff --git a/skills/dotnet-new-app-slnx/references/app.md b/skills/dotnet-new-app-slnx/references/app.md index d9752c2..b77defd 100644 --- a/skills/dotnet-new-app-slnx/references/app.md +++ b/skills/dotnet-new-app-slnx/references/app.md @@ -31,6 +31,8 @@ No `.nuget/` folder or `.snk` file (uncommon for apps). Copy the complete contents of `assets/shared/` into the generated repo root, preserving relative paths. +If the installation path drops dot-prefixed entries, treat that as an incomplete copy. Verify the upstream repository contents, then manually restore the missing files directly from the repository source tree into the matching relative paths before finalizing the scaffold. + Do not cherry-pick only the files that feel essential. The shared scaffold contract includes: - `.editorconfig` diff --git a/skills/dotnet-new-lib-slnx/SKILL.md b/skills/dotnet-new-lib-slnx/SKILL.md index 305cf7f..630efe6 100644 --- a/skills/dotnet-new-lib-slnx/SKILL.md +++ b/skills/dotnet-new-lib-slnx/SKILL.md @@ -90,6 +90,8 @@ Copy every file from `assets/shared/` to the project root, preserving directory Do this as a recursive, dotfile-aware copy. Hidden folders and files under `assets/shared/` are part of the scaffold and must not be skipped. In particular, copy `assets/shared/.bot/README.md` as a real file in the generated repo; do not replace it with a synthetic `.gitkeep` or placeholder note. +If the installed skill copy omits any required file or dot-prefixed path during a `npx skills add` copy, do not treat that as success or as a template defect. Verify the generated tree against the upstream repository contents, then manually restore the missing files directly from the repository source tree, preserving the same relative paths, before continuing. That includes hidden assets such as `.bot/`, `.github/`, `.docfx/`, and any other `.`-prefixed directories under `assets/library/` or `assets/shared/`. + Preserve UTF-8 when reading, copying, and writing text files. Do not transcode templates to ANSI, OEM, Windows-1252, or any system-default code page during generation. The shared `.editorconfig` in the scaffold declares `charset = utf-8`, and generated text files should match it from the start. When a text file does not need substitutions, prefer a byte-preserving file copy instead of read/transform/write. @@ -174,6 +176,7 @@ After generating, verify: - [ ] `.github/copilot-instructions.md` has project-specific patterns - [ ] `.bot/` folder exists and is listed in `.gitignore` - [ ] `.bot/README.md` exists in the generated repo and came from the shared asset template, not from a synthetic `.gitkeep` fallback +- [ ] If a `npx skills add` install omitted any required files, verify the upstream repository contents and manually restore the missing files directly from the repository source tree before generation continues - [ ] `.github/dependabot.yml` watches the repo root so central NuGet package management stays current after scaffolding Summarize what was generated and note any manual steps (e.g. registering with SonarCloud, populating `.docfx/images/` with logo/favicon). diff --git a/skills/dotnet-new-lib-slnx/references/library.md b/skills/dotnet-new-lib-slnx/references/library.md index 73e4db4..b1c95d5 100644 --- a/skills/dotnet-new-lib-slnx/references/library.md +++ b/skills/dotnet-new-lib-slnx/references/library.md @@ -82,6 +82,8 @@ Benchmark reports are written under `reports/`, not under the `tuning/` project Write generated text files as UTF-8 and preserve UTF-8 when copying from `assets/`. +If an installer path omits dot-prefixed files from the source tree, treat that as an incomplete copy and restore the missing files manually from the upstream repository contents directly into the matching relative paths before finalizing the scaffold. + - Do not transcode Markdown, JSON, YAML, XML, or source files to ANSI, OEM, Windows-1252, or another locale-dependent encoding - Respect the shared `.editorconfig`, which already sets `charset = utf-8` - If a tool cannot reliably preserve UTF-8 when rewriting a file, prefer keeping the original bytes intact over re-saving through a lossy default encoding From faa0d8e9a2836d9bd9dc566242c4b699b18be8e2 Mon Sep 17 00:00:00 2001 From: GitHub Copilot Date: Fri, 24 Apr 2026 20:12:24 +0200 Subject: [PATCH 2/2] =?UTF-8?q?=F0=9F=A7=AD=20standardize=20hidden-asset?= =?UTF-8?q?=20recovery=20phrase?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Use one canonical hidden-asset recovery sentence in the library scaffold reference and its validator assertion so the contract stays aligned if the wording changes again. --- scripts/validate-skill-templates.ps1 | 2 +- skills/dotnet-new-lib-slnx/references/library.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/scripts/validate-skill-templates.ps1 b/scripts/validate-skill-templates.ps1 index 4e2d790..79975db 100644 --- a/scripts/validate-skill-templates.ps1 +++ b/scripts/validate-skill-templates.ps1 @@ -586,7 +586,7 @@ Add-ValidationResult -Results $results -Name 'Shared .bot assets are tracked and Assert-Contains -Name 'app .bot README' -Content $appBot -Needle '# .bot Workspace' Assert-Contains -Name 'lib .bot README' -Content $libBot -Needle '# .bot Workspace' Assert-Contains -Name 'app shared copy guidance' -Content (Get-FileText -RepoRoot $repoRoot -RelativePath 'skills/dotnet-new-app-slnx/references/app.md' -GitRef $Ref) -Needle 'manually restore the missing files directly from the repository source tree' - Assert-Contains -Name 'lib shared copy guidance' -Content (Get-FileText -RepoRoot $repoRoot -RelativePath 'skills/dotnet-new-lib-slnx/references/library.md' -GitRef $Ref) -Needle 'restore the missing files manually from the upstream repository contents' + Assert-Contains -Name 'lib shared copy guidance' -Content (Get-FileText -RepoRoot $repoRoot -RelativePath 'skills/dotnet-new-lib-slnx/references/library.md' -GitRef $Ref) -Needle 'manually restore the missing files directly from the repository source tree' } Add-ValidationResult -Results $results -Name 'Library skill documents PROJECT_NAME and DOCFX target framework' -Action { diff --git a/skills/dotnet-new-lib-slnx/references/library.md b/skills/dotnet-new-lib-slnx/references/library.md index b1c95d5..7f4b464 100644 --- a/skills/dotnet-new-lib-slnx/references/library.md +++ b/skills/dotnet-new-lib-slnx/references/library.md @@ -82,7 +82,7 @@ Benchmark reports are written under `reports/`, not under the `tuning/` project Write generated text files as UTF-8 and preserve UTF-8 when copying from `assets/`. -If an installer path omits dot-prefixed files from the source tree, treat that as an incomplete copy and restore the missing files manually from the upstream repository contents directly into the matching relative paths before finalizing the scaffold. +If an installer path omits dot-prefixed files from the source tree, treat that as an incomplete copy. Verify the upstream repository contents, then manually restore the missing files directly from the repository source tree into the matching relative paths before finalizing the scaffold. - Do not transcode Markdown, JSON, YAML, XML, or source files to ANSI, OEM, Windows-1252, or another locale-dependent encoding - Respect the shared `.editorconfig`, which already sets `charset = utf-8`