From bc1641addec38476abb79def6c0a95bd63679a69 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Crist=C3=B3v=C3=A3o=20GUIHO?= Date: Sat, 12 Sep 2026 11:35:44 +0000 Subject: [PATCH 01/20] Plan explicit documentation authorization and complete trees --- .../explicit-documentation-authorization.md | 196 ++++++++++++++++++ 1 file changed, 196 insertions(+) create mode 100644 docs/plans/explicit-documentation-authorization.md diff --git a/docs/plans/explicit-documentation-authorization.md b/docs/plans/explicit-documentation-authorization.md new file mode 100644 index 0000000..bd4f313 --- /dev/null +++ b/docs/plans/explicit-documentation-authorization.md @@ -0,0 +1,196 @@ +--- +name: Explicit documentation authorization and complete trees +purpose: Plan issue 21 implementation without unsolicited Markdown changes. +description: Scoped requirements, architecture, execution units, and acceptance checks for opt-in documentation and complete descriptor discovery. +created: 2026-09-12 +owner: xdocs-plans +flags: [execution-authorized] +tags: [planning, authorization, tree] +keywords: [issue 21, named descriptors, frontmatter, Luna, Sol] +--- + +#### © 2026 [GUIHO](https://guiho.co) as represented by [Cristóvão GUIHO](https://guiho.co/cguiho) All Rights Reserved. + +# Explicit documentation authorization and complete trees + +## Authority and scope + +Source: [GitHub issue 21](https://github.com/CGuiho/xdocs/issues/21), created +2026-09-11, and the user's 2026-09-12 clarification in this session. +The user explicitly directed the coordinator to write the plan, hand the full +plan to Luna, have Sol review Luna's work, review the integrated result, and +push to GitHub. This authorizes this scoped planning-to-execution cycle without +another approval round. It supersedes the old migration plan's worker choices +and the issue's ambiguous use of bare `.xdocs.md`. + +The primary agent authors this plan and owns user communication and delivery. +Implementation uses `gpt-5.6-luna` at `max`; every Luna contribution receives +`gpt-5.6-sol` review at `xhigh`. Optional Terra uses `xhigh`. +Native collaboration tools are the user-selected worker mechanism. With four +concurrent slots, Luna schedules bounded child work and releases slots before +Sol reviews. Subagents commit only their own scoped changes and never push. + +Keep this work separate from the unfinished broad CLI Convention 0001 rewrite. +No version bump, release publication, installer migration, or issue comment is +authorized here. Preserve historical plans and existing user documentation. +The referenced Go CLI skill is unavailable locally; use existing Go patterns, +repository requirements, and the applicable GUIHO CLI convention. The local +general SWE convention file is empty, so it supplies no additional rules. + +## Required behavior and architecture + +1. A directory has at most one named descriptor, normally its basename plus + `.xdocs.md`: `technologies/technologies.xdocs.md`. Bare `.xdocs.md` and + `.docs.md` are not valid descriptor names. Reuse existing valid named + descriptors; do not rename, delete, consolidate, or overwrite existing + documents automatically. Report duplicate descriptors as errors. +2. Each descriptor contains meaningful directory context, file/document + descriptions, and parent/child metadata. Put overview and details in that + descriptor, not separate generated overview/detail Markdown files. +3. Add a strict `documentation` configuration section. `directories` is a + list of repository-relative directory paths, default `[]`. Each entry + authorizes descriptor maintenance in that directory and descendants; + `.` deliberately authorizes the entire non-excluded project. Reject empty, + absolute, escaping, or malformed entries. Missing permission never implies + repository-wide permission. Do not auto-populate this list during init. +4. `documentation.frontmatter` is a list of explicit rules with `pattern` + and `kind` (`file` or `directory`), default `[]`. These authorize companion + Markdown metadata only within `documentation.directories`. Use existing + validated glob semantics; legacy `ignore.rules` with `frontmatter: false` + always deny and take precedence. Existing configuration remains loadable, + but omitted opt-ins now mean ordinary Markdown metadata is not required. + `ai.mode: auto` controls timing only, never broadens authorization. +5. Ordinary Markdown remains discoverable and describable in a descriptor's + `documents` map without opening, rewriting, or requiring its frontmatter. + Keep metadata-policy output explicit (`frontmatterRequired: false`). Do not + create extra companion documents or advise header insertion unless the + user expressly permits the exact scope. Preserve existing frontmatter, + bodies, and files; do not strip historical metadata. +6. Discovery is read-only and independent of write permission. `tree` walks all + non-excluded directories to arbitrary depth, displays every named + descriptor exactly once with its relative path and available context, and + cannot silently lose orphans, multiple roots, duplicate subjects, or invalid + descriptors. Build the displayed hierarchy from directory containment and + nearest ancestor descriptors, using a synthetic project root when needed. + Keep metadata relationship validation in doctor; tree emits clear + diagnostics by default without hiding nodes or producing cyclic JSON. + Respect root/nested gitignore and scan exclusions, including file patterns. +7. `scan`, `meta`, `context`, `doctor`, and `tree` without `--output` never + mutate files. `generate` and `merge` remain report-producing commands, + printing to stdout unless an explicit output destination is supplied. + An explicit ordinary report destination authorizes that report, not + companion metadata edits or additional output files. Do not pretend a + rendered report without descriptor metadata is a valid descriptor. +8. Centralize safe output checks used by generate/merge/tree. Descriptor + destinations require directory permission, valid complete descriptor + metadata, and no second descriptor in the directory. Reject bare/legacy + descriptor destinations and malformed frontmatter before any write; + ordinary Markdown frontmatter destinations require explicit policy. + Reject symlink escapes/targets and excluded writes. Preflight before + truncation; use safe atomic replacement. No silent frontmatter repair. + Doctor reports existing invalid metadata; actual repair remains explicit. +9. Keep existing explicit init/root-index and agent-resource setup boundaries + intact; they do not authorize arbitrary Markdown metadata maintenance. + Existing `XDOCS.md` is the special root index, not a second per-directory + descriptor. Do not create additional indexes as part of discovery or + descriptor maintenance. + +Example user-authored policy: + +```yaml +documentation: + directories: + - technologies + frontmatter: + - pattern: technologies/approved-notes.md + kind: file +``` + +This allows named descriptors beneath `technologies`, and companion metadata +only for the listed file unless a legacy denial matches. It does not authorize +README header edits. For this implementation task, the user's requested source, +plan, documentation, tests, and descriptor maintenance are authorized; do not +add blanket permissions to the repository configuration to conceal failures. + +## Execution units + +### U1 — Configuration and read policy + +Files: `internal/config/config.go`, its tests and descriptor; +`internal/xdocs/ignore.go`, discovery/meta/doctor policy adapters and tests. +Implement typed strict defaults, decoding and semantic validation. Share policy +matching across readers/writers; keep ignores stronger than opt-ins. Update +tests that intentionally require companion metadata to opt in explicitly. +Validate default/no-config behavior, malformed rules, nested allow/deny, +directory boundaries, root grants, scoped metadata and explicit file exclusions. +Signal: focused config/domain tests, then Sol review. No external state impact. + +### U2 — Complete tree + +Independent of U1's policy implementation. Files: `internal/xdocs/tree.go`, +tree model only as necessary, dedicated tree tests, `cmd/domain.go` tree adapter +and command tests. Coordinate shared command/model files with Luna before edits. +Retain every discovered descriptor even when invalid; avoid duplicate-subject +map loss and metadata cycles; show paths and meaningful available descriptions. +Validate arbitrary depth, forests, orphans, duplicate subjects, malformed YAML, +cycles, missing ancestor descriptors, stable order and text/Markdown/JSON parity. +Signal: focused domain/command tests and Sol review. + +### U3 — Output authorization and validation + +Depends on U1; serialize edits to shared command/domain files after U2. +Files: shared output helper in `internal/xdocs` or `cmd`, `cmd/domain.go`, +generation/merge rendering only as needed, focused output tests and descriptors. +Validate exact single-file output, denied descriptor write, duplicate prevention, +malformed/invalid frontmatter, existing-target byte preservation on rejection, +symlink escape, excludes, stdout no writes, and explicitly requested plain report. +Reject invalid descriptor reports instead of inventing their metadata. +Signal: mutation regression tests, focused command tests and Sol review. + +### U4 — Embedded instructions and user documentation + +Depends on settled U1–U3 behavior. Files: `skills/guiho-s-xdocs/SKILL.md`, +`prompts/write.md`, `prompts/update.md`, `prompts/generate.md`, +`prompts/agents.md`, `prompts/guiho-i-xdocs.md` where applicable, +`internal/agent/agent.go` instruction template, `README.md`, `DOCS.md`, +`AGENTS.md` relevant policy prose, and owning descriptors. +Explicitly replace broad header and extra-file advice with named single-file +context and opt-in policy. Preserve unmanaged instruction content and skill +version. Do not insert YAML frontmatter into README/AGENTS or any opted-out +document. Test actual embedded resources and the managed instruction template +for policy consistency. Signal: resource tests, documentation review by Sol. + +### U5 — Integration and delivery + +Primary owns integration review and final push; Luna owns implementation fixes. +Run `gofmt` on changed Go files, `go test ./...`, `go vet ./...`, and +`CGO_ENABLED=0 go build -trimpath` into an isolated temporary output directory. +Use the newly built CLI with temporary project/home fixtures for smoke tests; +never bootstrap or update the user's global installation. Validate real repository +metadata/tree/doctor and compare any unrelated baseline failures honestly. +Run Linux-native checks; cross-platform compilation is not native validation. +Sol reviews the complete Luna diff and every corrective change. Primary verifies +acceptance against source and test evidence, then pushes the reviewed commits. +If a protected-branch gate rejects delivery, use a reviewable feature branch/PR; +never force or bypass a rejection. Do not publish a release. + +## Tracking, findings, and acceptance + +- [ ] U1 typed opt-in policy and read behavior implemented and reviewed. +- [ ] U2 complete deep tree implemented and reviewed. +- [ ] U3 guarded output implemented and reviewed. +- [ ] U4 embedded guidance and documentation consistent and reviewed. +- [ ] U5 full checks, integrated review, and GitHub push verified. + +Use this plan's checklist for executor progress and add one concise task to the +existing TODO index. Reuse existing documentation directories instead of +creating an unnecessary hierarchy of planning files. Record substantive +unforeseen decisions under `docs/questions/explicit-documentation-authorization/` +only when needed. Keep the final review and validation evidence in one focused +record under `docs/reviews/implementation/` with acceptance mapping and commands. +Never claim human acceptance before the user tests the result. + +Hard stops: potential unrecoverable data loss, secret access, or required +production mutation. Resolve routine implementation choices conservatively, +record them, and continue. No schema or output policy may silently expand +the user's authorization. From 5d8d1ce8c7b9dfc28beb713aa5bdfb34e5aadba6 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Crist=C3=B3v=C3=A3o=20GUIHO?= Date: Sat, 12 Sep 2026 11:37:54 +0000 Subject: [PATCH 02/20] Clarify legacy header audit and root index tree coverage --- docs/plans/explicit-documentation-authorization.md | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/docs/plans/explicit-documentation-authorization.md b/docs/plans/explicit-documentation-authorization.md index bd4f313..bec3f4a 100644 --- a/docs/plans/explicit-documentation-authorization.md +++ b/docs/plans/explicit-documentation-authorization.md @@ -74,7 +74,8 @@ general SWE convention file is empty, so it supplies no additional rules. nearest ancestor descriptors, using a synthetic project root when needed. Keep metadata relationship validation in doctor; tree emits clear diagnostics by default without hiding nodes or producing cyclic JSON. - Respect root/nested gitignore and scan exclusions, including file patterns. + Include the existing special root `XDOCS.md` index as a distinguishable path + entry. Respect root/nested gitignore and scan exclusions, including file patterns. 7. `scan`, `meta`, `context`, `doctor`, and `tree` without `--output` never mutate files. `generate` and `merge` remain report-producing commands, printing to stdout unless an explicit output destination is supplied. @@ -88,7 +89,12 @@ general SWE convention file is empty, so it supplies no additional rules. ordinary Markdown frontmatter destinations require explicit policy. Reject symlink escapes/targets and excluded writes. Preflight before truncation; use safe atomic replacement. No silent frontmatter repair. - Doctor reports existing invalid metadata; actual repair remains explicit. + Doctor reports existing invalid descriptor/authorized metadata. Add an + explicit read-only `doctor --existing-frontmatter` audit for malformed + historical headers on otherwise unauthorized ordinary Markdown: missing + headers remain valid, existing headers get YAML syntax/structure validation, + and no xdocs owner schema or write authorization is imposed. Actual repair + remains explicit. 9. Keep existing explicit init/root-index and agent-resource setup boundaries intact; they do not authorize arbitrary Markdown metadata maintenance. Existing `XDOCS.md` is the special root index, not a second per-directory From 1bc974538cd80103bf541cb914f1dfcd6763545f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Crist=C3=B3v=C3=A3o=20GUIHO?= Date: Sat, 12 Sep 2026 11:50:58 +0000 Subject: [PATCH 03/20] Add complete XDocs tree coverage tests --- internal/xdocs/tree_test.go | 268 ++++++++++++++++++++++++++++++++++++ 1 file changed, 268 insertions(+) create mode 100644 internal/xdocs/tree_test.go diff --git a/internal/xdocs/tree_test.go b/internal/xdocs/tree_test.go new file mode 100644 index 0000000..bf3887b --- /dev/null +++ b/internal/xdocs/tree_test.go @@ -0,0 +1,268 @@ +package xdocs + +import ( + "encoding/json" + "path/filepath" + "reflect" + "sort" + "strings" + "testing" +) + +func TestBuildTreeContainsEveryDiscoveredDescriptorByDirectory(t *testing.T) { + root := t.TempDir() + missingParent := "missing-parent" + files := []File{ + treeTestFile(root, "XDOCS.md", false, nil), + treeTestFile(root, "technologies/technologies.xdocs.md", true, treeTestMetadata("technologies", "Technology directory context.", nil)), + treeTestFile(root, "technologies/cloud/cloud.xdocs.md", true, treeTestMetadata("cloud", "Cloud directory context.", treeStringPointer("technologies"))), + treeTestFile(root, "technologies/cloud/observability/observability.xdocs.md", true, treeTestMetadata("observability", "Observability directory context.", treeStringPointer("cloud"))), + treeTestFile(root, "technologies/cloud/observability/runtime/runtime.xdocs.md", true, treeTestMetadata("runtime", "Runtime directory context.", treeStringPointer("observability"))), + treeTestFile(root, "technologies/cloud/broken/broken.xdocs.md", false, nil, "Missing YAML frontmatter."), + treeTestFile(root, "orphan/deep/orphan.xdocs.md", true, treeTestMetadata("orphan", "Orphan directory context.", &missingParent)), + treeTestFile(root, "duplicates/first/duplicate.xdocs.md", true, treeTestMetadata("duplicate", "First duplicate subject.", nil)), + treeTestFile(root, "duplicates/second/duplicate.xdocs.md", true, treeTestMetadata("duplicate", "Second duplicate subject.", nil)), + treeTestFile(root, "legacy/.xdocs.md", false, nil, `Invalid xdocs descriptor filename.`), + } + + tree := BuildTree(files) + nodes := treeTestNodes(tree) + if got, want := len(nodes), len(files)+1; got != want { + t.Fatalf("tree node count = %d, want synthetic root plus %d discovered entries", got, len(files)) + } + + byPath := map[string][]*TreeNode{} + for _, node := range nodes { + if node.Path != nil { + byPath[*node.Path] = append(byPath[*node.Path], node) + } + } + expectedPaths := make([]string, 0, len(files)) + for _, file := range files { + expectedPaths = append(expectedPaths, file.RelativePath) + } + sort.Strings(expectedPaths) + for _, path := range expectedPaths { + if got := len(byPath[path]); got != 1 { + t.Fatalf("tree path %q appears %d times, want exactly once", path, got) + } + } + + parentByPath := treeTestParentPaths(tree) + wantParents := map[string]string{ + "XDOCS.md": "(root)", + "technologies/technologies.xdocs.md": "(root)", + "technologies/cloud/cloud.xdocs.md": "technologies/technologies.xdocs.md", + "technologies/cloud/observability/observability.xdocs.md": "technologies/cloud/cloud.xdocs.md", + "technologies/cloud/observability/runtime/runtime.xdocs.md": "technologies/cloud/observability/observability.xdocs.md", + "technologies/cloud/broken/broken.xdocs.md": "technologies/cloud/cloud.xdocs.md", + "orphan/deep/orphan.xdocs.md": "(root)", + "duplicates/first/duplicate.xdocs.md": "(root)", + "duplicates/second/duplicate.xdocs.md": "(root)", + "legacy/.xdocs.md": "(root)", + } + for path, wantParent := range wantParents { + if got := parentByPath[path]; got != wantParent { + t.Errorf("parent of %q = %q, want %q", path, got, wantParent) + } + } + + index := byPath["XDOCS.md"][0] + if index.Kind != "index" { + t.Errorf("root index kind = %q, want index", index.Kind) + } + if index.Subject != "XDOCS.md" { + t.Errorf("root index subject = %q, want XDOCS.md", index.Subject) + } + if index.Valid { + t.Error("special root index should preserve its non-descriptor validity state") + } + + invalid := byPath["technologies/cloud/broken/broken.xdocs.md"][0] + if invalid.Kind != "descriptor" || invalid.Valid { + t.Fatalf("invalid descriptor was not retained as an invalid descriptor: %#v", invalid) + } + if !strings.Contains(invalid.Description, "Missing YAML frontmatter") { + t.Errorf("invalid descriptor lost its diagnostic context: %q", invalid.Description) + } + + orphan := byPath["orphan/deep/orphan.xdocs.md"][0] + if orphan.Description != "Orphan directory context." { + t.Errorf("orphan descriptor lost its available description: %q", orphan.Description) + } + if orphan.Valid != true { + t.Error("tree display should preserve a parsed orphan descriptor as available/valid metadata") + } +} + +func TestBuildTreeRetainsDuplicateSubjectsAndSemanticCyclesWithoutJSONCycles(t *testing.T) { + root := t.TempDir() + alphaParent := "beta" + betaParent := "alpha" + files := []File{ + treeTestFile(root, "alpha/alpha.xdocs.md", true, treeTestMetadata("alpha", "Alpha context.", &alphaParent)), + treeTestFile(root, "beta/beta.xdocs.md", true, treeTestMetadata("beta", "Beta context.", &betaParent)), + treeTestFile(root, "same/one.xdocs.md", true, treeTestMetadata("same", "First same subject.", nil)), + treeTestFile(root, "same/two.xdocs.md", true, treeTestMetadata("same", "Second same subject.", nil)), + treeTestFile(root, "invalid/deeper/invalid.xdocs.md", false, nil, "Invalid YAML frontmatter."), + } + + tree := BuildTree(files) + nodes := treeTestNodes(tree) + paths := make([]string, 0, len(files)) + for _, node := range nodes { + if node.Path != nil { + paths = append(paths, *node.Path) + } + } + sort.Strings(paths) + wantPaths := make([]string, 0, len(files)) + for _, file := range files { + wantPaths = append(wantPaths, file.RelativePath) + } + sort.Strings(wantPaths) + if !reflect.DeepEqual(paths, wantPaths) { + t.Fatalf("tree paths = %#v, want %#v", paths, wantPaths) + } + + validation := ValidateTree(files) + if validation.Valid { + t.Fatal("semantic cycle and duplicate subjects unexpectedly passed tree validation") + } + validationErrors := strings.Join(validation.Errors, "\n") + for _, want := range []string{"Duplicate subject", "Tree cycle detected"} { + if !strings.Contains(validationErrors, want) { + t.Errorf("validation errors omitted %q: %s", want, validationErrors) + } + } + + data, err := json.Marshal(tree) + if err != nil { + t.Fatalf("tree JSON contains a cycle: %v", err) + } + var decoded TreeNode + if err := json.Unmarshal(data, &decoded); err != nil { + t.Fatalf("tree JSON could not be decoded: %v", err) + } + if got := len(treeTestNodes(&decoded)); got != len(files)+1 { + t.Fatalf("decoded tree node count = %d, want %d", got, len(files)+1) + } +} + +func TestTreeRenderersAreDeterministicAndExposeAllPathsAndContext(t *testing.T) { + root := t.TempDir() + files := []File{ + treeTestFile(root, "zeta/zeta.xdocs.md", true, treeTestMetadata("zeta", "Zeta context.", nil)), + treeTestFile(root, "alpha/alpha.xdocs.md", true, treeTestMetadata("alpha", "Alpha context.", nil)), + treeTestFile(root, "alpha/deep/broken.xdocs.md", false, nil, "Missing YAML frontmatter."), + treeTestFile(root, "XDOCS.md", false, nil), + } + reversed := append([]File(nil), files...) + for left, right := 0, len(reversed)-1; left < right; left, right = left+1, right-1 { + reversed[left], reversed[right] = reversed[right], reversed[left] + } + + first := BuildTree(files) + second := BuildTree(reversed) + textFirst, textSecond := RenderTree(first), RenderTree(second) + markdownFirst, markdownSecond := RenderTreeMarkdown(first), RenderTreeMarkdown(second) + jsonFirstBytes, err := json.Marshal(first) + if err != nil { + t.Fatal(err) + } + jsonSecondBytes, err := json.Marshal(second) + if err != nil { + t.Fatal(err) + } + if textFirst != textSecond || markdownFirst != markdownSecond || string(jsonFirstBytes) != string(jsonSecondBytes) { + t.Fatalf("tree renderers are input-order dependent:\ntext:\n%s\n---\n%s\nmarkdown:\n%s\n---\n%s\njson:\n%s\n---\n%s", textFirst, textSecond, markdownFirst, markdownSecond, jsonFirstBytes, jsonSecondBytes) + } + + for _, path := range []string{"XDOCS.md", "alpha/alpha.xdocs.md", "alpha/deep/broken.xdocs.md", "zeta/zeta.xdocs.md"} { + for format, output := range map[string]string{"text": textFirst, "markdown": markdownFirst, "json": string(jsonFirstBytes)} { + if !strings.Contains(output, path) { + t.Errorf("%s tree omitted path %q:\n%s", format, path, output) + } + } + } + for _, description := range []string{"Alpha context.", "Zeta context.", "Missing YAML frontmatter."} { + for format, output := range map[string]string{"markdown": markdownFirst, "json": string(jsonFirstBytes)} { + if !strings.Contains(output, description) { + t.Errorf("%s tree omitted available context %q:\n%s", format, description, output) + } + } + } +} + +func treeTestFile(root, relativePath string, valid bool, metadata *Metadata, errors ...string) File { + path := filepath.Join(root, filepath.FromSlash(relativePath)) + return File{ + Path: path, + RelativePath: filepath.ToSlash(relativePath), + Directory: filepath.Dir(path), + Metadata: metadata, + Valid: valid, + Errors: append([]string(nil), errors...), + Documents: []MarkdownDocument{}, + } +} + +func treeTestMetadata(subject, description string, parent *string) *Metadata { + return &Metadata{ + Subject: subject, + Description: description, + Parent: parent, + Children: []string{}, + Files: map[string]string{}, + Documents: map[string]string{}, + Tags: []string{}, + Keywords: []string{}, + Flags: []string{}, + } +} + +func treeTestNodes(root *TreeNode) []*TreeNode { + var result []*TreeNode + seen := map[*TreeNode]bool{} + var visit func(*TreeNode) + visit = func(node *TreeNode) { + if node == nil || seen[node] { + return + } + seen[node] = true + result = append(result, node) + for _, child := range node.Children { + visit(child) + } + } + visit(root) + return result +} + +func treeTestParentPaths(root *TreeNode) map[string]string { + result := map[string]string{} + seen := map[*TreeNode]bool{} + var visit func(*TreeNode, string) + visit = func(node *TreeNode, parent string) { + if node == nil || seen[node] { + return + } + seen[node] = true + if node.Path != nil { + result[*node.Path] = parent + } + label := "(root)" + if node.Path != nil { + label = *node.Path + } + for _, child := range node.Children { + visit(child, label) + } + } + visit(root, "") + return result +} + +func treeStringPointer(value string) *string { + return &value +} From 5fbcfc8a761b7f45f4c256e9a5160d2a17dbf110 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Crist=C3=B3v=C3=A3o=20GUIHO?= Date: Sat, 12 Sep 2026 11:51:12 +0000 Subject: [PATCH 04/20] Add explicit documentation authorization policy --- cmd/domain.go | 68 +++++---- internal/config/config.go | 253 +++++++++++++++++++++++++++++---- internal/config/config_test.go | 69 +++++++++ internal/xdocs/discovery.go | 45 ++++-- internal/xdocs/doctor.go | 8 +- internal/xdocs/ignore.go | 103 ++++++++++++-- internal/xdocs/ignore_test.go | 8 ++ internal/xdocs/meta.go | 40 ++++-- internal/xdocs/metadata.go | 82 +++++++++-- internal/xdocs/model.go | 35 +++-- internal/xdocs/output.go | 153 ++++++++++++++++++++ internal/xdocs/xdocs_test.go | 4 +- 12 files changed, 738 insertions(+), 130 deletions(-) create mode 100644 internal/xdocs/output.go diff --git a/cmd/domain.go b/cmd/domain.go index eff5789..131b3ee 100644 --- a/cmd/domain.go +++ b/cmd/domain.go @@ -103,6 +103,9 @@ func newScanCommand(options *commonOptions) *cobra.Command { fmt.Fprintf(out, "covered directories: %d\n", result.CoveredDirectories) fmt.Fprintf(out, "uncovered directories: %d\n", result.UncoveredDirectories) fmt.Fprintf(out, "xdocs descriptors found: %d\n", len(result.XDocsFiles)) + for _, message := range result.Errors { + fmt.Fprintln(command.ErrOrStderr(), "error: "+message) + } if len(result.XDocsFiles) > 0 { fmt.Fprintln(out, "\nfiles:") for _, file := range result.XDocsFiles { @@ -162,6 +165,7 @@ type scanJSONResult struct { XDocsFiles []scanJSONFile `json:"xdocsFiles"` MarkdownDocuments []string `json:"markdownDocuments"` UncoveredPaths []string `json:"uncoveredPaths"` + Errors []string `json:"errors"` } func scanJSON(result domain.ScanResult) scanJSONResult { @@ -171,6 +175,7 @@ func scanJSON(result domain.ScanResult) scanJSONResult { CoveredDirectories: result.CoveredDirectories, UncoveredDirectories: result.UncoveredDirectories, XDocsFiles: []scanJSONFile{}, MarkdownDocuments: []string{}, UncoveredPaths: append([]string{}, result.UncoveredPaths...), + Errors: append([]string{}, result.Errors...), } for _, file := range result.XDocsFiles { entry := scanJSONFile{ @@ -219,11 +224,7 @@ func newGenerateCommand(options *commonOptions) *cobra.Command { return err } if output != "" { - path := output - if !filepath.IsAbs(path) { - path = filepath.Join(options.cwd, path) - } - if err := writeFile(path, []byte(content), 0o644); err != nil { + if err := domain.WriteReport(cfg, output, []byte(content)); err != nil { return err } if options.format == "json" { @@ -280,11 +281,7 @@ func newMergeCommand(options *commonOptions) *cobra.Command { return nil } if output != "" { - path := output - if !filepath.IsAbs(path) { - path = filepath.Join(options.cwd, path) - } - if err := writeFile(path, []byte(content), 0o644); err != nil { + if err := domain.WriteReport(cfg, output, []byte(content)); err != nil { return err } if options.format == "json" { @@ -326,14 +323,15 @@ func newTreeCommand(options *commonOptions) *cobra.Command { return err } tree := domain.BuildTree(scan.XDocsFiles) - if options.verbose { - validation := domain.ValidateTree(scan.XDocsFiles) - for _, message := range validation.Warnings { - fmt.Fprintln(command.ErrOrStderr(), "warning: "+message) - } - for _, message := range validation.Errors { - fmt.Fprintln(command.ErrOrStderr(), "error: "+message) - } + for _, message := range scan.Errors { + fmt.Fprintln(command.ErrOrStderr(), "error: "+message) + } + validation := domain.ValidateTree(scan.XDocsFiles) + for _, message := range validation.Warnings { + fmt.Fprintln(command.ErrOrStderr(), "warning: "+message) + } + for _, message := range validation.Errors { + fmt.Fprintln(command.ErrOrStderr(), "error: "+message) } var content string switch options.format { @@ -349,11 +347,7 @@ func newTreeCommand(options *commonOptions) *cobra.Command { content = domain.RenderTree(tree) + "\n" } if output != "" { - path := output - if !filepath.IsAbs(path) { - path = filepath.Join(options.cwd, path) - } - if err := writeFile(path, []byte(content), 0o644); err != nil { + if err := domain.WriteReport(cfg, output, []byte(content)); err != nil { return err } fmt.Fprintf(command.OutOrStdout(), "tree: %s\n", output) @@ -405,7 +399,7 @@ func newListCommand(options *commonOptions) *cobra.Command { } func newMetaCommand(options *commonOptions) *cobra.Command { - var includeDocuments, strict bool + var includeDocuments, existingFrontmatter, strict bool var owner, tag, keyword string command := &cobra.Command{ Use: "meta [path]", @@ -421,7 +415,7 @@ func newMetaCommand(options *commonOptions) *cobra.Command { target = args[0] } result, err := domain.ScanMetadata(cfg, domain.MetaOptions{ - TargetPath: target, IncludeDocuments: includeDocuments, Strict: strict, + TargetPath: target, IncludeDocuments: includeDocuments, ExistingFrontmatter: existingFrontmatter, Strict: strict, Filters: domain.Filters{Owner: owner, Tag: tag, Keyword: keyword}, }) if err != nil { @@ -435,6 +429,7 @@ func newMetaCommand(options *commonOptions) *cobra.Command { } flags := command.Flags() flags.BoolVar(&includeDocuments, "documents", false, "Include companion document frontmatter") + flags.BoolVar(&existingFrontmatter, "existing-frontmatter", false, "Audit existing companion frontmatter without requiring it") flags.BoolVar(&strict, "strict", false, "Fail when metadata is invalid") flags.StringVar(&owner, "owner", "", "Filter by descriptor subject or document owner") flags.StringVar(&tag, "tag", "", "Filter by tag") @@ -481,7 +476,7 @@ func newContextCommand(options *commonOptions) *cobra.Command { } func newDoctorCommand(options *commonOptions) *cobra.Command { - var noDocuments, warningsAsErrors bool + var noDocuments, existingFrontmatter, warningsAsErrors bool command := &cobra.Command{ Use: "doctor [path]", Short: "Run strict xdocs health checks.", @@ -496,7 +491,7 @@ func newDoctorCommand(options *commonOptions) *cobra.Command { target = args[0] } result, err := domain.Doctor(cfg, domain.DoctorOptions{ - TargetPath: target, IncludeDocuments: !noDocuments, WarningsAsErrors: warningsAsErrors, + TargetPath: target, IncludeDocuments: !noDocuments, ExistingFrontmatter: existingFrontmatter, WarningsAsErrors: warningsAsErrors, }) if err != nil { return err @@ -511,6 +506,7 @@ func newDoctorCommand(options *commonOptions) *cobra.Command { }, } command.Flags().BoolVar(&noDocuments, "no-documents", false, "Skip companion-document validation") + command.Flags().BoolVar(&existingFrontmatter, "existing-frontmatter", false, "Audit existing companion frontmatter without requiring it") command.Flags().BoolVar(&warningsAsErrors, "warnings-as-errors", false, "Treat warnings as errors") return command } @@ -670,19 +666,21 @@ type metaJSONDescriptor struct { } type metaJSONResult struct { - Root string `json:"root"` - TargetPath string `json:"targetPath"` - IncludeDocuments bool `json:"includeDocuments"` - Strict bool `json:"strict"` - Filters domain.Filters `json:"filters"` - Descriptors []metaJSONDescriptor `json:"descriptors"` - Errors []string `json:"errors"` + Root string `json:"root"` + TargetPath string `json:"targetPath"` + IncludeDocuments bool `json:"includeDocuments"` + ExistingFrontmatter bool `json:"existingFrontmatter"` + Strict bool `json:"strict"` + Filters domain.Filters `json:"filters"` + Descriptors []metaJSONDescriptor `json:"descriptors"` + Errors []string `json:"errors"` } func metaJSON(result domain.MetaResult) metaJSONResult { projected := metaJSONResult{ Root: result.Root, TargetPath: result.TargetPath, - IncludeDocuments: result.IncludeDocuments, Strict: result.Strict, Filters: result.Filters, + IncludeDocuments: result.IncludeDocuments, ExistingFrontmatter: result.ExistingFrontmatter, + Strict: result.Strict, Filters: result.Filters, Descriptors: []metaJSONDescriptor{}, Errors: append([]string{}, result.Errors...), } for _, descriptor := range result.Descriptors { diff --git a/internal/config/config.go b/internal/config/config.go index 2d4607c..cda6272 100644 --- a/internal/config/config.go +++ b/internal/config/config.go @@ -1,12 +1,14 @@ package config import ( + "bytes" "errors" "fmt" "io" "os" "path" "path/filepath" + "runtime" "strings" "github.com/CGuiho/xdocs/internal/apperror" @@ -28,12 +30,13 @@ var ( ) type rawConfig struct { - Schema yaml.Node `yaml:"schema"` - Extensions *extensions `yaml:"extensions"` - AI *ai `yaml:"ai"` - Ignore *ignore `yaml:"ignore"` - Scan *scan `yaml:"scan"` - Project *project `yaml:"project"` + Schema yaml.Node `yaml:"schema"` + Extensions *extensions `yaml:"extensions"` + AI *ai `yaml:"ai"` + Documentation *documentation `yaml:"documentation"` + Ignore *ignore `yaml:"ignore"` + Scan *scan `yaml:"scan"` + Project *project `yaml:"project"` } type extensions struct { @@ -44,6 +47,16 @@ type ai struct { Mode string `yaml:"mode"` } +type documentation struct { + Directories []string `yaml:"directories"` + Frontmatter []documentationRule `yaml:"frontmatter"` +} + +type documentationRule struct { + Pattern string `yaml:"pattern"` + Kind string `yaml:"kind"` +} + type ignore struct { Gitignore *bool `yaml:"gitignore"` Rules *[]ignoreRule `yaml:"rules"` @@ -69,16 +82,31 @@ type IgnoreRule struct { Frontmatter bool } +// DocumentationRule authorizes companion-document frontmatter within an +// explicitly authorized documentation directory. +type DocumentationRule struct { + Pattern string + Kind string +} + +// DocumentationConfig contains the opt-in write and metadata policy. Empty +// values are deliberate: discovering Markdown never grants write permission. +type DocumentationConfig struct { + Directories []string + Frontmatter []DocumentationRule +} + type Config struct { - Schema int - CWD string - Path string - Extensions []string - AIMode string - Gitignore bool - IgnoreRules []IgnoreRule - Exclude []string - Project string + Schema int + CWD string + Path string + Extensions []string + AIMode string + Documentation DocumentationConfig + Gitignore bool + IgnoreRules []IgnoreRule + Exclude []string + Project string } func Defaults(cwd string) (Config, error) { @@ -87,10 +115,14 @@ func Defaults(cwd string) (Config, error) { return Config{}, apperror.Wrap(apperror.Configuration, "resolve working directory", err) } return Config{ - Schema: 1, - CWD: absolute, - Extensions: append([]string(nil), defaultExtensions...), - AIMode: defaultAIMode, + Schema: 1, + CWD: absolute, + Extensions: append([]string(nil), defaultExtensions...), + AIMode: defaultAIMode, + Documentation: DocumentationConfig{ + Directories: []string{}, + Frontmatter: []DocumentationRule{}, + }, Gitignore: true, IgnoreRules: cloneIgnoreRules(defaultIgnoreRules), Exclude: append([]string(nil), defaultExclude...), @@ -188,7 +220,14 @@ func decode(path string) (Config, error) { } defer file.Close() - decoder := yaml.NewDecoder(file) + content, err := io.ReadAll(file) + if err != nil { + return Config{}, apperror.Wrap(apperror.Configuration, "read xdocs YAML configuration", err) + } + if err := validateDocumentationYAML(content); err != nil { + return Config{}, err + } + decoder := yaml.NewDecoder(bytes.NewReader(content)) decoder.KnownFields(true) var raw rawConfig if err := decoder.Decode(&raw); err != nil { @@ -228,6 +267,31 @@ func decode(path string) (Config, error) { if aiMode != "prompt" && aiMode != "auto" { return Config{}, apperror.New(apperror.Configuration, `invalid ai.mode: expected "prompt" or "auto"`) } + documentationPolicy := DocumentationConfig{ + Directories: []string{}, + Frontmatter: []DocumentationRule{}, + } + if raw.Documentation != nil { + for index, directory := range raw.Documentation.Directories { + validated, err := validateDocumentationDirectory(index, directory) + if err != nil { + return Config{}, err + } + for _, existing := range documentationPolicy.Directories { + if samePath(existing, validated) { + return Config{}, apperror.New(apperror.Configuration, fmt.Sprintf("documentation.directories[%d] duplicates %q", index, validated)) + } + } + documentationPolicy.Directories = append(documentationPolicy.Directories, validated) + } + for index, rule := range raw.Documentation.Frontmatter { + validated, err := validateDocumentationRule(index, rule) + if err != nil { + return Config{}, err + } + documentationPolicy.Frontmatter = append(documentationPolicy.Frontmatter, validated) + } + } gitignore := true ignoreRules := cloneIgnoreRules(defaultIgnoreRules) if raw.Ignore != nil { @@ -262,16 +326,148 @@ func decode(path string) (Config, error) { } } return Config{ - Schema: schema, - Extensions: supported, - AIMode: aiMode, - Gitignore: gitignore, - IgnoreRules: ignoreRules, - Exclude: exclude, - Project: projectName, + Schema: schema, + Extensions: supported, + AIMode: aiMode, + Documentation: documentationPolicy, + Gitignore: gitignore, + IgnoreRules: ignoreRules, + Exclude: exclude, + Project: projectName, }, nil } +func validateDocumentationDirectory(index int, value string) (string, error) { + label := fmt.Sprintf("documentation.directories[%d]", index) + value = strings.TrimSpace(value) + if value == "" { + return "", apperror.New(apperror.Configuration, label+" must be a non-empty repository-relative directory") + } + if value == "." { + return value, nil + } + if invalidRepositoryPath(value) { + return "", apperror.New(apperror.Configuration, label+" must use a repository-relative path with forward slashes") + } + if strings.HasSuffix(value, "/") { + return "", apperror.New(apperror.Configuration, label+" must not end with a slash") + } + for _, part := range strings.Split(value, "/") { + if part == "" || part == "." || part == ".." { + return "", apperror.New(apperror.Configuration, label+" must not contain empty, current-directory, or parent path segments") + } + if strings.ContainsAny(part, "*?[") { + return "", apperror.New(apperror.Configuration, label+" must name a directory, not a glob") + } + } + return value, nil +} + +func validateDocumentationRule(index int, rule documentationRule) (DocumentationRule, error) { + label := fmt.Sprintf("documentation.frontmatter[%d]", index) + pattern := strings.TrimSpace(rule.Pattern) + if pattern == "" { + return DocumentationRule{}, apperror.New(apperror.Configuration, label+".pattern must be a non-empty repository-relative glob") + } + if invalidRepositoryPath(pattern) { + return DocumentationRule{}, apperror.New(apperror.Configuration, label+".pattern must use forward slashes and be relative to the repository") + } + kind := strings.TrimSpace(rule.Kind) + if kind != "file" && kind != "directory" { + return DocumentationRule{}, apperror.New(apperror.Configuration, label+`.kind must be "file" or "directory"`) + } + if strings.HasSuffix(pattern, "/") { + if kind != "directory" { + return DocumentationRule{}, apperror.New(apperror.Configuration, label+".pattern may end with a slash only when kind is directory") + } + pattern = strings.TrimSuffix(pattern, "/") + } + for _, part := range strings.Split(pattern, "/") { + if part == "" || part == "." || part == ".." { + return DocumentationRule{}, apperror.New(apperror.Configuration, label+".pattern must not contain empty, current-directory, or parent path segments") + } + } + if _, err := path.Match(pattern, ""); err != nil { + return DocumentationRule{}, apperror.New(apperror.Configuration, label+".pattern is not a valid glob: "+err.Error()) + } + return DocumentationRule{Pattern: pattern, Kind: kind}, nil +} + +func invalidRepositoryPath(value string) bool { + if strings.Contains(value, `\`) || strings.HasPrefix(value, "/") || filepath.IsAbs(value) || filepath.VolumeName(value) != "" { + return true + } + // filepath.VolumeName only recognizes the host OS syntax. Reject Windows + // drive forms on every platform so a config copied between systems cannot + // turn a drive-qualified path into a relative grant. + return len(value) >= 2 && ((value[0] >= 'a' && value[0] <= 'z') || (value[0] >= 'A' && value[0] <= 'Z')) && value[1] == ':' +} + +func samePath(left, right string) bool { + if runtime.GOOS == "windows" { + return strings.EqualFold(left, right) + } + return left == right +} + +func validateDocumentationYAML(content []byte) error { + var document yaml.Node + decoder := yaml.NewDecoder(bytes.NewReader(content)) + if err := decoder.Decode(&document); err != nil { + return nil // The typed decoder reports the authoritative syntax error. + } + if len(document.Content) == 0 { + return nil + } + root := document.Content[0] + if root.Kind != yaml.MappingNode { + return nil + } + for index := 0; index+1 < len(root.Content); index += 2 { + if root.Content[index].Value != "documentation" { + continue + } + return validateDocumentationNode(root.Content[index+1]) + } + return nil +} + +func validateDocumentationNode(node *yaml.Node) error { + if node.Kind != yaml.MappingNode { + return apperror.New(apperror.Configuration, "documentation must be a YAML object") + } + for index := 0; index+1 < len(node.Content); index += 2 { + key, value := node.Content[index].Value, node.Content[index+1] + switch key { + case "directories": + if value.Kind != yaml.SequenceNode { + return apperror.New(apperror.Configuration, "documentation.directories must be an array of strings") + } + for itemIndex, item := range value.Content { + if item.Kind != yaml.ScalarNode || item.Tag != "!!str" { + return apperror.New(apperror.Configuration, fmt.Sprintf("documentation.directories[%d] must be a string", itemIndex)) + } + } + case "frontmatter": + if value.Kind != yaml.SequenceNode { + return apperror.New(apperror.Configuration, "documentation.frontmatter must be an array of rules") + } + for itemIndex, item := range value.Content { + if item.Kind != yaml.MappingNode { + return apperror.New(apperror.Configuration, fmt.Sprintf("documentation.frontmatter[%d] must be an object", itemIndex)) + } + for fieldIndex := 0; fieldIndex+1 < len(item.Content); fieldIndex += 2 { + field, fieldValue := item.Content[fieldIndex].Value, item.Content[fieldIndex+1] + if (field == "pattern" || field == "kind") && (fieldValue.Kind != yaml.ScalarNode || fieldValue.Tag != "!!str") { + return apperror.New(apperror.Configuration, fmt.Sprintf("documentation.frontmatter[%d].%s must be a string", itemIndex, field)) + } + } + } + } + } + return nil +} + func validateIgnoreRule(index int, rule ignoreRule) (IgnoreRule, error) { label := fmt.Sprintf("ignore.rules[%d]", index) pattern := strings.TrimSpace(rule.Pattern) @@ -323,6 +519,9 @@ extensions: - .xdocs.md ai: mode: auto +documentation: + directories: [] + frontmatter: [] ignore: gitignore: true rules: diff --git a/internal/config/config_test.go b/internal/config/config_test.go index 1447adb..43fed16 100644 --- a/internal/config/config_test.go +++ b/internal/config/config_test.go @@ -57,6 +57,75 @@ func TestAIModeDefaultsToAuto(t *testing.T) { } } +func TestDocumentationPolicyDefaultsToEmptyAndIsGenerated(t *testing.T) { + root := t.TempDir() + defaults, err := Defaults(root) + if err != nil { + t.Fatal(err) + } + if len(defaults.Documentation.Directories) != 0 || len(defaults.Documentation.Frontmatter) != 0 { + t.Fatalf("documentation defaults unexpectedly grant permission: %#v", defaults.Documentation) + } + content := DefaultContent(root) + if !strings.Contains(content, "documentation:\n directories: []\n frontmatter: []\n") { + t.Fatalf("generated configuration omitted explicit documentation opt-ins:\n%s", content) + } + + path := filepath.Join(root, Filename) + if err := os.WriteFile(path, []byte(`schema: 1 +documentation: + directories: + - . + - technologies + frontmatter: + - pattern: technologies/approved-notes.md + kind: file + - pattern: technologies/notes + kind: directory +`), 0o644); err != nil { + t.Fatal(err) + } + cfg, err := Load(root, "", true) + if err != nil { + t.Fatal(err) + } + if len(cfg.Documentation.Directories) != 2 || cfg.Documentation.Directories[1] != "technologies" { + t.Fatalf("documentation directories not loaded: %#v", cfg.Documentation) + } + if len(cfg.Documentation.Frontmatter) != 2 || cfg.Documentation.Frontmatter[0].Kind != "file" { + t.Fatalf("documentation frontmatter rules not loaded: %#v", cfg.Documentation) + } +} + +func TestDocumentationPolicyRejectsMalformedEntries(t *testing.T) { + root := t.TempDir() + path := filepath.Join(root, Filename) + tests := []string{ + "directories:\n - ''\n", + "directories:\n - /absolute\n", + "directories:\n - ../outside\n", + "directories:\n - ./technologies\n", + "directories:\n - technologies/\n", + "directories:\n - tech*\n", + "directories:\n - technologies\n - technologies\n", + "frontmatter:\n - pattern: ''\n kind: file\n", + "frontmatter:\n - pattern: /notes.md\n kind: file\n", + "frontmatter:\n - pattern: ../notes.md\n kind: file\n", + "frontmatter:\n - pattern: notes.md/\n kind: file\n", + "frontmatter:\n - pattern: notes.md\n kind: document\n", + "frontmatter:\n - pattern: notes/[abc\n kind: file\n", + } + for _, body := range tests { + content := "schema: 1\ndocumentation:\n " + strings.ReplaceAll(body, "\n", "\n ") + if err := os.WriteFile(path, []byte(content), 0o644); err != nil { + t.Fatal(err) + } + if _, err := Load(root, "", true); err == nil { + t.Fatalf("malformed documentation policy accepted:\n%s", content) + } + } +} + func TestIgnoreDefaultsAndGeneratedConfiguration(t *testing.T) { root := t.TempDir() defaults, err := Defaults(root) diff --git a/internal/xdocs/discovery.go b/internal/xdocs/discovery.go index afab07a..05ea955 100644 --- a/internal/xdocs/discovery.go +++ b/internal/xdocs/discovery.go @@ -20,6 +20,7 @@ func ScanProject(cfg config.Config) (ScanResult, error) { XDocsFiles: []File{}, MarkdownDocuments: []MarkdownDocument{}, UncoveredPaths: []string{}, + Errors: []string{}, } documentsByDirectory := map[string][]MarkdownDocument{} var directories []string @@ -30,6 +31,7 @@ func ScanProject(cfg config.Config) (ScanResult, error) { err = filepath.WalkDir(cfg.CWD, func(path string, entry os.DirEntry, walkErr error) error { if walkErr != nil { + result.Errors = append(result.Errors, fmt.Sprintf("Unable to inspect %s: %v", slashRelative(cfg.CWD, path), walkErr)) if entry != nil && entry.IsDir() { return filepath.SkipDir } @@ -48,9 +50,13 @@ func ScanProject(cfg config.Config) (ScanResult, error) { if policy.ignored(path, false) { return nil } + if entry.Type()&os.ModeSymlink != 0 { + result.Errors = append(result.Errors, fmt.Sprintf("Skipped symlink during discovery: %s", slashRelative(cfg.CWD, path))) + return nil + } result.TotalFiles++ switch { - case IsDescriptor(path): + case IsDescriptorCandidate(path): file := ParseFile(path, cfg.CWD) filterMetadata(file.Metadata, nil, file.Directory, policy) result.XDocsFiles = append(result.XDocsFiles, file) @@ -72,11 +78,20 @@ func ScanProject(cfg config.Config) (ScanResult, error) { } rootPath := filepath.Join(cfg.CWD, rootFilename) - if content, readErr := os.ReadFile(rootPath); readErr == nil && !policy.ignored(rootPath, false) { - result.XDocsFiles = append(result.XDocsFiles, File{ - Path: rootPath, RelativePath: rootFilename, Directory: cfg.CWD, - Documents: []MarkdownDocument{}, Body: string(content), Valid: false, Errors: []string{}, - }) + rootInfo, rootStatErr := os.Lstat(rootPath) + if rootStatErr == nil && rootInfo.Mode()&os.ModeSymlink != 0 { + if !policy.ignored(rootPath, false) { + result.Errors = append(result.Errors, fmt.Sprintf("Skipped symlink during discovery: %s", rootFilename)) + } + rootStatErr = os.ErrInvalid + } + if rootStatErr == nil && !policy.ignored(rootPath, false) { + if content, readErr := os.ReadFile(rootPath); readErr == nil { + result.XDocsFiles = append(result.XDocsFiles, File{ + Path: rootPath, RelativePath: rootFilename, Directory: cfg.CWD, + Documents: []MarkdownDocument{}, Body: string(content), Valid: false, Errors: []string{}, + }) + } } sort.Slice(result.XDocsFiles, func(i, j int) bool { return result.XDocsFiles[i].RelativePath < result.XDocsFiles[j].RelativePath }) sort.Slice(result.MarkdownDocuments, func(i, j int) bool { @@ -85,7 +100,7 @@ func ScanProject(cfg config.Config) (ScanResult, error) { enrichFiles(result.XDocsFiles, documentsByDirectory) covered := map[string]bool{} - if info, err := os.Stat(rootPath); err == nil && info.Mode().IsRegular() && !policy.ignored(rootPath, false) { + if info, err := os.Lstat(rootPath); err == nil && info.Mode().IsRegular() && !policy.ignored(rootPath, false) { covered[cfg.CWD] = true } for _, file := range result.XDocsFiles { @@ -106,13 +121,21 @@ func ScanProject(cfg config.Config) (ScanResult, error) { } func IsDescriptor(path string) bool { - return strings.HasSuffix(strings.ToLower(filepath.Base(path)), descriptorSuffix) + name := strings.ToLower(filepath.Base(path)) + return strings.HasSuffix(name, descriptorSuffix) && name != descriptorSuffix && name != ".docs.md" +} + +// IsDescriptorCandidate includes malformed legacy names so scanning and tree +// output can report them instead of silently treating them as ordinary notes. +func IsDescriptorCandidate(path string) bool { + name := strings.ToLower(filepath.Base(path)) + return IsDescriptor(path) || name == descriptorSuffix || strings.HasSuffix(name, ".docs.md") } func IsPlainMarkdown(path string) bool { name := filepath.Base(path) lower := strings.ToLower(name) - return strings.HasSuffix(lower, ".md") && !strings.EqualFold(name, rootFilename) && !IsDescriptor(path) + return strings.HasSuffix(lower, ".md") && !strings.EqualFold(name, rootFilename) && !IsDescriptorCandidate(path) } func excluded(name string, values []string) bool { @@ -127,7 +150,7 @@ func excluded(name string, values []string) bool { func enrichFiles(files []File, documents map[string][]MarkdownDocument) { byDirectory := map[string][]int{} for index := range files { - if !IsDescriptor(files[index].Path) { + if !IsDescriptorCandidate(files[index].Path) { continue } byDirectory[files[index].Directory] = append(byDirectory[files[index].Directory], index) @@ -143,7 +166,7 @@ func enrichFiles(files []File, documents map[string][]MarkdownDocument) { } for index := range files { file := &files[index] - if !IsDescriptor(file.Path) { + if !IsDescriptorCandidate(file.Path) { continue } file.Documents = append([]MarkdownDocument(nil), documents[file.Directory]...) diff --git a/internal/xdocs/doctor.go b/internal/xdocs/doctor.go index cd9a95f..3eb626b 100644 --- a/internal/xdocs/doctor.go +++ b/internal/xdocs/doctor.go @@ -24,13 +24,17 @@ func Doctor(cfg config.Config, options DoctorOptions) (DoctorResult, error) { return DoctorResult{}, err } meta, err := ScanMetadata(cfg, MetaOptions{ - TargetPath: options.TargetPath, - IncludeDocuments: options.IncludeDocuments, + TargetPath: options.TargetPath, + IncludeDocuments: options.IncludeDocuments, + ExistingFrontmatter: options.ExistingFrontmatter, }) if err != nil { return DoctorResult{}, err } issues := []DoctorIssue{} + for _, message := range scan.Errors { + issues = append(issues, DoctorIssue{Severity: "error", Code: "discovery", Message: message}) + } for _, file := range scan.XDocsFiles { if !inScope(file.Path, target) { continue diff --git a/internal/xdocs/ignore.go b/internal/xdocs/ignore.go index 5e4fcb6..c34c835 100644 --- a/internal/xdocs/ignore.go +++ b/internal/xdocs/ignore.go @@ -14,12 +14,14 @@ import ( ) type pathPolicy struct { - root string - useGitignore bool - gitignoreRules []gitignoreRule - frontmatterRules []configuredIgnoreRule - loaded map[string]bool - ignoreCase bool + root string + useGitignore bool + gitignoreRules []gitignoreRule + frontmatterRules []configuredIgnoreRule + documentationDirs []string + documentationRules []configuredDocumentationRule + loaded map[string]bool + ignoreCase bool } type gitignoreRule struct { @@ -36,20 +38,33 @@ type configuredIgnoreRule struct { pattern *regexp.Regexp } +type configuredDocumentationRule struct { + kind string + hasSlash bool + pattern *regexp.Regexp +} + func newPathPolicy(cfg config.Config, target string) (*pathPolicy, error) { policy := &pathPolicy{ - root: filepath.Clean(cfg.CWD), - useGitignore: cfg.Gitignore, - gitignoreRules: []gitignoreRule{}, - frontmatterRules: []configuredIgnoreRule{}, - loaded: map[string]bool{}, - ignoreCase: runtime.GOOS == "windows", + root: filepath.Clean(cfg.CWD), + useGitignore: cfg.Gitignore, + gitignoreRules: []gitignoreRule{}, + frontmatterRules: []configuredIgnoreRule{}, + documentationDirs: append([]string(nil), cfg.Documentation.Directories...), + documentationRules: []configuredDocumentationRule{}, + loaded: map[string]bool{}, + ignoreCase: runtime.GOOS == "windows", } for _, rule := range cfg.IgnoreRules { policy.frontmatterRules = append(policy.frontmatterRules, configuredIgnoreRule{ kind: rule.Kind, hasSlash: strings.Contains(rule.Pattern, "/"), pattern: compileGlob(rule.Pattern, policy.ignoreCase), }) } + for _, rule := range cfg.Documentation.Frontmatter { + policy.documentationRules = append(policy.documentationRules, configuredDocumentationRule{ + kind: rule.Kind, hasSlash: strings.Contains(rule.Pattern, "/"), pattern: compileGlob(rule.Pattern, policy.ignoreCase), + }) + } if !policy.useGitignore { return policy, nil } @@ -214,6 +229,8 @@ func relativeToBase(relative, base string) (string, bool) { func (policy *pathPolicy) frontmatterRequired(pathname string) bool { relative := slashRelative(policy.root, pathname) + // Legacy ignore rules are explicit denials and always win over the new + // opt-in policy. for _, rule := range policy.frontmatterRules { if rule.kind == "file" { if matchConfiguredPattern(rule, relative) { @@ -233,7 +250,60 @@ func (policy *pathPolicy) frontmatterRequired(pathname string) bool { directory = next } } - return true + if !policy.documentationDirectoryAuthorized(path.Dir(relative)) { + return false + } + for _, rule := range policy.documentationRules { + if rule.kind == "file" { + if matchConfiguredDocumentationPattern(rule, relative) { + return true + } + continue + } + directory := path.Dir(relative) + for directory != "." && directory != "/" && directory != "" { + if matchConfiguredDocumentationPattern(rule, directory) { + return true + } + next := path.Dir(directory) + if next == directory { + break + } + directory = next + } + } + return false +} + +func (policy *pathPolicy) documentationDirectoryAuthorized(directory string) bool { + directory = path.Clean(directory) + if directory == ".." || strings.HasPrefix(directory, "../") { + return false + } + for _, allowed := range policy.documentationDirs { + allowed = path.Clean(allowed) + if allowed == "." || pathEqual(allowed, directory, policy.ignoreCase) || pathPrefix(allowed, directory, policy.ignoreCase) { + return true + } + } + return false +} + +func pathEqual(left, right string, ignoreCase bool) bool { + if ignoreCase { + return strings.EqualFold(left, right) + } + return left == right +} + +func pathPrefix(prefix, value string, ignoreCase bool) bool { + if len(value) <= len(prefix) { + return false + } + if ignoreCase { + return strings.HasPrefix(strings.ToLower(value), strings.ToLower(prefix)+"/") + } + return strings.HasPrefix(value, prefix+"/") } func matchConfiguredPattern(rule configuredIgnoreRule, relative string) bool { @@ -243,6 +313,13 @@ func matchConfiguredPattern(rule configuredIgnoreRule, relative string) bool { return rule.pattern.MatchString(path.Base(relative)) } +func matchConfiguredDocumentationPattern(rule configuredDocumentationRule, relative string) bool { + if rule.hasSlash { + return rule.pattern.MatchString(relative) + } + return rule.pattern.MatchString(path.Base(relative)) +} + func compileGlob(pattern string, ignoreCase bool) *regexp.Regexp { var expression strings.Builder if ignoreCase { diff --git a/internal/xdocs/ignore_test.go b/internal/xdocs/ignore_test.go index bd92484..1bfe1eb 100644 --- a/internal/xdocs/ignore_test.go +++ b/internal/xdocs/ignore_test.go @@ -81,6 +81,10 @@ func TestPathPolicyMatchesFileAndDirectoryFrontmatterRules(t *testing.T) { {Pattern: "README.md", Kind: "file", Frontmatter: false}, {Pattern: "docs/private", Kind: "directory", Frontmatter: false}, }, + Documentation: config.DocumentationConfig{ + Directories: []string{"docs"}, + Frontmatter: []config.DocumentationRule{{Pattern: "docs/public", Kind: "directory"}}, + }, }, root) if err != nil { t.Fatal(err) @@ -94,6 +98,7 @@ func TestPathPolicyMatchesFileAndDirectoryFrontmatterRules(t *testing.T) { {filepath.Join(root, "docs", "private", "notes.md"), false}, {filepath.Join(root, "docs", "private", "nested", "notes.md"), false}, {filepath.Join(root, "docs", "public", "notes.md"), true}, + {filepath.Join(root, "docs", "other", "notes.md"), false}, } for _, test := range tests { if got := policy.frontmatterRequired(test.path); got != test.required { @@ -384,6 +389,9 @@ keywords: [visible] t.Fatal(err) } cfg.Project = "example" + cfg.Documentation = config.DocumentationConfig{ + Directories: []string{"."}, Frontmatter: []config.DocumentationRule{{Pattern: "**/*.md", Kind: "file"}}, + } scan, err := ScanProject(cfg) if err != nil { diff --git a/internal/xdocs/meta.go b/internal/xdocs/meta.go index 7ee044c..70ca991 100644 --- a/internal/xdocs/meta.go +++ b/internal/xdocs/meta.go @@ -29,13 +29,14 @@ func ScanMetadata(cfg config.Config, options MetaOptions) (MetaResult, error) { } result := MetaResult{ - Root: cfg.CWD, - TargetPath: displayTarget(cfg.CWD, target), - IncludeDocuments: options.IncludeDocuments, - Strict: options.Strict, - Filters: options.Filters, - Descriptors: []MetaDescriptor{}, - Errors: []string{}, + Root: cfg.CWD, + TargetPath: displayTarget(cfg.CWD, target), + IncludeDocuments: options.IncludeDocuments, + ExistingFrontmatter: options.ExistingFrontmatter, + Strict: options.Strict, + Filters: options.Filters, + Descriptors: []MetaDescriptor{}, + Errors: []string{}, } if scanExcludedPath(cfg.CWD, target, cfg.Exclude) { return result, nil @@ -64,7 +65,11 @@ func ScanMetadata(cfg config.Config, options MetaOptions) (MetaResult, error) { if policy.ignored(path, false) { return nil } - if IsDescriptor(path) { + if entry.Type()&os.ModeSymlink != 0 { + result.Errors = append(result.Errors, fmt.Sprintf("%s: skipped symlink during metadata discovery", slashRelative(cfg.CWD, path))) + return nil + } + if IsDescriptorCandidate(path) { descriptor := parseMetaDescriptor(path, cfg.CWD) filterMetadata(descriptor.Metadata, descriptor.Frontmatter, descriptor.Directory, policy) result.Descriptors = append(result.Descriptors, descriptor) @@ -81,7 +86,7 @@ func ScanMetadata(cfg config.Config, options MetaOptions) (MetaResult, error) { return MetaResult{}, err } sort.Slice(result.Descriptors, func(i, j int) bool { return result.Descriptors[i].RelativePath < result.Descriptors[j].RelativePath }) - enrichMeta(result.Descriptors, documents, options.IncludeDocuments, cfg.CWD) + enrichMeta(result.Descriptors, documents, options.IncludeDocuments, options.ExistingFrontmatter, cfg.CWD) result.Descriptors = filterDescriptors(result.Descriptors, options.Filters, options.IncludeDocuments) for _, descriptor := range result.Descriptors { for _, message := range descriptor.Errors { @@ -104,13 +109,22 @@ func parseMetaDescriptor(path, root string) MetaDescriptor { Documents: []MetaDocument{}, Errors: []string{}, } - raw, ok, err := ReadFrontmatter(path) + if strings.EqualFold(filepath.Base(path), ".xdocs.md") { + result.Errors = append(result.Errors, `Invalid xdocs descriptor filename. Use a named file such as "authentication.xdocs.md"; ".xdocs.md" is only the extension.`) + } else if strings.HasSuffix(strings.ToLower(filepath.Base(path)), ".docs.md") { + result.Errors = append(result.Errors, `Invalid xdocs descriptor filename. Use a named file such as "authentication.xdocs.md"; ".docs.md" is not a supported descriptor.`) + } + raw, ok, present, err := readFrontmatterDetailed(path) if err != nil { result.Errors = append(result.Errors, fmt.Sprintf("Read frontmatter: %v", err)) return result } if !ok { - result.Errors = append(result.Errors, "Missing YAML frontmatter.") + if present { + result.Errors = append(result.Errors, "Malformed YAML frontmatter: missing a closing delimiter.") + } else { + result.Errors = append(result.Errors, "Missing YAML frontmatter.") + } return result } metadata, frontmatter, errors := parseMetadata(raw) @@ -126,7 +140,7 @@ func parseMetaDescriptor(path, root string) MetaDescriptor { return result } -func enrichMeta(descriptors []MetaDescriptor, documents map[string][]MarkdownDocument, includeDocuments bool, root string) { +func enrichMeta(descriptors []MetaDescriptor, documents map[string][]MarkdownDocument, includeDocuments, existingFrontmatter bool, root string) { byDirectory := map[string][]int{} for index := range descriptors { byDirectory[descriptors[index].Directory] = append(byDirectory[descriptors[index].Directory], index) @@ -162,7 +176,7 @@ func enrichMeta(descriptors []MetaDescriptor, documents map[string][]MarkdownDoc continue } if includeDocuments { - descriptor.Documents = append(descriptor.Documents, parseDocument(document.Path, root, descriptor.Metadata.Subject, document.FrontmatterRequired)) + descriptor.Documents = append(descriptor.Documents, parseDocument(document.Path, root, descriptor.Metadata.Subject, document.FrontmatterRequired, existingFrontmatter)) } } sort.Slice(descriptor.Documents, func(i, j int) bool { diff --git a/internal/xdocs/metadata.go b/internal/xdocs/metadata.go index b57769e..16da2bb 100644 --- a/internal/xdocs/metadata.go +++ b/internal/xdocs/metadata.go @@ -32,11 +32,17 @@ func ParseFile(path, root string) File { } if strings.EqualFold(filepath.Base(path), ".xdocs.md") { result.Errors = append(result.Errors, `Invalid xdocs descriptor filename. Use a named file such as "authentication.xdocs.md"; ".xdocs.md" is only the extension.`) + } else if strings.HasSuffix(strings.ToLower(filepath.Base(path)), ".docs.md") { + result.Errors = append(result.Errors, `Invalid xdocs descriptor filename. Use a named file such as "authentication.xdocs.md"; ".docs.md" is not a supported descriptor.`) } - frontmatter, body, ok := ExtractFrontmatter(string(content)) + frontmatter, body, ok, present := ExtractFrontmatterDetailed(string(content)) result.Body = body if !ok { - result.Errors = append(result.Errors, "Missing YAML frontmatter.") + if present { + result.Errors = append(result.Errors, "Malformed YAML frontmatter: missing a closing delimiter.") + } else { + result.Errors = append(result.Errors, "Missing YAML frontmatter.") + } return result } metadata, _, errors := parseMetadata(frontmatter) @@ -47,43 +53,74 @@ func ParseFile(path, root string) File { } func ExtractFrontmatter(content string) (string, string, bool) { + frontmatter, body, ok, _ := ExtractFrontmatterDetailed(content) + return frontmatter, body, ok +} + +func ExtractFrontmatterDetailed(content string) (string, string, bool, bool) { trimmed := strings.TrimLeft(content, " \t\r\n") if !strings.HasPrefix(trimmed, "---") { - return "", content, false + return "", content, false, false } + present := true remaining := trimmed[3:] if strings.HasPrefix(remaining, "\r\n") { remaining = remaining[2:] } else if strings.HasPrefix(remaining, "\n") { remaining = remaining[1:] + } else { + return "", content, false, present + } + index := -1 + closingLength := 0 + for offset := 0; offset < len(remaining); { + lineEnd := strings.IndexByte(remaining[offset:], '\n') + line := remaining[offset:] + if lineEnd >= 0 { + line = remaining[offset : offset+lineEnd] + } + line = strings.TrimSuffix(line, "\r") + if line == "---" { + index = offset + closingLength = len(line) + break + } + if lineEnd < 0 { + break + } + offset += lineEnd + 1 } - index := strings.Index(remaining, "\n---") if index < 0 { - return "", content, false + return "", content, false, present } frontmatter := strings.TrimSpace(remaining[:index]) - bodyStart := index + len("\n---") + bodyStart := index + closingLength if bodyStart < len(remaining) && remaining[bodyStart] == '\r' { bodyStart++ } if bodyStart < len(remaining) && remaining[bodyStart] == '\n' { bodyStart++ } - return frontmatter, strings.TrimSpace(remaining[bodyStart:]), true + return frontmatter, strings.TrimSpace(remaining[bodyStart:]), true, present } func ReadFrontmatter(path string) (string, bool, error) { + frontmatter, ok, _, err := readFrontmatterDetailed(path) + return frontmatter, ok, err +} + +func readFrontmatterDetailed(path string) (string, bool, bool, error) { file, err := os.Open(path) if err != nil { - return "", false, err + return "", false, false, err } defer file.Close() content, err := io.ReadAll(io.LimitReader(file, frontmatterMaxBytes)) if err != nil { - return "", false, err + return "", false, false, err } - frontmatter, _, ok := ExtractFrontmatter(string(content)) - return frontmatter, ok, nil + frontmatter, _, ok, present := ExtractFrontmatterDetailed(string(content)) + return frontmatter, ok, present, nil } func parseMetadata(raw string) (*Metadata, Frontmatter, []string) { @@ -99,6 +136,7 @@ func parseMetadata(raw string) (*Metadata, Frontmatter, []string) { } var metadata Metadata decoder := yaml.NewDecoder(bytes.NewBufferString(raw)) + decoder.KnownFields(true) if err := decoder.Decode(&metadata); err != nil { return nil, object, []string{fmt.Sprintf("Invalid YAML frontmatter: %v", err)} } @@ -138,22 +176,30 @@ func validateMetadata(metadata Metadata) []string { return errors } -func parseDocument(path, root, expectedOwner string, frontmatterRequired bool) MetaDocument { +func parseDocument(path, root, expectedOwner string, frontmatterRequired, inspectExisting bool) MetaDocument { result := MetaDocument{ Path: path, RelativePath: slashRelative(root, path), Directory: filepath.Dir(path), Name: filepath.Base(path), Owner: expectedOwner, FrontmatterRequired: frontmatterRequired, Errors: []string{}, } - if !frontmatterRequired { + if !frontmatterRequired && !inspectExisting { result.Valid = true return result } - raw, ok, err := ReadFrontmatter(path) + raw, ok, present, err := readFrontmatterDetailed(path) if err != nil { result.Errors = append(result.Errors, fmt.Sprintf("Read frontmatter: %v", err)) return result } if !ok { + if present { + result.Errors = append(result.Errors, "Malformed YAML frontmatter: missing a closing delimiter.") + return result + } + if !frontmatterRequired { + result.Valid = true + return result + } result.Errors = append(result.Errors, "Missing YAML frontmatter.") return result } @@ -163,6 +209,14 @@ func parseDocument(path, root, expectedOwner string, frontmatterRequired bool) M return result } result.Frontmatter = frontmatter + if !frontmatterRequired { + if frontmatter == nil { + result.Errors = append(result.Errors, "Frontmatter must be a YAML object.") + return result + } + result.Valid = true + return result + } result.Owner = stringValue(frontmatter, "owner") for _, field := range []string{"name", "purpose", "description", "created", "owner"} { if stringValue(frontmatter, field) == "" { diff --git a/internal/xdocs/model.go b/internal/xdocs/model.go index 838de56..df759ae 100644 --- a/internal/xdocs/model.go +++ b/internal/xdocs/model.go @@ -43,12 +43,16 @@ type ScanResult struct { XDocsFiles []File `json:"xdocsFiles"` MarkdownDocuments []MarkdownDocument `json:"markdownDocuments"` UncoveredPaths []string `json:"uncoveredPaths"` + Errors []string `json:"errors"` } type TreeNode struct { Subject string `json:"subject"` Description string `json:"description"` Path *string `json:"path"` + Kind string `json:"kind"` + Valid bool `json:"valid"` + Errors []string `json:"errors,omitempty"` Children []*TreeNode `json:"children"` } @@ -89,20 +93,22 @@ type MetaDescriptor struct { } type MetaOptions struct { - TargetPath string - IncludeDocuments bool - Strict bool - Filters Filters + TargetPath string + IncludeDocuments bool + ExistingFrontmatter bool + Strict bool + Filters Filters } type MetaResult struct { - Root string `json:"root"` - TargetPath string `json:"targetPath"` - IncludeDocuments bool `json:"includeDocuments"` - Strict bool `json:"strict"` - Filters Filters `json:"filters"` - Descriptors []MetaDescriptor `json:"descriptors"` - Errors []string `json:"errors"` + Root string `json:"root"` + TargetPath string `json:"targetPath"` + IncludeDocuments bool `json:"includeDocuments"` + ExistingFrontmatter bool `json:"existingFrontmatter"` + Strict bool `json:"strict"` + Filters Filters `json:"filters"` + Descriptors []MetaDescriptor `json:"descriptors"` + Errors []string `json:"errors"` } type ContextOptions struct { @@ -135,9 +141,10 @@ type ContextResult struct { } type DoctorOptions struct { - TargetPath string - IncludeDocuments bool - WarningsAsErrors bool + TargetPath string + IncludeDocuments bool + ExistingFrontmatter bool + WarningsAsErrors bool } type DoctorIssue struct { diff --git a/internal/xdocs/output.go b/internal/xdocs/output.go new file mode 100644 index 0000000..9f7147b --- /dev/null +++ b/internal/xdocs/output.go @@ -0,0 +1,153 @@ +package xdocs + +import ( + "fmt" + "os" + "path/filepath" + "strings" + + "github.com/CGuiho/xdocs/internal/config" +) + +// WriteReport writes exactly one explicitly requested report destination. It +// performs every path and policy check before creating or truncating a file. +// Generated reports are intentionally never accepted as descriptor files. +func WriteReport(cfg config.Config, destination string, content []byte) error { + target, existing, err := preflightReportPath(cfg, destination) + if err != nil { + return err + } + mode := os.FileMode(0o644) + if existing != nil { + mode = existing.Mode().Perm() + } + parent := filepath.Dir(target) + temporary, err := os.CreateTemp(parent, ".xdocs-output-*") + if err != nil { + return fmt.Errorf("create atomic output in %s: %w", filepath.ToSlash(parent), err) + } + temporaryPath := temporary.Name() + removeTemporary := true + defer func() { + if removeTemporary { + _ = os.Remove(temporaryPath) + } + }() + if err := temporary.Chmod(mode); err != nil { + _ = temporary.Close() + return fmt.Errorf("set output permissions: %w", err) + } + if _, err := temporary.Write(content); err != nil { + _ = temporary.Close() + return fmt.Errorf("write output: %w", err) + } + if err := temporary.Sync(); err != nil { + _ = temporary.Close() + return fmt.Errorf("sync output: %w", err) + } + if err := temporary.Close(); err != nil { + return fmt.Errorf("close output: %w", err) + } + if err := os.Rename(temporaryPath, target); err != nil { + return fmt.Errorf("atomically replace output %s: %w", filepath.ToSlash(target), err) + } + removeTemporary = false + return nil +} + +func preflightReportPath(cfg config.Config, destination string) (string, os.FileInfo, error) { + if strings.TrimSpace(destination) == "" { + return "", nil, fmt.Errorf("output destination must be a non-empty file path") + } + target := destination + if !filepath.IsAbs(target) { + target = filepath.Join(cfg.CWD, target) + } + target, err := filepath.Abs(target) + if err != nil { + return "", nil, fmt.Errorf("resolve output destination: %w", err) + } + relative, err := filepath.Rel(cfg.CWD, target) + if err != nil || relative == ".." || strings.HasPrefix(relative, ".."+string(filepath.Separator)) || filepath.IsAbs(relative) { + return "", nil, fmt.Errorf("output destination must remain inside the project: %s", filepath.ToSlash(target)) + } + base := filepath.Base(target) + if IsDescriptorCandidate(target) || strings.EqualFold(base, rootFilename) { + return "", nil, fmt.Errorf("refusing to write a generated report to reserved descriptor destination %s; descriptor files require complete validated metadata", filepath.ToSlash(relative)) + } + if scanExcludedPath(cfg.CWD, target, cfg.Exclude) { + return "", nil, fmt.Errorf("output destination is excluded by scan policy: %s", filepath.ToSlash(relative)) + } + parent := filepath.Dir(target) + if err := ensureNoSymlinkPath(cfg.CWD, parent, false); err != nil { + return "", nil, err + } + policy, err := newPathPolicy(cfg, parent) + if err != nil { + return "", nil, err + } + if policy.ignored(parent, true) || policy.ignored(target, false) { + return "", nil, fmt.Errorf("output destination is excluded by .gitignore: %s", filepath.ToSlash(relative)) + } + parentInfo, err := os.Stat(parent) + if err != nil { + return "", nil, fmt.Errorf("output parent directory does not exist: %s", filepath.ToSlash(parent)) + } + if !parentInfo.IsDir() { + return "", nil, fmt.Errorf("output parent is not a directory: %s", filepath.ToSlash(parent)) + } + var existing os.FileInfo + info, err := os.Lstat(target) + if err == nil { + if info.Mode()&os.ModeSymlink != 0 { + return "", nil, fmt.Errorf("refusing to write through symlink output: %s", filepath.ToSlash(relative)) + } + if !info.Mode().IsRegular() { + return "", nil, fmt.Errorf("output destination is not a regular file: %s", filepath.ToSlash(relative)) + } + existing = info + } else if !os.IsNotExist(err) { + return "", nil, fmt.Errorf("inspect output destination: %w", err) + } + return target, existing, nil +} + +func ensureNoSymlinkPath(root, target string, allowMissingLeaf bool) error { + root, err := filepath.Abs(root) + if err != nil { + return fmt.Errorf("resolve project root: %w", err) + } + target, err = filepath.Abs(target) + if err != nil { + return fmt.Errorf("resolve output path: %w", err) + } + relative, err := filepath.Rel(root, target) + if err != nil || relative == ".." || strings.HasPrefix(relative, ".."+string(filepath.Separator)) { + return fmt.Errorf("output path escapes project root: %s", filepath.ToSlash(target)) + } + current := root + parts := []string{} + if relative != "." && relative != "" { + parts = strings.Split(relative, string(filepath.Separator)) + } + for index, part := range parts { + current = filepath.Join(current, part) + info, statErr := os.Lstat(current) + if os.IsNotExist(statErr) { + if allowMissingLeaf && index == len(parts)-1 { + return nil + } + return fmt.Errorf("output parent directory does not exist: %s", filepath.ToSlash(current)) + } + if statErr != nil { + return fmt.Errorf("inspect output path: %w", statErr) + } + if info.Mode()&os.ModeSymlink != 0 { + return fmt.Errorf("refusing to traverse symlink output path: %s", filepath.ToSlash(current)) + } + if index < len(parts)-1 && !info.IsDir() { + return fmt.Errorf("output path component is not a directory: %s", filepath.ToSlash(current)) + } + } + return nil +} diff --git a/internal/xdocs/xdocs_test.go b/internal/xdocs/xdocs_test.go index fb256f6..843c9cb 100644 --- a/internal/xdocs/xdocs_test.go +++ b/internal/xdocs/xdocs_test.go @@ -51,7 +51,9 @@ keywords: [login] if err := os.WriteFile(filepath.Join(module, "service.go"), []byte("package module\n"), 0o644); err != nil { t.Fatal(err) } - cfg := config.Config{Schema: 1, CWD: root, Extensions: []string{".xdocs.md"}, AIMode: "auto", Exclude: []string{".git"}, Project: "example"} + cfg := config.Config{Schema: 1, CWD: root, Extensions: []string{".xdocs.md"}, AIMode: "auto", Exclude: []string{".git"}, Project: "example", Documentation: config.DocumentationConfig{ + Directories: []string{"."}, Frontmatter: []config.DocumentationRule{{Pattern: "**/*.md", Kind: "file"}}, + }} scan, err := ScanProject(cfg) if err != nil { t.Fatal(err) From 06b7a3018921618fc5669d63ba73c501f2329c3b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Crist=C3=B3v=C3=A3o=20GUIHO?= Date: Sat, 12 Sep 2026 11:52:25 +0000 Subject: [PATCH 05/20] Test opt-in metadata and guarded reports --- internal/xdocs/ignore_test.go | 43 ++++++++++++++ internal/xdocs/output_test.go | 102 ++++++++++++++++++++++++++++++++++ internal/xdocs/xdocs_test.go | 45 +++++++++++++++ 3 files changed, 190 insertions(+) create mode 100644 internal/xdocs/output_test.go diff --git a/internal/xdocs/ignore_test.go b/internal/xdocs/ignore_test.go index 1bfe1eb..4831fc3 100644 --- a/internal/xdocs/ignore_test.go +++ b/internal/xdocs/ignore_test.go @@ -107,6 +107,49 @@ func TestPathPolicyMatchesFileAndDirectoryFrontmatterRules(t *testing.T) { } } +func TestPathPolicyRequiresExplicitDocumentationAuthorization(t *testing.T) { + root := t.TempDir() + policy, err := newPathPolicy(config.Config{ + CWD: root, + Documentation: config.DocumentationConfig{ + Directories: []string{"technologies"}, + Frontmatter: []config.DocumentationRule{{Pattern: "technologies/approved-notes.md", Kind: "file"}}, + }, + IgnoreRules: []config.IgnoreRule{{Pattern: "README.md", Kind: "file", Frontmatter: false}}, + }, root) + if err != nil { + t.Fatal(err) + } + tests := []struct { + path string + want bool + }{ + {filepath.Join(root, "technologies", "approved-notes.md"), true}, + {filepath.Join(root, "technologies", "other.md"), false}, + {filepath.Join(root, "other", "approved-notes.md"), false}, + {filepath.Join(root, "technologies", "README.md"), false}, + } + for _, test := range tests { + if got := policy.frontmatterRequired(test.path); got != test.want { + t.Errorf("frontmatterRequired(%s) = %t, want %t", test.path, got, test.want) + } + } + + rootPolicy, err := newPathPolicy(config.Config{ + CWD: root, + Documentation: config.DocumentationConfig{ + Directories: []string{"."}, + Frontmatter: []config.DocumentationRule{{Pattern: "**/*.md", Kind: "file"}}, + }, + }, root) + if err != nil { + t.Fatal(err) + } + if rootPolicy.frontmatterRequired(filepath.Join(root, "..", "outside.md")) { + t.Fatal("root documentation grant escaped the repository") + } +} + func TestPathPolicyDoesNotEnterIgnoredOrExcludedTarget(t *testing.T) { for _, test := range []struct { name string diff --git a/internal/xdocs/output_test.go b/internal/xdocs/output_test.go new file mode 100644 index 0000000..d93fedf --- /dev/null +++ b/internal/xdocs/output_test.go @@ -0,0 +1,102 @@ +package xdocs + +import ( + "os" + "path/filepath" + "runtime" + "strings" + "testing" + + "github.com/CGuiho/xdocs/internal/config" +) + +func TestWriteReportPreflightsReservedAndUnsafeDestinations(t *testing.T) { + root := t.TempDir() + cfg, err := config.Defaults(root) + if err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(root, "existing.md"), []byte("before\n"), 0o644); err != nil { + t.Fatal(err) + } + if err := WriteReport(cfg, "existing.md", []byte("after\n")); err != nil { + t.Fatal(err) + } + content, err := os.ReadFile(filepath.Join(root, "existing.md")) + if err != nil { + t.Fatal(err) + } + if string(content) != "after\n" { + t.Fatalf("report was not replaced atomically: %q", content) + } + + descriptorPath := filepath.Join(root, "module.xdocs.md") + original := []byte("existing descriptor\n") + if err := os.WriteFile(descriptorPath, original, 0o644); err != nil { + t.Fatal(err) + } + if err := WriteReport(cfg, "module.xdocs.md", []byte("report\n")); err == nil { + t.Fatal("descriptor report destination was accepted") + } + unchanged, err := os.ReadFile(descriptorPath) + if err != nil { + t.Fatal(err) + } + if string(unchanged) != string(original) { + t.Fatalf("rejected descriptor destination changed: %q", unchanged) + } + + if err := WriteReport(cfg, "missing/report.md", []byte("report\n")); err == nil { + t.Fatal("missing output parent was created implicitly") + } + if _, err := os.Stat(filepath.Join(root, "missing")); !os.IsNotExist(err) { + t.Fatalf("unexpected output parent after rejection: %v", err) + } + + if err := os.Mkdir(filepath.Join(root, "excluded"), 0o755); err != nil { + t.Fatal(err) + } + cfg.Exclude = []string{"excluded"} + if err := WriteReport(cfg, "excluded/report.md", []byte("report\n")); err == nil { + t.Fatal("excluded output destination was accepted") + } +} + +func TestWriteReportRejectsGitignoredAndSymlinkDestinations(t *testing.T) { + if runtime.GOOS == "windows" { + t.Skip("symlink permissions vary on Windows") + } + root := t.TempDir() + cfg, err := config.Defaults(root) + if err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(root, ".gitignore"), []byte("private/\n"), 0o644); err != nil { + t.Fatal(err) + } + if err := os.Mkdir(filepath.Join(root, "private"), 0o755); err != nil { + t.Fatal(err) + } + if err := WriteReport(cfg, "private/report.md", []byte("report\n")); err == nil { + t.Fatal("Git-ignored output destination was accepted") + } + + outside := t.TempDir() + if err := os.WriteFile(filepath.Join(outside, "report.md"), []byte("outside\n"), 0o644); err != nil { + t.Fatal(err) + } + if err := os.Symlink(outside, filepath.Join(root, "linked")); err != nil { + t.Fatal(err) + } + err = WriteReport(cfg, "linked/report.md", []byte("report\n")) + if err == nil || !strings.Contains(strings.ToLower(err.Error()), "symlink") { + t.Fatalf("symlink output was not rejected clearly: %v", err) + } + content, err := os.ReadFile(filepath.Join(outside, "report.md")) + if err != nil { + t.Fatal(err) + } + if string(content) != "outside\n" { + t.Fatalf("symlink target changed: %q", content) + } +} diff --git a/internal/xdocs/xdocs_test.go b/internal/xdocs/xdocs_test.go index 843c9cb..f5e42d5 100644 --- a/internal/xdocs/xdocs_test.go +++ b/internal/xdocs/xdocs_test.go @@ -118,6 +118,51 @@ flags: [] } } +func TestExistingFrontmatterAuditFindsMalformedHeadersWithoutRequiringHeaders(t *testing.T) { + root := t.TempDir() + descriptor := `--- +subject: example +description: Example. +parent: null +children: [] +files: {} +documents: + notes.md: Notes. + broken.md: Broken notes. +tags: [] +keywords: [] +flags: [] +--- +` + if err := os.WriteFile(filepath.Join(root, "example.xdocs.md"), []byte(descriptor), 0o644); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(root, "notes.md"), []byte("# Notes\n"), 0o644); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(root, "broken.md"), []byte("---\nname: [unterminated\n"), 0o644); err != nil { + t.Fatal(err) + } + cfg, err := config.Defaults(root) + if err != nil { + t.Fatal(err) + } + withoutAudit, err := ScanMetadata(cfg, MetaOptions{IncludeDocuments: true}) + if err != nil { + t.Fatal(err) + } + if len(withoutAudit.Errors) != 0 { + t.Fatalf("ordinary Markdown unexpectedly required frontmatter: %#v", withoutAudit.Errors) + } + withAudit, err := ScanMetadata(cfg, MetaOptions{IncludeDocuments: true, ExistingFrontmatter: true}) + if err != nil { + t.Fatal(err) + } + if len(withAudit.Errors) == 0 || !strings.Contains(strings.Join(withAudit.Errors, "\n"), "broken.md") { + t.Fatalf("existing-header audit missed malformed frontmatter: %#v", withAudit) + } +} + func TestContextRejectsEmptyQuery(t *testing.T) { cfg := config.Config{CWD: t.TempDir(), Extensions: []string{".xdocs.md"}, Exclude: []string{}} if _, err := FindContext(cfg, " ", ContextOptions{}); err == nil { From b078a2234e1c0ed77865b23ebb6bf933e7dc225c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Crist=C3=B3v=C3=A3o=20GUIHO?= Date: Sat, 12 Sep 2026 11:52:43 +0000 Subject: [PATCH 06/20] Reject malformed documentation policy YAML --- internal/config/config_test.go | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/internal/config/config_test.go b/internal/config/config_test.go index 43fed16..5a48fa4 100644 --- a/internal/config/config_test.go +++ b/internal/config/config_test.go @@ -101,6 +101,12 @@ func TestDocumentationPolicyRejectsMalformedEntries(t *testing.T) { root := t.TempDir() path := filepath.Join(root, Filename) tests := []string{ + "directories: 3\n", + "directories: null\n", + "frontmatter: {}\n", + "frontmatter:\n - pattern: 3\n kind: file\n", + "frontmatter:\n - pattern: notes.md\n kind: 3\n", + "frontmatter:\n - pattern: notes.md\n kind: file\n extra: true\n", "directories:\n - ''\n", "directories:\n - /absolute\n", "directories:\n - ../outside\n", From ec4c58a332c963ffd147f9b5a6a6abd2442563b0 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Crist=C3=B3v=C3=A3o=20GUIHO?= Date: Sat, 12 Sep 2026 11:54:36 +0000 Subject: [PATCH 07/20] Validate report frontmatter before output --- internal/xdocs/output.go | 30 ++++++++++++++++++++++++++++++ internal/xdocs/output_test.go | 31 +++++++++++++++++++++++++++++++ 2 files changed, 61 insertions(+) diff --git a/internal/xdocs/output.go b/internal/xdocs/output.go index 9f7147b..0e29caf 100644 --- a/internal/xdocs/output.go +++ b/internal/xdocs/output.go @@ -7,6 +7,7 @@ import ( "strings" "github.com/CGuiho/xdocs/internal/config" + "go.yaml.in/yaml/v3" ) // WriteReport writes exactly one explicitly requested report destination. It @@ -17,6 +18,9 @@ func WriteReport(cfg config.Config, destination string, content []byte) error { if err != nil { return err } + if err := validateReportContent(cfg, target, content); err != nil { + return err + } mode := os.FileMode(0o644) if existing != nil { mode = existing.Mode().Perm() @@ -55,6 +59,32 @@ func WriteReport(cfg config.Config, destination string, content []byte) error { return nil } +func validateReportContent(cfg config.Config, target string, content []byte) error { + _, _, ok, present := ExtractFrontmatterDetailed(string(content)) + if !present { + return nil + } + if !ok { + return fmt.Errorf("refusing to write malformed report frontmatter: missing a closing delimiter") + } + policy, err := newPathPolicy(cfg, filepath.Dir(target)) + if err != nil { + return err + } + if !policy.frontmatterRequired(target) { + return fmt.Errorf("refusing to write Markdown frontmatter without explicit documentation.frontmatter authorization: %s", filepath.ToSlash(slashRelative(cfg.CWD, target))) + } + frontmatter, _, _, _ := ExtractFrontmatterDetailed(string(content)) + var object map[string]any + if err := yaml.Unmarshal([]byte(frontmatter), &object); err != nil { + return fmt.Errorf("refusing to write invalid report frontmatter: %w", err) + } + if object == nil { + return fmt.Errorf("refusing to write report frontmatter: expected a YAML object") + } + return nil +} + func preflightReportPath(cfg config.Config, destination string) (string, os.FileInfo, error) { if strings.TrimSpace(destination) == "" { return "", nil, fmt.Errorf("output destination must be a non-empty file path") diff --git a/internal/xdocs/output_test.go b/internal/xdocs/output_test.go index d93fedf..08ab076 100644 --- a/internal/xdocs/output_test.go +++ b/internal/xdocs/output_test.go @@ -22,6 +22,12 @@ func TestWriteReportPreflightsReservedAndUnsafeDestinations(t *testing.T) { if err := WriteReport(cfg, "existing.md", []byte("after\n")); err != nil { t.Fatal(err) } + if err := WriteReport(cfg, "existing.md", []byte("---\nname: [broken\n")); err == nil { + t.Fatal("unterminated report frontmatter was accepted") + } + if err := WriteReport(cfg, "existing.md", []byte("---\nname: report\n---\nbody\n")); err == nil { + t.Fatal("unauthorized report frontmatter was accepted") + } content, err := os.ReadFile(filepath.Join(root, "existing.md")) if err != nil { t.Fatal(err) @@ -62,6 +68,31 @@ func TestWriteReportPreflightsReservedAndUnsafeDestinations(t *testing.T) { } } +func TestWriteReportAllowsExplicitlyAuthorizedGenericFrontmatter(t *testing.T) { + root := t.TempDir() + cfg, err := config.Defaults(root) + if err != nil { + t.Fatal(err) + } + cfg.Documentation = config.DocumentationConfig{ + Directories: []string{"notes"}, + Frontmatter: []config.DocumentationRule{{Pattern: "notes/report.md", Kind: "file"}}, + } + if err := os.Mkdir(filepath.Join(root, "notes"), 0o755); err != nil { + t.Fatal(err) + } + if err := WriteReport(cfg, "notes/report.md", []byte("---\nname: report\n---\nbody\n")); err != nil { + t.Fatal(err) + } + content, err := os.ReadFile(filepath.Join(root, "notes", "report.md")) + if err != nil { + t.Fatal(err) + } + if !strings.HasPrefix(string(content), "---\nname: report") { + t.Fatalf("authorized report was not written: %q", content) + } +} + func TestWriteReportRejectsGitignoredAndSymlinkDestinations(t *testing.T) { if runtime.GOOS == "windows" { t.Skip("symlink permissions vary on Windows") From eddf88e06be824e7c7b626a11352ddcc89a25ab9 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Crist=C3=B3v=C3=A3o=20GUIHO?= Date: Sat, 12 Sep 2026 11:54:40 +0000 Subject: [PATCH 08/20] Document explicit XDocs authorization policy --- AGENTS.md | 13 +++++++------ TODO.md | 10 +++++++++- cmd/cmd.xdocs.md | 2 +- docs/plans/plans.xdocs.md | 3 +++ internal/config/config.xdocs.md | 7 ++++--- internal/xdocs/xdocs.xdocs.md | 13 ++++++++----- xdocs.yaml | 3 +++ 7 files changed, 35 insertions(+), 16 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 6556963..49785dc 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -83,14 +83,14 @@ architecture, planning, execution, review, validation, and release work. idempotently bootstraps both global skill targets and the current repository's bounded instruction block; other agent-resource changes use explicit `xdocs agent` actions. -- `xdocs scan` walks the project tree while respecting `scan.exclude`, root and nested `.gitignore` files when enabled, and explicit frontmatter opt-outs; it reports named `*.xdocs.md` descriptor coverage plus same-directory Markdown companion-document coverage. +- `xdocs scan` walks the project tree while respecting `scan.exclude`, root and nested `.gitignore` files when enabled, and explicit descriptor candidates; it reports complete named `*.xdocs.md` coverage plus same-directory Markdown companion-document coverage without granting write permission. - `xdocs generate [path]` generates documentation for a specific directory or the entire project. - `xdocs merge [path]` merges xdocs descriptors from a directory into a single consolidated document. - `xdocs tree` builds and displays the project hierarchy from xdocs metadata. - `xdocs list [path]` lists files in a scope with descriptions from xdocs metadata. -- `xdocs meta [path]` scans top-down and reads only YAML frontmatter from named `*.xdocs.md` descriptors; `--documents` also reads required companion `.md` frontmatter and reports configured opt-outs with `frontmatterRequired: false`, while `--owner`, `--tag`, and `--keyword` filter metadata before agents read full files. +- `xdocs meta [path]` scans top-down and reads only YAML frontmatter from named `*.xdocs.md` descriptors; `--documents` reports ordinary companions with explicit `frontmatterRequired: false` unless `documentation.frontmatter` opts them in within `documentation.directories`; `--existing-frontmatter` audits existing headers read-only, while `--owner`, `--tag`, and `--keyword` filter metadata before agents read full files. - `xdocs context [path]` recommends a minimal reading set for a task from descriptor, file, and companion-document metadata; use `--documents`, `--files`, `--limit`, and `--explain` for agent workflows. -- `xdocs doctor [path]` runs CI-friendly health checks for descriptor validity, required companion-document metadata, tree integrity, and documented file existence. +- `xdocs doctor [path]` runs CI-friendly health checks for descriptor validity, explicitly required companion-document metadata, tree integrity, and documented file existence; `--existing-frontmatter` audits legacy headers without authorizing writes. - `xdocs agent skill install|uninstall|update|list|show`, `agent instruction apply|remove|update|show`, and `agent prompt list|show` implement explicit RFC 0034 agent integration. - Skill mutations default global, use `--local` for project scope, and always target both `.agents/skills` and `.claude/skills`. - A bare xdocs invocation bootstraps shared agent resources, then prints the @@ -122,10 +122,10 @@ architecture, planning, execution, review, validation, and release work. ## Key Concepts -- xdocs descriptors use Markdown with YAML frontmatter and must be named `*.xdocs.md`; `.docs.md` is not supported and `.xdocs.md` by itself is invalid. Same-directory non-excluded plain `*.md` files are companion documents listed in the descriptor's `documents` metadata. Companion frontmatter is required unless a matching `ignore.rules` file or directory rule sets `frontmatter: false`; those documents remain tracked and must not be modified to add frontmatter. The root file is always `XDOCS.md` (uppercase, no prefix, no frontmatter). Use `xdocs meta [path] --documents --format json` when an agent needs descriptor and companion-document policy without reading full Markdown bodies. +- xdocs descriptors use Markdown with YAML frontmatter and must be named `*.xdocs.md`; `.docs.md` and `.xdocs.md` by themselves are invalid candidates. Same-directory non-excluded plain `*.md` files are companion documents listed in the descriptor's `documents` metadata. Ordinary Markdown frontmatter is not required or written unless a matching `documentation.frontmatter` rule is explicitly authorized inside `documentation.directories`; legacy `ignore.rules` denials always win. The root file is always `XDOCS.md` (uppercase, no prefix, no frontmatter). Use `xdocs meta [path] --documents --format json` when an agent needs descriptor and companion-document policy without reading full Markdown bodies. - Metadata fields: `subject`, `description`, `parent`, `children`, `files`, `documents`, `tags`, `keywords`, `flags`, and optional `status`. - The tree is a parent-child containment hierarchy, not a dependency graph. Built from `subject`/`parent`/`children` fields. -- Configuration lives in `xdocs.yaml`. Sections: `extensions`, `ai`, `ignore`, `scan`, and `project`. `ignore.gitignore` defaults to `true`; strict `ignore.rules` objects use `pattern`, `kind`, and `frontmatter: false`. +- Configuration lives in `xdocs.yaml`. Sections: `extensions`, `ai`, `documentation`, `ignore`, `scan`, and `project`. `documentation.directories` defaults to `[]` and grants descriptor maintenance only within listed repository-relative directories and descendants; `documentation.frontmatter` defaults to `[]` and grants companion metadata only for matching files or directories inside those grants. `ignore.gitignore` defaults to `true`; strict `ignore.rules` objects use `pattern`, `kind`, and `frontmatter: false`. - Agent resource operations are not configuration-driven. The plain root bootstrap and `init` setup are the only implicit/setup boundaries; other mutations are explicit. @@ -217,7 +217,8 @@ companion documents, repository scanning, metadata discovery, or validation. The project configuration is `xdocs.yaml`. Respect `ai.mode`: `prompt` requires confirmation before documentation writes, while -`auto` permits immediate descriptor maintenance. Also respect +`auto` permits immediate descriptor maintenance only inside explicitly +authorized documentation directories. Also respect `ignore.gitignore` and every `ignore.rules` entry: excluded paths are outside the xdocs corpus, while `frontmatter: false` keeps matching documents tracked without adding or requiring YAML frontmatter. Use `xdocs scan`, diff --git a/TODO.md b/TODO.md index 8235ffc..c66a1d4 100644 --- a/TODO.md +++ b/TODO.md @@ -31,7 +31,7 @@ All Rights Reserved. | Status | Count | | --- | ---: | | todo | 1 | -| in progress | 1 | +| in progress | 2 | | testing | 0 | | stopped | 0 | | completed | 7 | @@ -153,6 +153,14 @@ All Rights Reserved. `guiho-a-0047-plan-reviewer` exact-tree verdict before implementation. - Audit: [docs/reviews/implementation/guiho-cli-convention-0001-compliance-review.md](docs/reviews/implementation/guiho-cli-convention-0001-compliance-review.md) +### 10. Enforce Explicit Documentation Authorization And Complete Trees + +- Status: in progress +- Created: `2026-09-12` +- Updated: `2026-09-12` +- Outcome: Require explicit documentation directory and companion-frontmatter opt-ins, preserve ordinary Markdown, guard report destinations, and display every discovered named descriptor through arbitrary depth. +- Plan: [docs/plans/explicit-documentation-authorization.md](docs/plans/explicit-documentation-authorization.md) + ## Completion Archive Completed tasks and their task-owned evidence are registered in diff --git a/cmd/cmd.xdocs.md b/cmd/cmd.xdocs.md index 5369721..7b6f889 100644 --- a/cmd/cmd.xdocs.md +++ b/cmd/cmd.xdocs.md @@ -6,7 +6,7 @@ children: [] files: root.go: Root command, persistent flags, plain-invocation agent bootstrap routing, startup lifecycle, exit mapping, and hidden worker routes. help.go: Command-tree and Markdown help generated from live Cobra definitions. - domain.go: Adapters for init, scan, generate, merge, tree, list, meta, context, doctor, and frontmatter-policy output. + domain.go: Adapters for init, scan, generate, merge, tree, list, meta, context, doctor, existing-header audits, complete-tree diagnostics, and guarded report output. agent.go: Explicit skill, instruction, and prompt commands. upgrade.go: Release listing, checking, upgrading, and uninstall command adapters. uninstall_unix.go: Unix executable removal behavior. diff --git a/docs/plans/plans.xdocs.md b/docs/plans/plans.xdocs.md index 2e466c5..6e4d235 100644 --- a/docs/plans/plans.xdocs.md +++ b/docs/plans/plans.xdocs.md @@ -12,6 +12,7 @@ documents: upgrade-reliability-implementation.md: Executable plan for complete release discovery, immediate verified self-upgrade, recovery guidance, installer hardening, and validation. xdocs-0.7.0-welcome-update-pagination.md: Approved execution plan for issues 15 and 16, Mirror YAML migration, release validation, and public acceptance. xdocs-go-rewrite.md: Step-by-step native Go rewrite plan covering architecture, Cobra, YAML, domain parity, agents, updates, upgrades, installers, CI, documentation, validation, and release. + explicit-documentation-authorization.md: Execution plan for explicit documentation permissions, read-only ordinary Markdown handling, guarded report output, and complete descriptor trees. tags: - plans - implementation @@ -33,6 +34,8 @@ keywords: - read-only Mirror plan - upgrade reliability - Go rewrite + - explicit documentation authorization + - complete descriptor tree - Git-only versioning flags: [] status: stable diff --git a/internal/config/config.xdocs.md b/internal/config/config.xdocs.md index 727d652..54aa08b 100644 --- a/internal/config/config.xdocs.md +++ b/internal/config/config.xdocs.md @@ -1,11 +1,11 @@ --- subject: xdocs-internal-config -description: Strict xdocs.yaml discovery, decoding, defaults, ignore rules, and semantic validation. +description: Strict xdocs.yaml discovery, decoding, defaults, opt-in documentation authorization, ignore rules, and semantic validation. parent: xdocs-internal children: [] files: - config.go: Configuration precedence, known-field YAML decoding, single-document enforcement, defaults, Git-aware ignore/frontmatter rules, and validation. - config_test.go: Precedence, unknown-field, multiple-document, extension, AI mode, ignore-rule, and exclusion tests. + config.go: Configuration precedence, known-field YAML decoding, single-document enforcement, opt-in documentation directories and frontmatter rules, defaults, Git-aware ignore policy, and semantic validation. + config_test.go: Precedence, unknown-field, multiple-document, extension, AI mode, documentation-policy, ignore-rule, and exclusion tests. documents: {} tags: - configuration @@ -16,6 +16,7 @@ keywords: - known fields - gitignore - frontmatter opt-out + - documentation authorization flags: [] status: stable --- diff --git a/internal/xdocs/xdocs.xdocs.md b/internal/xdocs/xdocs.xdocs.md index b474703..48fd434 100644 --- a/internal/xdocs/xdocs.xdocs.md +++ b/internal/xdocs/xdocs.xdocs.md @@ -5,16 +5,18 @@ parent: xdocs-internal children: [] files: model.go: Shared descriptor, document, scan, context, and health result models. - ignore.go: Dependency-free root/nested .gitignore matching and explicit file/directory frontmatter policy. - metadata.go: YAML frontmatter extraction and typed descriptor/document decoding. - discovery.go: Exclusion-aware descriptor and companion-document discovery. + ignore.go: Dependency-free root/nested .gitignore matching, opt-in documentation authorization, and explicit file/directory frontmatter policy. + metadata.go: Exact YAML frontmatter extraction and typed descriptor/document decoding with read-only existing-header auditing. + discovery.go: Exclusion-aware complete descriptor and companion-document discovery, including invalid descriptor candidates and traversal diagnostics. + output.go: Preflighted, project-bounded, atomic single-file report output. tree.go: Synchronized parent-child, single-root, duplicate, orphan, and cycle validation with deterministic tree construction. meta.go: Metadata-only scoped reads and filters. context.go: Ranked minimal reading-set recommendations. doctor.go: Descriptor, companion, tree, and documented-file health checks. render.go: Deterministic text, Markdown, and JSON domain rendering. - xdocs_test.go: Discovery, metadata, context, tree, and doctor regression tests. - ignore_test.go: Gitignore, negation, file/directory rule, tracking, context, and doctor regression tests. + xdocs_test.go: Discovery, metadata, context, tree, doctor, and existing-header audit regression tests. + ignore_test.go: Gitignore, negation, authorization, file/directory rule, tracking, context, and doctor regression tests. + output_test.go: Atomic report output, descriptor rejection, exclusion, missing-parent, and symlink safety tests. documents: {} tags: - structured-documentation @@ -24,6 +26,7 @@ keywords: - frontmatter - gitignore - frontmatter opt-out + - explicit authorization - context - doctor flags: [] diff --git a/xdocs.yaml b/xdocs.yaml index 8505098..ec16ec8 100644 --- a/xdocs.yaml +++ b/xdocs.yaml @@ -4,6 +4,9 @@ extensions: - .xdocs.md ai: mode: auto +documentation: + directories: [] + frontmatter: [] ignore: gitignore: true rules: From f7e06c4a16c627442f63316f201603a6a0525c8c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Crist=C3=B3v=C3=A3o=20GUIHO?= Date: Sat, 12 Sep 2026 11:55:04 +0000 Subject: [PATCH 09/20] Clarify XDocs authorization in embedded skill --- skills/guiho-s-xdocs/SKILL.md | 103 ++++++++++++++++++++++++++++------ 1 file changed, 85 insertions(+), 18 deletions(-) diff --git a/skills/guiho-s-xdocs/SKILL.md b/skills/guiho-s-xdocs/SKILL.md index 248e5b7..0b5c1cc 100644 --- a/skills/guiho-s-xdocs/SKILL.md +++ b/skills/guiho-s-xdocs/SKILL.md @@ -17,6 +17,8 @@ metadata: version: "0.11.0" --- +#### © 2026 [GUIHO](https://guiho.co) as represented by [Cristóvão GUIHO](https://guiho.co/cguiho) All Rights Reserved. + # xdocs Structured Documentation ## When to use @@ -39,11 +41,33 @@ xdocs uses YAML only. Resolve it in this order: 2. project `xdocs.yaml`; 3. `~/.guiho/xdocs/xdocs.yaml`. -Read `ai.mode` before writing documentation. The default is `auto`: +Read `ai.mode` and the complete `documentation` policy before writing +documentation. The default is `auto`: + +- `auto`: make already-authorized descriptor or companion-document changes in + the same work unit. +- `prompt`: announce the already-authorized changes and wait for confirmation. + +`ai.mode` controls timing only. It never grants a directory or Markdown +frontmatter permission. + +The write policy is explicit and opt-in: + +```yaml +documentation: + directories: [] + frontmatter: [] +``` -- `auto`: make relevant documentation changes in the same work unit. -- `prompt`: announce the descriptors/documents that need updates and wait for - confirmation. +Each `documentation.directories` entry is a repository-relative literal +directory whose whole non-excluded subtree may receive descriptor maintenance. +`.` grants the whole non-excluded project. An empty list grants no descriptor +writes. `documentation.frontmatter` contains explicit `{pattern, kind}` rules, +where `kind` is `file` or `directory`; a frontmatter rule is effective only +inside an authorized documentation directory. A matching legacy +`ignore.rules` entry with `frontmatter: false` always denies frontmatter and +wins over the opt-in rule. Never infer permission from discovery, `ai.mode`, or +an existing Markdown file. Read `ignore` before scanning or editing documentation: @@ -67,8 +91,10 @@ exception: it installs or refreshes the skill globally by default; pass One root `XDOCS.md` is the repository index and has no frontmatter. Every documented package, application, or module uses exactly one named -`*.xdocs.md` descriptor in its directory. `.xdocs.md` by itself and `.docs.md` -are invalid. +`*.xdocs.md` descriptor in its directory. Use the directory name when naming +the descriptor, for example `technologies/technologies.xdocs.md`. `.xdocs.md` +by itself and legacy `.docs.md` files are invalid descriptor names. The single +root `XDOCS.md` is a special index and is not a per-directory descriptor. Required descriptor frontmatter: @@ -89,12 +115,16 @@ status: stable --- ``` -The tree represents containment, not dependencies. `parent` and `children` -must agree. +The descriptor body must carry useful directory context; do not create a second +summary, overview, or detail Markdown file for that context. The tree represents +containment, not dependencies. `parent` and `children` must agree. Same-directory ordinary Markdown files are companion documents. List every -non-excluded document in the descriptor `documents` map. Give it frontmatter -unless a matching ignore rule sets `frontmatter: false`: +non-excluded document in the descriptor `documents` map, whether or not it has +frontmatter. Missing frontmatter is valid by default. Add or maintain ordinary +Markdown frontmatter only when both an explicit `documentation.frontmatter` +rule and an authorized `documentation.directories` entry match, and no legacy +`ignore.rules` denial matches: ```yaml --- @@ -104,9 +134,14 @@ keywords: [] --- ``` +The default legacy denials protect `AGENTS.md`, `README.md`, and `CLAUDE.md` at +any depth. Do not add frontmatter to those or other user Markdown files merely +because they are discovered or listed. An explicit project policy is required. + ## Workflow -1. Read `xdocs.yaml`, its `ai.mode`, and its complete `ignore` policy. +1. Read `xdocs.yaml`, its `ai.mode`, complete `documentation` policy, and + complete `ignore` policy. 2. Use metadata-first discovery: ```bash @@ -116,15 +151,19 @@ keywords: [] 3. Read only the recommended descriptors, implementation files, and companion documents. -4. Make the implementation/documentation change. -5. Update the owning descriptor without touching Git-ignored paths or adding - frontmatter to opted-out documents: +4. Before any write, verify that the target directory is covered by + `documentation.directories` and that the target Markdown frontmatter is + covered by `documentation.frontmatter` when applicable. A read-only scan, + tree, metadata, context, or doctor result never grants write permission. +5. Make the implementation/documentation change only within that authorization. +6. Update the owning descriptor without touching Git-ignored paths or adding + frontmatter to denied or non-opted-in documents: - add/remove/rename `files` entries; - add/remove/rename `documents` entries; - keep parent/children links synchronized; - refresh description, tags, keywords, flags, and status when behavior changed. -6. Validate the narrow touched scope: +7. Validate the narrow touched scope: ```bash xdocs meta --documents --strict @@ -134,6 +173,21 @@ keywords: [] 7. Widen validation only when the change affects repository-wide integrity. +`xdocs tree` is a read-only complete discovery view. It walks every +non-excluded directory to arbitrary depth, retains every named descriptor and +the special `XDOCS.md` path, and reports malformed, orphaned, duplicate, or +cyclic metadata without silently dropping a node. Its discovery is independent +of the descriptor write allowlist. + +`generate`, `merge`, and `tree` print their report to stdout unless the user +provides one exact `--output` path. A report destination authorizes that single +report only; it does not authorize descriptor or companion metadata edits. +Never use a report as a descriptor, create extra output files, or overwrite a +target after output validation fails. `meta`, `context`, and `doctor` are +read-only. `doctor --existing-frontmatter` explicitly audits existing headers +on ordinary Markdown without requiring headers, imposing xdocs ownership, or +writing repairs. + ## CLI catalog ```text @@ -155,6 +209,11 @@ xdocs upgrade list xdocs uninstall ``` +`xdocs meta --existing-frontmatter` and `xdocs doctor --existing-frontmatter` +validate only headers that already exist on otherwise non-required ordinary +Markdown. A missing header is valid in this audit mode; malformed existing +headers are reported. Neither command writes files. + Foreground commands read only `~/.guiho/xdocs/cache.json`. An expired cache may start one short-lived detached update worker. The cache-scoped lease coalesces simultaneous invocations, the entire remote check is bounded to 15 seconds, and @@ -237,6 +296,13 @@ xdocs agent prompt show write `ignore.gitignore: true`. - Keep `frontmatter: false` documents listed and trackable, but never add, rewrite, require, or recommend YAML frontmatter for them. +- Keep ordinary Markdown read-only unless an explicit frontmatter rule and an + authorized directory both match; legacy `frontmatter: false` denials win. +- Never create a bare `.xdocs.md`, a legacy `.docs.md`, an extra overview file, + or an unrequested report/output file. +- The explicit `xdocs init` root-index creation and agent-resource bootstrap + are setup exceptions. They do not authorize arbitrary Markdown metadata + maintenance. - Do not read whole repositories when metadata can select a smaller context. - Do not run skill or instruction mutations implicitly outside the documented plain-invocation bootstrap and explicit setup or agent-management actions. @@ -248,9 +314,10 @@ xdocs agent prompt show write - configuration and `ai.mode` were respected; - `.gitignore` and explicit ignore/frontmatter rules were respected; -- every changed module has accurate descriptor metadata; -- companion documents are listed; required frontmatter is owned and opted-out - documents were left untouched; +- every changed module has accurate descriptor metadata in an authorized + directory; +- companion documents are listed; ordinary Markdown without an explicit + frontmatter grant was left untouched; - tree links are consistent; - strict metadata and doctor checks pass for the touched scope; - all documentation references use `xdocs.yaml` and the singular `agent` From d81593d54d739ec22a8b1642d37ee13dac9929a3 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Crist=C3=B3v=C3=A3o=20GUIHO?= Date: Sat, 12 Sep 2026 11:57:51 +0000 Subject: [PATCH 10/20] Align XDocs prompts with explicit authorization --- prompts/agents.md | 47 +++++++++++++++++++++++++++++-------- prompts/generate.md | 29 ++++++++++++++++++----- prompts/update.md | 47 ++++++++++++++++++++++++++----------- prompts/write.md | 56 ++++++++++++++++++++++++++++++++------------- 4 files changed, 134 insertions(+), 45 deletions(-) diff --git a/prompts/agents.md b/prompts/agents.md index f0c9a44..0b175ba 100644 --- a/prompts/agents.md +++ b/prompts/agents.md @@ -14,26 +14,53 @@ keywords: - xdocs skill --- +#### © 2026 [GUIHO](https://guiho.co) as represented by [Cristóvão GUIHO](https://guiho.co/cguiho) All Rights Reserved. + # xdocs: Update AGENTS.md -You are an AI assistant tasked with updating the AGENTS.md file to include xdocs instructions. +You are an AI assistant tasked with applying the bounded, managed xdocs +instruction block to an agent instruction file. This is an agent-resource +setup operation, separate from documentation-corpus writes. ## Instructions 1. Read the existing AGENTS.md file. -2. Check if there is already an xdocs section (between `` and `` markers). -3. If the section exists, update it with the current xdocs configuration. -4. If the section does not exist, add it at the end of the file. +2. Check for the exact managed markers + `` and + ``. Preserve every byte outside that bounded block and + refuse malformed, duplicated, or reversed markers. +3. If the section exists, replace only its body with the current managed + instruction. If it does not exist, add the block at the end of the file. +4. Do not add YAML frontmatter to AGENTS.md, CLAUDE.md, README.md, or any + other ordinary Markdown file as part of this operation. The setup exception + permits only the bounded instruction block and the explicitly requested + agent-resource files. 5. The xdocs section should instruct AI agents to: - Read XDOCS.md and named `*.xdocs.md` descriptor files when entering the project - - Respect the configured AI behavior mode (prompt or auto) + - Use one named descriptor per directory, such as + `technologies/technologies.xdocs.md`; never use bare `.xdocs.md` or + legacy `.docs.md` + - Read `documentation.directories` before descriptor writes; directory + entries grant their non-excluded subtrees and `.` grants the project + - Treat an empty directory list as no write authorization + - Respect the configured AI behavior mode (prompt or auto) as timing only; + it never grants permission + - Read `documentation.frontmatter` before changing ordinary Markdown + frontmatter; it requires an explicit `{pattern, kind}` rule inside an + authorized directory - Respect `ignore.gitignore` exclusions and every file or directory rule in `ignore.rules` + - Let legacy `frontmatter: false` denials override frontmatter opt-ins - Use the xdocs CLI for documentation operations - - Maintain xdocs descriptors and companion-document metadata when modifying code - - Use only named `*.xdocs.md` descriptors, never nameless `.xdocs.md` files + - Keep ordinary Markdown listed in descriptor `documents` metadata even + when it has no frontmatter, without adding headers by default + - Keep directory context in the descriptor body; do not create extra + summary, overview, detail, or report files + - Use `xdocs tree` as a read-only complete tree of every non-excluded + descriptor at every depth, including the root `XDOCS.md` path + - Print `generate`, `merge`, and `tree` reports to stdout unless one exact + `--output` path is requested; a report is never a descriptor - List every same-directory plain `*.md` companion document in the descriptor's `documents` metadata map - - Maintain `keywords` in descriptor and required companion Markdown - frontmatter, but never add or require frontmatter when a matching rule sets - `frontmatter: false` + - Use `meta --existing-frontmatter` and `doctor --existing-frontmatter` for + read-only audits of existing headers; missing headers remain valid - Follow the metadata schema for frontmatter diff --git a/prompts/generate.md b/prompts/generate.md index 2e3f44e..5cc1a45 100644 --- a/prompts/generate.md +++ b/prompts/generate.md @@ -14,14 +14,21 @@ keywords: - project summary --- +#### © 2026 [GUIHO](https://guiho.co) as represented by [Cristóvão GUIHO](https://guiho.co/cguiho) All Rights Reserved. + # xdocs: Generate Comprehensive Documentation -You are an AI assistant tasked with generating comprehensive documentation for a domain or the entire project. +You are an AI assistant tasked with generating one comprehensive report from +the available xdocs metadata. Report generation is read-only unless the user +names one exact output path. ## Instructions -1. Read `xdocs.yaml`, including `ignore.gitignore` and `ignore.rules`. -2. Scan all non-excluded xdocs descriptors in the target scope (directory or project). +1. Read `xdocs.yaml`, including `documentation.directories`, + `documentation.frontmatter`, `ignore.gitignore`, and `ignore.rules`. +2. Scan all non-excluded named xdocs descriptors in the target scope, at every + depth. Discovery is independent of `documentation.directories`; do not omit + an existing descriptor because its directory is not write-authorized. 3. Read every named `*.xdocs.md` descriptor's YAML frontmatter first, then read non-excluded source files and same-directory plain `*.md` documents only when needed. Treat documents configured with `frontmatter: false` as tracked @@ -34,12 +41,22 @@ You are an AI assistant tasked with generating comprehensive documentation for a - What each file does - Which companion Markdown documents belong to each module - Which keywords identify each module and companion document -5. Generate a single comprehensive Markdown document that includes: +5. Include the complete deepest tree. Retain every named descriptor path, + including malformed, orphaned, duplicate, or otherwise diagnostically + available entries, and include the special root `XDOCS.md` index path. +6. Generate a single comprehensive Markdown report that includes: - Project or domain overview - Complete hierarchy tree - Detailed description of each module - File listings with descriptions - Companion document listings with descriptions - Cross-references between related modules -6. The output should be a self-contained document that fully describes the scope. -7. Use clear headings, consistent formatting, and concise language. +7. Print the report to stdout unless the user explicitly provides one exact + `--output` path. An output path authorizes that report only. Preflight the + destination before replacing it, reject descriptor destinations such as + `module/module.xdocs.md`, and preserve an existing target when validation + fails. +8. Never add frontmatter to README, AGENTS, or another ordinary Markdown file + as part of generation. Never create extra report files, descriptors, or + companion documents. A generated report is not a valid xdocs descriptor. +9. Use clear headings, consistent formatting, and concise language. diff --git a/prompts/update.md b/prompts/update.md index 5897536..c17ce8a 100644 --- a/prompts/update.md +++ b/prompts/update.md @@ -14,40 +14,61 @@ keywords: - documentation maintenance --- +#### © 2026 [GUIHO](https://guiho.co) as represented by [Cristóvão GUIHO](https://guiho.co/cguiho) All Rights Reserved. + # xdocs: Update Documentation -You are an AI assistant tasked with updating existing xdocs documentation after code changes. +You are an AI assistant tasked with synchronizing existing xdocs descriptors +after code or document changes, within the project's explicit authorization. ## Instructions -1. Read `xdocs.yaml`, including `ignore.gitignore` and `ignore.rules`. -2. Identify which non-excluded files have changed in the recent modifications. -3. Find the named `*.xdocs.md` descriptor files that document the directories - containing those changes. There must be exactly one descriptor per directory, - and it must not be named only `.xdocs.md`. -4. For each affected xdocs descriptor: +1. Read `xdocs.yaml`, including `documentation.directories`, + `documentation.frontmatter`, `ai.mode`, `ignore.gitignore`, and + `ignore.rules`. +2. Identify which non-excluded files changed. Discovery is read-only and must + include all non-excluded descriptor paths at every depth, regardless of the + write allowlist. +3. Find the existing named `*.xdocs.md` descriptor for each changed directory. + There must be exactly one descriptor per directory. A bare `.xdocs.md` and + legacy `.docs.md` are invalid; report them instead of repairing or reusing + them. +4. Before editing a descriptor, verify that its directory is covered by + `documentation.directories` or by `.`. If it is not covered, leave the + descriptor and every ordinary Markdown file unchanged and report the + required authorization. +5. For each authorized affected xdocs descriptor: a. Re-read the files listed in the metadata to check if descriptions are still accurate. b. Check if new files were added that need to be listed. c. Check if files were removed that should be unlisted. d. Check if non-excluded sibling plain `*.md` documents were added or removed, and keep the `documents` metadata map exact. - e. Keep documents matched by an `ignore.rules` entry with - `frontmatter: false` listed, but never add, rewrite, require, or recommend - YAML frontmatter for them. + e. Keep every discovered ordinary Markdown file listed even when it has no + frontmatter. Add or update its frontmatter only when an explicit + `documentation.frontmatter` `{pattern, kind}` rule matches inside an + authorized directory. A matching legacy `ignore.rules` denial with + `frontmatter: false` always wins. f. Update the description if the module's purpose has changed. g. Update `keywords` if the module's searchable concepts changed. h. Update children if subdirectories were added or removed. i. Update the body content if significant changes occurred. -5. Preserve the existing structure and style of the xdocs descriptor. -6. Do not remove information that is still accurate. +6. Preserve the existing descriptor structure, useful body context, and all + information that remains accurate. +7. Do not create extra summary, overview, detail, companion, report, or index + Markdown files. The descriptor body carries directory context. +8. `ai.mode: auto` permits only these already-authorized writes; `prompt` asks + before them. Neither mode grants permission. ## Checklist - [ ] All new files are listed in the files metadata - [ ] Removed files are no longer listed - [ ] All sibling plain Markdown documents are listed in the documents metadata -- [ ] Git-ignored paths are absent and frontmatter-opted-out documents remain listed without frontmatter edits +- [ ] Git-ignored paths are absent and documents without an explicit + frontmatter grant remain listed without frontmatter edits - [ ] Removed Markdown documents are no longer listed +- [ ] The changed descriptor directory is explicitly authorized +- [ ] Legacy `frontmatter: false` denials were applied before opt-ins - [ ] File descriptions are accurate - [ ] Module description reflects current state - [ ] Keywords reflect current searchable concepts diff --git a/prompts/write.md b/prompts/write.md index 9772a25..7e8480c 100644 --- a/prompts/write.md +++ b/prompts/write.md @@ -14,18 +14,33 @@ keywords: - module documentation --- +#### © 2026 [GUIHO](https://guiho.co) as represented by [Cristóvão GUIHO](https://guiho.co/cguiho) All Rights Reserved. + # xdocs: Write Documentation -You are an AI assistant tasked with writing xdocs documentation for a directory/module. +You are an AI assistant tasked with writing one xdocs descriptor for a directory +that the project has explicitly authorized for documentation. ## Instructions -1. Read `xdocs.yaml`, including `ignore.gitignore` and `ignore.rules`. -2. Scan the target directory and all its subdirectories without entering paths - excluded by `.gitignore` when enabled. -3. Read every non-excluded source file needed to understand what it does. -4. Identify the purpose of this module/directory. -5. Create a named xdocs descriptor with YAML frontmatter containing: +1. Read `xdocs.yaml`, including `documentation.directories`, + `documentation.frontmatter`, `ai.mode`, `ignore.gitignore`, and + `ignore.rules`. +2. Resolve the exact target directory. A descriptor write is allowed only when + `documentation.directories` contains that directory or an ancestor, or + contains `.`. An empty list grants no descriptor writes. If the target is + not authorized, report the missing authorization and do not create or edit + any file. +3. Scan the target and its non-excluded contents without entering paths + excluded by root or nested `.gitignore` files or `scan.exclude`. +4. Read only the non-excluded source and descriptor content needed to write + useful context for this directory. +5. Create or update exactly one descriptor for the requested directory. Name a + new descriptor after the directory, for example + `technologies/technologies.xdocs.md`. Preserve an existing valid named + descriptor instead of renaming it automatically. A bare `.xdocs.md` and a + legacy `.docs.md` are invalid and must be reported, not reused. +6. Create the descriptor with YAML frontmatter containing: - subject: A short identifier for this module - description: A concise description of what this module does - parent: The parent module's subject (or null if this is a root module) @@ -35,18 +50,27 @@ You are an AI assistant tasked with writing xdocs documentation for a directory/ - tags: Relevant tags (empty array if none) - keywords: Search terms and concepts that should help agents match requests - flags: Relevant flags (empty array if none) -6. Write a Markdown body below the frontmatter with: +7. Write meaningful directory context in the descriptor body below the + frontmatter, including: - An overview section explaining the module in more detail - Usage examples if relevant - Any important notes or caveats -7. Name the file as `.xdocs.md` in the target directory. Never name - a file only `.xdocs.md`; `.xdocs.md` is the extension, not the full filename. -8. Use only `.xdocs.md` for xdocs descriptors. Do not create `.docs.md` files. -9. If the directory contains non-excluded sibling plain `*.md` files, list each - one under `documents` in this descriptor. -10. When an `ignore.rules` file or directory rule matches a document with - `frontmatter: false`, keep the document listed but never add, rewrite, - require, or recommend YAML frontmatter for it. +8. List every non-excluded same-directory plain Markdown file in the + descriptor's `documents` map, including files without frontmatter. +9. Add or update frontmatter on an ordinary Markdown companion only when its + path matches an explicit `documentation.frontmatter` rule and is inside an + authorized documentation directory. A legacy `ignore.rules` entry with + `frontmatter: false` always denies the write. Missing frontmatter is valid by + default; never add it to `README.md`, `AGENTS.md`, or another user document + merely because it is discoverable or listed. +10. Do not create a separate overview, summary, detail, companion, report, or + index Markdown file. The descriptor is the directory's context. Do not + create descriptors for other directories unless each is explicitly + requested and covered by the same authorization. +11. Keep parent and child metadata consistent with the existing descriptor + tree. Reject the write before changing any target if the descriptor is + malformed, duplicated, excluded, outside the repository, or would create a + second descriptor in the directory. ## Frontmatter Template From 80481a92f31abfcf42a119a01acd568a62c23015 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Crist=C3=B3v=C3=A3o=20GUIHO?= Date: Sat, 12 Sep 2026 11:58:25 +0000 Subject: [PATCH 11/20] Expand XDocs tree regression coverage --- internal/xdocs/tree_test.go | 125 ++++++++++++++++++++++++++++++++++-- 1 file changed, 121 insertions(+), 4 deletions(-) diff --git a/internal/xdocs/tree_test.go b/internal/xdocs/tree_test.go index bf3887b..a8b92f1 100644 --- a/internal/xdocs/tree_test.go +++ b/internal/xdocs/tree_test.go @@ -2,18 +2,21 @@ package xdocs import ( "encoding/json" + "os" "path/filepath" "reflect" "sort" "strings" "testing" + + "github.com/CGuiho/xdocs/internal/config" ) func TestBuildTreeContainsEveryDiscoveredDescriptorByDirectory(t *testing.T) { root := t.TempDir() missingParent := "missing-parent" files := []File{ - treeTestFile(root, "XDOCS.md", false, nil), + treeTestFile(root, "XDOCS.md", true, nil), treeTestFile(root, "technologies/technologies.xdocs.md", true, treeTestMetadata("technologies", "Technology directory context.", nil)), treeTestFile(root, "technologies/cloud/cloud.xdocs.md", true, treeTestMetadata("cloud", "Cloud directory context.", treeStringPointer("technologies"))), treeTestFile(root, "technologies/cloud/observability/observability.xdocs.md", true, treeTestMetadata("observability", "Observability directory context.", treeStringPointer("cloud"))), @@ -74,8 +77,8 @@ func TestBuildTreeContainsEveryDiscoveredDescriptorByDirectory(t *testing.T) { if index.Subject != "XDOCS.md" { t.Errorf("root index subject = %q, want XDOCS.md", index.Subject) } - if index.Valid { - t.Error("special root index should preserve its non-descriptor validity state") + if !index.Valid { + t.Error("special root index should be a valid navigable tree entry") } invalid := byPath["technologies/cloud/broken/broken.xdocs.md"][0] @@ -155,7 +158,7 @@ func TestTreeRenderersAreDeterministicAndExposeAllPathsAndContext(t *testing.T) treeTestFile(root, "zeta/zeta.xdocs.md", true, treeTestMetadata("zeta", "Zeta context.", nil)), treeTestFile(root, "alpha/alpha.xdocs.md", true, treeTestMetadata("alpha", "Alpha context.", nil)), treeTestFile(root, "alpha/deep/broken.xdocs.md", false, nil, "Missing YAML frontmatter."), - treeTestFile(root, "XDOCS.md", false, nil), + treeTestFile(root, "XDOCS.md", true, nil), } reversed := append([]File(nil), files...) for left, right := 0, len(reversed)-1; left < right; left, right = left+1, right-1 { @@ -192,6 +195,103 @@ func TestTreeRenderersAreDeterministicAndExposeAllPathsAndContext(t *testing.T) } } } + for _, context := range []string{"Alpha context.", "Zeta context.", "Missing YAML frontmatter."} { + if !strings.Contains(textFirst, context) { + t.Errorf("text tree omitted available context or diagnostic %q:\n%s", context, textFirst) + } + } + + for path, wantPrefix := range map[string]string{ + "XDOCS.md": "| |- ", + "alpha/alpha.xdocs.md": "| |- ", + "alpha/deep/broken.xdocs.md": "| |- ", + "zeta/zeta.xdocs.md": "| |- ", + } { + line := treeTestRenderedLine(textFirst, path) + if line == "" { + t.Errorf("text tree omitted line for %q:\n%s", path, textFirst) + continue + } + if !strings.HasPrefix(line, wantPrefix) { + t.Errorf("text tree line for %q = %q, want prefix %q", path, line, wantPrefix) + } + } +} + +func TestScanTreeRetainsMalformedDescriptorsAndExcludesIgnoredDescriptors(t *testing.T) { + root := t.TempDir() + if err := os.MkdirAll(filepath.Join(root, "visible", "deep"), 0o755); err != nil { + t.Fatal(err) + } + if err := os.MkdirAll(filepath.Join(root, "ignored"), 0o755); err != nil { + t.Fatal(err) + } + writeTreeTestDiskFile(t, root, "XDOCS.md", "# Root\n") + writeTreeTestDiskFile(t, root, ".gitignore", "ignored/\n") + writeTreeTestDiskFile(t, root, "visible/visible.xdocs.md", `--- +subject: visible +description: Visible directory context. +parent: null +children: [] +files: {} +documents: {} +tags: [] +keywords: [] +flags: [] +--- +# Visible +`) + writeTreeTestDiskFile(t, root, "visible/deep/broken.xdocs.md", "---\nsubject: broken\n") + writeTreeTestDiskFile(t, root, "ignored/ignored.xdocs.md", `--- +subject: ignored +description: This descriptor is ignored. +parent: null +children: [] +files: {} +documents: {} +tags: [] +keywords: [] +flags: [] +--- +`) + + cfg, err := config.Defaults(root) + if err != nil { + t.Fatal(err) + } + scan, err := ScanProject(cfg) + if err != nil { + t.Fatal(err) + } + paths := make([]string, 0, len(scan.XDocsFiles)) + for _, file := range scan.XDocsFiles { + paths = append(paths, file.RelativePath) + } + sort.Strings(paths) + if !reflect.DeepEqual(paths, []string{"XDOCS.md", "visible/deep/broken.xdocs.md", "visible/visible.xdocs.md"}) { + t.Fatalf("scan descriptor paths = %#v", paths) + } + + tree := BuildTree(scan.XDocsFiles) + text := RenderTree(tree) + for _, path := range []string{"XDOCS.md", "visible/visible.xdocs.md", "visible/deep/broken.xdocs.md"} { + if !strings.Contains(text, path) { + t.Errorf("scanned tree omitted %q:\n%s", path, text) + } + } + if strings.Contains(text, "ignored/ignored.xdocs.md") { + t.Errorf("scanned tree included gitignored descriptor:\n%s", text) + } + if got := treeTestParentPaths(tree)["visible/deep/broken.xdocs.md"]; got != "visible/visible.xdocs.md" { + t.Errorf("scanned malformed descriptor parent = %q, want visible descriptor", got) + } + broken := treeTestRenderedLine(text, "visible/deep/broken.xdocs.md") + if broken == "" || !strings.Contains(broken, "Missing YAML frontmatter") { + t.Errorf("scanned tree lost malformed descriptor diagnostic: %q\n%s", broken, text) + } + if rootIndex := treeTestRenderedLine(text, "XDOCS.md"); rootIndex == "" || strings.Contains(rootIndex, "[invalid]") { + t.Errorf("scanned root index should be valid and distinguishable: %q", rootIndex) + } } func treeTestFile(root, relativePath string, valid bool, metadata *Metadata, errors ...string) File { @@ -207,6 +307,14 @@ func treeTestFile(root, relativePath string, valid bool, metadata *Metadata, err } } +func writeTreeTestDiskFile(t *testing.T, root, relativePath, content string) { + t.Helper() + path := filepath.Join(root, filepath.FromSlash(relativePath)) + if err := os.WriteFile(path, []byte(content), 0o644); err != nil { + t.Fatal(err) + } +} + func treeTestMetadata(subject, description string, parent *string) *Metadata { return &Metadata{ Subject: subject, @@ -263,6 +371,15 @@ func treeTestParentPaths(root *TreeNode) map[string]string { return result } +func treeTestRenderedLine(rendered, path string) string { + for _, line := range strings.Split(rendered, "\n") { + if strings.Contains(line, path) { + return line + } + } + return "" +} + func treeStringPointer(value string) *string { return &value } From 1f2fd3c946f39f48118a5c502a0dc2d10747a67c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Crist=C3=B3v=C3=A3o=20GUIHO?= Date: Sat, 12 Sep 2026 11:58:32 +0000 Subject: [PATCH 12/20] Align managed XDocs instruction with authorization policy --- internal/agent/agent.go | 36 +++++++++++++++++++++++++++++------- internal/agent/agent_test.go | 20 ++++++++++++++++++++ 2 files changed, 49 insertions(+), 7 deletions(-) diff --git a/internal/agent/agent.go b/internal/agent/agent.go index 35aff28..fc54d03 100644 --- a/internal/agent/agent.go +++ b/internal/agent/agent.go @@ -27,15 +27,37 @@ This project uses **xdocs** for structured, machine-readable documentation. Load the ` + "`guiho-s-xdocs`" + ` agent skill when working with structured documentation, ` + "`XDOCS.md`" + ` indexes, named ` + "`*.xdocs.md`" + ` descriptors, companion documents, repository scanning, metadata discovery, or validation. +Use exactly one named descriptor per directory, such as +` + "`technologies/technologies.xdocs.md`" + `. The bare ` + "`.xdocs.md`" + ` and legacy +` + "`.docs.md`" + ` names are invalid; the root ` + "`XDOCS.md`" + ` index is a special setup +file, not a second per-directory descriptor. The project configuration is ` + "`xdocs.yaml`" + `. Respect ` + "`ai.mode`" + `: -` + "`prompt`" + ` requires confirmation before documentation writes, while -` + "`auto`" + ` permits immediate descriptor maintenance. Also respect -` + "`ignore.gitignore`" + ` and every ` + "`ignore.rules`" + ` entry: excluded paths are -outside the xdocs corpus, while ` + "`frontmatter: false`" + ` keeps matching documents -tracked without adding or requiring YAML frontmatter. Use ` + "`xdocs scan`" + `, -` + "`xdocs meta`" + `, ` + "`xdocs context`" + `, ` + "`xdocs tree`" + `, and -` + "`xdocs doctor`" + ` to discover and validate documentation. +` + "`prompt`" + ` asks before already-authorized documentation writes, while +` + "`auto`" + ` performs them immediately. Neither mode grants permission. +` + "`documentation.directories`" + ` lists repository-relative directories whose +non-excluded subtrees may receive descriptor writes; ` + "`.`" + ` grants the whole +project and an empty list grants none. ` + "`documentation.frontmatter`" + ` contains +explicit ` + "`{pattern, kind}`" + ` rules for ordinary Markdown metadata, and those +rules work only inside an authorized directory. A legacy ` + "`ignore.rules`" + ` entry +with ` + "`frontmatter: false`" + ` always denies frontmatter and wins over an opt-in. +Keep ordinary Markdown listed in descriptor ` + "`documents`" + ` metadata without +adding headers unless both explicit grants match. Respect +` + "`ignore.gitignore`" + ` and every ` + "`ignore.rules`" + ` entry. + +Read-only ` + "`xdocs scan`" + `, ` + "`xdocs meta`" + `, ` + "`xdocs context`" + `, +` + "`xdocs tree`" + `, and ` + "`xdocs doctor`" + ` discovery is independent of the write +allowlist. ` + "`xdocs tree`" + ` walks every non-excluded directory to arbitrary depth, +retains every named descriptor and the special root index, and reports malformed +or orphaned metadata without hiding paths. ` + "`meta --existing-frontmatter`" + ` and +` + "`doctor --existing-frontmatter`" + ` audit existing ordinary Markdown headers +without requiring missing headers or writing repairs. ` + "`generate`" + `, +` + "`merge`" + `, and ` + "`tree`" + ` print reports to stdout unless one exact +` + "`--output`" + ` path is requested; that report path does not authorize any other +document writes and a generated report is never a descriptor. + +The explicit ` + "`xdocs init`" + ` root-index creation and agent-resource bootstrap +are setup exceptions. They do not authorize arbitrary Markdown metadata edits. ` type Service struct { diff --git a/internal/agent/agent_test.go b/internal/agent/agent_test.go index 8923568..c92cb9f 100644 --- a/internal/agent/agent_test.go +++ b/internal/agent/agent_test.go @@ -132,6 +132,26 @@ func TestPromptCatalogIsEmbedded(t *testing.T) { } } +func TestInstructionTemplateStatesExplicitDocumentationAuthorization(t *testing.T) { + for _, required := range []string{ + "documentation.directories", + "documentation.frontmatter", + "ai.mode", + "frontmatter: false", + "exactly one named descriptor per directory", + "xdocs tree", + "existing-frontmatter", + "--output", + } { + if !strings.Contains(InstructionTemplate, required) { + t.Errorf("instruction template omits %q", required) + } + } + if strings.Contains(InstructionTemplate, "requires confirmation before documentation writes") { + t.Error("instruction template still treats ai.mode as the authorization policy") + } +} + func TestSkillInstallStagesBothTargetsBeforeMutation(t *testing.T) { root := t.TempDir() agentsSkill := filepath.Join(root, ".agents", "skills", SkillName) From c982c18836c4c9f0a66fa78ca47d6ae353096e60 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Crist=C3=B3v=C3=A3o=20GUIHO?= Date: Sat, 12 Sep 2026 11:59:47 +0000 Subject: [PATCH 13/20] Document explicit XDocs write authorization --- README.md | 75 +++++++++++++++++++++++++++++++++++++++++++++---------- 1 file changed, 62 insertions(+), 13 deletions(-) diff --git a/README.md b/README.md index af917af..ed3e578 100644 --- a/README.md +++ b/README.md @@ -3,9 +3,10 @@ # GUIHO XDocs XDocs is a native Go CLI for structured repository documentation. It discovers -named `*.xdocs.md` descriptors, validates companion Markdown metadata, renders -containment trees, recommends minimal reading sets, and reports documentation -health issues. +every non-excluded named `*.xdocs.md` descriptor at every depth, renders a +complete containment tree, recommends minimal reading sets, and reports +documentation health issues. Descriptor and ordinary Markdown writes are +explicitly opt-in per project directory. The shipping runtime uses Go 1.26.5, Cobra, strict YAML structs, embedded agent resources, local cached update notices, safe self-upgrades, and exactly eleven @@ -52,6 +53,9 @@ xdocs doctor `xdocs init` creates `xdocs.yaml` and `XDOCS.md`, then installs the skill globally by default. Use `xdocs init --local` for project-local skill targets. +The generated configuration starts with no documentation write grants, so +initialization does not authorize descriptor or ordinary Markdown metadata +changes. ## Configuration @@ -68,6 +72,9 @@ extensions: - .xdocs.md ai: mode: auto +documentation: + directories: [] + frontmatter: [] ignore: gitignore: true rules: @@ -94,9 +101,24 @@ project: name: example ``` -`ai.mode` defaults to `auto`. The supported modes are `auto`, which makes -relevant documentation changes in the same work unit, and `prompt`, which -announces needed documentation changes and waits for confirmation. +`ai.mode` defaults to `auto`. `auto` performs documentation changes that are +already authorized by `documentation`; `prompt` announces those changes and +waits for confirmation. The mode controls timing only and never grants write +permission. + +`documentation.directories` is a list of repository-relative literal directory +paths. Each entry authorizes descriptor maintenance in that directory and its +non-excluded descendants. `.` authorizes the whole non-excluded project. An +empty list authorizes no descriptor writes. XDocs does not populate this list +during `init`. + +`documentation.frontmatter` is a list of explicit rules with `pattern` and +`kind` (`file` or `directory`). A rule authorizes ordinary Markdown frontmatter +only when the document is also below an authorized documentation directory. +Legacy `ignore.rules` entries with `frontmatter: false` always deny the +frontmatter write and take precedence. Discovery can still list those +documents in a descriptor's `documents` map, and missing frontmatter is valid +unless both opt-ins match. `ignore.gitignore` defaults to `true`, so Git-ignored files and directories do not enter xdocs discovery, metadata, context, or health checks. Each strict @@ -129,7 +151,8 @@ entries fail explicitly. ## Descriptor model -Each documented module directory has exactly one named descriptor: +Each documented module directory has exactly one named descriptor. Name a new +descriptor after its directory, such as `technologies/technologies.xdocs.md`: ```yaml --- @@ -149,10 +172,17 @@ flags: [] --- ``` -Every non-excluded plain sibling Markdown file must be declared in `documents`. -Unless a matching ignore rule sets `frontmatter: false`, it must have -frontmatter containing `name`, `purpose`, `description`, `created`, `owner`, -`flags`, `tags`, and `keywords`. +The descriptor body carries the directory's useful overview, details, and +usage context. Do not create a separate summary or detail Markdown file for +that context. The bare `.xdocs.md` name and legacy `.docs.md` files are invalid +descriptors. The root `XDOCS.md` is the single special project index. + +Every non-excluded plain sibling Markdown file must be declared in `documents`, +whether or not it has frontmatter. Ordinary Markdown needs the standard +frontmatter fields only when an explicit `documentation.frontmatter` rule and +an authorized directory match. The default legacy denials protect `README.md`, +`AGENTS.md`, and `CLAUDE.md`; do not add frontmatter to those or other user +documents merely because XDocs discovers them. ## Commands @@ -162,9 +192,9 @@ frontmatter containing `name`, `purpose`, `description`, `created`, `owner`, - `xdocs merge [path] [--output ]` - `xdocs tree [--output ]` - `xdocs list [path]` -- `xdocs meta [path] [--documents] [--strict]` +- `xdocs meta [path] [--documents] [--existing-frontmatter] [--strict]` - `xdocs context [path] [--documents] [--files]` -- `xdocs doctor [path] [--no-documents] [--warnings-as-errors]` +- `xdocs doctor [path] [--no-documents] [--existing-frontmatter] [--warnings-as-errors]` - `xdocs agent skill install|uninstall|update|list|show` - `xdocs agent instruction apply|remove|update|show` - `xdocs agent prompt list|show` @@ -177,6 +207,20 @@ Every scope supports `-h`/`--help`, `--help-tree`, `--help-tree-depth `, and `--help-docs`. Only the root supports `-v`/`--version`. Use `--format text|json|markdown` for stable output. +`scan`, `meta`, `context`, `doctor`, and `tree` are read-only. `tree` walks all +non-excluded directories to arbitrary depth, includes every discovered named +descriptor and the special `XDOCS.md` path, and reports malformed or orphaned +metadata without hiding a node. `meta --existing-frontmatter` and +`doctor --existing-frontmatter` audit existing ordinary Markdown headers only: +missing headers remain valid, malformed existing headers are reported, and no +repair or ownership is inferred. + +`generate`, `merge`, and `tree` print reports to stdout unless one exact +`--output` path is supplied. That path authorizes only the requested report; +it does not authorize metadata edits or additional files. Descriptor-shaped +destinations are rejected, and a rejected destination is left byte-for-byte +unchanged. + ## Agents The binary embeds: @@ -195,6 +239,11 @@ bootstrap boundary; help, version, data commands, explicit agent management, upgrade, and uninstall do not run bootstrap. Bootstrap never scans or changes `XDOCS.md`, descriptors, or companion documents. +The managed instruction block teaches agents to inspect the documentation +allowlist before writing. Applying it never adds frontmatter to `AGENTS.md` or +other ordinary Markdown and does not grant corpus write permission. The +explicit `init` root-index and agent bootstrap behavior is a setup exception. + ## Updates and upgrades Ordinary commands perform no foreground network request. They read a validated From b1645e01668e6ac6ca018c673f711aaf5d91161d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Crist=C3=B3v=C3=A3o=20GUIHO?= Date: Sat, 12 Sep 2026 12:00:36 +0000 Subject: [PATCH 14/20] Document complete tree and opt-in metadata policy --- DOCS.md | 87 ++++++++++++++++++++++++++++++++++++++++++--------------- 1 file changed, 65 insertions(+), 22 deletions(-) diff --git a/DOCS.md b/DOCS.md index 4a392df..eba5aa2 100644 --- a/DOCS.md +++ b/DOCS.md @@ -16,13 +16,16 @@ keywords: - agent resources --- +#### © 2026 [GUIHO](https://guiho.co) as represented by [Cristóvão GUIHO](https://guiho.co/cguiho) All Rights Reserved. + # xdocs Documentation ## Purpose xdocs gives humans and agents a deterministic map of a repository through one -root `XDOCS.md`, one named `*.xdocs.md` descriptor per documented module, and -declared companion Markdown documents. +root `XDOCS.md`, one named `*.xdocs.md` descriptor per documented directory, and +declared companion Markdown documents. Discovery is read-only and complete; +descriptor and ordinary Markdown writes require explicit project policy. The active implementation is a native Go CLI. The historical TypeScript tree is retained as migration reference only and is not used by the executable, @@ -83,6 +86,10 @@ multiple YAML documents, and then validates: - `schema` is `1`; - the only descriptor extension is `.xdocs.md`; - `ai.mode` defaults to `auto` and accepts only `auto` or `prompt`; +- `documentation.directories` defaults to an empty list and contains only + repository-relative literal directory paths; +- `documentation.frontmatter` defaults to an empty list and each rule contains + a repository-relative `pattern` and `kind` of `file` or `directory`; - `ignore.gitignore` defaults to `true`; - each `ignore.rules` entry has a non-empty, syntactically valid, repository-relative forward-slash glob, `kind` set to `file` or `directory`, @@ -92,9 +99,23 @@ multiple YAML documents, and then validates: Global state and update cache live under `~/.guiho/xdocs/`. -In `auto` mode, agents make relevant documentation changes in the same work -unit. In `prompt` mode, agents announce the needed documentation changes and -wait for confirmation. +In `auto` mode, agents make already-authorized documentation changes in the +same work unit. In `prompt` mode, agents announce those changes and wait for +confirmation. `ai.mode` controls timing only and never expands authorization. + +`documentation.directories` grants descriptor maintenance for each listed +directory and its non-excluded descendants. `.` grants the complete +non-excluded project; `[]` grants none. `xdocs init` writes the empty defaults +and does not populate this allowlist. Discovery, metadata reads, tree output, +and health checks remain available for every non-excluded directory regardless +of the allowlist. + +`documentation.frontmatter` grants ordinary Markdown frontmatter only when a +rule's `pattern` and `kind` match a document and the document is below an +authorized directory. A matching legacy `ignore.rules` entry with +`frontmatter: false` is an explicit denial and always wins. A plain Markdown +file can therefore remain listed in its descriptor's `documents` map without +frontmatter; missing headers are valid unless both opt-ins match. The default ignore contract is: @@ -155,16 +176,23 @@ Descriptors require: - `tags`, `keywords`, and `flags`: string arrays; - optional `status`. -The root `XDOCS.md` has no frontmatter. A bare `.xdocs.md` filename is invalid. -Multiple descriptors in one directory are invalid. Every plain sibling -Markdown document not excluded by `.gitignore` must be declared, and every -declared non-excluded document must exist. +The root `XDOCS.md` has no frontmatter and is the single special project index. +A new directory descriptor should use the directory name, such as +`technologies/technologies.xdocs.md`. A bare `.xdocs.md` filename and legacy +`.docs.md` files are invalid. Multiple named descriptors in one directory are +invalid. The descriptor body carries useful directory context; xdocs does not +create a separate summary or detail file for it. Every plain sibling Markdown +document not excluded by `.gitignore` must be declared, and every declared +non-excluded document must exist. Companion documents require `name`, `purpose`, `description`, `created` -(`YYYY-MM-DD`), `owner`, `flags`, `tags`, and `keywords` unless a matching -ignore rule sets `frontmatter: false`. `owner` must equal the owning descriptor -subject when frontmatter is required. Opted-out documents remain associated -with that descriptor subject without xdocs modifying their content. +(`YYYY-MM-DD`), `owner`, `flags`, `tags`, and `keywords` only when an explicit +`documentation.frontmatter` rule and an authorized directory match. The +`owner` must equal the owning descriptor subject when frontmatter is required. +Missing frontmatter is valid otherwise. The default legacy `ignore.rules` +denials protect `AGENTS.md`, `README.md`, and `CLAUDE.md`; XDocs leaves those and +other ordinary Markdown bodies and headers unchanged unless the project has +explicitly authorized the exact scope. ## Command catalog @@ -173,22 +201,28 @@ with that descriptor subject without xdocs modifying their content. - `init [--local]` creates missing root files and installs the embedded skill. - `scan` reports non-excluded descriptor and companion-document coverage and, in verbose output, identifies documents whose frontmatter is not required. -- `doctor [path]` validates descriptors, required companion metadata, tree - links, and documented files. `--warnings-as-errors` promotes warnings. +- `doctor [path]` validates descriptors, authorized companion metadata, tree + links, and documented files. `--existing-frontmatter` audits malformed + headers that already exist on otherwise non-required Markdown without + requiring ownership or writing repairs. `--warnings-as-errors` promotes + warnings. ### Documentation views -- `generate [path]` renders a project or module document. +- `generate [path]` renders a project or module report. - `merge [path]` combines descriptors with source markers. -- `tree` renders containment hierarchy as text, Markdown, or JSON. +- `tree` renders the complete deepest containment hierarchy as text, Markdown, + or JSON, including the special root index and diagnostically available + descriptor paths. - `list [path]` lists documented files and companion documents. ### Agent context -- `meta [path]` reads frontmatter only. `--documents` includes required - companion frontmatter and tracks opted-out documents with - `frontmatterRequired: false`; `--owner`, `--tag`, and `--keyword` filter - before full reads. +- `meta [path]` reads frontmatter only. `--documents` includes companion + frontmatter when it is required or already present and tracks ordinary + Markdown with `frontmatterRequired: false`; `--existing-frontmatter` audits + only existing headers. `--owner`, `--tag`, and `--keyword` filter before full + reads. - `context [path]` tokenizes a query, applies stable weighted ranking, and returns the smallest useful descriptor/file/document reading set. `--explain` includes match reasons. @@ -204,7 +238,8 @@ paths. `--local` chooses project scope. Instruction apply/update/remove is idempotent, preserves unmanaged content and line endings, and refuses malformed managed markers. The plain-invocation bootstrap uses the same embedded sources and mutation services; explicit agent commands remain available for deliberate -management. +management. These setup operations do not grant permission to edit the +documentation corpus or add frontmatter to agent instruction files. ### Upgrade and uninstall @@ -218,6 +253,14 @@ pagination, defaults to eight entries, and retains full machine-readable metadata in JSON. Direct upgrade uses the linker-embedded build target so ARMv6 and ARMv7 remain distinct. +All scan, meta, context, doctor, and tree operations are read-only. `generate`, +`merge`, and `tree` write nothing unless the user supplies one exact +`--output` path. That path authorizes only the requested report; it cannot +authorize descriptor or companion metadata changes, and a descriptor-shaped +destination is rejected. Output validation runs before replacement so a rejected +destination retains its original bytes. No command creates extra summary, +companion, or index files as a side effect. + ## Help and output Every command supports: From af9450e3d3a65d769902788326d08559ed10d08b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Crist=C3=B3v=C3=A3o=20GUIHO?= Date: Sat, 12 Sep 2026 12:01:08 +0000 Subject: [PATCH 15/20] Align tree tests with diagnostics and indentation --- internal/xdocs/tree_test.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/internal/xdocs/tree_test.go b/internal/xdocs/tree_test.go index a8b92f1..8b26166 100644 --- a/internal/xdocs/tree_test.go +++ b/internal/xdocs/tree_test.go @@ -286,7 +286,7 @@ flags: [] t.Errorf("scanned malformed descriptor parent = %q, want visible descriptor", got) } broken := treeTestRenderedLine(text, "visible/deep/broken.xdocs.md") - if broken == "" || !strings.Contains(broken, "Missing YAML frontmatter") { + if broken == "" || !strings.Contains(broken, "Malformed YAML frontmatter") { t.Errorf("scanned tree lost malformed descriptor diagnostic: %q\n%s", broken, text) } if rootIndex := treeTestRenderedLine(text, "XDOCS.md"); rootIndex == "" || strings.Contains(rootIndex, "[invalid]") { From e465fd8fabb0c5ddf1fd9959e344af2c6310c844 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Crist=C3=B3v=C3=A3o=20GUIHO?= Date: Sat, 12 Sep 2026 12:02:09 +0000 Subject: [PATCH 16/20] Tighten documentation policy guidance --- DOCS.md | 16 ++++++++-------- README.md | 8 ++++---- internal/agent/agent.go | 4 ++-- prompts/agents.md | 5 +++-- skills/guiho-s-xdocs/SKILL.md | 18 +++++++++++++----- 5 files changed, 30 insertions(+), 21 deletions(-) diff --git a/DOCS.md b/DOCS.md index eba5aa2..3e609a8 100644 --- a/DOCS.md +++ b/DOCS.md @@ -218,11 +218,11 @@ explicitly authorized the exact scope. ### Agent context -- `meta [path]` reads frontmatter only. `--documents` includes companion - frontmatter when it is required or already present and tracks ordinary - Markdown with `frontmatterRequired: false`; `--existing-frontmatter` audits - only existing headers. `--owner`, `--tag`, and `--keyword` filter before full - reads. +- `meta [path]` reads frontmatter only. `--documents` reads companion + frontmatter when it is required and tracks ordinary Markdown with + `frontmatterRequired: false`; `--existing-frontmatter` additionally reads + and validates headers that already exist on otherwise non-required documents. + `--owner`, `--tag`, and `--keyword` filter before full reads. - `context [path]` tokenizes a query, applies stable weighted ranking, and returns the smallest useful descriptor/file/document reading set. `--explain` includes match reasons. @@ -253,9 +253,9 @@ pagination, defaults to eight entries, and retains full machine-readable metadata in JSON. Direct upgrade uses the linker-embedded build target so ARMv6 and ARMv7 remain distinct. -All scan, meta, context, doctor, and tree operations are read-only. `generate`, -`merge`, and `tree` write nothing unless the user supplies one exact -`--output` path. That path authorizes only the requested report; it cannot +All scan, meta, context, and doctor operations are read-only. Without +`--output`, tree discovery is also read-only. `generate`, `merge`, and `tree` +write nothing unless the user supplies one exact `--output` path. That path authorizes only the requested report; it cannot authorize descriptor or companion metadata changes, and a descriptor-shaped destination is rejected. Output validation runs before replacement so a rejected destination retains its original bytes. No command creates extra summary, diff --git a/README.md b/README.md index ed3e578..fbf11db 100644 --- a/README.md +++ b/README.md @@ -207,10 +207,10 @@ Every scope supports `-h`/`--help`, `--help-tree`, `--help-tree-depth `, and `--help-docs`. Only the root supports `-v`/`--version`. Use `--format text|json|markdown` for stable output. -`scan`, `meta`, `context`, `doctor`, and `tree` are read-only. `tree` walks all -non-excluded directories to arbitrary depth, includes every discovered named -descriptor and the special `XDOCS.md` path, and reports malformed or orphaned -metadata without hiding a node. `meta --existing-frontmatter` and +`scan`, `meta`, `context`, and `doctor` are read-only. Without `--output`, +`tree` is also read-only: it walks all non-excluded directories to arbitrary +depth, includes every discovered named descriptor and the special `XDOCS.md` +path, and reports malformed or orphaned metadata without hiding a node. `meta --existing-frontmatter` and `doctor --existing-frontmatter` audit existing ordinary Markdown headers only: missing headers remain valid, malformed existing headers are reported, and no repair or ownership is inferred. diff --git a/internal/agent/agent.go b/internal/agent/agent.go index fc54d03..be81684 100644 --- a/internal/agent/agent.go +++ b/internal/agent/agent.go @@ -46,8 +46,8 @@ adding headers unless both explicit grants match. Respect ` + "`ignore.gitignore`" + ` and every ` + "`ignore.rules`" + ` entry. Read-only ` + "`xdocs scan`" + `, ` + "`xdocs meta`" + `, ` + "`xdocs context`" + `, -` + "`xdocs tree`" + `, and ` + "`xdocs doctor`" + ` discovery is independent of the write -allowlist. ` + "`xdocs tree`" + ` walks every non-excluded directory to arbitrary depth, +and ` + "`xdocs doctor`" + ` operations, plus ` + "`xdocs tree`" + ` without +` + "`--output`" + `, discover independently of the write allowlist. ` + "`xdocs tree`" + ` walks every non-excluded directory to arbitrary depth, retains every named descriptor and the special root index, and reports malformed or orphaned metadata without hiding paths. ` + "`meta --existing-frontmatter`" + ` and ` + "`doctor --existing-frontmatter`" + ` audit existing ordinary Markdown headers diff --git a/prompts/agents.md b/prompts/agents.md index 0b175ba..13544a5 100644 --- a/prompts/agents.md +++ b/prompts/agents.md @@ -56,8 +56,9 @@ setup operation, separate from documentation-corpus writes. when it has no frontmatter, without adding headers by default - Keep directory context in the descriptor body; do not create extra summary, overview, detail, or report files - - Use `xdocs tree` as a read-only complete tree of every non-excluded - descriptor at every depth, including the root `XDOCS.md` path + - Use `xdocs tree` without `--output` as the read-only complete tree of + every non-excluded descriptor at every depth, including the root + `XDOCS.md` path - Print `generate`, `merge`, and `tree` reports to stdout unless one exact `--output` path is requested; a report is never a descriptor - List every same-directory plain `*.md` companion document in the descriptor's `documents` metadata map diff --git a/skills/guiho-s-xdocs/SKILL.md b/skills/guiho-s-xdocs/SKILL.md index 0b5c1cc..9c7b52d 100644 --- a/skills/guiho-s-xdocs/SKILL.md +++ b/skills/guiho-s-xdocs/SKILL.md @@ -128,7 +128,12 @@ rule and an authorized `documentation.directories` entry match, and no legacy ```yaml --- +name: Companion Notes +purpose: Explain the companion document. +description: Context for the owning directory. +created: 2026-09-12 owner: descriptor-subject +flags: [] tags: [] keywords: [] --- @@ -151,11 +156,12 @@ because they are discovered or listed. An explicit project policy is required. 3. Read only the recommended descriptors, implementation files, and companion documents. -4. Before any write, verify that the target directory is covered by +4. Before any xdocs documentation write, verify that the target directory is covered by `documentation.directories` and that the target Markdown frontmatter is covered by `documentation.frontmatter` when applicable. A read-only scan, tree, metadata, context, or doctor result never grants write permission. -5. Make the implementation/documentation change only within that authorization. +5. Make only the documentation write within that authorization. Unrelated + implementation changes follow their own task and repository authorization. 6. Update the owning descriptor without touching Git-ignored paths or adding frontmatter to denied or non-opted-in documents: - add/remove/rename `files` entries; @@ -171,13 +177,15 @@ because they are discovered or listed. An explicit project policy is required. xdocs doctor ``` -7. Widen validation only when the change affects repository-wide integrity. +8. Widen validation only when the change affects repository-wide integrity. -`xdocs tree` is a read-only complete discovery view. It walks every +`xdocs tree` performs a read-only complete discovery when no `--output` is +provided. It walks every non-excluded directory to arbitrary depth, retains every named descriptor and the special `XDOCS.md` path, and reports malformed, orphaned, duplicate, or cyclic metadata without silently dropping a node. Its discovery is independent -of the descriptor write allowlist. +of the descriptor write allowlist. With one exact `--output` path, it writes +only that requested report after output validation. `generate`, `merge`, and `tree` print their report to stdout unless the user provides one exact `--output` path. A report destination authorizes that single From 7582fb613a68d718beb1ea1f2117241fc7bdf357 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Crist=C3=B3v=C3=A3o=20GUIHO?= Date: Sat, 12 Sep 2026 12:13:56 +0000 Subject: [PATCH 17/20] Harden XDocs authorization and complete metadata audits --- AGENTS.md | 9 +- DOCS.md | 8 +- cmd/cmd.xdocs.md | 1 + cmd/domain.go | 57 ++++++++- cmd/tree_output_test.go | 99 +++++++++++++++ internal/config/config.go | 24 ++++ internal/config/config_test.go | 31 +++++ internal/xdocs/discovery.go | 6 + internal/xdocs/doctor.go | 10 ++ internal/xdocs/ignore.go | 6 +- internal/xdocs/ignore_test.go | 5 + internal/xdocs/meta.go | 43 ++++++- internal/xdocs/metadata.go | 164 ++++++++++++++++++++++-- internal/xdocs/model.go | 1 + internal/xdocs/output.go | 13 +- internal/xdocs/output_test.go | 23 +++- internal/xdocs/tree.go | 225 ++++++++++++++++++++++++--------- internal/xdocs/xdocs.xdocs.md | 5 +- internal/xdocs/xdocs_test.go | 132 +++++++++++++++++++ 19 files changed, 768 insertions(+), 94 deletions(-) create mode 100644 cmd/tree_output_test.go diff --git a/AGENTS.md b/AGENTS.md index 49785dc..7c755e6 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -86,7 +86,9 @@ architecture, planning, execution, review, validation, and release work. - `xdocs scan` walks the project tree while respecting `scan.exclude`, root and nested `.gitignore` files when enabled, and explicit descriptor candidates; it reports complete named `*.xdocs.md` coverage plus same-directory Markdown companion-document coverage without granting write permission. - `xdocs generate [path]` generates documentation for a specific directory or the entire project. - `xdocs merge [path]` merges xdocs descriptors from a directory into a single consolidated document. -- `xdocs tree` builds and displays the project hierarchy from xdocs metadata. +- `xdocs tree` walks the complete non-excluded filesystem and displays every + descriptor by directory containment; metadata relationships are diagnosed + separately by `doctor`. - `xdocs list [path]` lists files in a scope with descriptions from xdocs metadata. - `xdocs meta [path]` scans top-down and reads only YAML frontmatter from named `*.xdocs.md` descriptors; `--documents` reports ordinary companions with explicit `frontmatterRequired: false` unless `documentation.frontmatter` opts them in within `documentation.directories`; `--existing-frontmatter` audits existing headers read-only, while `--owner`, `--tag`, and `--keyword` filter metadata before agents read full files. - `xdocs context [path]` recommends a minimal reading set for a task from descriptor, file, and companion-document metadata; use `--documents`, `--files`, `--limit`, and `--explain` for agent workflows. @@ -124,7 +126,10 @@ architecture, planning, execution, review, validation, and release work. - xdocs descriptors use Markdown with YAML frontmatter and must be named `*.xdocs.md`; `.docs.md` and `.xdocs.md` by themselves are invalid candidates. Same-directory non-excluded plain `*.md` files are companion documents listed in the descriptor's `documents` metadata. Ordinary Markdown frontmatter is not required or written unless a matching `documentation.frontmatter` rule is explicitly authorized inside `documentation.directories`; legacy `ignore.rules` denials always win. The root file is always `XDOCS.md` (uppercase, no prefix, no frontmatter). Use `xdocs meta [path] --documents --format json` when an agent needs descriptor and companion-document policy without reading full Markdown bodies. - Metadata fields: `subject`, `description`, `parent`, `children`, `files`, `documents`, `tags`, `keywords`, `flags`, and optional `status`. -- The tree is a parent-child containment hierarchy, not a dependency graph. Built from `subject`/`parent`/`children` fields. +- The displayed tree is a directory-containment hierarchy, not a dependency + graph. It uses the nearest ancestor descriptor so malformed, orphaned, and + duplicate metadata cannot hide a discovered file; `doctor` validates + `subject`/`parent`/`children` relationships separately. - Configuration lives in `xdocs.yaml`. Sections: `extensions`, `ai`, `documentation`, `ignore`, `scan`, and `project`. `documentation.directories` defaults to `[]` and grants descriptor maintenance only within listed repository-relative directories and descendants; `documentation.frontmatter` defaults to `[]` and grants companion metadata only for matching files or directories inside those grants. `ignore.gitignore` defaults to `true`; strict `ignore.rules` objects use `pattern`, `kind`, and `frontmatter: false`. - Agent resource operations are not configuration-driven. The plain root bootstrap and `init` setup are the only implicit/setup boundaries; other diff --git a/DOCS.md b/DOCS.md index 3e609a8..9ac4c79 100644 --- a/DOCS.md +++ b/DOCS.md @@ -220,8 +220,12 @@ explicitly authorized the exact scope. - `meta [path]` reads frontmatter only. `--documents` reads companion frontmatter when it is required and tracks ordinary Markdown with - `frontmatterRequired: false`; `--existing-frontmatter` additionally reads - and validates headers that already exist on otherwise non-required documents. + `frontmatterRequired: false`; `--existing-frontmatter` implies + `--documents`, audits every discovered ordinary Markdown file in scope + (including unlisted and descriptorless files), and returns those audited + documents in the top-level `documents` JSON array and matching text/Markdown + sections. Missing headers remain valid; existing headers are checked for + generic YAML object structure without imposing the companion owner schema. `--owner`, `--tag`, and `--keyword` filter before full reads. - `context [path]` tokenizes a query, applies stable weighted ranking, and returns the smallest useful descriptor/file/document reading set. diff --git a/cmd/cmd.xdocs.md b/cmd/cmd.xdocs.md index 7b6f889..cc23ced 100644 --- a/cmd/cmd.xdocs.md +++ b/cmd/cmd.xdocs.md @@ -13,6 +13,7 @@ files: uninstall_windows.go: Windows deferred executable removal behavior. root_test.go: Root bootstrap, ignore defaults, frontmatter-policy JSON, exclusion, idempotence, marker safety, banner, version, catalog, depth, upgrade-completion, and recovery tests. resources_test.go: Deterministic embedded-resource fixtures for command tests. + tree_output_test.go: Text, Markdown, and JSON complete-tree visibility and default diagnostic tests. documents: {} tags: - cobra diff --git a/cmd/domain.go b/cmd/domain.go index 131b3ee..25df820 100644 --- a/cmd/domain.go +++ b/cmd/domain.go @@ -326,6 +326,11 @@ func newTreeCommand(options *commonOptions) *cobra.Command { for _, message := range scan.Errors { fmt.Fprintln(command.ErrOrStderr(), "error: "+message) } + for _, file := range scan.XDocsFiles { + for _, message := range file.Errors { + fmt.Fprintf(command.ErrOrStderr(), "error: %s: %s\n", file.RelativePath, message) + } + } validation := domain.ValidateTree(scan.XDocsFiles) for _, message := range validation.Warnings { fmt.Fprintln(command.ErrOrStderr(), "warning: "+message) @@ -560,6 +565,25 @@ func renderMeta(command *cobra.Command, format string, result domain.MetaResult) fmt.Fprintln(out) } } + if len(result.Documents) > 0 { + fmt.Fprintln(out, "## Audited Documents") + fmt.Fprintln(out) + for _, document := range result.Documents { + status := "invalid" + if document.Valid { + status = "valid" + } + fmt.Fprintf(out, "- `%s` (%s", document.RelativePath, status) + if !document.FrontmatterRequired { + fmt.Fprint(out, ", frontmatter: not required") + } + fmt.Fprintln(out, ")") + for _, message := range document.Errors { + fmt.Fprintln(out, " - "+message) + } + } + fmt.Fprintln(out) + } if len(result.Errors) > 0 { fmt.Fprintln(out, "\n## Errors") for _, message := range result.Errors { @@ -596,6 +620,23 @@ func renderMeta(command *cobra.Command, format string, result domain.MetaResult) } } } + if len(result.Documents) > 0 { + fmt.Fprintln(out, "\naudited documents:") + for _, document := range result.Documents { + status := "invalid" + if document.Valid { + status = "valid" + } + fmt.Fprintf(out, " %s [%s]", document.RelativePath, status) + if !document.FrontmatterRequired { + fmt.Fprint(out, " frontmatter=not-required") + } + fmt.Fprintln(out) + for _, message := range document.Errors { + fmt.Fprintln(out, " error: "+message) + } + } + } if len(result.Errors) > 0 { fmt.Fprintln(out, "\nerrors:") for _, message := range result.Errors { @@ -673,6 +714,7 @@ type metaJSONResult struct { Strict bool `json:"strict"` Filters domain.Filters `json:"filters"` Descriptors []metaJSONDescriptor `json:"descriptors"` + Documents []metaJSONDocument `json:"documents"` Errors []string `json:"errors"` } @@ -681,7 +723,7 @@ func metaJSON(result domain.MetaResult) metaJSONResult { Root: result.Root, TargetPath: result.TargetPath, IncludeDocuments: result.IncludeDocuments, ExistingFrontmatter: result.ExistingFrontmatter, Strict: result.Strict, Filters: result.Filters, - Descriptors: []metaJSONDescriptor{}, Errors: append([]string{}, result.Errors...), + Descriptors: []metaJSONDescriptor{}, Documents: []metaJSONDocument{}, Errors: append([]string{}, result.Errors...), } for _, descriptor := range result.Descriptors { entry := metaJSONDescriptor{ @@ -708,6 +750,19 @@ func metaJSON(result domain.MetaResult) metaJSONResult { } projected.Descriptors = append(projected.Descriptors, entry) } + for _, document := range result.Documents { + item := metaJSONDocument{ + Path: document.Path, RelativePath: document.RelativePath, Directory: document.Directory, + Name: document.Name, FrontmatterRequired: document.FrontmatterRequired, + Valid: document.Valid, Frontmatter: document.Frontmatter, + Errors: append([]string{}, document.Errors...), + } + if document.Owner != "" { + owner := document.Owner + item.Owner = &owner + } + projected.Documents = append(projected.Documents, item) + } return projected } diff --git a/cmd/tree_output_test.go b/cmd/tree_output_test.go new file mode 100644 index 0000000..8784191 --- /dev/null +++ b/cmd/tree_output_test.go @@ -0,0 +1,99 @@ +package cmd + +import ( + "encoding/json" + "os" + "path/filepath" + "strings" + "testing" +) + +func TestTreeCommandKeepsInvalidDescriptorsVisibleAndReportsDiagnostics(t *testing.T) { + root := t.TempDir() + deep := filepath.Join(root, "one", "two", "three") + if err := os.MkdirAll(deep, 0o755); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(root, "XDOCS.md"), []byte("# Root\n"), 0o644); err != nil { + t.Fatal(err) + } + valid := `--- +subject: deep +description: Deep descriptor context. +parent: null +children: [] +files: {} +documents: {} +tags: [] +keywords: [] +flags: [] +--- +` + if err := os.WriteFile(filepath.Join(deep, "three.xdocs.md"), []byte(valid), 0o644); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(root, "one", "one.xdocs.md"), []byte("---\nsubject: [broken\n"), 0o644); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(root, "one", "two", "two.xdocs.md"), []byte("---\nsubject: orphan\ndescription: Orphan\nparent: missing\nchildren: []\nfiles: {}\ndocuments: {}\ntags: []\nkeywords: []\nflags: []\n---\n"), 0o644); err != nil { + t.Fatal(err) + } + + for _, format := range []string{"text", "markdown", "json"} { + args := []string{"--cwd", root, "--format", format, "tree"} + out, stderr, err := execute(t, args...) + if err != nil { + t.Fatalf("tree %s failed: %v\n%s", format, err, stderr) + } + if format == "json" { + var value map[string]any + if err := json.Unmarshal([]byte(out), &value); err != nil { + t.Fatalf("tree JSON is invalid: %v\n%s", err, out) + } + } + for _, path := range []string{"XDOCS.md", "one/one.xdocs.md", "one/two/two.xdocs.md", "one/two/three/three.xdocs.md"} { + var count int + switch format { + case "json": + count = treeJSONPathCount(t, out, path) + case "markdown": + count = strings.Count(out, "(`"+path+"`)") + default: + count = strings.Count(out, "["+path+"]") + } + if count != 1 { + t.Fatalf("tree %s did not expose %s exactly once (count %d):\n%s", format, path, count, out) + } + } + if !strings.Contains(stderr, "one.xdocs.md") || !strings.Contains(stderr, "orphan") { + t.Fatalf("tree %s omitted default diagnostics:\n%s", format, stderr) + } + } +} + +func treeJSONPathCount(t *testing.T, output, wanted string) int { + t.Helper() + var root map[string]any + if err := json.Unmarshal([]byte(output), &root); err != nil { + t.Fatalf("tree JSON is invalid: %v\n%s", err, output) + } + count := 0 + var visit func(map[string]any) + visit = func(node map[string]any) { + if path, ok := node["path"].(string); ok && path == wanted { + count++ + } + children, ok := node["children"].([]any) + if !ok { + return + } + for _, child := range children { + childNode, ok := child.(map[string]any) + if ok { + visit(childNode) + } + } + } + visit(root) + return count +} diff --git a/internal/config/config.go b/internal/config/config.go index cda6272..5bf5d10 100644 --- a/internal/config/config.go +++ b/internal/config/config.go @@ -423,6 +423,12 @@ func validateDocumentationYAML(content []byte) error { if root.Kind != yaml.MappingNode { return nil } + for index := 0; index+1 < len(root.Content); index += 2 { + key := root.Content[index] + if key.Value == "<<" || key.Tag == "!!merge" || key.Alias != nil { + return apperror.New(apperror.Configuration, "xdocs configuration must not use root YAML merge keys or aliases") + } + } for index := 0; index+1 < len(root.Content); index += 2 { if root.Content[index].Value != "documentation" { continue @@ -433,6 +439,9 @@ func validateDocumentationYAML(content []byte) error { } func validateDocumentationNode(node *yaml.Node) error { + if err := rejectDocumentationAliases(node); err != nil { + return err + } if node.Kind != yaml.MappingNode { return apperror.New(apperror.Configuration, "documentation must be a YAML object") } @@ -468,6 +477,21 @@ func validateDocumentationNode(node *yaml.Node) error { return nil } +func rejectDocumentationAliases(node *yaml.Node) error { + if node == nil { + return nil + } + if node.Anchor != "" || node.Alias != nil || node.Tag == "!!merge" || node.Value == "<<" { + return apperror.New(apperror.Configuration, "documentation policy must not use YAML anchors, aliases, or merge keys") + } + for _, child := range node.Content { + if err := rejectDocumentationAliases(child); err != nil { + return err + } + } + return nil +} + func validateIgnoreRule(index int, rule ignoreRule) (IgnoreRule, error) { label := fmt.Sprintf("ignore.rules[%d]", index) pattern := strings.TrimSpace(rule.Pattern) diff --git a/internal/config/config_test.go b/internal/config/config_test.go index 5a48fa4..9208160 100644 --- a/internal/config/config_test.go +++ b/internal/config/config_test.go @@ -132,6 +132,37 @@ func TestDocumentationPolicyRejectsMalformedEntries(t *testing.T) { } } +func TestDocumentationPolicyRejectsYAMLMergeAndAliases(t *testing.T) { + root := t.TempDir() + path := filepath.Join(root, Filename) + content := `schema: 1 +documentation: + <<: &grant + directories: + - . + frontmatter: [] +` + if err := os.WriteFile(path, []byte(content), 0o644); err != nil { + t.Fatal(err) + } + if _, err := Load(root, "", true); err == nil { + t.Fatal("documentation YAML merge key unexpectedly granted authorization") + } + content = `schema: 1 +<<: &grant + documentation: + directories: + - . + frontmatter: [] +` + if err := os.WriteFile(path, []byte(content), 0o644); err != nil { + t.Fatal(err) + } + if _, err := Load(root, "", true); err == nil { + t.Fatal("root YAML merge key unexpectedly granted documentation authorization") + } +} + func TestIgnoreDefaultsAndGeneratedConfiguration(t *testing.T) { root := t.TempDir() defaults, err := Defaults(root) diff --git a/internal/xdocs/discovery.go b/internal/xdocs/discovery.go index 05ea955..2a81308 100644 --- a/internal/xdocs/discovery.go +++ b/internal/xdocs/discovery.go @@ -1,6 +1,7 @@ package xdocs import ( + "errors" "fmt" "os" "path/filepath" @@ -85,12 +86,17 @@ func ScanProject(cfg config.Config) (ScanResult, error) { } rootStatErr = os.ErrInvalid } + if rootStatErr != nil && !errors.Is(rootStatErr, os.ErrNotExist) && !errors.Is(rootStatErr, os.ErrInvalid) { + result.Errors = append(result.Errors, fmt.Sprintf("Unable to inspect %s: %v", rootFilename, rootStatErr)) + } if rootStatErr == nil && !policy.ignored(rootPath, false) { if content, readErr := os.ReadFile(rootPath); readErr == nil { result.XDocsFiles = append(result.XDocsFiles, File{ Path: rootPath, RelativePath: rootFilename, Directory: cfg.CWD, Documents: []MarkdownDocument{}, Body: string(content), Valid: false, Errors: []string{}, }) + } else { + result.Errors = append(result.Errors, fmt.Sprintf("Unable to read %s: %v", rootFilename, readErr)) } } sort.Slice(result.XDocsFiles, func(i, j int) bool { return result.XDocsFiles[i].RelativePath < result.XDocsFiles[j].RelativePath }) diff --git a/internal/xdocs/doctor.go b/internal/xdocs/doctor.go index 3eb626b..04e49c0 100644 --- a/internal/xdocs/doctor.go +++ b/internal/xdocs/doctor.go @@ -72,6 +72,16 @@ func Doctor(cfg config.Config, options DoctorOptions) (DoctorResult, error) { } } } + for _, document := range meta.Documents { + documentPath := document.RelativePath + severity := "warning" + if options.WarningsAsErrors { + severity = "error" + } + for _, message := range document.Errors { + issues = append(issues, DoctorIssue{Severity: severity, Code: "document-metadata", Path: &documentPath, Message: message}) + } + } validation := ValidateTree(scan.XDocsFiles) for _, message := range validation.Errors { issues = append(issues, DoctorIssue{Severity: "error", Code: "tree-invalid", Message: message}) diff --git a/internal/xdocs/ignore.go b/internal/xdocs/ignore.go index c34c835..b6b50fd 100644 --- a/internal/xdocs/ignore.go +++ b/internal/xdocs/ignore.go @@ -277,7 +277,7 @@ func (policy *pathPolicy) frontmatterRequired(pathname string) bool { func (policy *pathPolicy) documentationDirectoryAuthorized(directory string) bool { directory = path.Clean(directory) - if directory == ".." || strings.HasPrefix(directory, "../") { + if directory == ".." || strings.HasPrefix(directory, "../") || path.IsAbs(directory) || windowsDrivePath(directory) { return false } for _, allowed := range policy.documentationDirs { @@ -289,6 +289,10 @@ func (policy *pathPolicy) documentationDirectoryAuthorized(directory string) boo return false } +func windowsDrivePath(value string) bool { + return len(value) >= 2 && ((value[0] >= 'a' && value[0] <= 'z') || (value[0] >= 'A' && value[0] <= 'Z')) && value[1] == ':' +} + func pathEqual(left, right string, ignoreCase bool) bool { if ignoreCase { return strings.EqualFold(left, right) diff --git a/internal/xdocs/ignore_test.go b/internal/xdocs/ignore_test.go index 4831fc3..d40bce2 100644 --- a/internal/xdocs/ignore_test.go +++ b/internal/xdocs/ignore_test.go @@ -148,6 +148,11 @@ func TestPathPolicyRequiresExplicitDocumentationAuthorization(t *testing.T) { if rootPolicy.frontmatterRequired(filepath.Join(root, "..", "outside.md")) { t.Fatal("root documentation grant escaped the repository") } + for _, outside := range []string{"../outside", "/absolute/outside", "C:/outside"} { + if rootPolicy.documentationDirectoryAuthorized(outside) { + t.Fatalf("root documentation grant accepted outside directory %q", outside) + } + } } func TestPathPolicyDoesNotEnterIgnoredOrExcludedTarget(t *testing.T) { diff --git a/internal/xdocs/meta.go b/internal/xdocs/meta.go index 70ca991..a85d61e 100644 --- a/internal/xdocs/meta.go +++ b/internal/xdocs/meta.go @@ -12,6 +12,7 @@ import ( ) func ScanMetadata(cfg config.Config, options MetaOptions) (MetaResult, error) { + includeDocuments := options.IncludeDocuments || options.ExistingFrontmatter target := cfg.CWD if options.TargetPath != "" { target = options.TargetPath @@ -31,11 +32,12 @@ func ScanMetadata(cfg config.Config, options MetaOptions) (MetaResult, error) { result := MetaResult{ Root: cfg.CWD, TargetPath: displayTarget(cfg.CWD, target), - IncludeDocuments: options.IncludeDocuments, + IncludeDocuments: includeDocuments, ExistingFrontmatter: options.ExistingFrontmatter, Strict: options.Strict, Filters: options.Filters, Descriptors: []MetaDescriptor{}, + Documents: []MetaDocument{}, Errors: []string{}, } if scanExcludedPath(cfg.CWD, target, cfg.Exclude) { @@ -86,8 +88,16 @@ func ScanMetadata(cfg config.Config, options MetaOptions) (MetaResult, error) { return MetaResult{}, err } sort.Slice(result.Descriptors, func(i, j int) bool { return result.Descriptors[i].RelativePath < result.Descriptors[j].RelativePath }) - enrichMeta(result.Descriptors, documents, options.IncludeDocuments, options.ExistingFrontmatter, cfg.CWD) - result.Descriptors = filterDescriptors(result.Descriptors, options.Filters, options.IncludeDocuments) + enrichMeta(result.Descriptors, documents, includeDocuments, options.ExistingFrontmatter, cfg.CWD) + if options.ExistingFrontmatter { + for _, document := range allMarkdownDocuments(documents) { + result.Documents = append(result.Documents, parseDocument(document.Path, cfg.CWD, "", document.FrontmatterRequired, true)) + } + } + result.Descriptors = filterDescriptors(result.Descriptors, options.Filters, includeDocuments) + if options.ExistingFrontmatter { + result.Documents = filterMetaDocuments(result.Documents, options.Filters) + } for _, descriptor := range result.Descriptors { for _, message := range descriptor.Errors { result.Errors = append(result.Errors, descriptor.RelativePath+": "+message) @@ -98,9 +108,36 @@ func ScanMetadata(cfg config.Config, options MetaOptions) (MetaResult, error) { } } } + for _, document := range result.Documents { + for _, message := range document.Errors { + result.Errors = append(result.Errors, document.RelativePath+": "+message) + } + } return result, nil } +func allMarkdownDocuments(documents map[string][]MarkdownDocument) []MarkdownDocument { + result := []MarkdownDocument{} + for _, values := range documents { + result = append(result, values...) + } + sort.Slice(result, func(i, j int) bool { return result[i].RelativePath < result[j].RelativePath }) + return result +} + +func filterMetaDocuments(documents []MetaDocument, filters Filters) []MetaDocument { + if filters.Owner == "" && filters.Tag == "" && filters.Keyword == "" { + return documents + } + result := []MetaDocument{} + for _, document := range documents { + if documentMatches(document, filters) { + result = append(result, document) + } + } + return result +} + func parseMetaDescriptor(path, root string) MetaDescriptor { result := MetaDescriptor{ Path: path, diff --git a/internal/xdocs/metadata.go b/internal/xdocs/metadata.go index 16da2bb..31694e0 100644 --- a/internal/xdocs/metadata.go +++ b/internal/xdocs/metadata.go @@ -58,12 +58,18 @@ func ExtractFrontmatter(content string) (string, string, bool) { } func ExtractFrontmatterDetailed(content string) (string, string, bool, bool) { - trimmed := strings.TrimLeft(content, " \t\r\n") - if !strings.HasPrefix(trimmed, "---") { + if !strings.HasPrefix(content, "---") { + // A frontmatter opener is only valid at byte zero. Keep a leading + // opener distinguishable from an ordinary Markdown body so writers and + // read-only audits reject the ambiguous form instead of accepting it. + trimmed := strings.TrimLeft(content, " \t\r\n") + if strings.HasPrefix(trimmed, "---") || strings.HasPrefix(content, "\ufeff---") { + return "", content, false, true + } return "", content, false, false } present := true - remaining := trimmed[3:] + remaining := content[3:] if strings.HasPrefix(remaining, "\r\n") { remaining = remaining[2:] } else if strings.HasPrefix(remaining, "\n") { @@ -124,13 +130,19 @@ func readFrontmatterDetailed(path string) (string, bool, bool, error) { } func parseMetadata(raw string) (*Metadata, Frontmatter, []string) { - var object Frontmatter - if err := yaml.Unmarshal([]byte(raw), &object); err != nil { + object, node, err := decodeFrontmatter(raw) + if err != nil { return nil, nil, []string{fmt.Sprintf("Invalid YAML frontmatter: %v", err)} } if object == nil { return nil, nil, []string{"Frontmatter must be a YAML object."} } + if err := rejectMetadataAliases(node); err != nil { + return nil, object, []string{err.Error()} + } + if errors := validateMetadataNode(node); len(errors) > 0 { + return nil, object, errors + } if _, present := object["parent"]; !present { return nil, object, []string{`frontmatter.parent: expected a parent subject string or null`} } @@ -147,6 +159,121 @@ func parseMetadata(raw string) (*Metadata, Frontmatter, []string) { return &metadata, object, nil } +func validateMetadataNode(root *yaml.Node) []string { + if root == nil || root.Kind != yaml.MappingNode { + return []string{"Frontmatter must be a YAML object."} + } + fields := map[string]*yaml.Node{} + for index := 0; index+1 < len(root.Content); index += 2 { + key, value := root.Content[index], root.Content[index+1] + if key.Kind == yaml.ScalarNode && key.Tag == "!!str" { + fields[key.Value] = value + } + } + var errors []string + for index := 0; index+1 < len(root.Content); index += 2 { + key := root.Content[index] + if key.Kind != yaml.ScalarNode || key.Tag != "!!str" { + errors = append(errors, "frontmatter: field names must be strings") + } + } + for _, field := range []string{"subject", "description", "status"} { + if value, ok := fields[field]; ok && !yamlStringNode(value) { + errors = append(errors, fmt.Sprintf("frontmatter.%s: expected a string", field)) + } + } + if value, ok := fields["parent"]; ok && !yamlStringNode(value) && !yamlNullNode(value) { + errors = append(errors, "frontmatter.parent: expected a parent subject string or null") + } + for _, field := range []string{"children", "tags", "keywords", "flags"} { + if value, ok := fields[field]; ok && !yamlStringSequenceNode(value) { + errors = append(errors, fmt.Sprintf("frontmatter.%s: expected an array of strings", field)) + } + } + for _, field := range []string{"files", "documents"} { + if value, ok := fields[field]; ok && !yamlStringMapNode(value) { + errors = append(errors, fmt.Sprintf("frontmatter.%s: expected a string map", field)) + } + } + return errors +} + +func rejectMetadataAliases(node *yaml.Node) error { + if node == nil { + return nil + } + if node.Anchor != "" || node.Alias != nil || node.Tag == "!!merge" { + return fmt.Errorf("frontmatter must not use YAML anchors, aliases, or merge keys") + } + for _, child := range node.Content { + if err := rejectMetadataAliases(child); err != nil { + return err + } + } + return nil +} + +func yamlStringNode(node *yaml.Node) bool { + return node != nil && node.Kind == yaml.ScalarNode && node.Tag == "!!str" +} + +func yamlNullNode(node *yaml.Node) bool { + return node != nil && node.Kind == yaml.ScalarNode && node.Tag == "!!null" +} + +func yamlStringSequenceNode(node *yaml.Node) bool { + if node == nil || node.Kind != yaml.SequenceNode { + return false + } + for _, item := range node.Content { + if !yamlStringNode(item) { + return false + } + } + return true +} + +func yamlStringMapNode(node *yaml.Node) bool { + if node == nil || node.Kind != yaml.MappingNode { + return false + } + for index := 0; index+1 < len(node.Content); index += 2 { + if !yamlStringNode(node.Content[index]) || !yamlStringNode(node.Content[index+1]) { + return false + } + } + return true +} + +func decodeFrontmatterObject(raw string) (Frontmatter, error) { + object, _, err := decodeFrontmatter(raw) + return object, err +} + +func decodeFrontmatter(raw string) (Frontmatter, *yaml.Node, error) { + decoder := yaml.NewDecoder(bytes.NewBufferString(raw)) + var document yaml.Node + if err := decoder.Decode(&document); err != nil { + return nil, nil, err + } + var trailing yaml.Node + if err := decoder.Decode(&trailing); err != io.EOF { + if err == nil { + return nil, nil, fmt.Errorf("frontmatter must contain exactly one YAML document") + } + return nil, nil, err + } + if len(document.Content) == 0 { + return nil, nil, nil + } + root := document.Content[0] + var object Frontmatter + if err := root.Decode(&object); err != nil { + return nil, root, err + } + return object, root, nil +} + func validateMetadata(metadata Metadata) []string { var errors []string if strings.TrimSpace(metadata.Subject) == "" { @@ -203,12 +330,15 @@ func parseDocument(path, root, expectedOwner string, frontmatterRequired, inspec result.Errors = append(result.Errors, "Missing YAML frontmatter.") return result } - var frontmatter Frontmatter - if err := yaml.Unmarshal([]byte(raw), &frontmatter); err != nil { + frontmatter, err := decodeFrontmatterObject(raw) + if err != nil { result.Errors = append(result.Errors, fmt.Sprintf("Invalid YAML frontmatter: %v", err)) return result } result.Frontmatter = frontmatter + if !frontmatterRequired { + result.Owner = stringValue(frontmatter, "owner") + } if !frontmatterRequired { if frontmatter == nil { result.Errors = append(result.Errors, "Frontmatter must be a YAML object.") @@ -218,24 +348,30 @@ func parseDocument(path, root, expectedOwner string, frontmatterRequired, inspec return result } result.Owner = stringValue(frontmatter, "owner") + result.Errors = append(result.Errors, validateCompanionFrontmatter(frontmatter, expectedOwner)...) + result.Valid = len(result.Errors) == 0 + return result +} + +func validateCompanionFrontmatter(frontmatter Frontmatter, expectedOwner string) []string { + var errors []string for _, field := range []string{"name", "purpose", "description", "created", "owner"} { if stringValue(frontmatter, field) == "" { - result.Errors = append(result.Errors, fmt.Sprintf(`Missing or invalid "%s" field. Expected a non-empty string.`, field)) + errors = append(errors, fmt.Sprintf(`Missing or invalid "%s" field. Expected a non-empty string.`, field)) } } for _, field := range []string{"flags", "tags", "keywords"} { if _, ok := stringSlice(frontmatter[field]); !ok { - result.Errors = append(result.Errors, fmt.Sprintf(`Missing or invalid "%s" field. Expected an array of strings.`, field)) + errors = append(errors, fmt.Sprintf(`Missing or invalid "%s" field. Expected an array of strings.`, field)) } } if created := stringValue(frontmatter, "created"); created != "" && !datePattern.MatchString(created) { - result.Errors = append(result.Errors, `Invalid "created" field. Expected YYYY-MM-DD.`) + errors = append(errors, `Invalid "created" field. Expected YYYY-MM-DD.`) } - if expectedOwner != "" && result.Owner != "" && result.Owner != expectedOwner { - result.Errors = append(result.Errors, fmt.Sprintf(`Invalid "owner" field. Expected "%s".`, expectedOwner)) + if expectedOwner != "" && stringValue(frontmatter, "owner") != "" && stringValue(frontmatter, "owner") != expectedOwner { + errors = append(errors, fmt.Sprintf(`Invalid "owner" field. Expected "%s".`, expectedOwner)) } - result.Valid = len(result.Errors) == 0 - return result + return errors } func stringValue(frontmatter Frontmatter, key string) string { diff --git a/internal/xdocs/model.go b/internal/xdocs/model.go index df759ae..88927eb 100644 --- a/internal/xdocs/model.go +++ b/internal/xdocs/model.go @@ -108,6 +108,7 @@ type MetaResult struct { Strict bool `json:"strict"` Filters Filters `json:"filters"` Descriptors []MetaDescriptor `json:"descriptors"` + Documents []MetaDocument `json:"documents"` Errors []string `json:"errors"` } diff --git a/internal/xdocs/output.go b/internal/xdocs/output.go index 0e29caf..5311078 100644 --- a/internal/xdocs/output.go +++ b/internal/xdocs/output.go @@ -7,7 +7,6 @@ import ( "strings" "github.com/CGuiho/xdocs/internal/config" - "go.yaml.in/yaml/v3" ) // WriteReport writes exactly one explicitly requested report destination. It @@ -60,7 +59,7 @@ func WriteReport(cfg config.Config, destination string, content []byte) error { } func validateReportContent(cfg config.Config, target string, content []byte) error { - _, _, ok, present := ExtractFrontmatterDetailed(string(content)) + raw, _, ok, present := ExtractFrontmatterDetailed(string(content)) if !present { return nil } @@ -74,14 +73,16 @@ func validateReportContent(cfg config.Config, target string, content []byte) err if !policy.frontmatterRequired(target) { return fmt.Errorf("refusing to write Markdown frontmatter without explicit documentation.frontmatter authorization: %s", filepath.ToSlash(slashRelative(cfg.CWD, target))) } - frontmatter, _, _, _ := ExtractFrontmatterDetailed(string(content)) - var object map[string]any - if err := yaml.Unmarshal([]byte(frontmatter), &object); err != nil { + frontmatter, err := decodeFrontmatterObject(raw) + if err != nil { return fmt.Errorf("refusing to write invalid report frontmatter: %w", err) } - if object == nil { + if frontmatter == nil { return fmt.Errorf("refusing to write report frontmatter: expected a YAML object") } + if errors := validateCompanionFrontmatter(frontmatter, ""); len(errors) > 0 { + return fmt.Errorf("refusing to write invalid companion frontmatter: %s", strings.Join(errors, "; ")) + } return nil } diff --git a/internal/xdocs/output_test.go b/internal/xdocs/output_test.go index 08ab076..7364f21 100644 --- a/internal/xdocs/output_test.go +++ b/internal/xdocs/output_test.go @@ -81,7 +81,8 @@ func TestWriteReportAllowsExplicitlyAuthorizedGenericFrontmatter(t *testing.T) { if err := os.Mkdir(filepath.Join(root, "notes"), 0o755); err != nil { t.Fatal(err) } - if err := WriteReport(cfg, "notes/report.md", []byte("---\nname: report\n---\nbody\n")); err != nil { + valid := "---\nname: report\npurpose: Explain the report.\ndescription: Report metadata.\ncreated: 2026-09-12\nowner: xdocs\nflags: []\ntags: []\nkeywords: []\n---\nbody\n" + if err := WriteReport(cfg, "notes/report.md", []byte(valid)); err != nil { t.Fatal(err) } content, err := os.ReadFile(filepath.Join(root, "notes", "report.md")) @@ -91,6 +92,26 @@ func TestWriteReportAllowsExplicitlyAuthorizedGenericFrontmatter(t *testing.T) { if !strings.HasPrefix(string(content), "---\nname: report") { t.Fatalf("authorized report was not written: %q", content) } + if err := WriteReport(cfg, "notes/report.md", []byte("---\nname: [invalid]\npurpose: Explain the report.\ndescription: Report metadata.\ncreated: 2026-09-12\nowner: xdocs\nflags: []\ntags: []\nkeywords: []\n---\nchanged\n")); err == nil { + t.Fatal("authorized report with invalid companion field type was accepted") + } + unchanged, err := os.ReadFile(filepath.Join(root, "notes", "report.md")) + if err != nil { + t.Fatal(err) + } + if string(unchanged) != valid { + t.Fatalf("invalid authorized report changed destination: %q", unchanged) + } + if err := WriteReport(cfg, "notes/report.md", []byte(" \n---\nname: report\npurpose: Explain the report.\ndescription: Report metadata.\ncreated: 2026-09-12\nowner: xdocs\nflags: []\ntags: []\nkeywords: []\n---\nchanged\n")); err == nil { + t.Fatal("report with a non-leading frontmatter opener was accepted") + } + unchanged, err = os.ReadFile(filepath.Join(root, "notes", "report.md")) + if err != nil { + t.Fatal(err) + } + if string(unchanged) != valid { + t.Fatalf("non-leading frontmatter rejection changed destination: %q", unchanged) + } } func TestWriteReportRejectsGitignoredAndSymlinkDestinations(t *testing.T) { diff --git a/internal/xdocs/tree.go b/internal/xdocs/tree.go index efb60bb..a3a0d2b 100644 --- a/internal/xdocs/tree.go +++ b/internal/xdocs/tree.go @@ -2,77 +2,151 @@ package xdocs import ( "fmt" + "path/filepath" "sort" "strings" ) +// BuildTree builds a display tree from directory containment. Descriptor +// metadata relationships remain a doctor concern; using them here would hide +// malformed, orphaned, or duplicate descriptors from the navigable tree. func BuildTree(files []File) *TreeNode { - nodes := map[string]*TreeNode{} + root := &TreeNode{ + Subject: "(root)", + Description: "Project XDocs tree.", + Kind: "project", + Valid: true, + Children: []*TreeNode{}, + } + + type entry struct { + file File + node *TreeNode + dir string + depth int + } + entries := make([]entry, 0, len(files)) for _, file := range files { - if file.Metadata == nil { - continue + kind := "descriptor" + valid := file.Valid + if !IsDescriptorCandidate(file.Path) { + if !strings.EqualFold(file.RelativePath, rootFilename) { + continue + } + kind = "index" + // XDOCS.md is a special navigable index. It has no descriptor + // frontmatter, so File.Valid does not describe its display state. + valid = true } path := file.RelativePath - nodes[file.Metadata.Subject] = &TreeNode{ - Subject: file.Metadata.Subject, - Description: file.Metadata.Description, - Path: &path, - Children: []*TreeNode{}, + subject := filepath.Base(file.RelativePath) + description := "" + errors := append([]string(nil), file.Errors...) + if file.Metadata != nil { + subject = file.Metadata.Subject + description = file.Metadata.Description + } else if len(errors) > 0 { + description = errors[0] } - } - var root *TreeNode - for _, file := range files { - if file.Metadata == nil { - continue + if subject == "" { + subject = file.RelativePath } - node := nodes[file.Metadata.Subject] - if file.Metadata.Parent == nil { - if root == nil { - root = node + if description == "" { + if kind == "index" { + description = "Repository root index." + } else { + description = "Descriptor metadata is invalid or unavailable." } - continue } - if parent := nodes[*file.Metadata.Parent]; parent != nil && node != nil { - addChild(parent, node) + node := &TreeNode{ + Subject: subject, Description: description, Path: &path, + Kind: kind, Valid: valid, Errors: errors, Children: []*TreeNode{}, } + directory := filepath.Clean(file.Directory) + entries = append(entries, entry{file: file, node: node, dir: directory, depth: pathDepth(directory)}) } - for _, file := range files { - if file.Metadata == nil { - continue + sort.SliceStable(entries, func(i, j int) bool { + return entries[i].file.RelativePath < entries[j].file.RelativePath + }) + + // Only named descriptors can contain descendant descriptors. The root + // index is retained as a project child and is never treated as a descriptor. + descriptorEntries := make([]int, 0, len(entries)) + for index, item := range entries { + if item.node.Kind == "descriptor" { + descriptorEntries = append(descriptorEntries, index) } - parent := nodes[file.Metadata.Subject] - for _, childName := range file.Metadata.Children { - if child := nodes[childName]; child != nil { - addChild(parent, child) + } + for index := range entries { + item := &entries[index] + parent := -1 + bestDepth := -1 + if item.node.Kind == "descriptor" { + for _, candidateIndex := range descriptorEntries { + if candidateIndex == index { + continue + } + candidate := entries[candidateIndex] + if candidate.depth >= item.depth || !directoryContains(candidate.dir, item.dir) { + continue + } + if candidate.depth > bestDepth { + parent = candidateIndex + bestDepth = candidate.depth + } } } + if parent >= 0 { + addChild(entries[parent].node, item.node) + } else { + addChild(root, item.node) + } } - for _, node := range nodes { - sort.Slice(node.Children, func(i, j int) bool { return node.Children[i].Subject < node.Children[j].Subject }) - } - if root != nil { - return root + for _, item := range entries { + sort.SliceStable(item.node.Children, func(i, j int) bool { + left, right := item.node.Children[i], item.node.Children[j] + leftPath, rightPath := "", "" + if left.Path != nil { + leftPath = *left.Path + } + if right.Path != nil { + rightPath = *right.Path + } + if leftPath != rightPath { + return leftPath < rightPath + } + return left.Subject < right.Subject + }) } - root = &TreeNode{Subject: "(root)", Description: "No root xdocs descriptor found.", Children: []*TreeNode{}} - children := map[string]bool{} - for _, file := range files { - if file.Metadata == nil { - continue + sort.SliceStable(root.Children, func(i, j int) bool { + left, right := root.Children[i], root.Children[j] + leftPath, rightPath := "", "" + if left.Path != nil { + leftPath = *left.Path } - for _, child := range file.Metadata.Children { - children[child] = true - } - if file.Metadata.Parent != nil { - children[file.Metadata.Subject] = true + if right.Path != nil { + rightPath = *right.Path } + return leftPath < rightPath + }) + return root +} + +func pathDepth(value string) int { + value = filepath.Clean(value) + if value == "." || value == string(filepath.Separator) { + return 0 } - for subject, node := range nodes { - if !children[subject] { - root.Children = append(root.Children, node) - } + parts := strings.Split(filepath.ToSlash(value), "/") + if len(parts) == 1 && parts[0] == "." { + return 0 } - sort.Slice(root.Children, func(i, j int) bool { return root.Children[i].Subject < root.Children[j].Subject }) - return root + return len(parts) +} + +func directoryContains(parent, child string) bool { + relative, err := filepath.Rel(parent, child) + return err == nil && relative != "." && relative != ".." && !strings.HasPrefix(relative, ".."+string(filepath.Separator)) && !filepath.IsAbs(relative) } func addChild(parent, child *TreeNode) { @@ -125,22 +199,22 @@ func ValidateTree(files []File) TreeValidation { } } seenChildren := map[string]bool{} - for _, child := range file.Metadata.Children { - if seenChildren[child] { - result.Errors = append(result.Errors, fmt.Sprintf(`Duplicate child: "%s" lists "%s" more than once in %s`, file.Metadata.Subject, child, file.RelativePath)) + for _, childName := range file.Metadata.Children { + if seenChildren[childName] { + result.Errors = append(result.Errors, fmt.Sprintf(`Duplicate child: "%s" lists "%s" more than once in %s`, file.Metadata.Subject, childName, file.RelativePath)) continue } - seenChildren[child] = true - childFile := subjects[child] + seenChildren[childName] = true + childFile := subjects[childName] if childFile == nil { - result.Errors = append(result.Errors, fmt.Sprintf(`Missing child: "%s" references non-existent child "%s" in %s`, file.Metadata.Subject, child, file.RelativePath)) + result.Errors = append(result.Errors, fmt.Sprintf(`Missing child: "%s" references non-existent child "%s" in %s`, file.Metadata.Subject, childName, file.RelativePath)) continue } if childFile.Metadata.Parent == nil || *childFile.Metadata.Parent != file.Metadata.Subject { result.Errors = append(result.Errors, fmt.Sprintf( `Parent-child mismatch: "%s" lists child "%s", whose parent does not point back.`, file.Metadata.Subject, - child, + childName, )) } } @@ -195,19 +269,39 @@ func RenderTree(root *TreeNode) string { return } seen[node] = true - if prefix == "" { - lines = append(lines, node.Subject) - } else { - lines = append(lines, prefix+"|- "+node.Subject) - } + lines = append(lines, treeLabel(node, prefix == "", prefix)) for _, child := range node.Children { - visit(child, prefix+"| ") + childPrefix := prefix + "| " + if prefix != "" { + childPrefix = prefix + " " + } + visit(child, childPrefix) } } visit(root, "") return strings.Join(lines, "\n") } +func treeLabel(node *TreeNode, root bool, prefix string) string { + label := node.Subject + if node.Path != nil { + label += " [" + *node.Path + "]" + } + if !node.Valid && node.Kind != "index" { + label += " [invalid]" + } + if node.Description != "" { + label += ": " + node.Description + } + if root { + return label + } + if len(prefix) > 1 { + prefix = prefix[:1] + strings.ReplaceAll(prefix[1:], "|", " ") + } + return prefix + "|- " + label +} + func RenderTreeMarkdown(root *TreeNode) string { var lines []string seen := map[*TreeNode]bool{} @@ -217,7 +311,14 @@ func RenderTreeMarkdown(root *TreeNode) string { return } seen[node] = true - lines = append(lines, fmt.Sprintf("%s- **%s**: %s", strings.Repeat(" ", depth), node.Subject, node.Description)) + label := "**" + node.Subject + "**" + if node.Path != nil { + label += " (`" + *node.Path + "`)" + } + if !node.Valid { + label += " [invalid]" + } + lines = append(lines, fmt.Sprintf("%s- %s: %s", strings.Repeat(" ", depth), label, node.Description)) for _, child := range node.Children { visit(child, depth+1) } diff --git a/internal/xdocs/xdocs.xdocs.md b/internal/xdocs/xdocs.xdocs.md index 48fd434..3361d9f 100644 --- a/internal/xdocs/xdocs.xdocs.md +++ b/internal/xdocs/xdocs.xdocs.md @@ -9,14 +9,15 @@ files: metadata.go: Exact YAML frontmatter extraction and typed descriptor/document decoding with read-only existing-header auditing. discovery.go: Exclusion-aware complete descriptor and companion-document discovery, including invalid descriptor candidates and traversal diagnostics. output.go: Preflighted, project-bounded, atomic single-file report output. - tree.go: Synchronized parent-child, single-root, duplicate, orphan, and cycle validation with deterministic tree construction. + tree.go: Directory-containment tree construction that retains every descriptor candidate, plus separate metadata relationship validation. meta.go: Metadata-only scoped reads and filters. context.go: Ranked minimal reading-set recommendations. doctor.go: Descriptor, companion, tree, and documented-file health checks. render.go: Deterministic text, Markdown, and JSON domain rendering. xdocs_test.go: Discovery, metadata, context, tree, doctor, and existing-header audit regression tests. ignore_test.go: Gitignore, negation, authorization, file/directory rule, tracking, context, and doctor regression tests. - output_test.go: Atomic report output, descriptor rejection, exclusion, missing-parent, and symlink safety tests. + tree_test.go: Deep filesystem tree coverage, malformed and duplicate descriptor retention, context, indentation, and JSON-cycle regressions. + output_test.go: Atomic report output, descriptor rejection, strict frontmatter schema, exclusion, missing-parent, and symlink safety tests. documents: {} tags: - structured-documentation diff --git a/internal/xdocs/xdocs_test.go b/internal/xdocs/xdocs_test.go index f5e42d5..b733962 100644 --- a/internal/xdocs/xdocs_test.go +++ b/internal/xdocs/xdocs_test.go @@ -163,6 +163,57 @@ flags: [] } } +func TestExistingFrontmatterAuditIncludesUnlistedAndDescriptorlessDocuments(t *testing.T) { + root := t.TempDir() + files := map[string]string{ + "unlisted.md": "---\nname: [unterminated\n", + "valid.md": "---\nsource: historical\nowner: historical-owner\n---\nbody\n", + "missing.md": "# No header\n", + } + for name, content := range files { + if err := os.WriteFile(filepath.Join(root, name), []byte(content), 0o644); err != nil { + t.Fatal(err) + } + } + cfg, err := config.Defaults(root) + if err != nil { + t.Fatal(err) + } + result, err := ScanMetadata(cfg, MetaOptions{ExistingFrontmatter: true}) + if err != nil { + t.Fatal(err) + } + if !result.IncludeDocuments || !result.ExistingFrontmatter { + t.Fatalf("existing-header audit did not imply document reads: %#v", result) + } + if len(result.Documents) != len(files) { + t.Fatalf("audited documents = %d, want %d: %#v", len(result.Documents), len(files), result.Documents) + } + byPath := map[string]MetaDocument{} + for _, document := range result.Documents { + byPath[document.RelativePath] = document + } + if byPath["missing.md"].Valid == false { + t.Fatalf("missing header was not valid in read-only audit: %#v", byPath["missing.md"]) + } + if byPath["valid.md"].Valid == false || byPath["valid.md"].Frontmatter["source"] != "historical" { + t.Fatalf("valid generic header was not audited: %#v", byPath["valid.md"]) + } + if byPath["unlisted.md"].Valid || len(byPath["unlisted.md"].Errors) == 0 { + t.Fatalf("malformed descriptorless header was not reported: %#v", byPath["unlisted.md"]) + } + if !strings.Contains(strings.Join(result.Errors, "\n"), "unlisted.md") { + t.Fatalf("audit errors omitted descriptorless malformed path: %#v", result.Errors) + } + filtered, err := ScanMetadata(cfg, MetaOptions{ExistingFrontmatter: true, Filters: Filters{Owner: "historical-owner"}}) + if err != nil { + t.Fatal(err) + } + if len(filtered.Documents) != 1 || filtered.Documents[0].RelativePath != "valid.md" { + t.Fatalf("standalone audit owner filter ignored raw owner: %#v", filtered.Documents) + } +} + func TestContextRejectsEmptyQuery(t *testing.T) { cfg := config.Config{CWD: t.TempDir(), Extensions: []string{".xdocs.md"}, Exclude: []string{}} if _, err := FindContext(cfg, " ", ContextOptions{}); err == nil { @@ -184,6 +235,87 @@ flags: []` } } +func TestFrontmatterDelimitersMustOccupyWholeLines(t *testing.T) { + for _, content := range []string{ + "---suffix\nsubject: example\n---\n", + "---\nsubject: example\n---suffix\n", + "---\nsubject: example\n", + " \n---\nsubject: example\n---\n", + "\ufeff---\nsubject: example\n---\n", + } { + if _, _, ok := ExtractFrontmatter(content); ok { + t.Fatalf("malformed frontmatter delimiter accepted: %q", content) + } + } + frontmatter, body, ok := ExtractFrontmatter("---\nsubject: example\n---\nbody\n") + if !ok || frontmatter != "subject: example" || body != "body" { + t.Fatalf("valid frontmatter was not extracted exactly: %q %q %t", frontmatter, body, ok) + } +} + +func TestDescriptorRejectsScalarCoercionAndWrongCollections(t *testing.T) { + root := t.TempDir() + path := filepath.Join(root, "module.xdocs.md") + content := `--- +subject: 123 +description: true +parent: 42 +children: [7] +files: + service.go: false +documents: [] +tags: [true] +keywords: [9] +flags: {} +--- +` + if err := os.WriteFile(path, []byte(content), 0o644); err != nil { + t.Fatal(err) + } + file := ParseFile(path, root) + if file.Valid || !strings.Contains(strings.Join(file.Errors, "\n"), "frontmatter.subject") { + t.Fatalf("scalar descriptor fields were accepted: %#v", file) + } + cfg, err := config.Defaults(root) + if err != nil { + t.Fatal(err) + } + meta, err := ScanMetadata(cfg, MetaOptions{}) + if err != nil { + t.Fatal(err) + } + if len(meta.Descriptors) != 1 || meta.Descriptors[0].Valid { + t.Fatalf("metadata scan accepted wrong descriptor types: %#v", meta.Descriptors) + } + doctor, err := Doctor(cfg, DoctorOptions{}) + if err != nil { + t.Fatal(err) + } + if doctor.Valid || doctor.Summary.Errors == 0 { + t.Fatalf("doctor missed wrong descriptor types: %#v", doctor) + } + content = `--- +<<: &metadata + subject: 123 + description: true + parent: null + children: [] + files: {} + documents: {} + tags: [] + keywords: [] + flags: [] +--- +` + if err := os.WriteFile(path, []byte(content), 0o644); err != nil { + t.Fatal(err) + } + file = ParseFile(path, root) + if file.Valid || !strings.Contains(strings.Join(file.Errors, "\n"), "anchors") { + t.Fatalf("descriptor YAML merge bypassed strict validation: %#v", file) + } +} + func TestValidateTreeRejectsMismatchesCyclesAndMultipleRoots(t *testing.T) { parent := "root" cycleParent := "cycle-b" From 55453abe61aa03c9a6f4fa0f1ef3e96c686692a0 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Crist=C3=B3v=C3=A3o=20GUIHO?= Date: Sat, 12 Sep 2026 12:21:51 +0000 Subject: [PATCH 18/20] Reject whitespace-only companion metadata --- internal/xdocs/metadata.go | 2 +- internal/xdocs/output_test.go | 11 +++++++ internal/xdocs/xdocs_test.go | 56 +++++++++++++++++++++++++++++++++++ 3 files changed, 68 insertions(+), 1 deletion(-) diff --git a/internal/xdocs/metadata.go b/internal/xdocs/metadata.go index 31694e0..f443baa 100644 --- a/internal/xdocs/metadata.go +++ b/internal/xdocs/metadata.go @@ -356,7 +356,7 @@ func parseDocument(path, root, expectedOwner string, frontmatterRequired, inspec func validateCompanionFrontmatter(frontmatter Frontmatter, expectedOwner string) []string { var errors []string for _, field := range []string{"name", "purpose", "description", "created", "owner"} { - if stringValue(frontmatter, field) == "" { + if strings.TrimSpace(stringValue(frontmatter, field)) == "" { errors = append(errors, fmt.Sprintf(`Missing or invalid "%s" field. Expected a non-empty string.`, field)) } } diff --git a/internal/xdocs/output_test.go b/internal/xdocs/output_test.go index 7364f21..a944654 100644 --- a/internal/xdocs/output_test.go +++ b/internal/xdocs/output_test.go @@ -112,6 +112,17 @@ func TestWriteReportAllowsExplicitlyAuthorizedGenericFrontmatter(t *testing.T) { if string(unchanged) != valid { t.Fatalf("non-leading frontmatter rejection changed destination: %q", unchanged) } + whitespace := "---\nname: \" \"\npurpose: \" \"\ndescription: \" \"\ncreated: 2026-09-12\nowner: \" \"\nflags: []\ntags: []\nkeywords: []\n---\nchanged\n" + if err := WriteReport(cfg, "notes/report.md", []byte(whitespace)); err == nil { + t.Fatal("report with whitespace-only required companion fields was accepted") + } + unchanged, err = os.ReadFile(filepath.Join(root, "notes", "report.md")) + if err != nil { + t.Fatal(err) + } + if string(unchanged) != valid { + t.Fatalf("whitespace-only companion rejection changed destination: %q", unchanged) + } } func TestWriteReportRejectsGitignoredAndSymlinkDestinations(t *testing.T) { diff --git a/internal/xdocs/xdocs_test.go b/internal/xdocs/xdocs_test.go index b733962..9c583d3 100644 --- a/internal/xdocs/xdocs_test.go +++ b/internal/xdocs/xdocs_test.go @@ -214,6 +214,62 @@ func TestExistingFrontmatterAuditIncludesUnlistedAndDescriptorlessDocuments(t *t } } +func TestRequiredCompanionWhitespaceFieldsFailMetadataAndDoctor(t *testing.T) { + root := t.TempDir() + descriptor := `--- +subject: example +description: Example descriptor. +parent: null +children: [] +files: {} +documents: + guide.md: Guide. +tags: [] +keywords: [] +flags: [] +--- +` + companion := `--- +name: " " +purpose: " " +description: " " +created: 2026-09-12 +owner: " " +flags: [] +tags: [] +keywords: [] +--- +` + if err := os.WriteFile(filepath.Join(root, "example.xdocs.md"), []byte(descriptor), 0o644); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(root, "guide.md"), []byte(companion), 0o644); err != nil { + t.Fatal(err) + } + cfg, err := config.Defaults(root) + if err != nil { + t.Fatal(err) + } + cfg.Documentation = config.DocumentationConfig{ + Directories: []string{"."}, + Frontmatter: []config.DocumentationRule{{Pattern: "guide.md", Kind: "file"}}, + } + meta, err := ScanMetadata(cfg, MetaOptions{IncludeDocuments: true}) + if err != nil { + t.Fatal(err) + } + if len(meta.Descriptors) != 1 || len(meta.Descriptors[0].Documents) != 1 || meta.Descriptors[0].Documents[0].Valid { + t.Fatalf("whitespace-only companion fields were accepted: %#v", meta) + } + doctor, err := Doctor(cfg, DoctorOptions{IncludeDocuments: true, WarningsAsErrors: true}) + if err != nil { + t.Fatal(err) + } + if doctor.Valid || doctor.Summary.Errors == 0 { + t.Fatalf("doctor did not report whitespace-only companion fields: %#v", doctor) + } +} + func TestContextRejectsEmptyQuery(t *testing.T) { cfg := config.Config{CWD: t.TempDir(), Extensions: []string{".xdocs.md"}, Exclude: []string{}} if _, err := FindContext(cfg, " ", ContextOptions{}); err == nil { From 6ba7eff928f332865636f37d6fc0ac2416340139 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Crist=C3=B3v=C3=A3o=20GUIHO?= Date: Sat, 12 Sep 2026 12:25:41 +0000 Subject: [PATCH 19/20] Record authorization and tree implementation review --- ...icit-documentation-authorization-review.md | 85 +++++++++++++++++++ .../implementation/implementation.xdocs.md | 1 + 2 files changed, 86 insertions(+) create mode 100644 docs/reviews/implementation/explicit-documentation-authorization-review.md diff --git a/docs/reviews/implementation/explicit-documentation-authorization-review.md b/docs/reviews/implementation/explicit-documentation-authorization-review.md new file mode 100644 index 0000000..15d14ee --- /dev/null +++ b/docs/reviews/implementation/explicit-documentation-authorization-review.md @@ -0,0 +1,85 @@ +--- +name: Explicit documentation authorization implementation review +purpose: Record implementation review and validation for issue 21. +description: Acceptance mapping, resolved findings, test evidence, and delivery status for opt-in metadata and complete trees. +created: 2026-09-12 +owner: xdocs-implementation-reviews +flags: [reviewed, human-review-pending] +tags: [review, authorization, tree] +keywords: [issue 21, Luna, Sol, frontmatter, report output] +--- + +#### © 2026 [GUIHO](https://guiho.co) as represented by [Cristóvão GUIHO](https://guiho.co/cguiho) All Rights Reserved. + +# Explicit documentation authorization implementation review + +## Verdict and delivery + +Sol (`gpt-5.6-sol`, xhigh) approved implementation commit `55453ab` with no +unresolved findings after reviewing the integrated changes and corrective +patches. Luna (`gpt-5.6-luna`, max) performed implementation and delegated +tree tests and documentation work. The primary coordinator authored the +[plan](../../plans/explicit-documentation-authorization.md), checked independent +CLI fixtures, and pushed the reviewed implementation to +`fix/issue-21-documentation-authorization` on `CGuiho/xdocs`. + +This addresses [issue 21](https://github.com/CGuiho/xdocs/issues/21), created +2026-09-11, and the user's subsequent scope clarification. Human acceptance +remains pending; TODO task 10 is in testing. No version bump or release was +performed. The separate CLI Convention 0001 migration remains unfinished. + +## Acceptance mapping + +| Requirement | Implementation | Evidence | +| --- | --- | --- | +| Explicit directory and companion metadata grants | Strict typed `documentation.directories` and `documentation.frontmatter`, empty by default; legacy denials take precedence | Config and domain tests cover boundaries, malformed grants, scalar coercion, aliases, merges, and default policy | +| Preserve ordinary Markdown and avoid unnecessary files | Metadata requirements apply only to explicit grants; embedded guidance keeps context in one named descriptor; report commands default to stdout | Resource tests and independent command snapshots preserve README and project bytes | +| One valid named descriptor per directory | Discovery and doctor diagnose duplicate and bare/legacy names; no automatic consolidation or repair | Descriptor, discovery, and doctor regression tests | +| Full tree at arbitrary depth | Filesystem containment with nearest descriptor ancestor, paths/context, root index, and diagnostics; invalid/orphan/duplicate nodes remain visible | Tree and command tests plus independent text/Markdown/JSON fixture with deep nesting and malformed metadata | +| Safe explicit report output | Shared preflight and atomic replacement reject descriptor destinations, symlinks, exclusions, invalid headers, and unauthorized companion metadata | Output tests prove destination preservation on rejection; independent CLI checks verify exact single report output | +| Explicit audit of historical headers | Standalone `meta --existing-frontmatter` and `doctor --existing-frontmatter` include ordinary documents without descriptors; absent headers remain valid | Focused audit tests and independent descriptorless-document fixture | +| Consistent user and embedded instructions | Updated README, DOCS, managed instruction template, skill, prompts, AGENTS, and descriptor indexes | Sol documentation review and embedded-resource tests | + +## Resolved review findings + +Review corrections closed path escape and portable-path validation gaps, +YAML scalar coercion and merge/alias authorization bypasses, incomplete +existing-header audit discovery, ambiguous frontmatter openers, flattened +tree rendering, and root-index classification. Report preflight now applies +the same companion schema validation as metadata and doctor. + +The final correction in `55453ab` rejects required companion string fields +containing only whitespace. Sol reran the adversarial report probe and tests +that verify both no destination creation and byte-for-byte preservation of an +existing destination. Meta and doctor consistency tests also passed. + +## Validation evidence + +Go 1.26.5 was used from an isolated temporary toolchain. Luna completed the +full suite, vet, and CGO-disabled native build at `55453ab`. Sol independently +ran the full suite on the preceding integrated revision and an uncached domain +suite plus focused adversarial tests on the final correction. Formatting and +`git diff --check` passed. + +Commands exercised: + +```sh +go test ./... +go vet ./... +CGO_ENABLED=0 go build -trimpath -o /tmp/xdocs-issue21-review . +xdocs meta --documents --strict --format json +xdocs doctor --format json +xdocs tree +``` + +The primary rebuilt the final revision and reran two independent temporary +project fixtures: complete deep-tree output and read-only command preservation; +and default metadata policy, standalone historical-header audit, strict +descriptor types, YAML merge rejection, and byte preservation. Both passed. +Repository doctor reported valid with zero errors and zero warnings. + +Windows AMD64 and macOS ARM64 cross-builds passed on `7582fb6`, before the +final platform-independent whitespace validation correction. Runtime smoke +tests ran on Linux only; native Windows and macOS execution was not tested. +Temporary binaries and smoke scripts are validation artifacts outside the +repository and are not shipped files. diff --git a/docs/reviews/implementation/implementation.xdocs.md b/docs/reviews/implementation/implementation.xdocs.md index 463f808..1d8d788 100644 --- a/docs/reviews/implementation/implementation.xdocs.md +++ b/docs/reviews/implementation/implementation.xdocs.md @@ -5,6 +5,7 @@ parent: xdocs-reviews children: [] files: {} documents: + explicit-documentation-authorization-review.md: Acceptance mapping and validation evidence for explicit documentation authorization, protected report output, and complete filesystem trees. guiho-cli-convention-0001-compliance-review.md: Findings-first audit of xdocs against GUIHO CLI Convention 0001 across tooling, flags, configuration, agents, lifecycle, releases, documentation, and validation. ignore-paths-and-frontmatter-review.md: Findings-first acceptance review of Git-aware exclusion, frontmatter opt-outs, raw output safety, command boundaries, and resolved review findings. open-issue-and-release-hardening-review.md: Findings-first review approving release assets, installers, issues 7 through 12, and exact-version release hardening for a patch release. From 7b58a3310c547a577b5ddd789e861e4e7a2f29fd Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Crist=C3=B3v=C3=A3o=20GUIHO?= Date: Sat, 12 Sep 2026 12:25:41 +0000 Subject: [PATCH 20/20] Move documentation authorization work to user testing --- TODO.md | 10 +++++++--- .../explicit-documentation-authorization.md | 16 ++++++++++------ 2 files changed, 17 insertions(+), 9 deletions(-) diff --git a/TODO.md b/TODO.md index c66a1d4..163ecd9 100644 --- a/TODO.md +++ b/TODO.md @@ -31,8 +31,8 @@ All Rights Reserved. | Status | Count | | --- | ---: | | todo | 1 | -| in progress | 2 | -| testing | 0 | +| in progress | 1 | +| testing | 1 | | stopped | 0 | | completed | 7 | @@ -155,12 +155,16 @@ All Rights Reserved. ### 10. Enforce Explicit Documentation Authorization And Complete Trees -- Status: in progress +- Status: testing - Created: `2026-09-12` - Updated: `2026-09-12` - Outcome: Require explicit documentation directory and companion-frontmatter opt-ins, preserve ordinary Markdown, guard report destinations, and display every discovered named descriptor through arbitrary depth. - Plan: [docs/plans/explicit-documentation-authorization.md](docs/plans/explicit-documentation-authorization.md) +- Review: [docs/reviews/implementation/explicit-documentation-authorization-review.md](docs/reviews/implementation/explicit-documentation-authorization-review.md) +- External: [CGuiho/xdocs#21](https://github.com/CGuiho/xdocs/issues/21) +- Acceptance: Implementation reviewed and branch pushed; user testing pending. + ## Completion Archive Completed tasks and their task-owned evidence are registered in diff --git a/docs/plans/explicit-documentation-authorization.md b/docs/plans/explicit-documentation-authorization.md index bec3f4a..047742b 100644 --- a/docs/plans/explicit-documentation-authorization.md +++ b/docs/plans/explicit-documentation-authorization.md @@ -4,7 +4,7 @@ purpose: Plan issue 21 implementation without unsolicited Markdown changes. description: Scoped requirements, architecture, execution units, and acceptance checks for opt-in documentation and complete descriptor discovery. created: 2026-09-12 owner: xdocs-plans -flags: [execution-authorized] +flags: [execution-complete, reviewed, human-review-pending] tags: [planning, authorization, tree] keywords: [issue 21, named descriptors, frontmatter, Luna, Sol] --- @@ -182,11 +182,11 @@ never force or bypass a rejection. Do not publish a release. ## Tracking, findings, and acceptance -- [ ] U1 typed opt-in policy and read behavior implemented and reviewed. -- [ ] U2 complete deep tree implemented and reviewed. -- [ ] U3 guarded output implemented and reviewed. -- [ ] U4 embedded guidance and documentation consistent and reviewed. -- [ ] U5 full checks, integrated review, and GitHub push verified. +- [x] U1 typed opt-in policy and read behavior implemented and reviewed. +- [x] U2 complete deep tree implemented and reviewed. +- [x] U3 guarded output implemented and reviewed. +- [x] U4 embedded guidance and documentation consistent and reviewed. +- [x] U5 full checks, integrated review, and GitHub push verified. Use this plan's checklist for executor progress and add one concise task to the existing TODO index. Reuse existing documentation directories instead of @@ -200,3 +200,7 @@ Hard stops: potential unrecoverable data loss, secret access, or required production mutation. Resolve routine implementation choices conservatively, record them, and continue. No schema or output policy may silently expand the user's authorization. + +Implementation and GitHub branch delivery are complete. Human acceptance remains +pending. See the [implementation review](../reviews/implementation/explicit-documentation-authorization-review.md) +for requirement mapping, review findings, and validation evidence.