Skip to content

Add --list-rules option for discovering the rule catalog - #25

Merged
Malcolmnixon merged 7 commits into
mainfrom
feat/list-rules-option
Sep 29, 2026
Merged

Malcolmnixon merged 7 commits into
mainfrom
feat/list-rules-option

Conversation

@Malcolmnixon

Copy link
Copy Markdown
Member

Pull Request

Description

Adds a --list-rules option that emits a JSON catalog of every rule code the
Linting subsystem can produce (code, title, official/advisory classification,
suggestion kind, and applicable mode(s)).

This addresses integrator feedback (item #8 of a third-party write-up on
embedding ste100mark in downstream tooling): the rule catalog could
previously only be discovered by running the tool over prose built to trip
every check and cross-referencing the assembly's string table. Callers can
now query --list-rules at startup to validate their assumptions instead of
hard-coding a rule list that goes stale when the tool adds a rule.

New unit: src/DemaConsulting.Ste100Mark/Linting/RuleCatalog.cs — a static,
hand-authored, pure data source with no dependency on Context,
StructuralRules, or DictionaryChecker.

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update
  • Code quality improvement

Related Issues

Closes #

Pre-Submission Checklist

Before submitting this pull request, ensure you have completed the following:

Build and Test

  • Code builds successfully and all tests pass: pwsh ./build.ps1 — 1320/1320 passed (net8.0/net9.0/net10.0)
  • Code produces zero warnings

Code Quality

  • New code has appropriate XML documentation comments
  • Static analyzer warnings have been addressed

Quality Checks

Please run the following checks before submitting:

  • All linters pass: pwsh ./lint.ps1 — clean (yamllint, cspell, markdownlint, dotnet format, reqstream, versionmark, reviewmark, sysml2tools)

Testing

  • Added unit tests for new functionality
  • Updated existing tests if behavior changed
  • All tests follow the AAA (Arrange, Act, Assert) pattern
  • Test coverage is maintained or improved

Documentation

  • Updated README.md (if applicable)
  • Updated docs/ documentation (if applicable)
  • Added code examples for new features (if applicable)
  • Updated requirements.yaml (if applicable)

Additional Notes

  • SysML2 architecture model updated in the same change: added
    docs/sysml2/model/ste100-mark/linting/rule-catalog.sysml and wired
    part ruleCatalog : RuleCatalog; into linting.sysml, per this repo's
    sysml2-modeling.md standard.
  • Independent formal review performed prior to opening this PR; the one
    finding raised (missing SysML2 model entry) was fixed before push.

Malcolm Nixon and others added 2 commits September 29, 2026 11:20
Adds a new RuleCatalog unit enumerating all 8 rule codes the tool can
emit (STE100-4.1, STE100-4.2, STE100-8.1, STE100-DICT, and the four
STE100-ADV-* advisory heuristics), each with code, title, official
flag, suggestionKind (advice/citationForm), and applicable modes.

- Context.cs: new --list-rules flag / ListRules property
- Program.cs: --list-rules dispatches before the main lint path,
  short-circuiting like --version/--help, requiring no Markdown files
- RuleCatalog.cs: JSON serialization via System.Text.Json source
  generation matching DiagnosticReporter's camelCase/indented style
- Tests: Context parsing, Program dispatch, RuleCatalog content, and
  an integration test running the published CLI in an empty directory
- Docs: design/verification/reqstream updates across cli, program,
  and linting docs, plus README and user guide, following the
  --allow-empty precedent; new system-level reqstream requirement
  links all new unit-level requirements as children
Address formal-review finding: the RuleCatalog unit backing --list-rules
was missing its mandatory SysML2 model counterpart per the
sysml2-modeling.md standard.

- Add docs/sysml2/model/ste100-mark/linting/rule-catalog.sysml (mirrors
  dictionary-checker.sysml conventions)
- Wire part ruleCatalog : RuleCatalog; into linting.sysml

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI lite review requested due to automatic review settings September 29, 2026 15:48

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Review findings remain in implementation accuracy, help coverage, and verification traceability.

Review effort: Lite
Findings: 1 Medium severity · 3 Low severity

Open (4)
What changed in this PR

Adds --list-rules to emit a JSON catalog of available linting rules for downstream tooling.

Changes:

  • Adds catalog metadata, serialization, CLI parsing, and dispatch.
  • Adds unit and integration tests.
  • Updates documentation, requirements, design artifacts, verification, and SysML2 models.
File Change
test/​DemaConsulting.Ste100Mark.Tests/​ProgramTests.cs Tests catalog-only program output.
test/​DemaConsulting.Ste100Mark.Tests/​Linting/​RuleCatalogTests.cs Tests catalog contents and serialization.
test/​DemaConsulting.Ste100Mark.Tests/​IntegrationTests.cs Tests published CLI behavior.
test/​DemaConsulting.Ste100Mark.Tests/​Cli/​ContextTests.cs Tests flag parsing.
test/​DemaConsulting.Ste100Mark.Tests/​Cli/​CliSubsystemTests.cs Tests CLI flow.
src/​DemaConsulting.Ste100Mark/​Program.cs Dispatches --list-rules and updates help.
src/​DemaConsulting.Ste100Mark/​Linting/​RuleCatalog.cs Defines and serializes rule metadata.
src/​DemaConsulting.Ste100Mark/​Cli/​Context.cs Parses and stores the new flag.
README.md Documents the new option.
docs/​verification/​ste100-mark/​program.md Documents program verification.
docs/​verification/​ste100-mark/​linting.md Documents catalog verification.
docs/​verification/​ste100-mark/​cli/​context.md Documents context verification.
docs/​verification/​ste100-mark/​cli.md Documents CLI verification.
docs/​user_guide/​introduction.md Adds user-facing catalog guidance.
docs/​sysml2/​model/​ste100-mark/​linting/​rule-catalog.sysml Models the catalog unit.
docs/​sysml2/​model/​ste100-mark/​linting.sysml Adds the catalog to Linting.
docs/​reqstream/​ste100-mark/​program.yaml Adds program requirements.
docs/​reqstream/​ste100-mark/​linting.yaml Adds linting requirements and traceability.
docs/​reqstream/​ste100-mark/​cli/​context.yaml Adds context requirements.
docs/​reqstream/​ste100-mark/​cli.yaml Adds CLI requirements.
docs/​reqstream/​ste100-mark.yaml Adds system-level requirements.
docs/​design/​ste100-mark/​program.md Updates program design.
docs/​design/​ste100-mark/​linting/​rule-catalog.md Documents catalog design.
docs/​design/​ste100-mark/​linting.md Updates linting design.
docs/​design/​ste100-mark/​cli/​context.md Updates context design.
docs/​design/​ste100-mark/​cli.md Updates CLI design.
docs/​design/​ste100-mark.md Updates system architecture.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/DemaConsulting.Ste100Mark/Program.cs
Comment thread docs/reqstream/ste100-mark/linting.yaml
Comment thread docs/verification/ste100-mark/linting.md Outdated
Comment thread src/DemaConsulting.Ste100Mark/Program.cs
- Assert --list-rules appears in help output
  (Program_Run_WithHelpFlag_DisplaysUsageInformation)
- Update Run's XML remarks to document the rule-catalog dispatch
  before help/validation
- Link RuleCatalog_Entries_ModesIncludeBothWritingModes into the
  Ste100Mark-Linting-RuleCatalog requirement's tests list
- Include that test in the linting verification doc's scenario methods

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings September 29, 2026 16:07

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Unresolved moderate findings concern inaccurate rule metadata and insufficient catalog completeness guarantees.

Review effort: Lite
Findings: 1 Medium severity

Open (1)
Resolved since last review (3)
Previously missed (3)

In code that hasn't changed since last review

Medium severity Catalog incorrectly claims dictionary is an approved ASD-STE100 list

src/​DemaConsulting.Ste100Mark/​Linting/​RuleCatalog.cs:63

This title overstates what the checker uses: the default embedded dictionary is explicitly illustrative rather than the official ASD-STE100 Part 2 Dictionary, and callers can supply arbitrary configured dictionaries. The catalog should describe the effective/configured dictionary instead of claiming it is an approved ASD-STE100 word list.

Medium severity Misclassified tool-defined check marked as an official rule

src/​DemaConsulting.Ste100Mark/​Linting/​RuleCatalog.cs:64

Official: true contradicts the metadata contract documented immediately below, which defines this field as an actual ASD-STE100 numbered rule. STE100-DICT is explicitly described elsewhere as a tool-defined dictionary check (docs/design/ste100-mark/linting.md:149-150) and has no numbered ASD-STE100 rule, so integrations consuming this catalog will misclassify it. Either report it as non-official and update the companion tests/docs, or replace this boolean with a classification that distinguishes official numbered rules, tool-defined mechanical checks, and advisory heuristics.

Medium severity Test fails to detect missing emitted rule codes

test/​DemaConsulting.Ste100Mark.Tests/​Linting/​RuleCatalogTests.cs:40

This test does not actually verify the advertised “every emitted rule code” invariant: ExpectedCodes is just a second hand-maintained copy of the catalog, so adding a new code to StructuralRules or DictionaryChecker while forgetting both lists still leaves this test green. Add a completeness guard that derives the emitted codes (or exercises each emitter and compares the observed codes with RuleCatalog.Entries) so future catalog drift fails CI.

…ness

- STE100-DICT is a tool-defined dictionary check, not an actual
  ASD-STE100 numbered rule; classify it Official: false rather than
  true, and drop the incorrect 'approved ASD-STE100 word list' claim
  from its title (the effective dictionary may be the illustrative
  default or a project-supplied configured dictionary)
- Update RuleCatalogEntry's Official XML doc to describe the field's
  actual true/false contract precisely
- Update the rule-catalog design doc's classification table and
  rationale to match
- Replace the hand-maintained-list-vs-hand-maintained-list
  RuleCatalog_Entries_ContainsEveryEmittedRuleCode test with one that
  exercises StructuralRules and DictionaryChecker directly against
  prose engineered to trip every rule, so an emitted code missing from
  (or a stale code left in) RuleCatalog.Entries now fails CI
- Update RuleCatalog_Entries_ClassifiesOfficialAndAdvisoryRulesCorrectly
  for the corrected STE100-DICT classification

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings September 29, 2026 16:31

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

One or more issues must be addressed before approval.

Review effort: Lite
Findings: 1 Medium severity

Open (1)
Previously missed (1)

In code that hasn't changed since last review

Medium severity Unify rule codes to prevent catalog drift

src/​DemaConsulting.Ste100Mark/​Linting/​RuleCatalog.cs:34

Entries is not actually the single source of truth: StructuralRules and DictionaryChecker still embed their own rule-code literals (for example, StructuralRules.cs:203 and DictionaryChecker.cs:326). A future rule can therefore be added to an emitter without being included in this catalog, causing the new discovery API to silently omit exactly the rule it is meant to expose. Share the codes through a common registry/constants source used by both emitters and the catalog, or add an exhaustive registration mechanism that cannot drift.

StructuralRules and DictionaryChecker each embedded their own
'STE100-...' string literals, independent of RuleCatalog.Entries, so a
future rule code could be added to an emitter without a corresponding
catalog entry.

- Add RuleCodes: a single, shared static class of rule-code string
  constants
- StructuralRules, DictionaryChecker, and RuleCatalog now all
  reference the same RuleCodes constants instead of independent
  literals
- Add the SysML2 model counterpart (rule-codes.sysml, wired into
  linting.sysml) and design doc (rule-codes.md) for the new unit,
  per the sysml2-modeling.md standard
- Update the linting design doc and RuleCatalog's remarks to describe
  the RuleCodes/RuleCatalog division of responsibility

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings September 29, 2026 16:50

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

The catalog classification, completeness test, and design inventory require updates before approval.

Review effort: Lite
Findings: 1 Low severity

Open (1)
Resolved since last review (1)
Previously missed (1)

In code that hasn't changed since last review

Medium severity Expose explicit advisory versus mechanical rule classification

src/​DemaConsulting.Ste100Mark/​Linting/​RuleCatalog.cs:68

official: false does not provide the promised official/advisory classification: STE100-DICT is explicitly documented by DictionaryChecker as a non-advisory mechanical error, yet it is indistinguishable in this schema from the advisory heuristics. A consumer cannot determine whether a false entry is advisory or mechanical; expose an explicit classification (or advisory flag) rather than overloading official for both cases.

Comment thread docs/design/ste100-mark/linting.md Outdated
…lassification and add RuleCodes to the unit inventory

RuleCatalogEntry.Official was a boolean, so STE100-DICT (a tool-defined
deterministic check) was indistinguishable from the STE100-ADV-* heuristics
(fallible pattern detection): both reported false, with the distinction only
in prose.

- Replace RuleCatalogEntry.Official (bool) with Classification (string):
  "official" for a numbered ASD-STE100 rule, "mechanical" for a tool-defined
  deterministic check that is not a numbered rule (STE100-DICT), or
  "advisory" for a fallible heuristic that is not a numbered rule
- Update the --list-rules JSON schema and README example accordingly
  (classification replaces official)
- Update RuleCatalogTests, the rule-catalog design doc, the linting design
  doc, reqstream requirement, and verification doc to match
- Add RuleCodes to the Linting subsystem's unit inventory (prose, unit list,
  and dependency diagram) in docs/design/ste100-mark/linting.md; it was
  introduced in the prior review round but never added to the subsystem
  overview

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings September 29, 2026 17:06

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

The catalog completeness test can miss newly emitted rules, and several documents do not match the three-value classification schema.

Review effort: Lite
Findings: 3 Low severity

Open (3)
Resolved since last review (1)

Comment thread docs/reqstream/ste100-mark.yaml Outdated
Comment thread docs/sysml2/model/ste100-mark/linting/rule-catalog.sysml Outdated
Comment thread docs/user_guide/introduction.md Outdated
… the three-value classification schema

The prior round renamed RuleCatalogEntry.Official (bool) to Classification
(string) and updated the source, tests, README, and four design/verification
docs, but missed three more prose copies of the old two-value schema:

- docs/reqstream/ste100-mark.yaml: the system-level requirement text still
  described the catalog's classification as official/advisory only
- docs/sysml2/model/ste100-mark/linting/rule-catalog.sysml: the architecture
  model's doc comment likewise omitted the mechanical value
- docs/user_guide/introduction.md: the --list-rules JSON example still
  showed the removed "official": true boolean field

Also updated RuleCodes.cs's remarks, which described RuleCatalog's metadata
as official/advisory only.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI lite review requested due to automatic review settings September 29, 2026 17:19

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

No unresolved blocking issues were identified.

Review effort: Lite
Findings: None

Resolved since last review (3)

@Malcolmnixon
Malcolmnixon merged commit a443dae into main Sep 29, 2026
16 checks passed
@Malcolmnixon
Malcolmnixon deleted the feat/list-rules-option branch September 29, 2026 17:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants