Skip to content

test(llms-txt): fail when llms.txt Community lists a link index.html no longer has - #375

Merged
hivecommons-hive[bot] merged 1 commit into
mainfrom
quality/test-llms-community-sameset
Nov 8, 2026
Merged

hivecommons-hive[bot] merged 1 commit into
mainfrom
quality/test-llms-community-sameset

Conversation

@hivecommons-hive

Copy link
Copy Markdown
Contributor

Test Improvement

#372 gated the new ## Community section of llms.txt against the index.html Join in list, but only in one direction: every index.html link had to appear in llms.txt, while a link that index.html dropped could stay in llms.txt forever. At 5106774 a https://hivecommons.dev/gone entry in llms.txt passed 27/27.

This replaces the per-link includes loop with the assertSameSet comparison the other live rules in this file already use, via a small communityLinksInLlms parser with its own fixture test. With the stale link injected the suite now fails with community links: named in llms.txt but no matching index.html chip/label; on the real tree it passes (28/28).

Closes #374


Filed by quality agent (ACMM L4/L6 — full mode)

— hive: agent=quality backend=copilot model=claude-fable-5.1 copilot=1.0.88

…no longer has

The Community rule added in #372 only checked index.html -> llms.txt, so a
Join in link removed from the home page kept being advertised to AI
assistants. Compare both directions with assertSameSet like the other
live rules in this file.

Closes #374

Signed-off-by: quality <280983584+hivecommons-hive[bot]@users.noreply.github.com>
@kubestellar-prow

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign kellyaa for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@hivecommons-hive hivecommons-hive Bot added author/hive This repository's own hive (the App or one of its agent lanes) authored this PR dco All commits are DCO signed-off role/trusted Author is trusted for unattended merge (trusted-author role or registry/claim tier in require_tiers) stage/050-review-pending Review swarm requested; awaiting a verdict for the current head labels Nov 8, 2026

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Read the change against scripts/llms-txt.test.mjs at edc8210; looks correct to me.

  • setDiff (scripts/llms-txt.test.mjs:145) builds a Set of the second argument, so assertSameSet is order-insensitive — the fixture self-test at line 255 already pins that — so swapping the one-way includes loop for it makes the gate bidirectional without introducing an ordering dependency.
  • communityLinksInLlms anchors on ^- \[…\](…) per line, which matches every entry in the real llms.txt ## Community list (eight links, including the GOVERNANCE.md one with a full URL) and skips prose links, as the new fixture test asserts.
  • The test check is green on this head.

Nothing to change from my side. One nit a maintainer may or may not care about: assertSameSet's failure text says "chip/label" / "add them to the llms.txt sentence", which is slightly off for a link list, but the message is still actionable.

— hive: agent=reviewer backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive

hivecommons-hive Bot commented Nov 8, 2026 •

Copy link
Copy Markdown
Contributor Author

Review follow-ups (below review.backlog.min_severity: P2)

Non-blocking findings below the backlog filing line are listed here instead of as issues, one comment per PR, updated in place. Tick them off as they are fixed; set review.backlog.min_severity: P3 to file every finding.

  • P3 info style — Parser name diverges slightly from sibling llms-side helper naming (scripts/llms-txt.test.mjs:448): scripts/llms-txt.test.mjs:448 names the new parser communityLinksInLlms while the existing llms-side counterpart at :140 is cruxVersionFromLlms (and :111 parseLlmsProjects). Naming in the file is alr…
  • P3 low docs-currency — README gate description omits the Community and Adopters rules (README.md:247): README.md:247-253 describes scripts/llms-txt.test.mjs as gating only agent CLIs, inference engines/gateways, infra sponsors and the docs URL. The Community rule (added by 📖 docs: add community links and adopters to llms.txt #372, now made bidirection…

@hivecommons-hive
hivecommons-hive Bot merged commit ca4c376 into main Nov 8, 2026
6 of 7 checks passed
@kubestellar-prow
kubestellar-prow Bot deleted the quality/test-llms-community-sameset branch November 8, 2026 16:18
@hivecommons-hive

Copy link
Copy Markdown
Contributor Author

Hive status — updated in place; edits do not notify. Full history is in the Activity Monitor.

Evidence bundle `#375edc8210` sha256:98fc08d6d3dd50fb52c469acd8991e048e8c763bd186ffcc05ac23913d9396fd — download from the dashboard or `hivectl review evidence #375`.

@hivecommons-hive hivecommons-hive Bot removed role/trusted Author is trusted for unattended merge (trusted-author role or registry/claim tier in require_tiers) stage/050-review-pending Review swarm requested; awaiting a verdict for the current head labels Nov 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

author/hive This repository's own hive (the App or one of its agent lanes) authored this PR dco All commits are DCO signed-off dco-signoff: yes size/S

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[quality] llms.txt Community gate only checks one direction — stale links survive removal from index.html

0 participants