test: restore stale registry key consistency check - #174
Merged
Conversation
Contributor
Author
1 similar comment
Contributor
Author
Contributor
|
Thanks for the patch. Could you please add more context to the currently blank Summary and Solution Description sections? In particular, it would help to explain:
That context will make it easier to confirm that this is still the intended registry invariant. |
Contributor
Author
|
Thanks for the review! I have filled in the Summary and Solution Description sections of the PR description with the regression context (check originally from #100, dropped in #114), the relation to the #92/#100 work, and the exact test commands/results including the negative case (injected stale key |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The bundled Cookiecutter template registry (
templates/index.json) must stay in sync with the actual template directories undertemplates/. This PR restores one of the registry consistency checks that #92 originally asked for: every key intemplates/index.jsonmust point to an existingtemplates/<type>/<name>/directory containingcookiecutter.json(a "stale key" check).Regression context
The original check shipped in #100 (commit
920c1cb) astest_registered_templates_point_to_cookiecutter_directories. During the lifecycle-toolkit refactor in #114 (commite87460d), it was removed and replaced bytest_registry_contains_expected_template_keys, which only asserts that a fixed set (EXPECTED_TEMPLATE_KEYS) is a subset of the registry. That check is one-directional: it cannot detect extra or unknown keys.As a result, since #114 the repo has had no test that fails when
templates/index.jsoncontains a stale key (e.g. a key pointing to a directory that was deleted, renamed, or never merged). Such a stale key would make the CLI advertise a template that cannot be rendered, and CI would pass silently — exactly the failure mode #92's acceptance criteria ("Tests fail whentemplates/index.jsoncontains a stale key") were meant to prevent.Relation to #92 / #100
bub/contextseekis unaffected: it is deliberately not registered, so it can never be reported as stale.Solution Description
Add
test_registered_templates_point_to_cookiecutter_directoriestotests/cli_commands/test_templates_registry.py:_template_dirs()walkstemplates/and collects every directory containing acookiecutter.json;_registered_templates()reads the keys oftemplates/index.json. The test fails exactly when the registry advertises a template that has no backing directory. The test name and assertion match the version originally shipped in #100, so the diff is minimal — one test, no production code, no new CLI command.Validation
Negative case — verifying the test actually catches a stale key: temporarily added
"langchain/nonexistent": "..."totemplates/index.jsonand re-ran:templates/index.jsonwas restored afterwards; the working tree contains only the test change.