Skip to content

test: restore stale registry key consistency check - #174

Merged
webup merged 1 commit into
ob-labs:mainfrom
popwatt:test/template-registry-stale-key
Aug 12, 2026
Merged

test: restore stale registry key consistency check#174
webup merged 1 commit into
ob-labs:mainfrom
popwatt:test/template-registry-stale-key

Conversation

@popwatt

@popwatt popwatt commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

Summary

The bundled Cookiecutter template registry (templates/index.json) must stay in sync with the actual template directories under templates/. This PR restores one of the registry consistency checks that #92 originally asked for: every key in templates/index.json must point to an existing templates/<type>/<name>/ directory containing cookiecutter.json (a "stale key" check).

Regression context

The original check shipped in #100 (commit 920c1cb) as test_registered_templates_point_to_cookiecutter_directories. During the lifecycle-toolkit refactor in #114 (commit e87460d), it was removed and replaced by test_registry_contains_expected_template_keys, which only asserts that a fixed set (EXPECTED_TEMPLATE_KEYS) is a subset of the registry. That check is one-directional: it cannot detect extra or unknown keys.

As a result, since #114 the repo has had no test that fails when templates/index.json contains a stale key (e.g. a key pointing to a directory that was deleted, renamed, or never merged). Such a stale key would make the CLI advertise a template that cannot be rendered, and CI would pass silently — exactly the failure mode #92's acceptance criteria ("Tests fail when templates/index.json contains a stale key") were meant to prevent.

Relation to #92 / #100

Solution Description

Add test_registered_templates_point_to_cookiecutter_directories to tests/cli_commands/test_templates_registry.py:

def test_registered_templates_point_to_cookiecutter_directories() -> None:
    """Every registry key points to a template directory with cookiecutter.json."""
    stale = sorted(_registered_templates() - _template_dirs())

    assert not stale, f"stale templates/index.json key(s): {stale}"

_template_dirs() walks templates/ and collects every directory containing a cookiecutter.json; _registered_templates() reads the keys of templates/index.json. The test fails exactly when the registry advertises a template that has no backing directory. The test name and assertion match the version originally shipped in #100, so the diff is minimal — one test, no production code, no new CLI command.

Validation

$ pytest tests/cli_commands/test_templates_registry.py -q
6 passed in 0.02s

$ pytest tests/cli_commands/test_templates_render.py -q   # existing render tests
50 passed in 12.64s

$ ruff check tests/cli_commands/test_templates_registry.py
All checks passed!

Negative case — verifying the test actually catches a stale key: temporarily added "langchain/nonexistent": "..." to templates/index.json and re-ran:

$ pytest tests/cli_commands/test_templates_registry.py -q
1 failed, 5 passed
>       assert not stale, f"stale templates/index.json key(s): {stale}"
E       AssertionError: stale templates/index.json key(s): ['langchain/nonexistent']

templates/index.json was restored afterwards; the working tree contains only the test change.

@popwatt

popwatt commented Aug 11, 2026

Copy link
Copy Markdown
Contributor Author

#92

1 similar comment
@popwatt

popwatt commented Aug 11, 2026

Copy link
Copy Markdown
Contributor Author

#92

@webup

webup commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Thanks for the patch. Could you please add more context to the currently blank Summary and Solution Description sections?

In particular, it would help to explain:

That context will make it easier to confirm that this is still the intended registry invariant.

@popwatt

popwatt commented Aug 12, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for the review! I have filled in the Summary and Solution Description sections of the PR description with the regression context (check originally from #100, dropped in #114), the relation to the #92/#100 work, and the exact test commands/results including the negative case (injected stale key langchain/nonexistent fails with AssertionError: stale templates/index.json key(s), then restored).

@webup
webup merged commit d0af630 into ob-labs:main Aug 12, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants