Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions .github/workflows/index.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ on:
- 'schema/**'
- 'scripts/build_comparison_search.py'
- 'scripts/build_comparisons.py'
- 'scripts/build_depth_discovery.py'
- 'scripts/build_index.py'
- 'scripts/catalog_utils.py'
- 'scripts/coverage_metadata.py'
Expand All @@ -39,6 +40,7 @@ on:
- 'schema/**'
- 'scripts/build_comparison_search.py'
- 'scripts/build_comparisons.py'
- 'scripts/build_depth_discovery.py'
- 'scripts/build_index.py'
- 'scripts/catalog_utils.py'
- 'scripts/coverage_metadata.py'
Expand Down Expand Up @@ -103,6 +105,13 @@ jobs:
--output-dir dist/index \
--comparison-index dist/comparisons/module-comparisons.json

- name: Build reviewed depth discovery artifacts
run: |
python scripts/build_depth_discovery.py \
--catalog-dir catalog \
--maturity-manifest quality/module-maturity.yaml \
--output-dir dist/depth

- name: Validate comparison discovery graph
run: |
python scripts/validate_comparison_discovery.py \
Expand Down Expand Up @@ -141,10 +150,12 @@ jobs:
comparison_search = json.loads(Path('dist/comparisons/search.json').read_text(encoding='utf-8'))
module_comparisons = json.loads(Path('dist/comparisons/module-comparisons.json').read_text(encoding='utf-8'))
discovery = json.loads(Path('dist/comparisons/discovery-validation.json').read_text(encoding='utf-8'))
depth = json.loads(Path('dist/depth/depth.json').read_text(encoding='utf-8'))
coverage = json.loads(Path('dist/coverage/coverage.json').read_text(encoding='utf-8'))
print('# OpenDevIndex Search Index')
print()
print(f"Generated **{data['module_count']}** validated modules.")
print(f"Reviewed depth discovery: **{depth['maturity_counts'].get('deep-dive', 0)}** deep dive(s), **{depth['maturity_counts'].get('guide', 0)}** guide(s).")
print(f"Generated **{comparisons['comparison_count']}** curated comparison view(s).")
print(f"Searchable comparison records: **{comparison_search['comparison_count']}**.")
print(f"Comparison-linked modules: **{module_comparisons['module_count']}**.")
Expand All @@ -162,6 +173,8 @@ jobs:
for kind, count in data['kind_counts'].items():
print(f'| `{kind}` | {count} |')
print()
print(Path('dist/depth/depth.md').read_text(encoding='utf-8'))
print()
print(Path('dist/coverage/coverage.md').read_text(encoding='utf-8'))
print()
print(Path('dist/quality/quality-report.md').read_text(encoding='utf-8'))
Expand All @@ -174,6 +187,7 @@ jobs:
path: |
dist/index
dist/comparisons
dist/depth
dist/coverage
dist/quality
if-no-files-found: error
Expand Down
17 changes: 13 additions & 4 deletions .github/workflows/public-index.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ on:
- 'quality/module-maturity.yaml'
- 'taxonomy/**'
- 'scripts/build_comparisons.py'
- 'scripts/build_depth_discovery.py'
- 'scripts/build_index.py'
- 'scripts/catalog_utils.py'
- 'scripts/module_maturity.py'
Expand Down Expand Up @@ -53,6 +54,14 @@ jobs:
--output-dir dist/comparisons \
--public-dir docs/comparisons

- name: Generate reviewed depth view
run: |
python scripts/build_depth_discovery.py \
--catalog-dir catalog \
--maturity-manifest quality/module-maturity.yaml \
--output-dir dist/depth \
--public-file docs/depth.md

- name: Generate comparison-aware public module index
run: |
python scripts/build_index.py \
Expand All @@ -63,13 +72,13 @@ jobs:

- name: Publish only when discovery views changed
run: |
if [ -z "$(git status --porcelain -- INDEX.md docs/comparisons)" ]; then
echo 'Public index and comparison views are already current.' >> "$GITHUB_STEP_SUMMARY"
if [ -z "$(git status --porcelain -- INDEX.md docs/comparisons docs/depth.md)" ]; then
echo 'Public index, comparison views, and reviewed depth view are already current.' >> "$GITHUB_STEP_SUMMARY"
exit 0
fi
git config user.name "OpenDevIndex Publisher"
git config user.email "207100624+BLCCoreStudio@users.noreply.github.com"
git add INDEX.md docs/comparisons
git add INDEX.md docs/comparisons docs/depth.md
git commit -m "docs: refresh public discovery views [skip ci]"
git push origin HEAD:main
echo 'Published updated INDEX.md and comparison views.' >> "$GITHUB_STEP_SUMMARY"
echo 'Published updated INDEX.md, comparison views, and reviewed depth view.' >> "$GITHUB_STEP_SUMMARY"
7 changes: 5 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@

<p align="center">
<a href="INDEX.md">Browse the Index</a> ·
<a href="docs/depth.md">Browse Reviewed Depth</a> ·
<a href="docs/COMPARISONS.md">Compare Technologies</a> ·
<a href="docs/COVERAGE.md">Explore Technology Areas</a> ·
<a href="docs/MODULE_STANDARD.md">Module Standard</a> ·
Expand All @@ -31,7 +32,7 @@ A module can grow from a concise source-backed overview into a full technical de

## Start exploring

The generated [`INDEX.md`](INDEX.md) is the main directory. It groups modules by what they are and shows the domains they belong to.
The generated [`INDEX.md`](INDEX.md) is the main directory. It groups modules by what they are and shows the domains they belong to. The generated [`docs/depth.md`](docs/depth.md) view focuses specifically on reviewed guide and deep-dive modules, with kind and domain facets for faster depth-oriented browsing.

A few examples:

Expand Down Expand Up @@ -180,11 +181,12 @@ OpenDevIndex generates machine-readable search and comparison artifacts alongsid

```bash
python scripts/build_index.py --catalog-dir catalog --output-dir dist/index --public-index INDEX.md
python scripts/build_depth_discovery.py --catalog-dir catalog --maturity-manifest quality/module-maturity.yaml --output-dir dist/depth
python scripts/search_index.py "local ai" --index dist/index/search.json
python scripts/build_comparisons.py --comparisons-dir comparisons --catalog-dir catalog --output-dir dist/comparisons
```

Search can use taxonomy-aware fields including maturity, kind, domains, tags, deployment type, licensing metadata, and Technology Universe coverage facets. Comparison artifacts expose reviewed dimensions, module metadata, and requirement-oriented decision guidance for downstream interfaces.
Search can use taxonomy-aware fields including maturity, kind, domains, tags, deployment type, licensing metadata, and Technology Universe coverage facets. Reviewed-depth artifacts provide a focused machine-readable and Markdown view of guide/deep-dive modules without redefining maturity. Comparison artifacts expose reviewed dimensions, module metadata, and requirement-oriented decision guidance for downstream interfaces.

The structured model is intended to work for:

Expand Down Expand Up @@ -262,6 +264,7 @@ See [`CONTRIBUTING.md`](CONTRIBUTING.md) and [`docs/EDITORIAL_POLICY.md`](docs/E
## Documentation

- [`docs/MODULE_STANDARD.md`](docs/MODULE_STANDARD.md) — module depth and editorial expectations
- [`docs/DEPTH_DISCOVERY.md`](docs/DEPTH_DISCOVERY.md) — reviewed guide/deep-dive discovery generation and source-of-truth rules
- [`docs/COMPARISONS.md`](docs/COMPARISONS.md) — curated comparison manifests, validation, and publication model
- [`docs/COVERAGE.md`](docs/COVERAGE.md) — technology-area coverage model
- [`docs/TAXONOMY.md`](docs/TAXONOMY.md) — stable addresses, kinds, domains, and relationships
Expand Down
2 changes: 2 additions & 0 deletions comparisons/_meta/search-smoke.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
schema_version: 1
cases:
- comparison: containers-docker-containerd
query: image build responsibility
- comparison: github-actions-argocd
query: runner jobs drift reconciliation
- comparison: kubernetes-helm-argocd
Expand Down
51 changes: 51 additions & 0 deletions docs/DEPTH_DISCOVERY.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
# Reviewed depth discovery

OpenDevIndex keeps module coverage separate from editorial depth. The reviewed-depth discovery view exposes only modules that have been explicitly classified as `guide` or `deep-dive` in `quality/module-maturity.yaml`; overview modules remain valid catalog entries and are not treated as lower quality.

## Source of truth

The generator does not infer maturity from page length, schema version, branch history, or catalog wording. It joins validated catalog records with the existing maturity manifest and uses that reviewed metadata as the only maturity source of truth.

This keeps discovery output derived and reproducible instead of introducing a second hand-maintained list of mature modules.

## Build locally

```bash
python scripts/build_depth_discovery.py \
--catalog-dir catalog \
--maturity-manifest quality/module-maturity.yaml \
--output-dir dist/depth
```

The command writes:

- `dist/depth/depth.json` — machine-readable reviewed-depth records and maturity/kind/domain counts;
- `dist/depth/depth.md` — deterministic human-readable discovery view.

To render the same Markdown to a public path, pass `--public-file`:

```bash
python scripts/build_depth_discovery.py \
--catalog-dir catalog \
--maturity-manifest quality/module-maturity.yaml \
--output-dir dist/depth \
--public-file docs/depth.md
```

## Discovery fields

Each reviewed record includes the stable module ref, display name, kind, domains, maturity, review date, summary, and canonical module URL. The Markdown view groups modules by maturity and also exposes kind/domain facets with direct module links.

Output ordering is deterministic so identical validated inputs produce identical files.

## CI and publication

The **Build Search Index** workflow builds the reviewed-depth artifacts, reports guide/deep-dive counts in the Actions summary, and includes `dist/depth` in the discovery artifact bundle.

The trusted **Publish Public Index** workflow renders `docs/depth.md` from the same generator and only commits public discovery output when validated generated views actually change.

Generated public Markdown is therefore a projection of reviewed catalog and maturity data. It must not become an independent editorial source of truth.

## Editorial boundary

A module enters this view only after its maturity metadata is deliberately reviewed. Adding a module to the catalog, increasing its word count, or changing generated discovery files must never promote maturity by itself.
48 changes: 38 additions & 10 deletions docs/SEARCH.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,26 @@ The index builder also joins `quality/module-maturity.yaml`, so generated record

The generator intentionally avoids timestamps in generated artifacts so the same validated inputs produce stable output.

## Build reviewed-depth discovery

To generate the focused guide/deep-dive discovery view from the same reviewed maturity source of truth:

```bash
python scripts/build_depth_discovery.py \
--catalog-dir catalog \
--maturity-manifest quality/module-maturity.yaml \
--output-dir dist/depth
```

Generated files:

- `dist/depth/depth.json` — machine-readable reviewed guide/deep-dive records plus deterministic maturity, kind, and domain facets;
- `dist/depth/depth.md` — human-readable reviewed-depth view with direct module navigation.

The trusted public publisher uses the same renderer to produce `docs/depth.md`. Overview modules are deliberately excluded from this focused view; that is a discovery filter, not a lower-quality classification. Maturity remains owned by `quality/module-maturity.yaml`, and generated depth files are never an editorial source of truth.

See [`DEPTH_DISCOVERY.md`](DEPTH_DISCOVERY.md) for the generation contract and source-of-truth boundary.

## Search modules locally

```bash
Expand Down Expand Up @@ -185,7 +205,7 @@ Successful validation writes `dist/comparisons/discovery-validation.json` with c

## Public discovery build

The public publisher renders comparison views first, then the comparison-aware module index:
The public publisher renders comparison views first, then the comparison-aware module index and reviewed-depth view:

```bash
python scripts/build_comparisons.py \
Expand All @@ -199,11 +219,17 @@ python scripts/build_index.py \
--output-dir dist/index \
--public-index INDEX.md \
--comparison-index dist/comparisons/module-comparisons.json

python scripts/build_depth_discovery.py \
--catalog-dir catalog \
--maturity-manifest quality/module-maturity.yaml \
--output-dir dist/depth \
--public-markdown docs/depth.md
```

Compact comparison search and discovery-validation JSON are CI/downstream artifacts rather than committed public Markdown. Public comparison pages remain generated under `docs/comparisons/` from the same reviewed manifests.
Compact comparison search and discovery-validation JSON are CI/downstream artifacts rather than committed public Markdown. Public comparison pages remain generated under `docs/comparisons/`, while `docs/depth.md` is generated from the reviewed maturity manifest and catalog through the same deterministic depth renderer used by CI.

See [`COMPARISONS.md`](COMPARISONS.md) for the editorial, bidirectional-discovery, semantic-search, and validation model.
See [`COMPARISONS.md`](COMPARISONS.md) for the editorial, bidirectional-discovery, semantic-search, and validation model, and [`DEPTH_DISCOVERY.md`](DEPTH_DISCOVERY.md) for the reviewed-depth publication contract.

## Coverage progress

Expand All @@ -223,22 +249,24 @@ The report compares real mapped modules with the 10,000-module Technology Univer

The **Build Search Index** workflow:

1. runs unit tests for module search, curated comparisons, reverse indexes, joins/filters, comparison search, and discovery validation;
1. runs unit tests for module search, curated comparisons, reverse indexes, joins/filters, comparison search, discovery validation, and reviewed-depth generation;
2. builds full comparison records and the compact comparison-search artifact;
3. builds module catalog/search artifacts with comparison backlinks joined in;
4. runs the generic comparison discovery validator against all generated graph surfaces;
5. executes every semantic rank-1 query registered in `comparisons/_meta/search-smoke.yaml`;
6. builds coverage-progress artifacts and audits editorial quality;
7. smoke-tests representative non-comparison module facets;
8. uploads module, comparison, validation, quality, and coverage discovery output as workflow artifacts.
6. builds the reviewed-depth JSON/Markdown artifacts from the reviewed maturity manifest;
7. builds coverage-progress artifacts and audits editorial quality;
8. smoke-tests representative non-comparison module facets;
9. uploads module, comparison, depth, validation, quality, and coverage discovery output as workflow artifacts.

The **Publish Public Index** workflow rebuilds `INDEX.md` plus generated `docs/comparisons/` views when their validated inputs change.
The **Publish Public Index** workflow rebuilds `INDEX.md`, generated `docs/comparisons/` views, and `docs/depth.md` when their validated inputs change.

OpenDevIndex therefore exposes two complementary deterministic search surfaces:
OpenDevIndex therefore exposes three complementary deterministic discovery surfaces:

```text
module search -> technologies + comparison backlinks
comparison search -> curated trade-off views + participating technologies
reviewed depth -> reviewed guides/deep dives + kind/domain facets
```

A separate end-to-end validator ensures those surfaces remain one consistent graph rather than drifting independent indexes.
Separate validation and generation contracts keep those surfaces consistent with their reviewed source data rather than allowing generated discovery files to become independent sources of truth.
Loading
Loading