Add optional submodules input to reusable CI (public submodules)

[kazmosahebi]

Problem

Consumer repos whose tests read content from a submodule fail in the reusable CI because the workflow checkouts (python-ci.yml and the other *-ci.yml) fetch no submodules. Example: dfe-engine resolves ClickHouse schema profiles from its schemas submodule (schemas/common-header/*.yaml); when the submodule is absent it falls back to stale bundled profiles in src/, so test_schema_builder_v2.py::...test_source_specific_overrides_in_views fails in CI (Version '1.1.0' not found ... Available versions: 1.0.0) while passing locally.

Current guidance

docs/LESSONS.md advises, for private submodules (e.g. dfe-schemas):

mark dependent tests with @pytest.mark.skipif(not schemas_dir.exists(), ...) rather than trying to check out private submodules in CI (GITHUB_TOKEN can't)

That stands for private submodules. But it leaves no supported path for a repo whose submodule is (or becomes) public and which wants those tests to actually run in CI.

Proposal

Add an optional submodules input to the reusable workflow(s):

submodules:
  type: string
  default: ""   # no-op for every current consumer

When set (e.g. submodules: schemas), the test job runs git submodule update --init --depth 1 <paths> after checkout (value passed via env to avoid script injection). Default empty -> nothing changes for repos that don't set it.

• Public submodule: works with the default GITHUB_TOKEN, no secret needed.
• Private submodule: still needs skipif (per LESSONS) or a configured cross-repo token -- out of scope here.

Docs to update (part of the fix)

• docs/LESSONS.md: reconcile -- skipif for private submodules; the new submodules input for public ones.

Scope / open questions

• Start with python-ci.yml (the dfe-engine case). Extend to rust-ci.yml / ts-ci.yml / go-ci.yml for parity, or add on demand?
• Only the test job consumes submodule content today (quality lints src/, build packages src/). Keep it test-job-only, or apply to all tree-using jobs?

A draft branch implementing the python-ci.yml input + the test-job init step is ready.

[kazmosahebi]

The python merge has been completed for this (to unblock dfe-engine) but keeping open for potential rust/typescript/go implementations if required
@catinspace-au

[kazmosahebi]

Follow-up: PR #40 added the submodules input to python-ci.yml, but the scaffold (hyperi-ci init -> _render_workflow) doesn't emit it -- so the generated consumer ci.yml never carries submodules:, and a hand-added line gets clobbered on the next init --force. That's only half the feature.

Second PR adds the scaffold half: a submodules: field in .hyperi-ci.yaml is read by init and rendered as the submodules input in the consumer ci.yml with: block (mirroring the existing publish-target conditional), and round-tripped back into .hyperi-ci.yaml so re-init preserves it. The field is ignored by the runtime config loader (CIConfig is .get()-based, no strict validation), so it's init-only.

Example -- .hyperi-ci.yaml:

submodules: schemas

renders into ci.yml:

jobs:
  ci:
    uses: hyperi-io/hyperi-ci/.github/workflows/python-ci.yml@main
    with:
      tag: ${{ inputs.tag || '' }}
      from-head: ${{ inputs.from-head || '' }}
      bump: ${{ inputs.bump || 'auto' }}
      submodules: schemas
    secrets: inherit