Skip to content

feat(test): support multi-service validation subsets - #3707

Closed
tang-vu wants to merge 1 commit into
Osmantic:mainfrom
tang-vu:feat/ods-test-multi-service-selection
Closed

tang-vu wants to merge 1 commit into
Osmantic:mainfrom
tang-vu:feat/ods-test-multi-service-selection

Conversation

@tang-vu

@tang-vu tang-vu commented Sep 4, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • let ods-test.sh --service/-s accept comma-separated service lists
  • allow the option to be repeated while preserving first-request order and deduplicating suites
  • validate the complete selection before running tests and report missing/empty entries as configuration errors
  • continue through the selected subset so the final summary remains a useful validation receipt

Why this matters

Operators and CI jobs often need a focused cross-service check (for example LLM + Whisper + TTS) without paying for the entire validation suite or launching three separate processes. The previous single-service contract made that workflow awkward and produced fragmented summaries.

The behavioral invariant is: a requested service suite runs at most once, in request order, and malformed input never falls through to the all-services path.

Overlap check

Searched open and closed upstream PR titles/bodies for ods-test multiple services service selection, reviewed the latest 1,000 open PRs and their changed production files, and inspected history for ods/scripts/ods-test.sh. No matching PR or currently open production-file overlap was found; upstream history for this script only contains the repository rename. PR #2651 mentions functional-test JSON output but changes a different script and behavior.

Test plan

  • bats ods-test-service-selection.bats ? 4/4 passing at the public CLI boundary
    • comma-separated and repeated selections run once in order
    • unselected suites do not run
    • unknown, missing, and empty service entries exit 2 before service dispatch
  • bash ods/tests/test-ods-test.sh ? 6/6 error-handling contract checks passing
  • bash -n ods/scripts/ods-test.sh
  • ShellCheck: no new findings (the file retains three pre-existing SC2034 unused-variable warnings)
  • git diff --check

Design and operational notes

  • Existing no-argument and single-service invocations are unchanged.
  • Deduplication uses an indexed array rather than associative arrays, retaining compatibility with older Bash environments.
  • A failing selected suite is recorded and the remaining requested suites still execute; the aggregate failure counter continues to determine exit status.
  • Rollback only restores the one-service parser and has no persisted-state impact.

Generated with Codex.

Batch compatibility

@Lightheartdevs

Copy link
Copy Markdown
Collaborator

Thanks for this contribution. We're closing outside-contributor pull requests that have had no activity for 14 days or more, so the queue reflects work that can still land. This isn't a judgment on the change itself. If it's still needed, please rebase onto main and reopen it or open a focused PR. See #7253 for details and the contribution policy.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants