Skip to content

test(bdd): invoke an LLM function in the single-cluster helmfile feature - #1055

Open
along-2017 wants to merge 2 commits into
mainfrom
test/bdd/llm-function-single-cluster
Open

test(bdd): invoke an LLM function in the single-cluster helmfile feature#1055
along-2017 wants to merge 2 commits into
mainfrom
test/bdd/llm-function-single-cluster

Conversation

@along-2017

@along-2017 along-2017 commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Why

The single-cluster Helmfile feature installs the LLM gateway and
router but never routes a request through them. A regression that
leaves the releases deployed while the serve path is broken passes
the suite. This adds the first LLM-type function coverage.

What changed

  • tests/bdd/features/single-cluster-helmfile.feature: new
    @llm-function-type scenario after the echo function lifecycle. It
    creates a function of type LLM backed by the OpenAI-compatible
    sample image, deploys it, invokes a chat completion through the LLM
    gateway asserting the response envelope, and asserts an
    unauthenticated gateway request returns 401. Existing DSL steps
    only.
  • tests/bdd/godog_test.go: wiring test seeds canned results for the
    LLM invoke and the curl check, and asserts the LLM create, invoke,
    and no-auth commands ran.

Customer Release Notes

Not customer visible.

Plan Summary

Not applicable

Usage

Not applicable

Testing

  • tests/bdd: go test -short ./... green (wiring test executes the new
    scenario against the fake runner); go vet green.
  • Live TestSingleClusterHelmfile not yet run: the scenario references
    nvcf-openai-compatible-sample:1.0.0 under the sample org/team, which
    must be published as a multi-arch tag first. The live run happens
    once the image exists; the invoke content assertion is tightened
    then.

Notes

The image tag 1.0.0 is a placeholder for the first published
multi-arch tag; update the scenario if the published tag differs.

References

Relates to #1019

Related Pull Requests

None

Dependencies

None

Summary by CodeRabbit

  • Tests
    • Added end-to-end coverage for the LLM function lifecycle, including creation, deployment, API-key generation, and synchronous OpenAI-compatible inference.
    • Added response validation to confirm successful invocation behavior.
    • Added coverage verifying that unauthenticated gateway requests are rejected with HTTP 401.
    • Expanded deployment wiring checks for model and runtime configuration.

Add an @llm-function-type scenario after the echo function lifecycle:
create a function of type LLM backed by the OpenAI-compatible sample
image, deploy it, invoke a chat completion through the LLM gateway,
and assert an unauthenticated gateway request returns 401. The wiring
test seeds canned invoke and curl results and asserts the LLM create,
invoke, and no-auth commands ran.

Relates to #1019

Signed-off-by: along <along@nvidia.com>
Strengthen the invoke assertion with a choices check, assert the curl
exit code, correct the comment describing the LLM Host-header routing,
and document why the gateway invoke has no poll duration.

Relates to #1019

Signed-off-by: along <along@nvidia.com>
@along-2017
along-2017 requested a review from a team as a code owner August 21, 2026 02:39
@coderabbitai

coderabbitai Bot commented Aug 21, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 027bf123-173d-462e-b6a9-ffdabc18cf37

📥 Commits

Reviewing files that changed from the base of the PR and between 171757e and da58706.

📒 Files selected for processing (2)
  • tests/bdd/features/single-cluster-helmfile.feature
  • tests/bdd/godog_test.go

Included review availability: Your plan provides up to 12 included reviews per hour; 11 remain after this review.


📝 Walkthrough

Walkthrough

The BDD tests add an OpenAI-compatible LLM function lifecycle. They verify deployment configuration, API-key generation, synchronous invocation, response validation, and HTTP 401 rejection for unauthenticated gateway requests.

Changes

LLM gateway testing

Layer / File(s) Summary
LLM function lifecycle and authentication
tests/bdd/features/single-cluster-helmfile.feature, tests/bdd/godog_test.go
The scenario creates and deploys an LLM function with the expected image, function type, and model. Fake-runner responses support authenticated chat-completions invocation and unauthenticated HTTP 401 validation.

Estimated code review effort: 2 (Simple) | ~10 minutes

Merge Risk: 🔵 Low · up to da587

The PR adds coverage for LLM invocation and gateway authentication without changing product runtime code. The live scenario depends on publishing and pulling the sample image first, so it is mergeable with explicit owner follow-up to verify that dependency before live execution.

Sequence Diagram(s)

sequenceDiagram
  participant BDDScenario
  participant FakeRunner
  participant LLMGateway
  BDDScenario->>FakeRunner: create and deploy LLM function
  BDDScenario->>LLMGateway: invoke chat-completions endpoint with API key
  LLMGateway-->>BDDScenario: synchronous OpenAI-compatible response
  BDDScenario->>LLMGateway: invoke gateway without API key
  LLMGateway-->>BDDScenario: HTTP 401
Loading

Suggested reviewers: nvaghela-oss

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title follows Conventional Commits with the test(bdd) type and accurately describes the added BDD coverage for invoking an LLM function.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 1 functions across 1 files. (1 skipped: 1 unsupported.)
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch test/bdd/llm-function-single-cluster

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant