Skip to content

test: phase 1 — add missing integration tests (validate, runtime-endpoint, import)#1118

Closed
jariy17 wants to merge 4 commits intomainfrom
test/phase1-integ-tests
Closed

test: phase 1 — add missing integration tests (validate, runtime-endpoint, import)#1118
jariy17 wants to merge 4 commits intomainfrom
test/phase1-integ-tests

Conversation

@jariy17
Copy link
Copy Markdown
Contributor

@jariy17 jariy17 commented May 4, 2026

What This PR Does

Closes the Phase 1 integration test gaps identified in the missing test coverage analysis.
All tests in this PR run locally — no AWS credentials required.

📋 Full missing test coverage analysis: https://gist.github.com/jariy17/a03bfb46ac9b32195aa3382919e3a913

Tests Added

integ-tests/validate.test.ts (extended — 4 new cases)

  • reports error for corrupted aws-targets.json
  • reports error for corrupted deployed-state.json
  • reports error for empty aws-targets.json
  • reports error for invalid schema in aws-targets.json

integ-tests/add-remove-runtime-endpoint.test.ts (new — 4 cases)

  • adds a runtime endpoint and verifies it appears in agentcore.json
  • rejects duplicate endpoint name
  • removes a runtime endpoint and verifies it is gone from agentcore.json
  • returns error for non-existent endpoint

integ-tests/import.test.ts (new — 2 cases)

  • returns error when source file does not exist
  • returns error for invalid YAML syntax

Test Results

194 passed | 0 skipped | 0 regressions

Phases

This PR is Phase 1 of a multi-phase plan to close missing test coverage across the CLI.

Phase Scope Status
Phase 1 (this PR) validate, runtime-endpoint, import error paths — local only, no AWS ✅ Done
Phase 2 package, update, fetch local state path — local only, no AWS 🔜 Next
Phase 3 E2E: deployed-state.json validation after deploy, logs filtering 🔜 Planned
Phase 4 E2E: fetch token generation, status all resource types, traces get 🔜 Planned
Phase 5 E2E: run batch-evaluation, run recommendation, run eval ARN mode 🔜 Planned
Phase 6 E2E: BYO auth types, import subcommands, multi-target deployments 🔜 Planned

Known Gaps Not In This PR

  • tagaction.ts exists but no command.ts and not registered in the CLI; agentcore tag is not a real command yet
  • runtime-endpoint dependency check — removing an endpoint referenced by an online-eval config is not validated in production code; needs a source change before it can be tested
  • import valid-YAML path — the happy path where valid YAML mutates agentcore.json was held back pending confirmation that Phase 1 is fully local with no physical resource IDs

Notes

  • deployed-state.json lives at agentcore/.cli/deployed-state.json per PathResolver.getStatePath; formatError labels it .cli/state.json in error output — tests assert on that label verbatim
  • agentcore import --source ... does not expose --json; import tests assert on combined stdout/stderr strings

@jariy17 jariy17 requested a review from a team May 4, 2026 22:35
@github-actions github-actions Bot added size/m PR size: M agentcore-harness-reviewing AgentCore Harness review in progress labels May 4, 2026
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 4, 2026

Coverage Report

Status Category Percentage Covered / Total
🔵 Lines 43.09% 9009 / 20907
🔵 Statements 42.36% 9564 / 22574
🔵 Functions 39.87% 1551 / 3890
🔵 Branches 39.95% 5804 / 14525
Generated in workflow #2402 for commit b7b79f9 by the Vitest Coverage Report Action

@agentcore-cli-automation
Copy link
Copy Markdown

Reviewed the changes end-to-end. Traced each assertion back to the production code paths:

  • RuntimeEndpointPrimitive.add/remove in src/cli/primitives/RuntimeEndpointPrimitive.ts emits the exact error strings the add-remove-runtime-endpoint tests match against, and the cliCommandRun path produces {success, error} JSON as expected.
  • handleImport in src/cli/commands/import/actions.ts aborts at the zero-agents check (line 159) before any AWS-touching code runs, so the import tests are safely AWS-free as documented.
  • validate/action.ts formatError does label the state file as .cli/state.json regardless of actual filename, so that assertion is correct.
  • tag/ is indeed missing command.ts and index.ts, so the skipped placeholder is accurate.

Tests-only change, no production behavior impact. LGTM to merge.

@github-actions github-actions Bot removed the agentcore-harness-reviewing AgentCore Harness review in progress label May 4, 2026
@github-actions github-actions Bot added size/m PR size: M and removed size/m PR size: M labels May 4, 2026
@jariy17 jariy17 changed the title test: add phase 1 missing integration tests test: phase 1 — add missing integration tests (validate, runtime-endpoint, import) May 4, 2026
@github-actions github-actions Bot added size/m PR size: M and removed size/m PR size: M labels May 5, 2026
@github-actions github-actions Bot added size/m PR size: M and removed size/m PR size: M labels May 5, 2026
@jariy17 jariy17 closed this May 5, 2026
@jariy17 jariy17 deleted the test/phase1-integ-tests branch May 5, 2026 18:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/m PR size: M

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants