fix: sync e2e IAM policy and fix run eval flag

[jariy17]

Summary

• Syncs docs/policies/iam-policy-user.json fully with the live e2e-github-actions role — the doc was significantly out of date
• Adds missing permissions that were causing e2e test failures:
  • bedrock-agentcore:CreateConfigurationBundle + CRUD (fixes config-bundle-eval-rec.test.ts 403s)
  • iam:CreateRole/DeleteRole/GetRole/PutRolePolicy/DeleteRolePolicy scoped to AgentCore-* (fixes HTTP gateway role creation in ab-test-target-based.test.ts)
  • bedrock-agentcore:StartBatchEvaluation, StartRecommendation + related Get/List actions (fixes 403s in batch eval and recommendation tests)
• Fixes run eval CLI flag: test was passing --lookback but the correct flag is --days

Root cause

Several new post-deploy operations (config bundle sync, HTTP gateway IAM role creation, batch evaluation, recommendations) were added without updating the e2e role policy or the docs. The doc had also drifted significantly from the live role.

Test plan

• config-bundle-eval-rec.test.ts — deploy warnings and 403s should clear
• ab-test-target-based.test.ts — HTTP gateway creation should succeed
• run eval test step — --days flag resolves the unknown option error

🤖 Generated with Claude Code

[agentcore-cli-automation]

Thanks for adding these. I found two IAM actions that look missing for the HTTP gateway flow — without them the e2e test will still hit AccessDenied in the same ab-test-target-based path you're trying to fix.

1. Missing iam:TagRole

post-deploy-http-gateways.ts calls CreateRoleCommand with a Tags array (lines ~554–565):

new CreateRoleCommand({
  RoleName: roleName,
  AssumeRolePolicyDocument: trustPolicy,
  Description: ...,
  Tags: [
    { Key: 'agentcore:created-by', Value: 'agentcore-cli' },
    { Key: 'agentcore:project-name', Value: projectName },
    { Key: 'agentcore:http-gateway-name', Value: gatewayName },
  ],
})

Per IAM docs, attaching tags via CreateRole requires iam:TagRole on the caller in addition to iam:CreateRole. Without it, CreateRole will fail with AccessDenied even though iam:CreateRole is present.

This is also consistent with the docs/PERMISSIONS.md table, which already lists iam:TagRole as a required action for execution role creation.

2. Missing iam:PassRole

After creating the role, the code calls createHttpGateway(...) which POSTs to /gateways with roleArn in the body (src/cli/aws/agentcore-http-gateways.ts line 223). Passing a role ARN to an AWS service API requires iam:PassRole on the caller, scoped to that role.

This path is new — unlike runtime execution roles that are created by CloudFormation via the CDK execution role (which already has iam:PassRole in iam-policy-cfn-execution.json), HTTP gateway roles are now created/passed by the user credentials directly via SDK, so the user policy needs iam:PassRole too. docs/PERMISSIONS.md also lists this as a required action.

Suggested fix

Add both actions to the HttpGatewayIamRoleManagement statement (keeping the AgentCore-* resource scope):

{
  "Sid": "HttpGatewayIamRoleManagement",
  "Effect": "Allow",
  "Action": [
    "iam:CreateRole",
    "iam:DeleteRole",
    "iam:GetRole",
    "iam:PutRolePolicy",
    "iam:DeleteRolePolicy",
    "iam:TagRole",
    "iam:PassRole"
  ],
  "Resource": "arn:aws:iam::*:role/AgentCore-*"
}

Alternatively, if you prefer to avoid iam:TagRole, you could drop the Tags argument from the CreateRoleCommand call in post-deploy-http-gateways.ts (though losing the tags would hurt debuggability/attribution — I'd recommend the policy change instead).

You may want to manually test ab-test-target-based.test.ts end-to-end after applying the updated policy, since CreateRole-succeeds-without-tags is not obviously caught by CI unless the e2e role is updated first.

[github-actions]

Coverage Report

Status	Category	Percentage	Covered / Total
🔵	Lines	42.89%	8936 / 20830
🔵	Statements	42.17%	9485 / 22491
🔵	Functions	39.66%	1537 / 3875
🔵	Branches	39.82%	5769 / 14486

Generated in workflow #2331 for commit ba54415 by the Vitest Coverage Report Action