Skip to content

fix(ecs): vend task-role sessions from the ECS credentials endpoint - #2621

Merged
vieiralucas merged 5 commits into
mainfrom
fix/issue-2601
Sep 30, 2026
Merged

vieiralucas merged 5 commits into
mainfrom
fix/issue-2601

Conversation

@vieiralucas

@vieiralucas vieiralucas commented Sep 30, 2026 •

Copy link
Copy Markdown
Member

Summary

Closes #2601.

  • GET /_fakecloud/ecs/creds/{task_id} (new fakecloud-server ecs_creds module) returns an STS session for the task's taskRoleArn, session name = task id (assumed-role/<role>/<task-id>), minted via the same mint_session_credentials path as Lambda (feat(lambda): give function containers the AWS environment #2562), so it verifies under --verify-sigv4 and acts as the role under --iam.
  • Cached per task (new WorkloadCredentialCache, shared rotation logic) and re-minted near expiry; revoked when the task stops (1s sweep, task lookup held through the mint so a stopping task can't get a fresh session); sessions left over from before a restart are revoked at startup.
  • Stopped / unknown / role-less task -> the ECS agent's exact response: HTTP 400 {"code":"InvalidIdInRequest","message":"CredentialsV2Request: Credentials not found","HTTPErrorCode":400}; empty id -> NoIdInRequest. The fallback to a made-up default role is gone.
  • Bare role name as taskRoleArn resolves in the task's account. RunTask overrides.taskRoleArn / executionRoleArn now actually apply (they were validated but ignored).
  • fakecloud_ecs::validate_task_role shared by RegisterTaskDefinition, RunTask overrides and CFN AWS::ECS::TaskDefinition: trust policy must name ecs-tasks.amazonaws.com (looked up in the role's owning account); cross-account role refused under --iam strict, logged under soft; error is ECS's ClientException ("ECS was unable to assume the role '...'").
  • --imds-link-local's 169.254.170.2 listener also serves the agent path /v2/credentials/<task-id>.

Known pre-existing limitation (documented, not changed here): the injected AWS_CONTAINER_CREDENTIALS_FULL_URI host is host.docker.internal (or the in-cluster URL), which AWS SDKs reject for plain HTTP (only loopback / 169.254.170.2 allowed). HTTP clients get the credentials; SDK default chains inside the container don't. docs/services/ecs.md no longer claims it works out of the box.

Surfaces: docs/services/ecs.md, docs/reference/introspection.md, docs/guides/instance-credentials.md, docs/reference/configuration.md, --imds-link-local help text, AGENTS.md service notes, doc comments in all seven SDKs (no code change).

Test plan

  • Unit: ECS / IAM / server / CloudFormation crates (129/526/133/565) incl. 11 server tests for the endpoint and link-local route.
  • New e2e ecs_task_role_credentials.rs (3): running task under --verify-sigv4 + --iam strict -> GetCallerIdentity = assumed-role/app-task-role/<task-id>, allowed SQS call works, disallowed denied, container fetched creds itself (wget), after StopTask endpoint 400 and old keys stop authenticating; role-less and unknown task -> agent error; RegisterTaskDefinition rejects untrusted and cross-account roles.
  • Updated ecs_task_role_secrets.rs and iam_pass_role.rs for the new behavior; ecs, ecs_enforce, cloudformation_ecs, cloudformation_ecs_backing, ecs_metadata_endpoint, instance_credentials, imds e2e pass.
  • fmt + workspace clippy clean; local /code-review medium x5, last clean (fixed: sweep race, cross-account trust lookup, SDK docs, post-restart sessions, stop/mint race, bare role account).

Summary by cubic

GET /_fakecloud/ecs/creds/{task_id} now vends a real STS session for the task's taskRoleArn (session name = task ID) instead of static credentials, minted and registered like an AssumeRole session so it verifies under --verify-sigv4 and acts as the role under --iam. Closes #2601.

New Features

  • Sessions are cached per task, rotated near expiry, and revoked once the task stops; stopped, unknown, or role-less tasks get the ECS agent's exact HTTP 400 InvalidIdInRequest response, and the fallback to a made-up default role is gone.
  • RunTask overrides.taskRoleArn/executionRoleArn now apply, and bare role names resolve in the task's account.
  • RegisterTaskDefinition, RunTask overrides, and CloudFormation AWS::ECS::TaskDefinition share role validation: the trust policy must name ecs-tasks.amazonaws.com, cross-account roles are refused under --iam strict, both with ECS's ClientException.
  • --imds-link-local's 169.254.170.2 listener also serves the agent path /v2/credentials/<task-id>.
  • Sessions left registered by a previous restart are revoked at startup.

Migration

  • The injected AWS_CONTAINER_CREDENTIALS_FULL_URI host (host.docker.internal or the in-cluster URL) is still rejected by AWS SDKs for plain HTTP; HTTP clients get credentials, SDK default chains in containers don't, and the docs no longer claim it works out of the box.

Written for commit 749baf3. Summary will update on new commits.

Review in cubic

GET /_fakecloud/ecs/creds/{task_id} returned static placeholder
credentials. It now returns a session for the task's taskRoleArn, named
after the task ID (assumed-role/<role>/<task-id>), minted and registered
like an AssumeRole session so it verifies under --verify-sigv4 and is
evaluated as the role under --iam. Sets are cached per task until near
expiry, then rotated, and revoked once the task stops. Stopped, unknown
and role-less tasks get the ECS agent's HTTP 400 InvalidIdInRequest.

RunTask overrides.taskRoleArn/executionRoleArn now replace the task
definition's roles for the task. RegisterTaskDefinition, RunTask role
overrides and CloudFormation AWS::ECS::TaskDefinition share
validate_task_role: the trust policy must name ecs-tasks.amazonaws.com,
and under --iam strict another account's role is refused, both with
ECS's ClientException message.

The --imds-link-local listener on 169.254.170.2 also serves task
credentials at the agent's /v2/credentials/<task-id>.

Closes #2601
- Hold the ECS read lock through the revocation sweep so a task started
  mid-sweep is not revoked against a stale snapshot.
- Check another account's task role against its own trust policy.
- Document the endpoint's new behavior in every SDK.
With persistence on, IAM state keeps the sessions handed to tasks while
the per-task cache that revokes them starts empty, and the restart stops
every restored task. Revoke the sessions of stopped tasks on startup.
A task stopping (and being swept) between the running check and the
mint could still be handed a fresh session until the next sweep.
ECS accepts a bare role name for taskRoleArn; its session was minted
(and looked for after a restart) in the server's default account rather
than the task's. Resolve it to the full role ARN in the task's account
and partition, which is also what the endpoint now reports as RoleArn.
@vieiralucas
vieiralucas merged commit 0737f40 into main Sep 30, 2026
159 checks passed
@vieiralucas
vieiralucas deleted the fix/issue-2601 branch September 30, 2026 23:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ECS credentials endpoint returns static credentials instead of a task-role session

1 participant