Skip to content

fix(ecs,lambda): pull fakecloud's ECR registry over plain HTTP under podman - #2595

Merged
vieiralucas merged 1 commit into
mainfrom
fix/podman-local-ecr-plain-http
Sep 30, 2026
Merged

vieiralucas merged 1 commit into
mainfrom
fix/podman-local-ecr-plain-http

Conversation

@vieiralucas

@vieiralucas vieiralucas commented Sep 29, 2026 •

Copy link
Copy Markdown
Member

Summary

Closes #2585.

When an ECS task or Lambda function uses an AWS ECR image URI, fakecloud rewrites it to its own OCI registry at 127.0.0.1:<port> (or the sibling host), and that registry serves plain HTTP. Docker treats a loopback registry as insecure automatically. Podman doesn't: it requires TLS for every registry. So under the podman backend the pull failed with server gave HTTP response to HTTPS client.

  • container_image::pull_image now takes a RegistryTransport. A pull of a rewritten ECR reference is PlainHttp, and podman then gets --tls-verify=false. Upstream registries keep TLS verification. Docker's arguments don't change, since docker pull has no such flag.
  • The ECS task launch and the Lambda docker backend (both container start and prepull) pick the transport based on whether the URI was rewritten.
  • Podman already honors DOCKER_CONFIG, which I verified: an empty config gets authentication required and a filled one pulls. So the existing isolated registry auth keeps working, and TLS was the only thing missing.
  • The ECS docs now describe the podman behavior.

Surfaces checked: there's no new API, introspection endpoint, flag or count, so the SDKs, README, parity, counts and repo description don't change. Only the ECS service doc needed an update.

Test plan

  • Unit tests (fakecloud-core container_image): podman plus plain HTTP gets --tls-verify=false, podman plus HTTPS doesn't, docker plus plain HTTP doesn't, and the transport is derived from the rewrite.
  • New e2e ecs_podman_ecr: pushes an image to fakecloud ECR, runs an ECS task that references the AWS URI under FAKECLOUD_CONTAINER_CLI=podman, and checks the task's logs and exit code 0. It runs in the podman E2E partition, which already installs podman, and e2e_nextest_partitions check passes.
  • With the fix disabled, the e2e fails locally (real podman 6.0) with the exact error from the issue. With the fix, it passes.
  • Reproduced manually with the unpatched binary (task STOPPED, image pull failed ... HTTPS client). With the patched binary, the task reaches RUNNING and the container prints its output.
  • clippy (-D warnings) and fmt are clean.

Summary by cubic

Fixes ECS tasks and image-based Lambda functions failing to pull images from fakecloud's ECR registry under the podman backend with server gave HTTP response to HTTPS client (closes #2585).

  • Adds RegistryTransport to container_image::pull_image; pulls of rewritten ECR references use plain HTTP, so podman gets --tls-verify=false, while upstream registries keep TLS verification.
  • ECS task launch and the Lambda docker backend (container start and prepull) derive the transport from whether the URI was rewritten; Docker's arguments are unchanged.
  • Adds an ecs_podman_ecr e2e test that pushes to fakecloud ECR and runs an ECS task under podman, routed to the podman E2E partition.
  • Updates the ECS docs to note the podman behavior; no API, flag, or count surfaces change.

Written for commit 7940137. Summary will update on new commits.

Review in cubic

…podman

fakecloud rewrites an AWS ECR image URI to its own OCI registry at
127.0.0.1:<port>, which serves plain HTTP. Docker treats a loopback
registry as insecure on its own, but podman insists on TLS for every
registry, so ECS tasks and image-based Lambda functions failed to pull
with "server gave HTTP response to HTTPS client" (#2585).

- container_image::pull_image takes a RegistryTransport; a pull of a
  rewritten ECR reference is PlainHttp, and podman then gets
  --tls-verify=false. Upstream registries keep TLS; docker is unchanged
  (its pull has no such flag).
- ECS task launch and the Lambda docker backend (start + prepull) pass
  the transport from whether the URI was rewritten.
- New ecs_podman_ecr e2e pushes to fakecloud ECR and runs an ECS task
  under podman; routed to the podman E2E partition.
- ECS docs note the podman behavior.

Closes #2585
@vieiralucas
vieiralucas requested a balanced review from Copilot September 29, 2026 23:12

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@vieiralucas
vieiralucas merged commit ed71975 into main Sep 30, 2026
159 checks passed
@vieiralucas
vieiralucas deleted the fix/podman-local-ecr-plain-http branch September 30, 2026 04:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Podman-backed ECS tries to connect to 127.0.0.1 over HTTPS instead of HTTP

2 participants