fix(s3): Garage/self-hosted S3 compatibility — path-style + SigV4, per-origin CORS rules#180
fix(s3): Garage/self-hosted S3 compatibility — path-style + SigV4, per-origin CORS rules#180nickgnat wants to merge 2 commits into
Conversation
…r-origin CORS rules Non-AWS mode (S3_STORAGE != s3) now forces path-style addressing and SigV4 on both the server-side and presign clients. boto3 could otherwise emit virtual-host-style URLs (broken behind reverse proxies without wildcard bucket DNS) and SigV2 presigned URLs, which Garage rejects. Startup bucket CORS is emitted as one rule per origin: Garage joins a rule's AllowedOrigins into a single comma-separated Access-Control-Allow-Origin header, which browsers reject — every HLS segment fetch and presigned upload then fails CORS. Per-origin rules behave identically on AWS-style backends. AWS mode is unchanged in both cases.
botocore's Config.merge lets the *argument* win, so _NON_AWS_COMPAT_CONFIG.merge(config) let a caller override the path-style and SigV4 settings the surrounding comment describes as forced — and replaced the `s3` sub-dict wholesale rather than deep-merging it. Merge onto the caller's config instead, so the compat baseline holds while non-conflicting caller options (the startup timeouts) still apply. Cover it with a test that passes a deliberately conflicting config; it fails on the previous merge direction. Also document the one-rule-per-origin requirement in the manual bucket CORS section, which still showed a single-rule example — the automatic startup path already emits per-origin rules.
ravirajsinh45
left a comment
There was a problem hiding this comment.
Approving — well-diagnosed and unusually well-prepared. Thanks @nickgnat.
I ran this locally against Postgres 15 + MinIO rather than reviewing it on paper, and the SigV4 half turns out to matter more than the PR claims. On the real multipart part-upload path against MinIO:
no Content-Type |
Content-Type: video/mp4 |
|
|---|---|---|
main (SigV2) |
200 | 403 SignatureDoesNotMatch |
| this PR (SigV4) | 200 | 200 |
SigV2's string-to-sign includes Content-Type and boto3 presigns it empty, so browser uploads to MinIO survive today only because upload-store.ts:192 calls file.slice(start, end) without a contentType argument — that returns a typeless Blob, so the browser omits the header. Anyone who "fixes" that line to preserve file.type would break every part upload on MinIO with an opaque 403. Your change removes that landmine for MinIO as well as Garage, so it's worth more than the Garage-only framing in the description.
Rest of what I verified: full API suite at 205 passed against a real Postgres (your 204 + one test I added); presigned URLs are SigV4 + path-style with a real PUT/GET round-trip against MinIO; every boto3.client site in the repo goes through the two constructors you patched; the CORS change is still a single put_bucket_cors call carrying N rules, and ensure_bucket_exists() stays non-fatal when the backend rejects it — this MinIO build answers PutBucketCors with NotImplemented, so that path no-ops with a warning there and your Garage testing is the only real coverage it has. The payload itself is correct: two rules, one origin each, ETag exposed on both.
I pushed one commit to your branch rather than sending you round again for two small things — shout if you'd rather I hadn't:
_NON_AWS_COMPAT_CONFIG.merge(config)gives the caller precedence, not the baseline — botocore'smergedoesconfig_options.update(other_config._user_provided_options), and thes3dict is replaced wholesale rather than deep-merged, so a caller passingsignature_versionor its owns3={...}could silently downgrade the settings the comment calls forced. No live bug, since_STARTUP_S3_CONFIGonly sets timeouts and retries. Flipped toconfig.merge(_NON_AWS_COMPAT_CONFIG)and added a test that passes a deliberately conflicting config — it fails on the old direction withassert 's3' == 's3v4'.- Added a line to the manual bucket-CORS section of
docs/deployment.md, which still showed a single-rule example — the same shape you just fixed in the automatic path.
One thing worth recording, not a change request: the addressing_style: "path" half is already botocore's default once endpoint_url is set (botocore/args.py:836 — force_path_style = s3_config.get('addressing_style') != 'virtual'). Still worth pinning, since an ~/.aws/config with addressing_style = virtual would flip it and an explicit Config wins over the shared config file — but SigV4 is doing the real work. I left your CHANGELOG wording alone since "could emit virtual-host-style URLs" stays true in that case.
Two adjacent things I found that are not yours to fix, noted so they're on record: AWS mode has the identical SigV2 presign behavior in us-east-1 (our default region), since it deliberately gets no config — same 403-on-Content-Type exposure, against a bucket policy that's stricter than MinIO's. And the startup bucket CORS ignores the CORS_ALLOW_ORIGINS setting from #173 while rewriting bucket CORS on every boot, so there's no manual workaround for an extra origin. We'll pick both up separately.
Summary
Makes self-hosted S3 backends — specifically Garage, but also MinIO-behind-a-proxy setups — work out of the box, fixing two issues found running FreeFrame against Garage v2.3.0 in production:
Received an unknown query parameter: 'AWSAccessKeyId'.AllowedOriginsinto one comma-separatedAccess-Control-Allow-Originheader, which browsers hard-reject — so every HLS segment fetch failed withHLS error: networkError, and browser-direct presigned uploads failed the same way.Changes
_build_s3_client()/_get_presign_client(): non-AWS mode (S3_STORAGE≠s3) now appliesConfig(s3={"addressing_style": "path"}, signature_version="s3v4"), merged with any caller-supplied config (the startup-timeout config keeps its timeouts). AWS mode is untouched — no endpoint, no forced config.ensure_bucket_exists(): startup bucket CORS emits one rule per origin (deduped). AWS-style backends echo only the matching origin either way, so behavior there is identical; on Garage this is the difference between a valid single-origin ACAO header and an invalid joined one.docs/deployment.md: Garage row in the provider endpoint table + a note that non-AWS mode is path-style + SigV4.test_s3_non_aws_compat.py): client config forced/merged in non-AWS mode, AWS mode untouched, per-origin CORS rules incl. dedupe.Testing
alembic upgrade head, mirroring CI)HeadBucket, and per-origin CORS echo (matching origin returned single-valued; unknown origins get no CORS grant)Checklist
CHANGELOG.mdentry added under## [Unreleased](for user-facing changes)Screenshots
n/a — backend only
🤖 Generated with Claude Code