Skip to content

Support uploading package attestations during publish --from-archive - #4876

Open
mosuem wants to merge 10 commits into
mainfrom
publish-with-attestation
Open

mosuem wants to merge 10 commits into
mainfrom
publish-with-attestation

Conversation

@mosuem

@mosuem mosuem commented Aug 19, 2026 •

Copy link
Copy Markdown
Member

Adds client support for publishing package attestations during pub publish:

  • Adds --with-attestation <bundle.sigstore.json> flag to dart pub publish --from-archive.
  • Enforces that a repository is specified in pubspec.yaml (repository field) when publishing with an attestation.
  • Validates that --with-attestation can only be used with --from-archive, and that the file is non-empty and contains a JSON object, before asking the user to confirm the publication.
  • Adds unit tests in test/lish/publishing_to_and_from_archive_test.dart.

Protocol

doc/repository-spec-v2.md gets a new Publishing with an Attestation section:

  • A repository that supports publishing with attestations returns an <attestation-upload-url> in the attestationUrl property of the response from <hosted-url>/api/packages/versions/new. A repository that does not support it simply omits the property.
  • The client POSTs the bundle as application/json to <attestation-upload-url> before uploading the package archive, attaching Authorization under the same prefix rule as archive_url.
  • <finalize-upload-url> remains a plain GET.

attestationUrl is an opaque URL: a repository may point it at its own endpoint or at a signed URL on a storage service. On pub.dev it is an endpoint on the app server, so no second signed upload policy is needed (see dart-lang/pub-dev#9545).

The alternative considered was to POST the attestation to <finalize-upload-url> and use 405 Method Not Allowed for feature detection. A separate URL was chosen because:

  • the client learns that a repository cannot take attestations before uploading the archive, instead of after; a missing JSON property is also a more reliable signal than 405, which proxies and frameworks do not produce consistently, and
  • the spec keeps one method per URL. This matters here because the archive upload is allowed to answer with 303 See Other pointing at <finalize-upload-url>, and a 303 is followed with a GET — any client that simply follows the redirect would drop the attestation body.

Server side: dart-lang/pub-dev#9545

@mosuem
mosuem force-pushed the publish-with-attestation branch 2 times, most recently from 4c73e6a to 591d116 Compare August 24, 2026 12:22
@mosuem
mosuem changed the base branch from master to reproducible-package-archives August 24, 2026 12:22
@mosuem
mosuem force-pushed the publish-with-attestation branch from 591d116 to f780f40 Compare August 24, 2026 12:40
@mosuem
mosuem force-pushed the publish-with-attestation branch from f780f40 to 4b0da87 Compare August 24, 2026 12:49
@mosuem
mosuem force-pushed the publish-with-attestation branch from 4b0da87 to f013f26 Compare August 24, 2026 12:59
@mosuem
mosuem force-pushed the publish-with-attestation branch from f013f26 to 7851679 Compare August 24, 2026 13:04
@mosuem
mosuem force-pushed the publish-with-attestation branch from 7851679 to c2757eb Compare August 24, 2026 13:28
@mosuem
mosuem force-pushed the publish-with-attestation branch 2 times, most recently from 92254ea to 12f6ab3 Compare August 24, 2026 15:46
@mosuem
mosuem force-pushed the publish-with-attestation branch from 12f6ab3 to 37388ff Compare August 24, 2026 15:50
@mosuem
mosuem force-pushed the publish-with-attestation branch 2 times, most recently from 0601f58 to 34cdf77 Compare August 24, 2026 15:57
@mosuem
mosuem force-pushed the publish-with-attestation branch from 34cdf77 to da83ed9 Compare August 24, 2026 16:19
@mosuem
mosuem force-pushed the publish-with-attestation branch 2 times, most recently from 08d1cc1 to ff724dd Compare August 24, 2026 16:26
@mosuem
mosuem requested a review from sigurdm August 28, 2026 12:48
@mosuem
mosuem force-pushed the publish-with-attestation branch from eb1d0bf to 568a844 Compare August 28, 2026 12:49

@sigurdm sigurdm left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could use also an update to ## Publishing Packages in doc/repository-spec-v2.md

@mosuem
mosuem force-pushed the publish-with-attestation branch 2 times, most recently from 0568241 to f239f47 Compare August 31, 2026 14:16
Comment thread doc/repository-spec-v2.md Outdated
Comment thread doc/repository-spec-v2.md Outdated
Comment thread doc/repository-spec-v2.md Outdated
Comment thread doc/repository-spec-v2.md Outdated
@mosuem
mosuem requested review from jonasfj and sigurdm September 3, 2026 15:36

@sigurdm sigurdm left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a proposal for how to communicate the attestation that would simplify both the spec and the server implementation:

The issue with uploading attestations to GCS

Google Cloud Storage's form POST API only allows uploading a single object per request. To upload both the archive and attestation to GCS, the server must generate two separately signed policies.

On pub.dev, signing a policy requires an IAM signBlob RPC. If GET /api/packages/versions/new returns two signed URLs unconditionally, we incur 2x IAM latency (50–200ms) and 2x quota on every publish, even though 99.9% of publishes don't have an attestation. In addition, during finalization the server has no record of whether the client intended to upload an attestation, so a dropped upload could lead to silently publishing without provenance.

Proposal: Send the attestation in the finalize request

An attestation bundle (.sigstore.json) is only 3–10 KB. Unlike the 100 MB archive, which must be uploaded directly to GCS to avoid buffering in App Engine / Cloud Run memory, a 5 KB JSON bundle can be sent directly to the server.

  1. GET /api/packages/versions/new stays completely unchanged. No new fields, no capability negotiation, and only 1 IAM call to sign the archive upload policy.
  2. The client uploads package.tar.gz to GCS as today.
  3. During finalization, if publishing with an attestation, the client issues a POST to <finalize-upload-url> with { "attestation": <sigstore JSON bundle> }. If publishing without an attestation, it issues a GET as today.

Benefits:

  • Zero changes to versions/new: No extra IAM calls or latency on pub.dev.
  • No multi-object storage complexity: No second signed URL, no orphan objects in GCS.
  • Atomic delivery: The attestation arrives directly with the finalize call, so the server knows unambiguously that this publish includes an attestation.
  • Compatibility: If a server doesn't support attestations, it will return 405 Method Not Allowed on the POST, which the client can report as "The repository does not support publishing with attestations."

Comment thread doc/repository-spec-v2.md Outdated
Comment thread lib/src/command/lish.dart Outdated
Comment thread lib/src/command/lish.dart Outdated
Comment thread lib/src/command/lish.dart
Comment thread lib/src/command/lish.dart
Comment thread test/lish/publishing_to_and_from_archive_test.dart Outdated

@sigurdm sigurdm left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM (once rebased onto main after #4885 lands).

Comment thread lib/src/command/lish.dart
mosuem added 10 commits October 12, 2026 16:49
- Adds `--with-attestation <bundle.sigstore.json>` flag to `dart pub publish --from-archive`.
- Enforces that a GitHub repository is specified in `pubspec.yaml` (`repository` field) when publishing with an attestation.
- In `_publishUsingClient`, uploads the attestation JSON bundle alongside `package.tar.gz` to the hosting cloud storage bucket.
- Adds comprehensive unit tests for `--with-attestation` and validation checks.
- In doc/repository-spec-v2.md, integrate package attestation uploads directly into ## Publishing Packages.
- Document optional attestationUrl and attestationFields returned by GET /api/packages/versions/new.
- Update lib/src/command/lish.dart to upload to attestationUrl with attestationFields prior to archive upload, without mutating form fields.
- Update test/lish/utils.dart and test/lish/publishing_to_and_from_archive_test.dart to verify new upload flow and server compatibility error.
- In repository-spec-v2.md, specify that attestations are sent via HTTP POST to finalize-upload-url with body {'attestation': <bundle>}, and 405 Method Not Allowed indicates lack of support.
- In lib/src/command/lish.dart, remove separate multipart form upload for attestation and send POST request on finalization if attestation is provided.
- Add client-side validation of attestation file (non-empty, valid JSON object) before publishing.
- Handle 405 Method Not Allowed cleanly.
- Update and add tests in test/lish/publishing_to_and_from_archive_test.dart.
Instead of overloading <finalize-upload-url> with a POST carrying the
attestation, the repository advertises support for publishing with
attestations by returning an <attestation-upload-url> in the
'attestationUrl' property of the response from
'/api/packages/versions/new'.

The attestation is POSTed there before the package archive is uploaded,
and <finalize-upload-url> remains a plain GET.

This means:
 * a client with an attestation learns that the repository does not
   support attestations before uploading the archive, rather than from a
   405 after the archive has been uploaded, and
 * the specification no longer has a GET and a POST doing the same thing
   on the same URL, which also avoids the conflict with following the
   303 redirect to <finalize-upload-url> (a 303 is followed with a GET,
   dropping the body).

The 'attestationUrl' is an opaque URL: a repository may point it at its
own endpoint or at a signed URL on a storage service.
The repository may direct the attestation upload to another origin (e.g. a
storage service). A non-2xx response from there did not match the
repository's origin and was rethrown as an unexpected error. Track the
attestation URL like the cloud storage URL and fail with the response body
and `Failed to upload the attestation.` instead.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants