[mosuem]

Note

This PR was generated by an AI coding agent (Jetski) on behalf of @mosuem.

First PR in stack for package attestation support:

• GET /api/packages/versions/new now returns an attestationUrl, pointing at POST /api/packages/versions/newUploadAttestation/<uploadId>. Repositories that do not support publishing with attestations omit the property, which is how pub detects support before uploading the archive (Support uploading package attestations during publish --from-archive pub#4876).
• The new endpoint requires an authenticated client, validates that the upload id is a uuid, that the bundle is at most maxAttestationContentLength (128 KB, enforced while reading the request body via foldBytes(maxSize:)), and that it parses as a JSON object, and then stores it as tmp/<guid>.attestation.json in the incoming bucket.
• newUploadFinish remains a GET-only endpoint.
• The attestation is canonical data (it is uploaded alongside the archive, not extracted from it), so _performTarballUpload copies it into the canonical bucket next to the archive, as packages/<package>-<version>.tar.gz.sigstore.json. It is not stored as a PackageVersionAsset. removePackageTarball removes both objects.
• The attestation is exported to the exported API bucket by ApiExporter.synchronizePackage, as /api/packages/<package>/versions/<version>/attestation (verbatim bytes), like the archives: ExportedPackage.synchronizeAttestations mirrors synchronizeTarballs, moderated/removed versions get their export deleted, and attestations are synchronized before the version listing is written, since clients fetch the attestation when the listing says a version has one and do not retry. Serving this path from the bucket needs a GCLB URL map change (ops).
• GET /api/packages/<package>/versions/<version>/attestation is kept as the handler for hosts that do not route the path to the bucket. It checks package and version visibility (404 for moderated content) and serves the canonical copy.
• Updates PubApiClientExt and adds tests in upload_test.dart (storage, export, verbatim retrieval, moderation), exported_api_test.dart and utils_test.dart.

Since the attestation is uploaded to the app server, publishing with an attestation does not require a second signed upload policy, and therefore no extra iam.signBlob call per publish.

Verifying the bundle against the uploaded archive (digest and provenance) follows in the next PR in the stack.

[sigurdm]

Note: since this PR advertises attestationUrl and accepts attestation uploads without verifying them yet, we should hold off on merging it to main until #9546 is ready to land right alongside it.

[sigurdm]

Storing the attestation only as a PackageVersionAsset and serving it directly from the database in getPackageVersionAttestation has a few architectural issues:

1. Canonical vs. derived storage: PackageVersionAsset is for derived data extracted from the package tarball (derivePackageVersionEntities for README, CHANGELOG, pubspec, license; see doc/entities.md). Because the attestation bundle is uploaded alongside the tarball rather than inside it, it is canonical data. If we only store it in PackageVersionAsset and delete tmp/<guid>.attestation.json from _incomingBucket, it is never stored in _canonicalBucket (packages/<package>-<version>...), and re-running derivePackageVersionEntities from the canonical archive would lose the attestation. It should be copied into _canonicalBucket alongside the tarball in _performTarballUpload.
2. Serving via ExportedApi / GCS: GET /api/packages/<package>/versions/<version>/attestation will be called during dart pub get (in pub#4875). All other pub get endpoints (/api/packages/<pkg>, /api/packages/<pkg>/advisories, /api/archives/...) are exported to _exportedApiBucket via ExportedApi (app/lib/package/api_export/exported_api.dart) and served directly from GCS by GCLB (and exempted from Cloud Armor /api/ rate limits). Exporting attestations via ExportedApi also means ApiExporter.synchronizePackage will automatically remove exported attestations if a package or version is moderated (lookupPackageVersionAsset currently does not check isPackageVisible / pv.isNotVisible).
3. Cloud Armor POST rate limit on upload: In production, Cloud Armor rate-limits non-GET requests to pub.dev to 5 requests per 3 minutes per IP (which wasn't hit by pub publish previously because the tarball POST goes to storage.googleapis.com, while new and newUploadFinish on pub.dev are GET requests with higher per-route limits). Sending POST /api/packages/versions/newUploadAttestation/<uploadId> to pub.dev will hit that limit when CI runners publish >5 packages from a monorepo in 3 minutes, so we'll need to adjust our Cloud Armor rules (or upload to GCS).

[mosuem]

Done for 1. and 2.; 3. needs a decision from you.

1. Canonical storage. The attestation is now copied from the incoming bucket to the canonical bucket inside _performTarballUpload, right after the archive, as packages/<package>-<version>.tar.gz.sigstore.json (the Sigstore convention of naming a bundle after the artifact it attests to). AssetKind.attestation is gone: the attestation is not derived from the archive, so it no longer pretends to be one. removePackageTarball deletes both objects, listVersionsInCanonicalBucket now filters by the .tar.gz suffix (it used to accept anything under packages/<pkg>-), and listAttestationsInCanonicalBucket lists the bundles.

2. Exported API. ExportedPackage.attestation(version) writes <prefix>/api/packages/<package>/versions/<version>/attestation (verbatim bytes, application/json, 2h max-age like the archives, no Content-Disposition), synchronizeAttestations mirrors synchronizeTarballs (they share one implementation now), and GC / delete() cover the new objects. ApiExporter.synchronizePackage syncs attestations after the archives and before writing the version listing — this ordering matters for the client: pub (dart-lang/pub#4875) fetches the attestation when the listing says the version has one and fails the install on a 404, it does not retry. So once slsa_level is set in #9552 the order is part of the contract (there is a note in the spec about it). Moderation is handled like for archives: synchronizePackage deletes exported attestations of versions that are no longer listed, and the app handler (kept as the fallback for hosts that do not route the path to the bucket, like listVersions) now checks package and version visibility and serves the canonical copy.

What I cannot do from here: the GCLB URL map has to route /api/packages/*/versions/*/attestation to the bucket before this is actually served from GCS (and exempt from the /api/ rate limits). Until then the handler serves it, which is also what staging will do. An alternative that avoids the URL-map change would be to export it under /api/archives/<package>-<version>.tar.gz.sigstore.json and advertise the URL in the version listing (attestation_url, next to archive_url). That is a protocol change in the pub spec, so I did not do it, but I am happy to go that way if you prefer it — it would also make life easier for mirrors.

3. Cloud Armor. Agreed that a monorepo publishing >5 packages in 3 minutes would hit the POST limit. Two options: (a) keep the POST and exempt /api/packages/versions/newUploadAttestation/* in the Cloud Armor policy — it is authenticated, ≤128 KB, and bounded by the upload rate limit; or (b) hand out a second signed GCS upload policy like for the archive, which keeps pub.dev out of the upload path but costs an extra iam.signBlob per publish, loses the early JSON check, and changes the client protocol to a multipart POST. I would go with (a) since it is a configuration change and keeps the simpler protocol, but either way it needs someone with access to the policy — which do you prefer?

[sigurdm]

Updating the GCLB URL map and Cloud Armor policy in Terraform is fine — let's keep /api/packages/<package>/versions/<version>/attestation and option (a) (POST /api/packages/versions/newUploadAttestation/<uploadId>). Please file a tracking issue (or note it in the rollout checklist) so we remember to land the gclb.tf and security_policy.tf rules before enabling this in prod.

One small thing in publishUploadedBlob (app/lib/package/backend.dart lines 1503–1528, 1792–1806): for the archive, we copy uploadObjectName to workObjectName first so the client cannot overwrite tmp/$uploadGuid between verification and copying to _canonicalBucket. For the attestation, publishUploadedBlob reads attestationObjectName (which #9546 verifies) and then _performTarballUpload copies attestationObjectName directly from _incomingBucket, so a concurrent POST /api/packages/versions/newUploadAttestation/<uploadId> could swap the bundle after verification. Since we already read the bytes into memory in publishUploadedBlob (≤128 KB), we can either upload those exact bytes to _canonicalBucket in _performTarballUpload or copy attestationObjectName to a workAttestationObjectName upfront.