Skip to content

gsutil in the download instructions is deprecated - causes crash when downloading on macOS #120

Closed

Description

@jcooper036

Hello

Thank for the nice tutorials on using the data set, they are very helpful.

I ran into an issue when following the data set tutorial (https://plinder-org.github.io/plinder/tutorial/dataset.html). I think there is a simple fix. I am happy to contribute a PR - but I wanted to check that it was something that the maintainers actually want before I do that.

Issue

The data set tutorial instructs users to use gsutil to download the dataset from the google bucket (if the user takes that route). However, gsutil is a legacy tool, replaced by gcloud storage, per Google's documentation.

Specifically, this causes modern Mac hardware to crash the process when trying to download the PLINDER dataset due to differences in how parallelism is implemented.

Suggestion

Replace:

gsutil -m cp -r "gs://plinder/${PLINDER_RELEASE}/${PLINDER_ITERATION}/*" ~/.local/share/plinder/${PLINDER_RELEASE}/${PLINDER_ITERATION}/

with:

gcloud storage cp -r "gs://plinder/${PLINDER_RELEASE}/${PLINDER_ITERATION}/*" ~/.local/share/plinder/${PLINDER_RELEASE}/${PLINDER_ITERATION}/

In the tutorial instructions.

Note that is drops the -m flag, which in gsutil was for parallelism. gcloud storage cp has this parallelism by default, and it's implemented in such a way that it avoids the macOS fork crash.

This would also require changing the link of the "installing gsutil" instructions, which I think is just part of the gcloud SDK now (https://docs.cloud.google.com/sdk/docs/install-sdk).

Alternatives

Dropping the -m flag and still using gsutil for mac users might work - I didn't test that though.

Testing

I tried using the gcloud storage cp approach, and the download completed in a few minutes for me.

Activity

  1. OleinikovasV commented on Jul 21, 2026

    @OleinikovasV
    Member

    Thanks for highlighting that. There's already some work in progress in #118 to update the dataset and its location.
    Would be great to get you to test the new version very soon!

  2. OleinikovasV commented on Sep 21, 2026

    @OleinikovasV
    Member

    Dataset downloads moved from GCS to Cloudflare R2: #124

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    documentationImprovements or additions to documentation

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions