Skip to content

better feedback when chunk size is adjusted #291

@burritojustice

Description

@burritojustice

During upload, if a chunk is too large for the API gateway, we cut it in half and try again (and if it's still too large, we cut it in half and try again (and if…)))

While this is quite robust (assuming there isn't a single feature that is larger than the size of the API gateway), we should give more meaningful feedback when we get to a chunk size that does work:

here xyz upload ee9Ap2Hw -f /Users/joram/Downloads/LROC_5TO20KM_CRATERS_0TO90W_180.geojson,/Users/joram/Downloads/LROC_5TO20KM_CRATERS_0TO90E_180.geojson,/Users/joram/Downloads/LROC_5TO20KM_CRATERS_90ETO180E_180.geojson,/Users/joram/Downloads/LROC_5TO20KM_CRATERS_90WTO180W_180.geojson -s
uploading file - /Users/joram/Downloads/LROC_5TO20KM_CRATERS_0TO90W_180.geojson

uploading chunk size of 200 features failed with 413 Request Entity too large error, trying upload again with smaller chunk of 100

uploading chunk size of 200 features failed with 413 Request Entity too large error, trying upload again with smaller chunk of 100

uploading chunk size of 100 features failed with 413 Request Entity too large error, trying upload again with smaller chunk of 50

uploading chunk size of 200 features failed with 413 Request Entity too large error, trying upload again with smaller chunk of 100

uploading chunk size of 50 features failed with 413 Request Entity too large error, trying upload again with smaller chunk of 25

uploading chunk size of 100 features failed with 413 Request Entity too large error, trying upload again with smaller chunk of 50

uploading chunk size of 100 features failed with 413 Request Entity too large error, trying upload again with smaller chunk of 50

uploading chunk size of 200 features failed with 413 Request Entity too large error, trying upload again with smaller chunk of 100

While it's clear the chunk size is being adjusted, it's not clear what and when actually got uploaded. We should fix that.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions