Title
Add a Decoder.SetMaxFieldLength() function to allow overriding the default limit of 2GB.
Description
The Decoder enforces a limit of 2GB on variable length fields like strings, bytes and nested messages.
We hit this limit for large snapshots in PowerDNS Lightning Stream as described in PowerDNS/lightningstream#112
We propose adding a Decoder.SetMaxFieldLength() function to allow overriding this limit, while still retaining the default of 2GB when not overridden (PR incoming).
Additional Info
This limit is recommended by https://protobuf.dev/programming-guides/proto-limits/ as a way to guarantee interoperability.
It is also the only limit on that page that is not specified as being language dependent.
Go internally uses a 64-bit int for sizes on 64ibit platforms, and the actual length in the wire format is encoded in a varint that is decoded as a uint64 and can represent its full range, so there is no technical reason to impose a 2GB limit in a Go implementation.
Title
Add a
Decoder.SetMaxFieldLength()function to allow overriding the default limit of 2GB.Description
The Decoder enforces a limit of 2GB on variable length fields like strings, bytes and nested messages.
We hit this limit for large snapshots in PowerDNS Lightning Stream as described in PowerDNS/lightningstream#112
We propose adding a
Decoder.SetMaxFieldLength()function to allow overriding this limit, while still retaining the default of 2GB when not overridden (PR incoming).Additional Info
This limit is recommended by https://protobuf.dev/programming-guides/proto-limits/ as a way to guarantee interoperability.
It is also the only limit on that page that is not specified as being language dependent.
Go internally uses a 64-bit int for sizes on 64ibit platforms, and the actual length in the wire format is encoded in a varint that is decoded as a uint64 and can represent its full range, so there is no technical reason to impose a 2GB limit in a Go implementation.