Rust change policy

[CasperN]

We have a bunch of work lined up, developing Flatbuffers in Rust. (Issues). Some of these, such as fixing #5589 would require breaking changes. We need a stability and change policy.

Some precedent:

• In Rust: policy on minimum support Rust compiler version #5812, we decided that we should only support compilers up to 12 weeks old.
• In Rework enums in rust. #6098, I fixed UB in enums with a breaking change. I also renamed some constants (leaving a 3 month deprecation warning for the old names).

Proposed policy

1. Critical Rust code-breaking changes may happen with 1 weeks notice.

   • These would be bugs that would prevent widespread use of the library.
   • This includes, fixing UB, and broken critical features like namespaces ([Rust] Missing 'use' statements when generating from .fbs that has imports #5589).
   • This does mean HEAD will be somewhat unstable while we fix the UB stuff

2. Minor Rust code-breaking changes may happen with 12 weeks notice

   • This includes adopting new compiler features (thus forcing a compiler upgrade), renaming constants to be more "Rust-y", etc
   • Change due to new features like the verifier Implement Flatbuffer Verifier for Rust #6161 fall into this category.

"notice" basically means mentioning it on an issue tagged rust in this repo.

Thoughts?

@rw @aardappel @krojew @jean-airoldie @andygrove @jcrevier @plwalsh @dvtomas @maxburke

[jcrevier]

The time frames suggested here seem reasonable to me. One thought though: It might be wise to publish or document somewhere which commit of flatc can generate working code for the corresponding rust flatbuffer crate. The crate documentation instructs users to use HEAD. But if the changes in generated code are significant and there's a bit of churn, it might take library users a bit longer to upgrade / transition. Being able to generate working code for any given version of the flatbuffers crate seems important.

Ideally we pin a version of the flatbuffers crate to a version of flatc but given how infrequently versions are released, I think we could just tie it to a specific commit.

[plwalsh]

I'm all for this. Breaking changes for the sake of active development is always okay in my book. Thanks for keeping us in the loop. And I agree with @jcrevier's suggestions above.

[krojew]

I think we should also consider #6149 which makes things more consistent.

[CasperN]

I think we should also consider #6149 which makes things more consistent.

I see the following tradeoffs to making the crate's version match flatc's:
(+) We work (and test) from HEAD anyway so it makes sense to snapshot them at the same point.
(+) We'd maintain a simpler compatibility mapping.
(-) Rust's version could be bumped due to irrelevant changes in flatc
(-) I'm not sure if Flatbuffers' version follows https://semver.org/ as rigorously as some Rust folk would like

Overall, it sounds good to me.

[krojew]

(-) Rust's version could be bumped due to irrelevant changes in flatc

That's really not an issue, since FB rarely uses patch version and we can leverage that (or any metadata). Only the major and minor versions should be relevant to the API.

(-) I'm not sure if Flatbuffers' version follows https://semver.org/ as rigorously as some Rust folk would like

I could rant for an hour about how crate authors fail to follow semver, but that's not the place for that. Nevertheless, it would be great if FB actually followed it, right @aardappel ? :)

[jcrevier]

(-) I'm not sure if Flatbuffers' version follows https://semver.org/ as rigorously as some Rust folk would like

I agree the simpler compatibility mapping would be a big win. However, I think the semver concerns are a problem. If we release a 1.12 version of the crate and then want to make backwards incompatible changes to just the crate, we will need a 2.0 release of the crate for things to play nicely with semver and cargo -- but is that a good enough reason for @aardappel to bump the major version of the entire project? I suspect we're better off overall without trying to couple these together.

[CasperN]

and cargo

Oh yea if automation enforces rigorous semver within the Rust ecosystem then I agree we should avoid coupling.

[jcrevier]

and cargo

Oh yea if automation enforces rigorous semver within the Rust ecosystem then I agree we should avoid coupling.

Yep, cargo will happily grab 1.13 if you have flatbuffers = "1.12" specified in your cargo.toml

[krojew]

Given the changes coming to FB, I would be happy with a semver-compatible 2.0 release.

[CasperN]

I would be happy with a semver-compatible 2.0 release.

That'd be an @aardappel decision, but having a single rigorous semver for all flatbuffers languages and flatc seems like a bad idea to me: There'll be far too many irrelevant "breaking changes" that are not observable in most projects since they don't use all the features/languages.

[krojew]

A breaking change is never irrelevant. If something fundamental is changing very often, there's something really wrong with the design.

[aardappel]

Up to y'all.. so far, FlatBuffer has not been strict about breaking changes, in that, we try pretty hard to avoid them, but when they have to happen because of critical safety/performance or even major usability changes, we do them anyway. I think it is unavoidable, and better than the alternatives. My general rule of thumb has been that the more people we're likely to break the more critical the change must be for it to be ok. But that's hard to encode in a rule.

We're a small enough project (in terms of regular maintainers) that I personally wouldn't want to commit to strict timespans for giving notice.. I like to keep process as simple as possible. But if the people interested in Rust want to do this, I have no problem with it either.

As for versioning, I am famously apathetic to any versioning scheme that's not just dates or commit ids. I feel SemVer is pretty meaningless, and impossible to make work in an exact way in a project spanning public, internal and code-genned APIs across 10+ languages. Most of the time I can't even predict what can all break by looking at a PR.

Versions so far are just tied to global FlatBuffers releases, and we're not going to do these more often for the needs of 1 language, as least as long as I am doing them (anyone want to take over FlatBuffers releases, inquire within ;) That said, if y'all wanna push to Cargo more often than that and keep your own versioning, you're of course welcome to.

[maxburke]

We're fine with breaking changes as long the changes are breaking only in API and not data as we rely on FB for inter-language serialization, but I assume that's what you mean anyways 🙂

[krojew]

I feel SemVer is pretty meaningless, and impossible to make work in an exact way in a project spanning public, internal and code-genned APIs across 10+ languages. Most of the time I can't even predict what can all break by looking at a PR.

I think trying to introduce semver can't hurt - after all, if it fails for us, we can still say we're not semver compatible, as it is now. On the other hand, if we can make it work, it's a good scheme to follow, which people will appreciate on upgrading.