(De)serializer from/to JSON

[Kriskras99]

Original issue #449 by @PookieBuns

Currently we only support (de)serializing from/to JSON via the TryFrom<serde_json::Value> for avro::types::Value and TryFrom<avro::types::Value> for serde_json::Value implementations.
These implementations are not according to the spec:

#[test]
fn avro_rs_518_union_int_is_json_int_value() {
    let v = Value::Union(1, Box::new(Value::Int(42)));
    let json: JsonValue = v.try_into().unwrap();
    assert_eq!(json, json!({"int": 42}));
}
#[test]
fn avro_rs_518_union_enum_is_json_enum_name_value() {
    let v = Value::Union(1, Box::new(Value::Enum(5, "Problematic")));
    let json: JsonValue = v.try_into().unwrap();
    // The Value::Enum doesn't contain the enum name (some_name) so can't do the correct thing
    assert_eq!(json, json!({"some_name": "Problematic"}));
}

Both these tests will fail. The first test can be fixed using the TryFrom implementations, but the second test cannot be fixed as avro::types::Value does not contain the necessary information for named types.

If we decide to support the JSON encoding of Avro, I suggest the following:

1. Create a new module json that will contain the from_reader(reader, schema)/to_writer(writer, schema, value).
2. Create a new serializer and deserializer that wrap around serde_json's serializer and deserializer, using the schema to call the right functions on serde_json's serializer and deserializer.

• The alternative is (de)serializing to serde_json::Value first, and then implementing our serializer and deserializer around that. However, this is really slow.
• If there is a usecase for (de)serializing to serde_json::Value, support can always be added later with a to_value/from_value function (it can reuse the logic of SchemaAwareRecordFieldDefault for the serialisation part).

[ignatz]

I've been quietly stalking this issue for a while after running into it myself. Just curious is this is something on the short-list? Either way, I wanted to express my support explicitly. Thanks for avro 🙏

[hwse]

I would also like to express my interest in this feature. It would also be helpful to know if the authors believe that JSON encoding and decoding should be included in this library.

[martin-g]

Any help is always welcome!

You could test current main and all PRs by @Kriskras99 and report issues/feedback.

[ignatz]

Any help is always welcome!

You could test current main and all PRs by @Kriskras99 and report issues/feedback.

Sorry, I don't understand the suggestion. I looked at main, I looked at pending PRs... likely I'm blind but I didn't notice anything immediately related to this issue

[Kriskras99]

There is quite a lot of work required to implement this feature properly (would require a new serializer and deserializer as outlined in the first post). Currently my limited time is focused on getting enum support into main so we can start a new release. After that I can start looking at implementing this, however that will not be before end of summer.

So currently there's not a lot to be done, unless you want to implement this feature yourself.