Skip to content

Draft: New schema parser - #672

Open
Kriskras99 wants to merge 1 commit into
mainfrom
feat/schema_parser_v2
Open

Draft: New schema parser#672
Kriskras99 wants to merge 1 commit into
mainfrom
feat/schema_parser_v2

Conversation

@Kriskras99

Copy link
Copy Markdown
Contributor

This PR is not ready for review, see it more as a progress update.

The new parser uses a different JSON library sonic which allows the parser to iterate over borrowed strings from the file. This saves a ton of heap allocations. sonic also has SIMD optimisations when RUSTFLAGS="-C target-cpu=native" is used, but on the small files we generally have, the impact of that is small.

TODO:

  • Proper error handling instead of panicking everywhere
  • Recursion and allocation limits
  • Test against all the parser tests
Parser Small record Large record
Old 23.5 us 25.7 ms
New 15.8 us 16.0 ms
Old (native) 24.0 us 26.1 ms
New (native) 13.8 us 15.6 ms
  1. The large record is large_schema.asvc
  2. The small record is RecordWithRequiredFields.avsc
  3. Take the benchmarks with a grain of salt, laptops are quite bad for benchmarking.

@Kriskras99
Kriskras99 force-pushed the feat/schema_parser_v2 branch from c10d80c to 0990ec6 Compare September 9, 2026 20:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant