test: recursive struct ordering — order-independence + byte-exact conformance

[kurodo3]

Summary

Closes PLT-840: Change schema hashing to be recursively ordered.

The existing implementation already performs recursive alphabetical ordering of struct fields at every nesting level (via data_type_to_value for schema hashing and BTreeMap-keyed paths for data hashing). This PR adds the missing test coverage to document, verify, and lock in that behaviour across all relevant nesting shapes.

New tests — order independence (tests/arrow_digester.rs)

Test	What it checks
deeply_nested_struct_schema_hash_is_field_order_independent	3-level deep Struct<Struct<fields>> — inner-field insertion order must not affect hash_schema
deeply_nested_struct_record_batch_hash_is_field_order_independent	Same for record-batch data hashing
hash_array_nested_struct_is_field_order_independent	hash_array API with a struct containing a nested struct
list_of_struct_schema_hash_is_field_order_independent	List<Struct<fields>> schema hash independent of inner struct field order
list_of_struct_record_batch_hash_is_field_order_independent	Same for actual record-batch data

New tests — byte-exact conformance (tests/digest_bytes.rs)

Example	What it checks
O: Nested struct record batch	Pins the canonical JSON and exact leaf paths (s/a, s/nested/p, s/nested/q) for a two-level struct column; verifies each leaf is hashed with the correct little-endian bytes
P: Nested struct field-order independence	Pins the expected canonical JSON for a schema whose struct fields are defined out of alphabetical order at both nesting levels; asserts both orderings produce that exact hash

Test plan

• All 60 tests pass (cargo test)
• cargo fmt applied, no formatting changes
• New tests fail if recursive ordering is removed (confirmed by tracing the code path)

Closes PLT-840

[Copilot]

Pull request overview

Adds regression tests and specification examples to lock in Starfix’s recursive struct field-order normalization (schema/type hashing and record-batch/array data hashing) so that nested struct insertion order cannot affect digests.

Changes:

• Add order-independence tests for deeply-nested structs and list-of-struct shapes across hash_schema, hash_record_batch, hash_array, and streaming update().
• Add byte-exact conformance tests for nested structs (record batch + hash_array) and a pinned canonical JSON example for nested struct ordering.
• Extend the byte-layout spec with new Examples O–Q documenting canonical JSON, leaf paths, and byte-level hashing steps.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File	Description
tests/digest_bytes.rs	Adds byte-exact examples O–Q for nested structs, including pinned canonical JSON and leaf-path hashing.
tests/arrow_digester.rs	Adds order-independence tests for nested structs (schema, record batches, hash_array, streaming update) and list-of-struct cases.
docs/byte-layout-spec.md	Documents Examples O–Q to match/anchor the new conformance tests and recursive ordering behavior.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.