Skip to content

to_json refactor#2848

Open
mikes-lunarg wants to merge 8 commits intoLunarG:devfrom
mikes-lunarg:to-json-refactor
Open

to_json refactor#2848
mikes-lunarg wants to merge 8 commits intoLunarG:devfrom
mikes-lunarg:to-json-refactor

Conversation

@mikes-lunarg
Copy link
Copy Markdown
Contributor

This PR is an attempt to move away from FieldToJson overloaded functions for converting GFXR types into nlohmann::ordered_json objects. The clang compiler (particularly on macOS) consumes a lot of memory resolving the large number of FieldToJson overloads in the codebase and has previously forced us to disable the macOS GitHub Actions runners (#2640).

The replacement solution is to define to_json functions for all of the types we want to convert, as described in the nlohmann/json docs: https://github.com/nlohmann/json?tab=readme-ov-file#arbitrary-types-conversions. While it seems similar to the FieldToJson approach, using to_json goes through the nlohmann::adl_serializer and doesn't suffer the same (Many call site)x(Many implementation) explosion in the compiler.

This changeset only migrates enums, flags, and integer types from FieldToJson to to_json as they represent a bulk of the FieldToJson call sites. Other types from the gfxrecon::decode namespace could be added in the future to simplify the code generators and make things more consistent.

Results

For the memory-constrained GitHub runners the change makes a significant difference:

Before:
Screenshot 2026-03-31 at 12 28 50 PM

After:
Screenshot 2026-03-31 at 12 28 42 PM

On higher spec'd developer machines the difference is smaller, but still noticable:

Before

Built 285 targets in 10.15 min total.
   47.24 s framework/decode/CMakeFiles/gfxrecon_decode.dir/__/generated/generated_vulkan_struct_to_json.cpp.o
   31.44 s framework/decode/CMakeFiles/gfxrecon_decode.dir/__/generated/generated_vulkan_replay_consumer.cpp.o
   25.51 s framework/decode/CMakeFiles/gfxrecon_decode.dir/__/generated/generated_openxr_struct_to_json.cpp.o
   24.64 s framework/decode/CMakeFiles/gfxrecon_decode.dir/__/generated/generated_vulkan_json_consumer.cpp.o
   13.08 s framework/decode/CMakeFiles/gfxrecon_decode.dir/__/generated/generated_openxr_json_consumer.cpp.o
   11.88 s framework/decode/CMakeFiles/gfxrecon_decode.dir/__/generated/generated_vulkan_cpp_structs.cpp.o
   11.62 s framework/decode/CMakeFiles/gfxrecon_decode.dir/__/generated/generated_vulkan_pnext_struct_decoder.cpp.o
    9.82 s framework/decode/CMakeFiles/gfxrecon_decode.dir/vulkan_replay_consumer_base.cpp.o
    9.49 s framework/encode/CMakeFiles/gfxrecon_encode.dir/__/generated/generated_vulkan_pnext_struct_encoder.cpp.o
    9.42 s framework/encode/CMakeFiles/gfxrecon_encode.dir/__/generated/generated_vulkan_api_call_encoders.cpp.o

Peak memory usage: 7079.78 MB
    6277.66 MB framework/decode/CMakeFiles/gfxrecon_decode.dir/__/generated/generated_vulkan_struct_to_json.cpp.o
    3311.78 MB framework/decode/CMakeFiles/gfxrecon_decode.dir/__/generated/generated_vulkan_json_consumer.cpp.o
    3127.98 MB framework/decode/CMakeFiles/gfxrecon_decode.dir/__/generated/generated_openxr_struct_to_json.cpp.o
    1849.25 MB framework/decode/CMakeFiles/gfxrecon_decode.dir/__/generated/generated_openxr_json_consumer.cpp.o
    1477.75 MB framework/decode/CMakeFiles/gfxrecon_decode.dir/vulkan_replay_consumer_base.cpp.o
    1348.52 MB tools/replay/CMakeFiles/gfxrecon-replay.dir/desktop_main.cpp.o
    1289.88 MB framework/decode/CMakeFiles/gfxrecon_decode.dir/__/generated/generated_vulkan_pnext_struct_decoder.cpp.o
    1263.95 MB framework/decode/CMakeFiles/gfxrecon_decode.dir/__/generated/generated_vulkan_cpp_structs.cpp.o
    1219.22 MB tools/convert/CMakeFiles/gfxrecon-convert.dir/main.cpp.o
    1199.89 MB framework/encode/CMakeFiles/gfxrecon_encode.dir/vulkan_state_writer.cpp.o

After

Built 285 targets in 8.03 min total.
   31.38 s framework/decode/CMakeFiles/gfxrecon_decode.dir/__/generated/generated_vulkan_replay_consumer.cpp.o
   12.14 s framework/decode/CMakeFiles/gfxrecon_decode.dir/__/generated/generated_vulkan_cpp_structs.cpp.o
   11.91 s framework/decode/CMakeFiles/gfxrecon_decode.dir/__/generated/generated_vulkan_pnext_struct_decoder.cpp.o
   10.03 s framework/decode/CMakeFiles/gfxrecon_decode.dir/vulkan_replay_consumer_base.cpp.o
    9.74 s framework/decode/CMakeFiles/gfxrecon_decode.dir/__/generated/generated_vulkan_struct_to_json.cpp.o
    9.60 s framework/encode/CMakeFiles/gfxrecon_encode.dir/__/generated/generated_vulkan_api_call_encoders.cpp.o
    9.57 s framework/encode/CMakeFiles/gfxrecon_encode.dir/__/generated/generated_vulkan_pnext_struct_encoder.cpp.o
    8.05 s framework/decode/CMakeFiles/gfxrecon_decode.dir/__/generated/generated_openxr_struct_to_json.cpp.o
    6.98 s framework/encode/CMakeFiles/gfxrecon_encode.dir/__/generated/generated_openxr_api_call_encoders.cpp.o
    6.67 s framework/decode/CMakeFiles/gfxrecon_decode.dir/__/generated/generated_vulkan_struct_handle_mappers.cpp.o

Peak memory usage: 3404.72 MB
    1476.06 MB framework/decode/CMakeFiles/gfxrecon_decode.dir/vulkan_replay_consumer_base.cpp.o
    1361.81 MB tools/replay/CMakeFiles/gfxrecon-replay.dir/desktop_main.cpp.o
    1285.27 MB framework/decode/CMakeFiles/gfxrecon_decode.dir/__/generated/generated_vulkan_pnext_struct_decoder.cpp.o
    1262.20 MB framework/decode/CMakeFiles/gfxrecon_decode.dir/__/generated/generated_vulkan_cpp_structs.cpp.o
    1199.86 MB framework/encode/CMakeFiles/gfxrecon_encode.dir/vulkan_state_writer.cpp.o
    1133.64 MB framework/encode/CMakeFiles/gfxrecon_encode.dir/vulkan_capture_manager.cpp.o
    1098.56 MB framework/encode/CMakeFiles/gfxrecon_encode.dir/__/generated/generated_vulkan_api_call_encoders.cpp.o
    1018.27 MB tools/convert/CMakeFiles/gfxrecon-convert.dir/main.cpp.o
    976.16 MB framework/encode/CMakeFiles/gfxrecon_encode.dir/vulkan_state_tracker.cpp.o
    963.42 MB framework/decode/CMakeFiles/gfxrecon_decode.dir/__/generated/generated_vulkan_replay_consumer.cpp.o```


This forces us to use the jdata["key"] = value syntax and makes sure
none of the enum conversion were missed
Also fix some DX12 fixed-width integer conversions that were broken by
the last change
Add a wrapper to assert that the now-global JsonOptions are set exactly once
and before any use.
@mikes-lunarg mikes-lunarg requested a review from a team as a code owner April 2, 2026 01:12
@mikes-lunarg mikes-lunarg added the approved-to-run-ci Can run CI check on internal LunarG machines label Apr 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved-to-run-ci Can run CI check on internal LunarG machines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant