Skip to content

Cut stack use on the JSON receive path - #104

Merged
kahrendt merged 2 commits into
mainfrom
reduce-httpd-stack
Aug 10, 2026
Merged

Cut stack use on the JSON receive path#104
kahrendt merged 2 commits into
mainfrom
reduce-httpd-stack

Conversation

@kahrendt

Copy link
Copy Markdown
Contributor

The ESP httpd task runs with the ESP-IDF default 4 KB stack (HTTPD_DEFAULT_CONFIG), and every inbound JSON message is parsed and dispatched on it, underneath httpd's own call chain. Measured with -fstack-usage on xtensa-esp32s3 g++ 14.2 at -Os, a server/state message carrying metadata cost about 1250 bytes of library frames alone, before httpd's own usage below it and before any logging on top. That is a third of the whole stack spent in this library, and it is what the reported httpd stack overflow (issue #102) was eating into.

Almost all of it was avoidable duplication rather than real working set:

  • server/state was parsed into one ServerStateMessage aggregate (288 bytes), so every section's storage stayed live in the caller's frame for the whole parse while each section parser built a second copy of the same fields in its own. A ServerMetadataStateDelta is 200 bytes and was materialized twice at once, then a third time by the by-value handle_server_state parameter.

  • GCC inlined the section parsers into process_server_state_message and did not overlap their locals, so metadata, color, and controller storage all coexisted in a single 736 byte frame, the largest in the library.

server/state is now parsed one section at a time. Each section is an out-of-line function that fills a caller-owned struct in place, the client scopes each section separately so the compiler can reuse the slots, and a section is only parsed when its role is present. The three handle_server_state overloads take an rvalue reference instead of a by-value copy. stream/start gets the same in-place treatment for its player, artwork, and visualizer sections.

Measured frames, before -> after:

process_json_message              512 -> 320
process_server_state_message      736 -> gone
  process_server_state_metadata          288
  process_server_state_controller        224
  process_server_state_color             192
process_stream_start_message      496 -> 448

Deepest server/state path: 1248 -> 608 bytes (51% less). Deepest
stream/start path: 1008 -> 768.

No behavior change: same fields, same validation, same tri-state delta semantics. tests/test_protocol.cpp moves to the per-section entry points.

Closes #102. It seems like this is a healthy enough margin without needing to increase the default task stack. The encryption work will require a small bump to the httpd task stack.

The ESP httpd task runs with the ESP-IDF default 4 KB stack
(HTTPD_DEFAULT_CONFIG), and every inbound JSON message is parsed and
dispatched on it, underneath httpd's own call chain. Measured with
-fstack-usage on xtensa-esp32s3 g++ 14.2 at -Os, a server/state message
carrying metadata cost about 1250 bytes of library frames alone, before
httpd's own usage below it and before any logging on top. That is a
third of the whole stack spent in this library, and it is what the
reported httpd stack overflow (issue #102) was eating into.

Almost all of it was avoidable duplication rather than real working set:

- server/state was parsed into one ServerStateMessage aggregate (288
  bytes), so every section's storage stayed live in the caller's frame
  for the whole parse while each section parser built a second copy of
  the same fields in its own. A ServerMetadataStateDelta is 200 bytes
  and was materialized twice at once, then a third time by the by-value
  handle_server_state parameter.

- GCC inlined the section parsers into process_server_state_message and
  did not overlap their locals, so metadata, color, and controller
  storage all coexisted in a single 736 byte frame, the largest in the
  library.

server/state is now parsed one section at a time. Each section is an
out-of-line function that fills a caller-owned struct in place, the
client scopes each section separately so the compiler can reuse the
slots, and a section is only parsed when its role is present. The three
handle_server_state overloads take an rvalue reference instead of a
by-value copy. stream/start gets the same in-place treatment for its
player, artwork, and visualizer sections.

Measured frames, before -> after:

    process_json_message              512 -> 320
    process_server_state_message      736 -> gone
      process_server_state_metadata          288
      process_server_state_controller        224
      process_server_state_color             192
    process_stream_start_message      496 -> 448

Deepest server/state path: 1248 -> 608 bytes (51% less). Deepest
stream/start path: 1008 -> 768.

No behavior change: same fields, same validation, same tri-state
delta semantics. tests/test_protocol.cpp moves to the per-section
entry points.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Reduces ESP HTTP task stack usage by parsing JSON role sections independently and in place.

Changes:

  • Splits server/state parsing into per-role functions.
  • Eliminates intermediate copies through rvalue-reference handlers.
  • Updates stream/start parsing and protocol tests.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated no comments.

Show a summary per file
File Description
src/client.cpp Parses and dispatches state sections individually.
src/protocol.cpp Implements in-place section parsing.
src/protocol_messages.h Replaces aggregate parser API.
src/metadata_role.cpp Accepts metadata deltas by rvalue reference.
src/metadata_role_impl.h Updates metadata handler declaration.
src/controller_role.cpp Accepts controller state by rvalue reference.
src/controller_role_impl.h Updates controller handler declaration.
src/color_role.cpp Accepts color deltas by rvalue reference.
src/color_role_impl.h Updates color handler declaration.
tests/test_protocol.cpp Tests the new per-section parsers.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

clang-tidy (performance-move-const-arg) rejects the std::move added in
the previous commit: ServerColorStateDelta is trivially copyable, so
there is nothing for the rvalue reference to move out of. The other two
handle_server_state overloads keep theirs, since a
ServerStateControllerObject holds a vector and a ServerMetadataStateDelta
holds strings.

Copy count is unchanged. The body already passed the parameter as an
lvalue into InboxSlot::merge, which takes its delta by value.
@kahrendt
kahrendt merged commit 30998ad into main Aug 10, 2026
5 checks passed
@kahrendt
kahrendt deleted the reduce-httpd-stack branch August 10, 2026 11:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ESP32-S3: HTTPD task stack overflow during Music Assistant playback; increasing config.stack_size appears to resolve

2 participants