Skip to content

Fix macro loading and UART data loss on wired pendants - #67

Open
f1adang wants to merge 7 commits into
bdring:mainfrom
f1adang:fix/uart-macro-loading
Open

Fix macro loading and UART data loss on wired pendants#67
f1adang wants to merge 7 commits into
bdring:mainfrom
f1adang:fix/uart-macro-loading

Conversation

@f1adang

@f1adang f1adang commented Sep 9, 2026

Copy link
Copy Markdown

Summary

On an M5Dial wired to FluidNC's UART, the Macros screen showed "No Macros"
even with macros present, and the link carried a continuous flood of error:3.
Tracing it turned up six separate defects, most of them on the shared path that
streams $File/SendJSON, $Files/ListGCode and $File/ShowSome, so file
listing and file preview were affected too, not just macros.

Verified on hardware (M5Dial, FNC_BAUD 1000000, UART transport). Each commit
builds m5dial, cyddial, m5dial_trace and the macos simulator.

The defects

1. The WiFi stack starved the UART reader. fnc_poll() reads a single byte
then calls poll_extra(), which drove wifi_poll() unconditionally whenever
USE_WIFI was compiled in. At 1 Mbaud a byte arrives every 10 µs, so a
UART-connected pendant entered the WiFi stack ~100k times a second. The RX ring
overflowed and bytes vanished mid-document — runs of text disappeared, and the
tail of a status report ended up spliced into a JSON chunk:

[json] len=82 d=4 | d": "btnZX",    "name": "btnZX<Idle|WPos:59.591,6.000,8.000|...

Nothing in wifi_poll() is on the UART data path, so in UART mode it now runs on
a 20 ms timer. Telnet and ESP-NOW keep the per-byte call, where the socket refill
genuinely is on the data path.

Two buffers on the same path were also undersized for 1 Mbaud: the UART RX ring
was 256 bytes (~2.5 ms of headroom, less than one frame render) and the drain
loop in loop() stopped after 64 bytes per tick against 100-byte chunks. Now
2048 and 1024 — the loop already exits the moment RX is empty, so the larger
bound costs nothing at idle.

2. vsend_linef() formatted into a static buffer. send_line()
fnc_send_line() spins in while (_ackwait) { fnc_poll(); } before it reads
the string, and fnc_poll() dispatches reports whose handlers call
send_linef() again. The inner call overwrote the buffer while the outer one was
still about to transmit from it, so commands went out mangled:

{"files":[],"path":"/sd$G","error":"Bad path"}

$Files/ListGCode=/sd lost its terminator and $G landed on the end of it. This
is the source of most of the error:3 (Bad $ statement) traffic. Long-standing
bug; it became reachable as handler-driven traffic increased.

3. Config-item requests retried forever. service_config_requests() re-sent
configRequests.front() every 500 ms until FluidNC answered with a matching
$name=value. A setting the machine doesn't have — an unconfigured axis, a key
this build lacks — is answered with a bare error:3 that never matches
parse_dollar(), so the item never left the queue. detect_homing_info() queues
twelve of these on connect, and since only front() is ever sent, one
unanswerable item blocked every request behind it while flooding the link.
Retries are now capped, and an error arriving while a query is outstanding drops
that item immediately.

4. show_error() made its own guard dead code. It reset the JSON depth and
then called file_request_failed_advance(), whose first act is to check
json_in_progress() — which could no longer be true. That guard exists because
FluidNC interleaves messages: an error from an already-failed request can land
after the next request's document has begun, and advancing there both tears
down a healthy document and skips past it.

5. Echo-off followed the build, not the transport. The Ctrl-L was guarded
#ifndef USE_WIFI and commented "UART only", but USE_WIFI means a network
transport is available, not in use — so a pendant built with it and wired to
UART stopped sending echo-off. With echo on, an echoed $... line arriving
mid-document trips the json_reset_depth() in handle_other().

6. preferences.json macros were never parsed. Three things:

  • PreferencesListener is installed part-way through the document, on the
    "result" key of the $File/SendJSON wrapper, so its _level is relative to
    where it took over. preferences.json puts settings/macros at _level 1
    there, not 2 — and the _level < 2 bail swallowed the macros key before the
    _level == 2 check ever saw it.
  • Unlike the other two listeners, startObject() never cleared
    _name/_target/_filename, so an entry missing action inherited the
    previous entry's filename — already prefixed — and got prefixed again,
    surfacing as //localfs/. An entry missing type inherited the previous type
    and was added when it should have been skipped.
  • Only one key spelling was accepted. WebUI versions differ: older exports use
    filename/target (what MacroListListener expects), newer ones
    action/type. Both are now accepted, ESP is treated as a localfs target
    alongside FS, and a leading slash is normalised before prefixing so neither
    convention yields /localfs//foo.g.

Additionally, $RI auto-reporting kept emitting <Idle|MPos:...> on the same
channel during a transfer, so any byte lost to a stall welded the two together.
It is now suspended for the duration of a transfer, resumed on a condition that
holds down every exit path (including the list and preview transfers, which have
no chain state of their own). And service_macro_chain() puts a deadline on an
in-flight request so the menu can no longer sit on "Reading Macros" forever.

Also included

m5dial_trace env. FNC_RX_TRACE already existed but printed nothing on the
M5Dial — dbg_print/dbg_write compile to no-ops unless DEBUG_TO_USB is also
defined, which the m5dial env doesn't set. Trace output is buffered and flushed
only when nothing is streaming: printing inline is self-defeating, since
dbg_print blocks waiting for USB buffer space and 50 ms of blocking at 1 Mbaud
is ~5 KB of arriving UART data — more than the RX ring holds, so the act of
tracing destroys the transfer being traced.

Command macros are now runnable from the green button. A cmd: macro has no
file behind it, so invoke() with no argument (which means "open the file
preview") left green and touch doing nothing at all — such macros were reachable
only from the dial. onGreenButtonPress() also returned early unless state was
Idle, and $Job/Resume exists precisely for when the machine is not idle.

Not included

  • No change to the !python ./git-version.py invocation. It fails on macOS,
    which has only python3, but hardcoding python3 would break Windows.
    The portable fix is extra_scripts = pre:./git-version.py — the script writes
    src/version.cpp and prints nothing to stdout, so it contributes no build
    flags and is purely a pre-build hook — but that has to be added to every env,
    so it belongs in its own PR.
  • MacrocfgListener has if (++_level = 2) and if (--_level = 1)
    assignment, not comparison, so both branches always taken. Left alone
    deliberately: the broken behaviour looks load-bearing, and correcting it needs
    a real macrocfg.json to test against. Not on the path this PR fixes.
  • Building without USE_WIFI is currently broken on main: SystemScene is
    inside #ifdef USE_WIFI, but BrightnessScene.cpp:32-33 and
    DisplaySettingsScene.cpp:35,39 call activate_scene(&systemScene)
    unconditionally. Fixing it means deciding where those screens navigate back to
    when there is no System scene, which is a UI call.

🤖 Generated with Claude Code

urscale and others added 7 commits September 9, 2026 23:43
FNC_RX_TRACE already existed but printed nothing on the M5Dial: dbg_print and
dbg_write compile to no-ops unless DEBUG_TO_USB is also defined, and the m5dial
env does not define it. The new m5dial_trace env sets both.

Also stop the trace dropping lines. dbg_print discarded output whenever the USB
TX buffer was full, which is exactly what happens during a burst -- so the log
went quiet precisely where the interesting thing occurred and read as "the
transfer stopped here" when it had not. Under FNC_RX_TRACE it now waits for
room, bounded to 50 ms so a detached USB host cannot wedge the firmware.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
(cherry picked from commit 087dfa06716bf49f1acf3e975c356b3841818c68)
fnc_poll() reads a single byte and then calls poll_extra(), which drove
wifi_poll() unconditionally whenever USE_WIFI was compiled in. At FNC_BAUD
1000000 a byte arrives every 10 us, so a UART-connected pendant entered the WiFi
stack roughly 100k times a second. The reader could not keep up, the RX ring
overflowed, and bytes vanished from the middle of a streamed document: runs of
text disappeared and the tail of a status report ended up spliced into a JSON
chunk, derailing the parser.

Nothing in wifi_poll() is on the UART data path -- it services the OTA server
and its AP-mode DNS -- so in UART mode it now runs on a 20 ms timer. Telnet and
ESP-NOW keep the per-byte call, where the socket refill genuinely is on the data
path.

Two buffers on the same path were also undersized for 1 Mbaud:

  - the UART RX ring was 256 bytes, only twice the hardware FIFO and about
    2.5 ms of headroom, less than a single frame render
  - the drain loop in loop() stopped after 64 bytes per tick, which could not
    keep pace with 100-byte file-transfer chunks, so the remainder aged in the
    ring until it overflowed

Now 2048 and 1024. The drain loop already exits the moment RX is empty, so the
larger bound costs nothing at idle.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
(cherry picked from commit 37488d7fdc8842417d23f6df94148c3bc53ecf27)
The Ctrl-L that turns FluidNC's command echo off was guarded by #ifndef
USE_WIFI and commented "UART only". But USE_WIFI means a network transport is
available in this build, not that one is in use, so a pendant built with it and
wired to FluidNC's UART stopped sending echo-off entirely.

With echo left on, FluidNC echoes every command back, and an echoed "$..." line
arriving mid-document trips the json_reset_depth() in handle_other(), tearing
down in-flight macro or file JSON.

Gate it on wifi_use_uart_mode() instead.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
(cherry picked from commit 1ba93872660ba6255981bcdf5ae00ef4c3bbe1c2)
vsend_linef() formatted into a static buffer and passed it to send_line().
send_line() -> fnc_send_line() spins in a "while (_ackwait) { fnc_poll(); }"
wait loop BEFORE it reads the string, and fnc_poll() dispatches received
reports whose handlers call send_linef() again. The inner call overwrote the
buffer while the outer one was still about to transmit from it, so the outer
command went out mangled or missing its terminator.

Observed on the wire as "$Files/ListGCode=/sd" arriving as "/sd$G" -- FluidNC
answering {"files":[],"path":"/sd$G","error":"Bad path"} -- and as a steady
stream of error:3 (Bad $ statement) from garbled commands.

Give it automatic storage so a re-entrant call cannot touch the outer call's
string. This bug is long-standing; it became reachable when the volume of
handler-driven traffic increased.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
(cherry picked from commit 76262f239f48dfe825adb401c0b42aa8b42cf0a0)
Two problems in error handling, both reachable from the same code path.

Config requests retried forever. service_config_requests() re-sent
configRequests.front() every 500 ms until FluidNC answered with a matching
"$name=value". A setting the machine does not have -- an axis that is not
configured, a key this firmware build lacks -- is answered with a bare
"error:3", which never matches parse_dollar(), so the item never left the queue.
detect_homing_info() queues twelve of these on connect, and since only front()
is ever sent, one unanswerable item blocked every request behind it while
flooding the link. Retries are now capped, and an error arriving while a query
is outstanding drops that item immediately rather than burning the whole budget.

show_error() also made its own guard dead. It reset the JSON depth and then
called file_request_failed_advance(), whose first act is to check
json_in_progress() -- which could no longer be true. That guard exists because
FluidNC interleaves messages: an error from an already-failed request can land
after the NEXT request's document has begun, and advancing there both tears
down a healthy document and skips past it. Ask the question while the answer is
still true.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
(cherry picked from commit 768e934b27c38e422857820fea49dc537d403fbb)
The Macros screen showed "No Macros" on a UART-connected pendant even with
macros present. Several distinct causes, all on the preferences.json path.

The macros key was never matched. PreferencesListener is installed part-way
through the document, on the "result" key of the $File/SendJSON wrapper, so its
_level is relative to wherever it took over: preferences.json puts
"settings"/"macros" at _level 1 there, not 2. The check was pinned to
_level == 2, and the _level < 2 bail above it swallowed the key before it was
ever tested. Match it at whatever depth it appears, and record that depth so
entries are the objects exactly one deeper -- nested structure inside an entry
must not be mistaken for an entry.

Per-entry state was never reset. Unlike the other two listeners,
PreferencesListener::startObject() did not clear _name/_target/_filename, so an
entry missing "action" inherited the previous entry's filename -- which had
ALREADY had its "/localfs/" prefix inserted -- and was prefixed again, showing
as "//localfs/". An entry missing "type" inherited the previous type and was
added when it should have been skipped.

Only one key spelling was accepted. WebUI versions disagree: older exports use
filename/target (what MacroListListener expects), newer ones action/type. Accept
either, treat ESP as a localfs target alongside FS, normalise a leading slash
before prefixing so neither convention yields "/localfs//foo.g", and skip
entries with an empty action rather than adding a row that does nothing.

Auto-reporting corrupted transfers. $RI keeps emitting <Idle|MPos:...> status
lines while a file streams on the same channel, so any byte lost to a stall
welds the two together and a chunk ends up carrying the tail of a status report.
Suspend it for the duration of $File/SendJSON, $Files/ListGCode and
$File/ShowSome. The resume is keyed on "no document in flight and the last
request is old enough" rather than on the terminal branches, so it comes back
down every route home, including for the list and preview transfers which have
no chain state of their own.

The chain could stall forever. With show_error() no longer advancing on an error
that lands mid-document, a genuinely torn transfer left the menu on "Reading
Macros" indefinitely. service_macro_chain(), polled from dispatch_events(), puts
a deadline on an in-flight request.

Trace output is now buffered and flushed only when nothing is streaming.
Printing inline was self-defeating: dbg_print blocks waiting for USB buffer
space, and 50 ms of blocking at 1 Mbaud is ~5 KB of arriving UART data, more
than the RX ring holds -- so the act of tracing overflowed the ring and
destroyed the transfer being traced.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
(cherry picked from commit d6b2bd9396188f7cbabf4eaeec4568ed97ac8ada)
A "cmd:" macro carries a command line rather than a path, so there is no file
behind it. invoke() with no argument means "open the file preview", which left
the green button and touch doing nothing at all for a command macro -- it was
reachable only from the dial. onGreenButtonPress() also returned early unless
state was Idle, and $Job/Resume and friends exist precisely for when the machine
is NOT idle, which is the one state that guard refuses.

Give MacroItem an is_command() accessor, run command macros directly from the
green button and touch, and do not gate them on Idle. A command macro is a line
of text on the channel; FluidNC rejects what it will not accept in the current
state, so blocking it in the pendant adds nothing. File macros keep the existing
Load -> Run flow, gated on Idle, unchanged.

The legend follows: a command macro shows Run on both buttons and keeps them
offered while the machine is busy.

Also handle the cmd: prefix in invoke() itself -- splitting the body on newlines
and ';' so a multi-statement macro is sent as separate lines, and skipping the
preview scene, which has nothing to show for a command.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
(cherry picked from commit 6114f331e690650e4bfd7949ab94edf239cf8bf7)
@f1adang

f1adang commented Sep 9, 2026

Copy link
Copy Markdown
Author

Yay, more vibe contributions ahoy! 🤪

PR containing some fixes against current main I discovered while debugging some issues with Brushograph remote control

@f1adang
f1adang marked this pull request as ready for review September 9, 2026 21:55
@f1adang

f1adang commented Sep 11, 2026

Copy link
Copy Markdown
Author

If you'd rather have this in separate PRs per commit, let me know

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants