Skip to content

swbor bench runs are killed by a spurious ez-FET reset a few seconds after release #72

Description

@byeongjee

Symptom

ckpt bench milp dijkstra --cap 10uF (halt-mode swbor, continuous power over the ez-FET) fails intermittently and leaves the board in the region-violation park (LED1 blinking). The same binary fails ~60% of the time, and the failure point is nearly fixed for a given clock (16 MHz: boundary 63–66 of 95, 8 MHz: boundary 45).

Investigation

  • At the moment boot code is entered, SYSRSTIV == 0 — no hardware reset (BOR/PUC/WDT/…) is recorded. (Sanity check: the BOR build records 0x06 = PMMSWBOR at every boundary.) Registers and SRAM are intact; only the PC has moved to the reset vector. cnt_boundary did not increment, so __region_boundary was not executed either. Only a JTAG/SBW-issued reset has these properties.
  • Ruled out:
    • board unit (two boards, identical behavior)
    • clock/FRAM timing (8 MHz fails as well)
    • the hw_init re-configuration in the recovery path (skipping it still fails)
    • a real BOR once right after the fresh boot (still fails)
    • CPU semantics (the same ELF runs 68 boundaries in mspdebug sim without issue)
    • a code regression: at 993e792 (before the in-region check existed) the same swbor build restarts from a fresh boot at boundary 171 and then runs to completion, so cnt_boundary ends at 430 instead of 259 while __nvm_done=1. The phenomenon was already there; it was silently reported as success.
  • Decisive experiment: with the SBWTDIO/SBWTCK jumpers removed and the run started via the RESET button, the same build completed 5/5; with the jumpers connected it fails ~60%.

Conclusion so far

After mspdebug tilib exits, the ez-FET sends a reset-like event to the target over the SBW lines a few seconds later. In swbor mode the program keeps running without any reset, so this event lands mid-region and is flagged as a violation. The bor mode is not affected, presumably because the PMMSWBOR at every boundary resets the target's debug-interface state. The exact cause on the FET side has not been identified.

Impact

  • swbor bench runs fail/retry intermittently.
  • swbor execution times measured before the in-region check existed may be inflated by silent restarts.

Open questions (goal of this issue)

  • Identify what exactly the ez-FET does on the SBW lines after mspdebug tilib exits (delayed release in MSP430_Close, periodic target detection, …) and why it arrives seconds after the run starts. A Saleae capture on SBWTCK/SBWTDIO aligned with the failure time should show it.
  • Why the bor mode is immune (PMMSWBOR resetting the debug-interface state is a hypothesis, not verified).
  • Fix the root cause once known (e.g. a different release sequence, or a way to make the target ignore the event).

Workaround (until fixed)

For bench/verify, use the Otii switchboard relays to disconnect both the SBW lines and the ez-FET 3V3 rail after flashing, and have the Otii supply 3.3 V during the run (reuse the relay control from the intermittent runner).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions