Commit b4469ef
authored
fix(dns): publish Technitium cluster port 53443 on the secondary (#3067)
* feat(benchmarks): full-roster cold re-run, and enforce the cold protocol in the driver
The matrix mixes two regimes, and #1947 rule 2 says two vintages must never
share a table:
phases 1+2 89 models contended (hp-llm-worker live), N=2 + escalation
phase 4 5 models cold, workers stopped, N=3, every cell +-0
The #3023 cold probe showed contention did not move scores -- a fully resident
control reproduced [63,63] exactly, and an escalated spilling cell resolved to
the majority the protocol had already chosen. So this is not a correction of
wrong numbers. It makes the regime uniform, which matters because the top of the
field is separated by ONE point across sixteen models, and a promotion decided on
that margin cannot rest on a mixed protocol.
N=2, not N=3. Part 4 found "22.1 of the 37.4 model-wall hours re-derived
identical bytes" and concluded repeats belong on the axes that are NOT fixed.
#3036 escalates 2 -> 3 -> 5 automatically wherever runs actually disagree, so
genuine variance is caught without a third run on every deterministic cell --
about a third cheaper for the same information.
sweep_extra.sh gains two things it should always have had:
- STOP_WORKERS stops hp-llm-worker and ghidra-revdeck-1 for the run and restores
them via a trap on EXIT/INT/TERM. #3023's finding was not that contention broke
the scores, but that the protocol depended on an operator remembering; this
moves it into the driver. The trap covers INT/TERM because this script is
routinely killed between models.
- KEEP_WEIGHTS_ABOVE_GB gates the post-model `ollama rm` on free space. That
removal was correct at 92% full and is actively harmful with terabytes free: it
destroyed all ten of the requant plan's source models, and it is the reason a
cold re-run of 89 models has to re-download 80 of them.
Results go to 1947cold/, not over 1947full/. The contended numbers are not
garbage: contended-vs-cold across 89 models is the largest evidence anyone will
have on whether the regime matters, and discarding it to save disk would repeat
the mistake that lost the weights.
chain_cold.sh waits on a POSITIVE condition -- every tag in models_requant.txt
has both tier files -- rather than "the GPU is idle". chain_phase3.sh is already
armed on the same card; two idle-checks would fire in the same poll window and
double-book it. coldrun.sh keeps its own running-sweep guard as a second line,
because the failure mode is a wasted day rather than an error message.
Guards verified live: coldrun aborts while phase 2 runs, chain_cold aborts at
0/7 ladder tags, and the combined roster de-duplicates to 96 unique entries
across the phase 1, phase 2 and ladder lists.
Refs #1947, #3023, #2245, #3031, #3036, #2641
* fix(dns): publish Technitium cluster port 53443 on the secondary
The homeserver node published only 53/tcp, 53/udp and 5380/tcp. The cluster
replication port was never published -- not in this file nor in any backup of
it -- so the primary could not reach the secondary and reported
Error! No route to host (supermicro.cluster.xore.lan:53443)
and replication was silently dead: settings changed on the primary never
reached this node.
"No route to host" rather than "connection refused" because firewalld REJECTs
it; 53 and 5380 are reachable only because Docker's DNAT/FORWARD rules bypass
the firewalld zone, whose public zone allows just cockpit, dhcpv6-client and
ssh. The image merely EXPOSEs 53443, and an exposed-but-unpublished port never
binds on the host in bridge mode. The asymmetry hid it: the primary runs with
network_mode: host, so its own 53443 listens on all interfaces and the reverse
direction looked healthy.
Publishing through Docker bypasses firewalld the same way the existing ports
do, so no zone change is needed.
Verified after recreating the container: 53443 accepts connections from the
primary, TLSv1.3 handshake completes, and the secondary picked up the primary's
state within 15 seconds -- updates.maxmind.com went from NXDOMAIN to NOERROR
there, which in turn cleared hp-geoipupdate (healthy, FailingStreak=0).
Closes #30661 parent 162925a commit b4469ef
1 file changed
Lines changed: 11 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
29 | 29 | | |
30 | 30 | | |
31 | 31 | | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
32 | 43 | | |
33 | 44 | | |
34 | 45 | | |
| |||
0 commit comments