From 0e461e5242e011e3ca3435baa71f566f7c414595 Mon Sep 17 00:00:00 2001 From: Alfie Date: Thu, 30 Jul 2026 18:14:27 +0800 Subject: [PATCH] docs: iDRAC unreachable (no IP), gps3 smartd verified, archive push planned MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit iDRAC: BMC alive but IP 0.0.0.0 โ€” DHCP got nothing. Blocks the patrol-read check (no vendor CLI available either) and, more urgently, means there is no remote recovery path before the pending kernel reboot that will also be the first real test of the rewritten fstab. Includes the ipmitool fix and the security steps that must precede it. smartd: complete and proven โ€” 16/16 members, alert path tested end to end into both sinks. Concedes that gps3 was right about -m root: -M exec replaces mailing, so 10mail failing is noise not loss. Records their honestly-flagged real gap: no long self-tests, on unverified patrol-read reasoning. Archive push: planned, not started. Three corrections from verifying on the T420 โ€” remount,ro is unreliable on fuseblk, the scope is 4 provenance-named directories (~157G) not one, and it is ~4.4h at the measured 10 MB/s over a wifi link that has dropped twice today. --- RESUME_NEXT.md | 96 ++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 96 insertions(+) diff --git a/RESUME_NEXT.md b/RESUME_NEXT.md index 546b616..df776e8 100644 --- a/RESUME_NEXT.md +++ b/RESUME_NEXT.md @@ -8,6 +8,102 @@ diagnosis corrected), 07-15 (migration/scan kicked off), 07-14 (clean shutdown, VADASE PRs, EVACUATE verdict), 07-13 (RAW done), 07-08 (freeze), 07-07 (excavation+crossref), 07-04 (DA-005).** +## ๐Ÿ”ด 2026-07-30 (late) โ€” iDRAC HAS NO IP; blocks two things + +`sudo ipmitool lan print 1` on gps3: + +``` +IP Address Source : DHCP Address +IP Address : 0.0.0.0 <-- asked DHCP, got nothing +Default Gateway : 0.0.0.0 +MAC Address : b0:7b:25:fe:2c:38 (all four OS NICs are e4:3d:1a:... โ€” + confirms a genuinely separate NIC) +``` + +The BMC is alive (`/dev/ipmi0`, 5 ipmi modules loaded, PowerEdge R740) but has +**no network address**, so iDRAC is unreachable. Either the dedicated port is +uncabled or it is on a VLAN with no DHCP. **Consequences:** + +1. **The patrol-read question cannot be answered** (see gps3's session log + ยง12.4) โ€” and no vendor CLI exists on the box to answer it another way + (`storcli`/`perccli`/`megacli` have no apt candidate; backplane not exposed + via SES). If patrol read is disabled then **nothing performs a full-surface + read of the 16 members**, and a latent bad sector surfacing during a rebuild + is exactly what kills 16-wide RAID 5. +2. **No remote recovery before the pending reboot.** Kernel `6.8.0-136` is + installed, `6.8.0-111` running; that first reboot is also the first real test + of the rewritten fstab. If it drops to an emergency shell, SSH cannot help โ€” + SSH needs a booted OS. **Fix iDRAC BEFORE rebooting**, or accept a trip to + the server room. + +Fix (needs sudo on gps3; `ipmitool` is in the Ubuntu archive): +```bash +sudo ipmitool lan set 1 ipsrc static +sudo ipmitool lan set 1 ipaddr 192.168.48. +sudo ipmitool lan set 1 netmask 255.255.255.0 +sudo ipmitool lan set 1 defgw ipaddr 192.168.48.5 +sudo ipmitool lan set 1 access on +sudo ipmitool user list 1 # then CHANGE PASSWORDS +``` +โš  **Security first:** iDRAC has full control of the server (power, console, +boot device). Dell's default login is well known and unprotected BMCs are a +standard target. Change the password and the SNMP community string (currently +the factory default `public`) BEFORE it touches a network, and keep it off +anything internet-facing. Auth is MD5-only on this firmware. + +## โœ… 2026-07-30 โ€” gps3 smartd COMPLETE, alert path proven + +All 16 members monitored (`smartd -q onecheck` = 16/16), `20log` hook installed, +and the alert path **tested end to end** โ€” a synthetic alert landed in both +journald and `/var/log/smartd-alerts.log`. That test mattered: the log was 0 +bytes beforehand, so the path had never actually run. + +**Conceded to gps3 on `-m root`:** `COORDINATION.md` ยง7 told them not to use it. +They used `-m root -M exec smartd-runner` anyway and were right โ€” `-M exec` +*replaces* mailing rather than supplementing it, `smartd-runner` run-parts +`run.d/` where `20log` writes both sinks, and `10mail` merely exits 1 (noise, +not lost alerts). It is also the distro-idiomatic form. ยง7's substance โ€” an +externally observable non-mail sink, proven working โ€” is met. + +**Real remaining gap (theirs, honestly flagged):** no long self-tests are +scheduled at all, only daily staggered short tests. The justification (PERC +patrol read covers surface scanning) is **unverified** and cannot be verified +until iDRAC works. If patrol read is off, add staggered long tests: +`-s (S/../.././NN|L/../../6/NN)`. + +Housekeeping: `sudo truncate -s 0 /var/log/smartd-alerts.log` to clear the +synthetic TEST entry so no future reader mistakes it for a real warning. + +## โธ 2026-07-30 โ€” archive push to gps3: PLANNED, NOT STARTED + +gps3 proposed the push; receiving side is ready (idle, no BPE, 20 TB free). +**Three corrections to that plan, from verifying it on the T420:** + +1. **`mount -o remount,ro` will likely fail.** DOSTB is `fuseblk` (ntfs-3g via + udisks2), and FUSE does not reliably honour `remount,ro`. Verify it actually + took effect (`findmnt -no OPTIONS`) rather than assuming; if it silently + stays rw, that is acceptable โ€” rsync only reads the source โ€” but do not + *believe* the source is protected when it is not. +2. **Scope is 4 directories, not one.** `RECOVERED_SEAGATE_W2A0W9T2_DATA0` + (131 G), `RECOVERED_HD-LBU2_...` (14 G), `RECOVERED_DOSTB20150918_from_BackupPlus` + (9 G), `RECOVERED_GPS_1TB_2_...` (3 G) = **~157 G total**. Decide whether all + four go, and preserve the directory names โ€” they encode provenance + (which drive each was recovered from), which is exactly what an archive + manifest needs. +3. **Duration: ~4.4 h for all four** at the measured **10 MB/s** over + `GNSS_5G2` wifi. That link has dropped twice in one day. Use tmux (their + plan does) and expect at least one resume. A direct cable into one of gps3's + three unused gigabit NICs would cut this to ~30 min โ€” worth considering + before committing 4+ hours to wifi. + +Their flag choices are sound: `-aH` (hardlinks), `--partial` (resumable), and +**no `-z`** because the bottleneck is the drive and GNSS data is already +largely compressed. Agreed on all three. Two additions: +- Census source **and** destination and compare files/symlinks/dirs/bytes โ€” + `rsync` exit=0 is not proof (this project lost a session to that). +- Generate `sha256` manifests after landing. That is the Tier-0 fixity item and + the copy is worthless as a preservation copy without it. + ## โœ… 2026-07-30 โ€” branch split closed, branching policy adopted **PR #57 merged.** `main` advanced `bc7b5b9` โ†’ `11315ee` and now holds the