feat(bernese): capture PAGENET_DLY.PCF and PGN_WK panels from the T420 - #65
Merged
Merged
Conversation
The gps3 session log §14.5 records this as the one asset blocking the BRN-001
acceptance test: the orchestrator is ready, $U is provisioned, maxjobs is
corrected and the validator now reads the real DATAPOOL — but PAGENET_DLY.PCF
existed only on the T420, the machine that drove the training week. Captured
byte-for-byte, not re-derived, for the reason that log gives: a re-derived PCF
would be a different PCF, and the acceptance test would then be exercising
something nobody has ever run.
Three of the expectations recorded in §14.5 do not match the actual files.
Checked against the files rather than assumed; details in PROVENANCE.md.
- No dangling WAIT. §14.5 anticipated `599 DUMMY` waiting on `512 514 522`
with 522 undefined. This PCF's 599 waits on `512 514` only; there is no
521/522 in the file and no 9xx tail. The adaptation §14.5 warned against
performing by eye had already been done deliberately. The provisioner's
dangling-WAIT check should pass it unmodified.
- PGN_WK/ADDNEQ2.INP is already sanitized, not pending sanitization. A
2026-07-03 pass converted separators, replaced the C:\Bernese ENVIRONMENT
block, remapped INPFILE from the instructor's 3 demo sessions to the 7 real
PAGENET sessions, and raised MAXPAR 5000 -> 10000. MAXPAR is past the
overflow the readiness notes describe; do not revert it.
- The Windows separators are in MENU.INP, not ADDNEQ2.INP:
SESSION_TABLE 1 "${P}/SOB\GEN\SESSIONS.SES" — instructor's campaign name,
backslashes intact.
Two demo-week literals do survive in ADDNEQ2.INP (COORD line 66 and FREESTA_F
line 935, both $(FIN)_20261040), and need remapping to the session being
processed. Those are the rejections to expect, and them being caught is the
provisioner working.
ADDNEQ2.bck and ADDNEQ2.INP.pre_phaseB_20260703 are not committed — 186 KB
each of pre-sanitization state now recorded in diff form in PROVENANCE.md,
and still on the T420 if the original is ever needed as evidence.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Important Review skippedToo many files! This PR contains 230 files, which is 130 over the limit of 100. To get a review, reduce the PR to 100 files or fewer by splitting it into smaller PRs or changing its base branch. Upgrade to a paid plan to raise the limit. This review couldn't start because sufficient usage credits or metered capacity aren't available. Add credits or update usage-based reviews in the billing tab, then retry. ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (230)
You can disable this status message by setting the Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
alfieprojectsdev
added a commit
that referenced
this pull request
Aug 4, 2026
Eleven findings, all confirmed against real data before being accepted. Two of them defeated guarantees this code advertises in its own docstrings. provision_opt_dir had no dry-run mode. provision_gpsuser.py printed "DRY RUN — nothing will be written" and then wrote every panel, because there was no parameter to ask it not to. Verified against PR #65's gold tree: the default invocation wrote all six PGN_WK panels into $U. Dryness is now a real parameter and the dry run writes zero files. The strict gate was also inverted and its refusal exited 0. strict=apply tied hazard checking to the write mode, so checks were OFF in dry run (which wrote) and ON only under --apply (which did not). Worse, a refusal was returned in the warnings list, so a run that provisioned nothing at all still exited 0 — the sixth instance in this project of a swallowed exit status reporting success. Strictness is now unconditional and refusals are errors; a refusing run exits 1. --stations silently reverted MAXPAR. compute_maxpar is max(1000, N*4+500), which for the ~72-station PAGENET network yields 1000 — overwriting the "10000" the real ADDNEQ2 panel carries, raised deliberately after an observed parameter overflow and flagged in PR #65 as not to be reverted. MAXPAR is a ceiling: it is now raised, never lowered. The existing test asserted the lowering behaviour and has been corrected. add_station_to_campaign.py took the station code from MARKER NAME — the exact bug fixed in the validator hours earlier on this same branch. Against a PAGENET header it derived name="BOGO", dome="PBOG" and would have written "BOGO PBOG" into five reference files. It now calls _resolve_station_code. The DOMES is additionally format-checked rather than taken as "whichever marker field is not the code", which produced "BOGO CITY" as a DOMES number; stations without one are written as a bare 4-char name, matching how PGN.STA already lists them. A partially-added station reported "nothing to do" and returned 0, leaving the campaign inconsistent and RXOBV3 still aborting. Partial adds now refuse loudly, and the presence test is anchored per line rather than a substring search that matched codes inside DOMES numbers. Added a hardcoded_campaign hazard class. PR #65's PROVENANCE.md listed MENU.INP under "expect the provisioner to catch these", but "${P}/SOB" — the instructor's demo campaign — matched no hazard class; the backslashes were auto-converted and the panel reported clean. Both MENU.INP and MENU_CMP.INP are now flagged. Two existing tests had been using that same broken line as their example of a CLEAN panel. require_stations now defaults to True. Fixing _is_rinex_obs removed the instance of the vacuous pass; leaving the safe behaviour opt-in left the failure mode. Production already passes it explicitly, so only direct callers change. One test conflated "this station has no data" with "there is no data at all" by proving the former with an empty directory; it now stages data for a different station. Also: verify_patrol_check.sh verified ~/patrol_check.sh rather than the repo copy this branch fixes, so it could have exercised a stale script and reported the old wrong numbers; inspect_member6.sh redirected as root into a predictable /tmp path open to a symlink attack, now mktemp with a trap; patrol_check.sh reported one arbitrary drive's power-on hours as an array-wide fact, now a cross-member range; and add_station's duplicated .Z reader leaked the child gzip, so it now reuses the validator's context manager instead of keeping a second copy that had already diverged. 194 tests pass, up from 189. ruff clean.
…rence set Abegail (Luzon network) delivered a 22 GB Bernese 5.2 processed set on the DOSTB drive for the 5.2-vs-5.4 comparison. The configuration half is captured here; the data half stays on the drive (PROVENANCE.md §5 inventories it). This is the production workflow of record, not a training example: GPSTEMP holds 1,203 BPE_PHIVOL_REL_* work directories, and PHIVOL_REL.PCF is dated 2025-09-12. It answers two open questions. Session log §14.5 suspected PAGENET_DLY.PCF might be an unsafe by-eye truncation leaving 599 DUMMY waiting on an undefined 522. It is not. PAGENET_DLY (46 PIDs) is a deliberate reduction of this lineage (84 PIDs), with 521/522 removed and 599 rewired to compensate. PAGENET.PCF (2015) and PAGENET2.PCF (2019) are also captured, so the whole lineage is now in the repo. Readiness item M -- decide Module 15/16 scope before building weekly/monthly plumbing -- has a reference implementation: 530 ADD_WK / PHI_WK and 531 ADD_MON / PHI_MO, plus the 901/902/903/991 save-summarise-clean tail that PAGENET_DLY lacks entirely. Operator names scrubbed. Bernese panels carry a free-text "USER" field naming whoever last saved the panel; this tree held four staff names and workstation accounts across 72 files, all now "<operator>". This repo is public and a colleague's name published without her knowing is not something a capture is entitled to do. The field is menu metadata and does not affect processing. Everything else is byte-for-byte, minus 44 *.bck editor backups. Two cautions, both in PROVENANCE.md: - Six LIVE .INP panels (not backups) carry C:\Bernese\... absolute paths -- gap #8, and on Linux a backslash is a literal character, so these resolve to filenames containing backslashes rather than failing loudly. Point provision_gpsuser.py at this tree before any use. The ENVIRONMENT block needs more than separator conversion; U and T are another machine's layout. - The two sets do NOT share models. 5.2 LUZON uses I14 and V_GNSSAR=ALL; 5.4 PAGENET uses I20 and GRE. Baselines and V_CLU agree. I14->I20 is a cm-level mostly-vertical systematic shift, well inside the range a tuning change could produce -- so reproduce her numbers under I14 first, then vary the model deliberately. Otherwise the frame change lands in the results looking like a PCF improvement. Also note the OPT tree is not uniformly a LUZON set: of the live panels, 38 name campaign ${P}/PHIVOLCS, 9 LUZON, 2 SAMR_LYT, 1 EXAMPLE. These are last-saved menu states from different work, not a curated gold standard. Mostly cosmetic since the BPE sets the campaign at runtime, but the directory name overstates what is in it. Version and run method needed no asking: BERN52 carries update_2020-08-27.zip and exe_aiub_64_2021.zip, and the BPE work directories settle GUI-vs-BPE. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
alfieprojectsdev
added a commit
that referenced
this pull request
Aug 4, 2026
Moves README_FOR_GPS3_CLAUDE.md into the repo. It is the T420 session's instructions for the DOSTB20150918 external drive, written immediately before the drive was unplugged and carried to the R740: what is on it, what must not be touched, and why it must be mounted read-only. The operative point is that the drive being on this machine's USB bus removes the 6 MB/s wifi bottleneck that has kept /srv/gnss-archive/legacy empty. 26 GB of RECOVERED_* is minutes rather than a day. That is the highest-value thing available while it is attached, and it is the only irreversible one — the drive is currently the sole complete copy of material rescued from three dead or dying disks. Two corrections to the session log, both found by the T420 rather than by me: Section 15.4 claimed "all work reached main". It had not and has not. origin/main is still at 1d1082e (PR #60); PRs #61-#65 are all open. Rule 5 says to verify origin/main actually advanced rather than infer it from a successful push, and this was written in the section meant to be the authoritative end-of-session state. Section 14.5 predicted that PAGENET_DLY.PCF, being RNX2SNX truncated at 514, would leave 599 DUMMY waiting on an undefined 522. Now that the real file is available, that is wrong: its 599 waits on 512 514, there is no 521/522, and find_dangling_waits reports zero. The reduction was done properly. The advice to capture rather than re-derive survives, but on the weaker and more honest ground that the captured file is the one actually validated and the 9xx tail involves choices a reconstruction would guess at. One further note on method. The script applying these two corrections printed "both corrections applied" unconditionally, and the second replacement had silently failed to match on a backtick. A success message from something that verified nothing — the same defect the log catalogues five times in section 15.5, committed while correcting an entry about it. Both edits are now confirmed present by grep.
Both provenance documents understated the hazards they exist to record, and both did so in the direction of false reassurance. gpsuser52-luzon/PROVENANCE.md said "Six live .INP files carry C:\Bernese\... absolute paths" and listed six. Measured by running sanitize_panel_text over all 105 live panels in that tree: 50 carry foreign absolute paths across 200 lines, and 72 of 105 carry a hazard of some kind (820 hardcoded_campaign, 200 foreign_abs_path, 95 hardcoded_date). An eight-fold understatement in the hazard document is worse than no document, because someone remediating six panels stops after six and believes the tree is clean. All 50 are now listed, with the command to reproduce the census. gpsuser/PROVENANCE.md listed MENU.INP under "expect the provisioner to catch these". It did not. sanitize_panel_text returned changed=True, warnings=0 for that file: the backslashes were converted and the panel reported clean, because the hardcoded campaign SOB matched no hazard class. A hardcoded_campaign class was added in PR #64 to close it, and MENU_CMP.INP — absent from the list entirely — turns out to carry the same defect. Merges origin/main into this branch, which is what made the second finding verifiable: checked on the branch alone, the panels reported clean, because the branch predated the sanitizer that detects the hazard. Verifying a claim about tooling against the wrong version of that tooling produces exactly the reassurance the claim was supposed to justify. Verified on the merged tree: MENU.INP and MENU_CMP.INP raise 2 hardcoded_campaign warnings each, ADDNEQ2.INP raises 20 across three classes, provision_gpsuser.py --apply refuses the tree and exits 1, and the 194 bernese-workflow tests pass.
alfieprojectsdev
added a commit
that referenced
this pull request
Aug 5, 2026
…ions do not overlap Cherry-picked from 6399c9a, written by the T420 session. Recovered and re-landed here because it was stranded: it was authored 2026-08-05 08:10 on parent bd0929f, the PR #65 head *before* the hazard-count correction, and PR #65 had already merged at 40f0409 the previous evening with the branch deleted. The T420 recreated the branch and pushed there, leaving two divergent corrections to the same file -- main had the six-to-fifty hazard fix without the overlap fix, and the recreated branch had the reverse. They merge cleanly; both are now present. Original message follows. PROVENANCE.md called SOL/ "the comparison target" next to the DATAPOOL RINEX, implying they cover the same days. They do not: RINEX in DATAPOOL/LUZON/ 2025 DOY 121-151 (31 days) Converted OBS/ and F1_ finals 2025 DOY 029-033, 2026 DOY 106-110 Solutions for DOY 121-151 none The raw data on that drive was never processed into the solutions on that drive, so "reproduce her SOL/ numbers" cannot be run against the RINEX we have. The viable set is the ten days where OBS/ and F1_ coincide. Worth recording how the error was made. Both facts were observed correctly and written down separately -- the DOY range in one inventory, the F1_ filenames in another -- and never compared against each other. Two accurate observations, one wrong conclusion, produced by holding them apart. (The AI co-author trailer on the original was dropped: CLAUDE.md forbids Claude or AI references in commit messages. Authorship is unchanged.)
alfieprojectsdev
added a commit
that referenced
this pull request
Aug 5, 2026
Corrects both the runbook and the PROVENANCE correction landed an hour ago, and lands the two T420 reply files. SAVEDISK/2025/SOL holds all 365 daily finals for 2025, gzipped, including every day of DOY 121-151. Verified on gps3: 730 files, 365 distinct DOYs, 31 of 31 days in the window. CAMPAIGN/LUZON/SOL looked empty of them because PHIVOL_REL.PCF archives to SAVEDISK with 902 R2S_SAV and then cleans the campaign with 903 R2S_DEL. A campaign holding only the most recent days is normal operation. The usable window is therefore 31 contiguous days from raw RINEX, not ten from converted observations. That is materially better: it exercises RXOBV3, station matching and QC, so this exercise can produce BRN-001 acceptance evidence, which the OBS path could not. New finding neither session had: seven IGS fiducials in her DOY 121 solution have no RINEX locally. F1_251210.SNX carries 30 stations, the DOY 121 RINEX carries 24, and AIRA ALIC BASC DAEJ DARW MCIL PNGM are absent. DATAPOOL_IGS holds only orbit products. A run from local RINEX alone would process 24 stations against a 30-station reference and measure the wrong thing. This is the same gap as one of the eight missing scripts: FTP_DWLD is exactly what fetched them. Three passes over one question, each more careful and each wrong. The DOY ranges were catalogued but never compared; then compared, but only within one directory; then answered by asking where the software writes finished solutions -- which the PCF had stated plainly, and which both sessions had read while checking something else. Also recovers commit 6399c9a from the T420, which was stranded: authored on the pre-correction PR #65 head after that PR had merged and its branch was deleted. The branch was recreated, leaving two divergent corrections to one file. They merge cleanly and both are now present, with the AI co-author trailer dropped per CLAUDE.md.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes the asset gap recorded in
docs/gps3-sessions/SESSION_LOG_20260729_storage.md§14.5 / §15.3 — "
PAGENET_DLY.PCFexists only on the T420 … the only thingbetween here and an acceptance test." Captured byte-for-byte from
/home/finch/GPSUSER/, the machine that drove the training week.Based on
main, not on #64. The files are disjoint from that PR(
README.md,SCRIPT/pagenet_pcs.pl), so either order merges cleanly.Three §14.5 expectations that the files contradict
Checked against the files, not inherited from the prose. Full detail in the new
config/bernese/gpsuser/PROVENANCE.md.No dangling
WAIT. §14.5 expected599 DUMMYwaiting on512 514 522with 522 undefined — that describes stock
RNX2SNX.PCF. Here 599 waits on512 514, there is no 521/522 in the file, and there is no9xxtail. Thetruncation §14.5 warned against doing by eye had already been done
deliberately. The provisioner should accept this PCF unmodified.
PGN_WK/ADDNEQ2.INPis already sanitized. A 2026-07-03 pass convertedevery separator, replaced the
C:\Bernese\…/"USER" "LAB-06"ENVIRONMENTblock, remapped
INPFILEfrom the instructor's three demo sessions(
20261030/40/50) to the seven real PAGENET sessions (20260840–20260900),and raised
MAXPAR5000 → 10000. That last one is past the ADDNEQ2parameter overflow the readiness notes describe — please do not revert it.
The Windows separators are in
MENU.INP, the one panel that pass missed:SESSION_TABLE 1 "${P}/SOB\GEN\SESSIONS.SES"— backslashes intact, andSOBis the instructor's campaign name.
What will still be rejected, correctly
Two demo-week literals survive in
ADDNEQ2.INP, both pointing at DOY 104 ratherthan any PAGENET session:
COORD(line 66) andFREESTA_F(line 935), both$(FIN)_20261040. PlusMENU.INPabove. Remap, then re-run — the rejection isthe tool working.
Not included
ADDNEQ2.bckandADDNEQ2.INP.pre_phaseB_20260703— 186 KB each ofpre-sanitization state, now recorded in diff form in
PROVENANCE.mdand stillon the T420 if the original is ever needed as evidence.
Note for the gps3 session
§15.4 states "All work reached
mainthrough PR #64." It has not —origin/mainis still at
1d1082e(PR #60), and #61, #62, #63 and #64 are all open. Wortha look given Rule 5 is specifically about verifying that
origin/mainactuallyadvanced rather than trusting the command.
Second commit (
bd0929f) — the Bernese 5.2 LUZON production setAbegail delivered a 22 GB processed run of the LUZON network under Bernese
5.2 for the 5.2-vs-5.4 comparison. Configuration captured to
config/bernese/gpsuser52-luzon/; the data half stays on the drive and isinventoried in that tree's
PROVENANCE.md§5.This is the production workflow of record —
GPSTEMP/holds 1,203BPE_PHIVOL_REL_*work directories, andPHIVOL_REL.PCFis dated 2025-09-12.It closes two open questions
PHIVOL_REL.PCFhas 84 PIDs including the
521/522R2S_RED branch.PAGENET_DLY.PCF(46) isa deliberate reduction of this lineage with 599 rewired to match — not an
unsafe truncation.
PAGENET.PCF(2015) andPAGENET2.PCF(2019) are alsocaptured, so the whole lineage is now versioned.
implementation:
530 ADD_WK/PHI_WK,531 ADD_MON/PHI_MO, plus the901/902/903/991save-summarise-clean tailPAGENET_DLYlacks entirely.⚠ Two cautions before anyone runs this
.INPpanels carryC:\Bernese\…— gap 🧹 Code Health: Remove commented import in bootstrap_v2.py #8. On Linux a backslashis a literal character, so they resolve to filenames containing backslashes
rather than failing loudly. Point
provision_gpsuser.pyat the tree first.The
ENVIRONMENTblock needs more than separator conversion.V_GNSSAR=ALL;5.4 PAGENET uses I20 and
GRE. Baselines andV_CLUagree. I14→I20 is acm-level, mostly-vertical systematic shift — comfortably inside the range a
PCF tuning change could produce. Reproduce her numbers under I14 first, or
the frame change lands in the results looking like a tuning win.
Also worth knowing: the OPT tree is not uniformly a LUZON set — of the live
panels, 38 name campaign
${P}/PHIVOLCS, 9LUZON, 2SAMR_LYT, 1EXAMPLE.Last-saved menu states from different work, not a curated gold standard. Mostly
cosmetic since the BPE sets the campaign at runtime, but the directory name
overstates what is in it.
Privacy change — the one non-byte-for-byte edit
Bernese panels carry a free-text
"USER"field naming whoever last saved them.This tree held four different staff names and workstation accounts across 72
files. All replaced with
"<operator>", at the repo owner's direction.This repo is public. A colleague's name published without her knowing is not
something a capture is entitled to do, and the field is menu metadata that does
not affect processing. Everything else is byte-for-byte, minus 44
*.bckeditor backups.
Version and run method — no follow-up needed
BERN52/carriesupdate_2020-08-27.zipandexe_aiub_64_2021.zip; the 1,203BPE work directories settle GUI-vs-BPE. Neither had to be asked.
A separate handover for the gps3 session is on that machine at
~/README_FOR_GPS3_CLAUDE.md— the drive is being carried to the R740's USBport, which unblocks §13.6:
/srv/gnss-archive/legacyis still empty andthe transfer was blocked on a 6 MB/s wifi push. 26 GB of
RECOVERED_*at USBspeed is minutes.