Add a local interface, served by the package itself - #18
Merged
Merged
Conversation
You wanted a GUI a lab could install and use on its own machine. What existed could not do that: it was a static page that read JSON someone dragged into it, with no way to reach a drive or run anything. A browser page cannot, so the interface is now served by a small local server that calls the same functions the command line does — one implementation of the standard rather than two. ndos gui That binds to the loopback address, prints a URL and opens it. Nothing is exposed to the network and nothing leaves the machine. Because this serves a program that will read any path it is pointed at, it is treated as that: loopback only, a token generated at startup and given only to the page it opened, a check that the request claims a local Host, and refusal of anything carrying an Origin from elsewhere — which is what stops a site you happen to have open from reading your disk. All four are tested. The interface itself is the React app, converted to a plain single-page build. It was written on TanStack Start, whose default build produces a Node server and whose static export was broken inside a third-party wrapper, which is why it had never deployed. The routes turned out to use no server features at all, so dropping Start, Nitro and the service worker left all of the components intact and produced a static bundle. That bundle is committed, so installing the package is the whole install: no Node, no npm, no build step, and the zero-dependency promise holds. The field definitions are now generated from ndos_table rather than copied by hand. The copy had drifted: the interface offered "male" and "female" where the standard says F and M, and could not express a lesion, a stimulation, "other" or "unknown" at all — a lab doing lesion studies had no way to record one. A test fails if the checked-in file stops matching. Two things found by testing rather than assuming. The first wheel contained no interface files whatsoever, because a wheel only carries what belongs to a package and it was a loose directory; installing would have given a working command line and an interface reporting itself missing. And scan could report progress only to a terminal, which for a three-hour scan would have left the page looking dead; it now accepts a callback, and the page shows a live rate and time remaining. Adds 29 tests; suite now 348.
The interface ships built so that installing the package is the whole install. The cost is that the shipped copy can fall behind its sources with nothing to show for it: the bundle is unreadable and its filenames are hashes, so an interface a version out of date looks exactly like a current one. So it records what it was built from. scripts/vendor_gui.py builds, copies the result into the package, and writes a fingerprint of the sources beside it; --check recompares and fails when they diverge. The check needs no Node, so it runs everywhere the tests do rather than only where a toolchain happens to be installed. CI additionally builds from source and diffs the result against the committed copy. Typechecking found two real faults, neither of which a build would have reported, because vite does not typecheck: The router was never handed the query client its root route declares. It was putting a provider around the whole tree with undefined in it, so the first component to ask for a query client would have thrown. Nothing asks yet, which is the only reason it had not. And the field adapter set optional properties to undefined explicitly, which is not the same as leaving them out. Three other things found while writing the instructions: The API was fetched at a relative path. Opened at /local/ rather than /local, every call the page made would have resolved to /local/api/ and 404ed. It is absolute now, which is where the server actually mounts it. The startup banner was not flushed, so running with the output redirected printed nothing until the server stopped -- and the address it prints is the only way in, since it carries the token. And the dev server could not do anything: no token, and /api pointing at itself. It now proxies to a running `ndos gui` and injects that server's token, so the page can be worked on against real data. Documented in gui/README.md, which was two lines. Also removed the error reporting left over from the editor this was started in. It forwarded errors to a vendor's global hook that only exists inside their preview, so it was dead here -- but a tool that reads research drives should not carry a reporting path nobody asked for. README, QUICKSTART and RELEASING now cover `ndos gui`, including the one release step no test can perform: rebuilding the interface, which needs Node. The tests will say it is needed; only a rebuild fixes it. 348 -> 350 tests.
Two faults the first CI run found, both in yesterday's commit. The fingerprint hashed the source files as they sit on disk. Git hands them to a Windows checkout with CRLF and to every other one with LF, so the same commit fingerprinted differently depending on where it had been cloned, and both Windows jobs failed against a tree nobody had touched. Line endings are normalised before hashing now. A CRLF checkout produces the same fingerprint as this one. And `npm ci` could not install: package.json and package-lock.json disagreed about ajv, and had before any of this work -- it was invisible because an existing node_modules made `npm run build` succeed anyway. Anyone cloning this and running the documented commands would have hit it. Only dev tooling moved, and the built bundle is byte-identical.
The fingerprint sorted Path objects. Comparing them folds case on Windows and nowhere else, so the same files were hashed in a different order there and the two platforms disagreed about an untouched tree. Sorting by the path as text fixes it: the Windows ordering reproduces the fingerprint CI reported, and the text ordering reproduces this one, which is unchanged. Both Windows failures on this branch were this check misfiring, not the interface being stale.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A GUI a lab can install and use on its own machine.
Why the old one could not be that
It was a static page that read JSON someone dragged into it — no way to reach a drive or run anything. A browser page cannot do either. So the interface is now served by a small local server that calls the same functions the command line does, rather than reimplementing the standard in TypeScript.
Binds to loopback, prints a URL, opens it. Nothing is exposed to the network.
The query page was giving a wrong answer
This is the part worth reading.
The page decided for itself which sessions matched, by comparing strings in the browser. A session whose species had never been entered compared unequal to
mouseand was dropped. So it answered a question about missing evidence as though it were a question about contradicting evidence — and showed a cohort cleaner than the data supports. That is the one mistake this project exists to prevent.It happened because the standard was implemented twice: once in Python, once in TypeScript. The query now runs where the rules live. The page sends constraints and renders what comes back — matched, excluded, and cannot be ruled out — with the evidence that placed each session in its group.
On a real project:
The old page showed
0 of 11, which reads as none of these are mice. None of them are anything yet. Nobody has said.Four new endpoints, all read-only
openlinktable check --emitwrites a file; nothing here needs that file to existcheckquerybrowsenow offers files as well as folders, so a manifest is chosen the way a person refers to it: by name. Every other page follows — Manifest opens from disk, Validate checks the tables a project already has, and the home page no longer calls itself browser-only, because it reads drives now.Security
This serves a program that will read any path it is pointed at, so it is treated as one: loopback only; a token generated at startup and given only to the page it opened; a local
Hostrequired; any foreignOriginrefused — which is what stops a site you have open in another tab from reading your disk. All four are tested.Keeping the shipped copy honest
The interface ships built, so installing the package is the whole install: no Node, no npm, no build step. The risk is that the bundle falls behind its sources invisibly — hashed filenames, unreadable contents.
scripts/vendor_gui.pyrecords a fingerprint ofgui/beside the build and a test fails when they diverge. The check needs no Node, so it runs everywhere the tests do; CI separately builds from source and byte-diffs against the committed copy, which passes.The drift is fixed at the source
Field definitions are generated from
ndos_tableinstead of copied by hand. The copy had drifted: the interface offeredmale/femalewhere the standard saysF/M, and could not expresslesion— a lab doing lesion studies had no way to record one.What testing found rather than assuming
scanreported a file count climbing towards no total and no time remaining.progressasks for terminal output, but was also silently deciding whether to count the work up front — so a page passingprogress=Falseplus a callback got neither. Now3,598 of 28,345 · ~7s left.vite buildcannot report because it does not typecheck: the router was never handed the query client its root route declares, so it wrapped the tree in a provider containingundefined; and the field adapter set optional properties toundefinedrather than omitting them./local/would have 404ed every call.ValueError.npm cicould not install:package.jsonandpackage-lock.jsondisagreed aboutajv, and had before any of this work. Invisible because an existingnode_modulesmadenpm run buildsucceed.Pathobjects (case-folded on Windows only). Both fixed by reproducing the exact fingerprints CI reported.Also removed
The error reporting left over from the editor this was started in — a vendor's global hook, dead outside their preview. A tool that reads research drives should not carry a reporting path nobody asked for.
Verified
Every page rendered in a real browser, headless: all four mount, every picker lists actual directories, and each endpoint answers over HTTP. A query nobody can parse comes back as words, not a stack trace. A wheel installs into a clean venv, pulls in nothing third-party, and serves the page.
319 → 364 tests. All ten CI jobs green.