chore(scripts): add copy_measure_content.py for cross-server measure content transfer - #394
Open
blakenan-bellese wants to merge 1 commit into
Open
chore(scripts): add copy_measure_content.py for cross-server measure content transfer#394blakenan-bellese wants to merge 1 commit into
blakenan-bellese wants to merge 1 commit into
Conversation
…content transfer Copies Measure / Library / ValueSet / CodeSystem from one FHIR server to another as batch Bundles of PUT entries, reusing the shape of fhir_client.push_resources (batch not transaction, so the target does not enforce cross-entry referential integrity). Written to load Lenny's connectathon measure content onto an external MCS. Used to move 185 resources to the CMS connectathon server. Safety properties, each of which came from a bug found while building it: - A read-only collision check runs before any write and aborts without --overwrite when the target holds different content at the same IDs. - Only 404/410 counts as "absent". A first version treated any error as "not there", so an expired token made an unreadable server look like an empty one and reported "185 new, 0 collisions" — a green light to overwrite based on 401s. - A preflight does one authenticated search before real work, because /metadata is frequently unsecured and proves nothing about the token. - Verification is by direct read, not search count. HAPI's index refresh is async; a measured 40s lag on the CMS server made a successful write look like a failure. Search count is still reported, labelled as informational. Secrets are never command-line arguments (argv is world-readable via ps and lands in shell history). The access token comes from $TARGET_FHIR_TOKEN, a 0600 cache file, or an OAuth2 client_credentials grant; client credentials come from env vars or the macOS keychain. Expired JWTs are skipped by `exp`, and a 401 mid-run triggers one refresh-and-retry. Tests cover the token-expiry and collision logic. CI's unit job runs `cd backend && pytest tests/`, so it does not collect scripts/tests/*.py — run them manually when touching the script.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
scripts/copy_measure_content.py— copiesMeasure/Library/ValueSet/CodeSystemfrom one FHIR server to another as batch Bundles of PUT entries, reusing the shape offhir_client.push_resources(batch not transaction, so the target does not enforce cross-entry referential integrity).Related issue
None — utility script extracted from operational work.
Type of change
Checklist
Safety properties
Each of these came from a bug found while building the script, not from foresight:
--overwritewhen the target holds different content at the same IDs, printing each conflict with both versions.185 new, 0 collisionsoff the back of 401s, which is a green light to overwrite. Now 401/403/5xx raises./metadataon the CMS server is unsecured and returns 200 regardless, so it proves nothing about the token.SHORTand exit 1. Direct reads decide pass/fail now; the search count is still shown, labelled as informational when it lags. Consistent with the async-indexing triage rule in CLAUDE.md.Security implications
psand lands in shell history. Thecurl -u CLIENT_ID:CLIENT_SECRETform this replaces leaks both ways.$TARGET_FHIR_TOKEN→0600cache file → OAuth2client_credentialsgrant. Client credentials come from env vars or the macOS keychain.O_CREATat0600so it is never briefly world-readable.expclaim (no signature verification — that is the server's job; we only decide whether it is worth sending). A 401 mid-run triggers exactly one refresh-and-retry.Test plan
Known gap: CI's unit job runs
cd backend && pytest tests/, so it does not collectscripts/tests/*.py. These tests run only when invoked manually. Wiring script tests into CI is out of scope here; flagging rather than leaving it implicit.Dry-run and check-only modes write nothing and are safe to run against any target:
Verified end to end against the CMS connectathon server: 185/185 written in 24.6s, all 185 confirmed by direct read (
--verify-all), and everyMeasure.librarycanonical resolved to a Library present on the target.🤖 Generated with Claude Code