fix(registration): accept hf artifacts in the validator - #294
Merged
Conversation
`roar put … hf://` stamps the published artifact `source_type="hf"` (put_execution.py, since 0.4.2), the publish path's own allowlist has accepted `hf` since #35, and the receiver has accepted it since June. The validator applied to every primitive artifact registration never learned it: VALID_SOURCE_TYPES has not changed since the initial commit. Nothing surfaced the gap because the failure is quiet. A staged artifact that fails validation is skipped, not rejected -- its message is collected into a warnings list and registration continues -- so an HF publish uploaded the file, reported success, and silently dropped that artifact's source from the lineage. The row still looked green: the output was already registered as a local artifact during the traced step, and the download location rides on a scoped label rather than on source_type. Also derives the rejection message from the set instead of hardcoding it. The old text still read "must be 's3', 'gs', 'https', or None" and would have gone stale again on the next scheme. The test pins the validator's set against the publish path's, so a scheme cannot be half-added a third time. Verified against prod glaas-api on a real delegated publish: previously "Artifact 0 (...): source_type must be ... got 'hf'" followed by "Registration completed with errors"; now a clean publish, 7/7, lineage published. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
roar put … hf://stamps the published artifactsource_type="hf"(put_execution.py:656, since 0.4.2). The publish path's own allowlist_VALID_REMOTE_SOURCE_TYPEShas acceptedhfsince #35, and glaas-api has accepted it since June.VALID_SOURCE_TYPESincore/validation.py— applied to every primitive artifact registration — never learned it, and has not changed since the initial commit.Why nobody noticed
The failure is quiet. A staged artifact that fails validation is skipped, not rejected:
So an HF publish uploaded the file, reported success, and dropped that artifact's source from the registration. Rows still looked green — the output is already registered as a local artifact (
source_type=None) during the traced step, and the download location rides on a scoped label (a514e77) rather than onsource_type. The only symptom is aRegistration completed with errorsline that reads as noise.Changes
"hf"toVALID_SOURCE_TYPES"must be 's3', 'gs', 'https', or None"and would have gone stale again on the next schemetests/core/test_validation_source_types.pypins the validator's set against the publish path's, so a scheme cannot be half-added a third timeVerification
Built a wheel from this branch and ran a real delegated publish to HF against prod glaas-api.
Before —
Artifact 0 (672314521e56): source_type must be 's3', 'gs', 'https', or None, got 'hf', thenRegistration completed with errors.After — clean publish,
Reproducibility 7/7,lineage saved on glaas.ai · 1 files · published to hf://…, job exit 0,lineagePublicationStatus: published.201 tests pass across
tests/coreandtests/application/publish; the new tests fail if the constant is reverted.Worth a separate look
continueon a failed staged artifact means an artifact the operator explicitly asked to publish is dropped inside a run that reports success — the same shape as P0-39. Raising, or at least a non-zero exit, seems more honest. Not changed here.