create a registry of models used in wPY tests. - #1160
Draft
mborodii-prog wants to merge 2 commits into
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add a distinct error for a model that doesn't exist, separate from access-denied
Problem
wrangles/data.py::_raise_model_response_error()mapped API responses to exceptions for401/403, but had no404handling at all - any response outside401/403fell through to a generic RuntimeError('Something went wrong trying to {action} model {id}'), which doesn't distinguish a model that plain doesn't exist (bad ID, typo, deleted model) from a model that exists but the caller has no access to (AuthorizationError,"Access denied to model {id}"`).This was part of investigating
tests/connectors/test_train.py::TestTrainLookup::test_missing_columns_error_messageintermittently surfacingAccess denied to model bc3ee6a0-e104-4700in a credentialed environment - the error message alone couldn't tell you whether that meant "fix the test's mocking" or "this model was deleted and the fixture needs replacing."Fix
Added
ModelNotFoundErrorand a404branch to_raise_model_response_error:This is the client-side half of a companion API-Core change that makes
model_roles_requiredactually return404for a nonexistent model instead of a403indistinguishable from "exists but forbidden." Without that API-Core change, the backend still only ever sends403for both cases - this WranglesPY change is forward-compatible with it (a404from the API is now handled correctly) and is harmless before it (no behavior change for existing403/401responses).Tests
Added
test_model_endpoints_raise_not_found_error_for_404totests/test_data.py, parametrized acrossdata.model,data.model_update, anddata.model_content, mirroring the existing401/403tests. All 12 tests intests/test_data.pypass.Follow-up: model registry + preflight access check
While investigating the original failing test, also added:
tests/model_registry.py- a registry of model IDs referenced across the test suite (seeded by scanningtests/**/test_*.py, 39 entries, with two known-mocked-only IDs excluded and a note that it needs further human review for others).scripts/check_model_registry_access.py- checks access to every registered model up front, using the newModelNotFoundErrordistinction to report "not found" vs "access denied" separately, exiting non-zero if anything is inaccessible..github/workflows/ci.yml, before the main credentialedpytestrun, so a broken model fixture fails fast with a clear report instead of a cascade of individual test failures.tests/test_check_model_registry_access.py- fully mocked unit tests for the checker's categorization logic (4 tests, no network).