Repository navigation
Fix Claude --model without a managed catalog - #1037
Merged
Merged
Conversation
JTSIV1
approved these changes
Oct 8, 2026
JTSIV1
left a comment
JTSIV1
left a comment
Collaborator
There was a problem hiding this comment.
this makes sense to me. thanks!
andy-xu-db
enabled auto-merge
October 8, 2026 15:14
GirishS2093
pushed a commit
to GirishS2093/unity-gateway
that referenced
this pull request
Oct 11, 2026
…atabricks#930) ## Summary Add fresh-launch journeys and dedicated CUJ7 model discovery/inference coverage. Tracks AIGTWY-4882. - Keep fresh `--workspace` file tasks and `--provider` launch checks in the Full agent suites. Provider cases consume existing services and do not perform inference. - Add five CUJ7 journeys / eight collected cases: configured Claude picker discovery, configured Codex app-server discovery, fresh scoped file tasks for both agents, and preservation of preexisting Claude OS-managed family defaults during explicit model selection. - Exercise Claude's UG-owned `--model` before `--` and native argument forwarding after it. Codex covers model flags before the separator and within native `exec`. Require completed file answers and model attribution, not just startup or routing banners. - Rebase onto databricks#1040 and reuse its shared bearer, cleanup, served-response checks, and absolute-path Claude prompts. `test_cuj7_model_discovery.py` defines `CUJ_NAME`, so automatic CI discovery runs it as `E2E CUJs · CUJ 7 · Unmanaged model discovery` without workflow changes. - Keep CUJ7 sessions function-scoped for fresh-launch coverage; the shared class fixture requires a published config, while this workspace must remain unmanaged. Preserve existing test names. No production changes, fixture provisioning, or remote configuration mutations. ## Workspace prerequisites CUJ7 is pinned to `https://dbc-14e376e8-6541.cloud.databricks.com` (workspace `7474651332480761`). It must publish no CodingAgentConfig, expose discoverable `system.ai` models, and retain `ug_e2e.models.claude_haiku`, `ug_e2e.models.claude_sonnet`, and `ug_e2e.models.gpt_luna`. The shared `UG_CUJ_SP_CLIENT_ID` / `UG_CUJ_SP_CLIENT_SECRET` identity needs workspace access, read access to config/model discovery, and use access to those services. Only a short-lived bearer reaches agent processes. Each case checks that the workspace remains unmanaged. The preservation cases seed distinct Sonnet and other-family defaults on a clean disposable runner, assert Sonnet inference and unchanged defaults, then remove that local input. ## Regression verification: before and after databricks#1037 Ran the **same post-fix component test files** (`tests/test_cli.py`, `tests/test_agent_claude.py`, and `tests/test_agent_codex.py`) against isolated source snapshots of the merged databricks#1037 fix and its immediate parent. Confirmed imports came from the intended snapshot in each run. | Source revision | Identical updated component suite | | --- | --- | | Before fix: `abf85db12d39c6c78a30b503cc0c7632d26245c6` | **957 passed, 6 failed** | | After fix (databricks#1037): `de4b119d9e8678131b0e6bd15addeb136164d72f` | **963 passed** | All four `test_unmanaged_claude_model_reaches_native_launcher` regressions (Claude/GPT IDs × separate/equals spellings) fail before the fix specifically because the native command omits `--model MODEL`; all four pass after it. The pre-fix revision's original component suite passes **959 tests**, demonstrating that the previous assertions missed this boundary. The older live explicit-model test only placed `--model` after `--`, bypassing UG's model-selection path. Coverage now includes the normal UG-owned option before the separator as well as supplemental native forwarding. **The before/after comparison is component evidence, not a live CUJ or real-inference comparison.** ## Current validation - After rebasing onto databricks#1040 and migrating helpers: **1,192 component/helper/contract tests passed**. - All **eight CUJ7 cases** collect; the CI discovery contract recognizes the file and its unique shard name. - Repository-wide Ruff checks and formatting, source type checks, and `git diff --check`: passed. - Live inference was **not run locally**: this host has `/etc/claude-code/managed-settings.json`, which can override isolated agent settings and model selection. The suite requires a clean disposable runner; CI must validate live inference and SP permissions. --------- Co-authored-by: Isaac <no-reply@databricks.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fix
ug claude --model <id>without a managed catalog by forwarding the exact ID to Claude's native--model. Remove #1004's catalog-gated alias fallback, which could resolve to the OS-managed Opus default instead. Keep selection launch-scoped and preserve family defaults, model restrictions, and #1004's URL fixes.Merged current main and trimmed duplicate catalog permutations and documentation. Production diff: 4 additions / 46 deletions. Regression coverage retains Claude/GPT IDs, both flag spellings, saved settings, managed restrictions, and native argument precedence. The existing live journey now covers both sides of UG's separator and checks JSON
modelUsage.Verification
forkptywarning.7deeaf7a; all four pass with the fix.src/ tests/,ty check src/, runner help, andgit diff --check: passed.scripts/run_integration.pyreports two existing TID251 violations (lines 212 and 242), reproduced on current main. Left out of scope.Screen.Recording.2026-10-08.at.10.56.19.AM.mov