[AIGTWY-4877] Exercise CUJ2 MCP tool use in both agents

tests/README.md

@@ -151,7 +151,7 @@ integration utilities; only CUJ-specific evidence correlation stays in a test fi
 
 | Test | User action | Expected evidence |
 | --- | --- | --- |
-| `test_cuj_configuration`, `test_cuj_codex_inference`, `test_cuj_claude_inference` in `test_cuj2_mps_mcp.py` | Read the permanently preconfigured two-agent MPS/MCP config; run separate configure, Codex TUI, and Claude TUI cases | Configuration verifies both native MPS APIs, generated settings, and sandbox inclusion with web_search excluded; each TUI case records its exact MPS target header and model, paired HTTP 200 response, and final marker. Live sandbox execution is deferred because MAS cannot downscope the CI service principal. No workspace config CRUD is performed |
+| `test_cuj_configuration`, `test_cuj_codex_inference`, `test_cuj_claude_inference` in `test_cuj2_mps_mcp.py` | Read the permanently preconfigured two-agent MPS/MCP config; run separate configure, Codex TUI, and Claude TUI cases | Configuration verifies both native MPS APIs, generated settings, and the `ug_e2e.tools` MCP selector with `read_fixture`; each TUI case records its connected native MCP inventory, calls `read_fixture` with a fresh nonce, checks the SHA-256 receipt in the parent transcript, and records the exact MPS target header/model with a paired HTTP 200 response. No workspace config CRUD is performed |
 | `test_ug_configure_claude_databricks` | Configure Databricks Hosted; execute the generated auth helper; launch plain `ug claude`, read a file, and open `/model` | Generated helper invokes `ug` with clean token stdout; assistant returns an unpredictable file value; native discovery caches `system.ai` models and the picker shows a discovered model without an opt-in flag; normal exit; reopen with working keyboard input |
 | `test_ug_configure_claude_anthropic_mps` | Select Anthropic MPS in the real configure picker; launch Claude | Saved provider in status; completed TUI file task; normal exit |
 | `test_ug_configure_codex_databricks` | Configure Databricks Hosted; execute the generated auth helper; open Codex TUI and read a file | Generated helper invokes `ug` with clean token stdout; completed assistant answer contains the file value; normal exit and reopen |
@@ -227,13 +227,15 @@ Managed discovery expectations come from separate read-only, provider-scoped
 model-list requests; they do not rely solely on ug's generated catalog.
 
 The dedicated-workspace CUJ suite adds three independently runnable CUJ2 MPS+MCP cases:
-one configuration case and separate Codex and Claude TUI task cases. The workspace is
+one configuration case and separate Codex and Claude TUI MCP-task cases. The workspace is
 preconfigured for CUJ2 and is read-only; every case validates it, and the fixture compares
 it again during teardown. The TUI cases forward agent requests through the shared per-test
 loopback recorder to the real workspace and assert the provider-service header, model,
-paired HTTP 200 response, and final answer. The configuration case verifies sandbox MCP
-selection and generated client listings; live sandbox execution remains deferred because
-MAS cannot downscope the CI service principal. Failed TUI runs
+paired HTTP 200 response, and final answer. The configuration case verifies the
+`ug_e2e.tools` MCP selector and generated `fixture_reader` listings. Each TUI case opens
+the native MCP inventory, requires the connected `read_fixture` tool, calls it with a fresh
+nonce, and checks the nonce-derived receipt and actual tool-call record in the parent transcript.
+Failed TUI runs
 retain terminal output/actions/screen and best-effort agent transcript snapshots under the
 runner artifact directory before the temporary homes are deleted. The suite requires
 `UG_CUJ_SP_CLIENT_ID` and `UG_CUJ_SP_CLIENT_SECRET`, with `ug`, `claude`, `codex`, and
@@ -322,7 +324,7 @@ These are unit/component checks; they do not establish live sudo password-prompt
 
 | Scenario | Status / requirement |
 | --- | --- |
-| Broad live MCP functionality | CUJ2 covers sandbox MCP configuration and generated client listings. Live sandbox execution is deferred because MAS cannot downscope the CI service principal. Installation tests cover the local web-search MCP handshake and tool listing, not upstream proxying or a real search request. Other MCP services, live parent/child search, and permission decisions remain deferred |
+| Broad live MCP functionality | CUJ2 covers the preconfigured `ug_e2e.tools` MCP schema, native inventory connection/tool listing, and real `read_fixture` calls with nonce-derived parent-transcript receipts. Installation tests cover the local web-search MCP handshake and tool listing, not upstream proxying or a real search request. Other MCP services, live parent/child search, and permission decisions remain deferred |
 | Dedicated CUJ Skill coverage | Deferred because of a backend storage-path issue; CUJ2 has no Skill configuration, download, invocation, or assertion path |
 | Broad configure flags, multiple workspaces, and PAT flows | Deferred while focusing on basic CUJs |
 | Workspace-switch MCP cleanup | The `workspace_switch` CUJ covers real registration, cleanup, repeat configure, and a completed Claude task. Unit/component tests cover duplicate attempts and injected removal failures; the CUJ does not force an agent timeout. It runs in the required managed CI lane for full/live runs. |

tests/e2e_cuj/AGENTS.md

@@ -47,13 +47,14 @@ Model evidence combines observed gateway requests/responses with native complete
 records; neither routing banners nor native records alone prove an applied decision.
 
 `test_cuj2_mps_mcp.py` runs three independently collected cases against a permanently
-preconfigured workspace: configuration, Codex inference, and Claude inference. Each case
-reads and validates the exact two-agent MPS/MCP shape. No case creates, updates, deletes,
+preconfigured workspace: configuration, Codex MCP tool use, and Claude MCP tool use. Each
+case reads and validates the exact two-agent MPS/MCP shape. No case creates, updates, deletes,
 or restores CodingAgentConfig. The fixture's final read-only equality check is the only
 workspace mutation guard. The test uses the shared `UserSession`, `Workspace`, and
-`TuiRequestRecorder`, `AgentTerminal`, and integration transcript readers directly. The
-configuration case verifies the selected sandbox service and generated MCP listings; live
-`run_code` execution remains deferred because MAS cannot downscope the CI service principal.
+`TuiRequestRecorder`, `AgentTerminal`, MCP inventory helper, and integration transcript readers
+directly. The configuration case verifies the `ug_e2e.tools` selector and generated
+`fixture_reader` listings; each TUI case verifies the connected native inventory, calls
+`read_fixture` with a fresh nonce, and checks its SHA-256 receipt in the parent transcript.
 
 Use a clean disposable POSIX runner without existing machine-wide agent settings.
 Install the intended versions of `ug`, `claude`, `codex`, and `databricks` on PATH.

tests/e2e_cuj/helpers/constants.py

@@ -34,3 +34,6 @@ class CodingAgent(StrEnum):
 
 SANDBOX_MCP_SERVICE_NAME = "system.ai.sandbox"
 WEB_SEARCH_MCP_SERVICE_NAME = "system.ai.web_search"
+MCP_FIXTURE_LOCATION = "ug_e2e.tools"
+MCP_FIXTURE_SERVICE_NAME = "ug_e2e.tools.fixture_reader"
+MCP_FIXTURE_TOOL_NAME = "read_fixture"

tests/e2e_cuj/test_cuj2_mps_mcp.py

@@ -2,6 +2,7 @@
 
 from __future__ import annotations
 
+import hashlib
 import json
 import shutil
 import tomllib
@@ -15,6 +16,7 @@
     assistant_answers,
     is_child_session,
 )
+from tests.integration.utils.mcp import open_claude_mcp_inventory, open_codex_mcp_inventory
 from tests.integration.utils.terminal import AgentTerminal
 
 from .base import BaseCujTest
@@ -23,8 +25,10 @@
     CODEX,
     CODING_AGENT_BY_CLI_NAME,
     INFERENCE_PATHS,
+    MCP_FIXTURE_LOCATION,
+    MCP_FIXTURE_SERVICE_NAME,
+    MCP_FIXTURE_TOOL_NAME,
     MODEL_PROVIDER_SERVICE_FIXTURES,
-    SANDBOX_MCP_SERVICE_NAME,
     WEB_SEARCH_MCP_SERVICE_NAME,
 )
 from .helpers.workspace import Workspace
@@ -52,7 +56,7 @@ def _assert_cuj2_config(config: dict) -> None:
         "spec_version": 1,
         "default_agent": CODING_AGENT_BY_CLI_NAME[CODEX],
         "enabled_agents": [expected_agent_config(CLAUDE), expected_agent_config(CODEX)],
-        "mcp_servers": {"names": [SANDBOX_MCP_SERVICE_NAME]},
+        "mcp_servers": {"unity_catalog_location": MCP_FIXTURE_LOCATION},
     }
     assert actual == expected, config
 
@@ -99,9 +103,10 @@ def _assert_mps_catalog(workspace) -> None:
 
 
 def _assert_mcp_services(workspace) -> None:
-    for name in (SANDBOX_MCP_SERVICE_NAME, WEB_SEARCH_MCP_SERVICE_NAME):
-        service = workspace.client.ai_gateway.get_mcp_service(f"mcp-services/{name}")
-        assert service.name and service.name.endswith(name), service
+    service = workspace.client.ai_gateway.get_mcp_service(
+        f"mcp-services/{MCP_FIXTURE_SERVICE_NAME}"
+    )
+    assert service.name and service.name.endswith(MCP_FIXTURE_SERVICE_NAME), service
 
 
 def _assert_configured_files(session, workspace_url: str) -> None:
@@ -149,7 +154,7 @@ def _assert_generated_mcp_listings(session) -> None:
         assert binary, f"Required agent is not on the isolated PATH: {agent}"
         listing = session.run("mcp", "list", binary=binary, timeout=120)
         output = f"{listing.stdout}\n{listing.stderr}"
-        assert _mcp_name_listed(output, SANDBOX_MCP_SERVICE_NAME), output
+        assert _mcp_name_listed(output, MCP_FIXTURE_SERVICE_NAME), output
         assert not _mcp_name_listed(output, WEB_SEARCH_MCP_SERVICE_NAME), output
 
 
@@ -161,29 +166,81 @@ def _parent_transcripts(session, agent: str) -> dict[str, list[dict]]:
     }
 
 
-def _task_complete(session, agent: str, marker: str) -> bool:
+def _task_complete(session, agent: str, task) -> bool:
     transcripts = _parent_transcripts(session, agent)
-    return any(
-        marker in answer
-        for records in transcripts.values()
-        for answer in assistant_answers(agent, records)
-    )
+    return any(task.completed(records, agent) for records in transcripts.values())
 
 
-def _assert_task_evidence(session, agent: str, marker: str) -> None:
+def _assert_task_evidence(session, agent: str, task) -> None:
     transcripts = _parent_transcripts(session, agent)
     answers = [
         answer for records in transcripts.values() for answer in assistant_answers(agent, records)
     ]
-    assert any(marker in answer for answer in answers), answers
+    assert task.completed_in_answers(answers), answers
+    assert any(task.tool_call_observed(records) for records in transcripts.values()), transcripts
     session.record(
         f"cuj2-{agent}-evidence",
-        {"marker": marker, "answers": answers},
+        {
+            "expected": task.expected,
+            "run_id": task.run_id,
+            "answers": answers,
+            "agent_sessions": transcripts,
+        },
     )
 
 
-def _agent_prompt(marker: str) -> str:
-    return f"Reply with the exact marker `{marker}` and no other text. Do not use tools."
+class _McpFixtureTask:
+    """A nonce-bound MCP call whose receipt is checked in the native parent transcript."""
+
+    def __init__(self, marker: str):
+        self.marker = marker
+        self.run_id = uuid.uuid4().hex
+        self.receipt = hashlib.sha256(
+            b"unity-gateway-cuj-fixture\0"
+            + MCP_FIXTURE_TOOL_NAME.encode("ascii")
+            + b"\0"
+            + self.run_id.encode("ascii")
+        ).hexdigest()
+        self.expected = {"marker": marker, MCP_FIXTURE_TOOL_NAME: self.receipt}
+        self.prompt = (
+            f"Call the read-only MCP tool {MCP_FIXTURE_TOOL_NAME} on "
+            f"{MCP_FIXTURE_SERVICE_NAME.replace('.', '-')}, with run_id={self.run_id}. "
+            f"Reply with only this JSON object: include marker {marker!r} and the exact text "
+            f"returned by the tool under the {MCP_FIXTURE_TOOL_NAME!r} key. Do not use shell "
+            "commands, files, or subagents."
+        )
+
+    def completed_in_answers(self, answers: list[str]) -> bool:
+        for answer in answers:
+            try:
+                if json.loads(answer) == self.expected:
+                    return True
+            except json.JSONDecodeError:
+                continue
+        return False
+
+    def completed(self, records: list[dict], agent: str) -> bool:
+        return self.completed_in_answers(assistant_answers(agent, records))
+
+    def tool_call_observed(self, records: list[dict]) -> bool:
+        def visit(value):
+            if isinstance(value, dict):
+                kind = value.get("type", "")
+                name = value.get("name", "")
+                if kind in {
+                    "tool_use",
+                    "tool_call",
+                    "function_call",
+                    "custom_tool_call",
+                } and MCP_FIXTURE_TOOL_NAME in str(name):
+                    arguments = value.get("input", value.get("arguments", {}))
+                    return self.run_id in json.dumps(arguments, sort_keys=True)
+                return any(visit(child) for child in value.values())
+            if isinstance(value, list):
+                return any(visit(child) for child in value)
+            return False
+
+        return visit(records)
 
 
 def _assert_inference_request(recorder, request, *, provider: str, model: str, marker: str) -> None:
@@ -203,9 +260,10 @@ class TestCuj2Configuration(_Cuj2Base):
     def test_cuj_configuration(self, cuj):
         """Scenario: configure ug from the preconfigured two-agent MPS/MCP workspace.
 
-        Expected: the read-only CodingAgentConfig selects the two exact MPS resources and sandbox
-        MCP; both native MPS APIs advertise their selected model, generated agent settings use
-        the exact provider/model values, and agent MCP listings exclude web_search.
+        Expected: the read-only CodingAgentConfig selects the two exact MPS resources and the
+        fixture MCP schema; both native MPS APIs advertise their selected model, generated agent
+        settings use the exact provider/model values, and agent MCP listings expose the fixture
+        reader while excluding web_search.
         """
         session, workspace, _ = cuj
         published = workspace.config()
@@ -228,11 +286,12 @@ def test_cuj_configuration(self, cuj):
 
 class TestCuj2CodexInference(_Cuj2Base):
     def test_cuj_codex_inference(self, cuj):
-        """Scenario: configure through the recorder and complete a real Codex inference.
+        """Scenario: configure through the recorder and complete a real Codex MCP task.
 
         Expected: Codex sends a Responses request for gpt-5-nano with the
         ug_e2e.providers.openai target header and receives its paired HTTP 200 response; its
-        parent transcript contains the final marker.
+        native inventory connects the fixture MCP and lists read_fixture; its parent transcript
+        records the nonce-bound receipt returned by the tool.
         """
         session, workspace, recorder = cuj
         published = workspace.config()
@@ -252,16 +311,20 @@ def test_cuj_codex_inference(self, cuj):
         _assert_generated_mcp_listings(session)
 
         recorder.prepare_launch()
-        marker = f"CUJ2-CODEX-{uuid.uuid4().hex}"
+        task = _McpFixtureTask(f"CUJ2-CODEX-{uuid.uuid4().hex}")
         checkpoint = recorder.checkpoint()
         with AgentTerminal(session, CODEX, [str(session.binary), CODEX], "cuj2-codex") as tui:
             tui.boot()
-            tui.submit(_agent_prompt(marker))
+            inventory = open_codex_mcp_inventory(
+                tui, {MCP_FIXTURE_SERVICE_NAME: MCP_FIXTURE_TOOL_NAME}
+            )
+            assert _mcp_name_listed(inventory, MCP_FIXTURE_SERVICE_NAME), inventory
+            tui.submit(task.prompt)
 
             def completed(screen):
                 assert_no_terminal_api_error(screen)
                 assert "Do you want to proceed?" not in screen, screen
-                return _task_complete(session, CODEX, marker)
+                return _task_complete(session, CODEX, task)
 
             tui.wait_for(completed, "completed Codex task", timeout=240)
             tui.exit_normally()
@@ -277,18 +340,19 @@ def completed(screen):
             request,
             provider=MODEL_PROVIDER_SERVICE_FIXTURES[CODEX][0],
             model=MODEL_PROVIDER_SERVICE_FIXTURES[CODEX][1],
-            marker=marker,
+            marker=task.marker,
         )
-        _assert_task_evidence(session, CODEX, marker)
+        _assert_task_evidence(session, CODEX, task)
 
 
 class TestCuj2ClaudeInference(_Cuj2Base):
     def test_cuj_claude_inference(self, cuj):
-        """Scenario: configure through the recorder and complete a real Claude inference.
+        """Scenario: configure through the recorder and complete a real Claude MCP task.
 
         Expected: Claude sends a Messages request for claude-haiku-4-5-20251001 with the
         ug_e2e.providers.anthropic target header and receives its paired HTTP 200 response; its
-        parent transcript contains the final marker.
+        native inventory connects the fixture MCP and lists read_fixture; its parent transcript
+        records the nonce-bound receipt returned by the tool.
         """
         session, workspace, recorder = cuj
         published = workspace.config()
@@ -308,15 +372,20 @@ def test_cuj_claude_inference(self, cuj):
         _assert_generated_mcp_listings(session)
 
         recorder.prepare_launch()
-        marker = f"CUJ2-CLAUDE-{uuid.uuid4().hex}"
+        task = _McpFixtureTask(f"CUJ2-CLAUDE-{uuid.uuid4().hex}")
         checkpoint = recorder.checkpoint()
         with AgentTerminal(session, CLAUDE, [str(session.binary), CLAUDE], "cuj2-claude") as tui:
             tui.boot()
-            tui.submit(_agent_prompt(marker))
+            inventory = open_claude_mcp_inventory(
+                tui, {MCP_FIXTURE_SERVICE_NAME: MCP_FIXTURE_TOOL_NAME}
+            )
+            assert _mcp_name_listed(inventory, MCP_FIXTURE_SERVICE_NAME), inventory
+            tui.submit(task.prompt)
 
             def completed(screen):
                 assert_no_terminal_api_error(screen)
-                return _task_complete(session, CLAUDE, marker)
+                assert "Do you want to proceed?" not in screen, screen
+                return _task_complete(session, CLAUDE, task)
 
             tui.wait_for(completed, "completed Claude task", timeout=240)
             tui.exit_normally()
@@ -332,6 +401,6 @@ def completed(screen):
             request,
             provider=MODEL_PROVIDER_SERVICE_FIXTURES[CLAUDE][0],
             model=MODEL_PROVIDER_SERVICE_FIXTURES[CLAUDE][1],
-            marker=marker,
+            marker=task.marker,
         )
-        _assert_task_evidence(session, CLAUDE, marker)
+        _assert_task_evidence(session, CLAUDE, task)