From de789e67117bc8b4da8fbe34e14bebfeb2ebad48 Mon Sep 17 00:00:00 2001 From: nahumkorda Date: Fri, 31 Jul 2026 20:42:03 +0300 Subject: [PATCH 1/2] feat: AWS Bedrock provider adapter MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add a `bedrock` provider type that runs OpenAnt scans against Claude models on AWS Bedrock with native AWS credentials — no proxy needed. Bedrock is on the README roadmap ("More provider adapters"); this implements it Python-only, with no config-schema or Go CLI changes. Design: the registry constructs every adapter as `cls(api_key=..., base_url=...)`, and AWS needs SigV4 credentials plus a region, which don't fit that signature. Rather than widening ProviderConfig (and touching Go config parsing), BedrockAdapter keeps the same constructor shape and delegates credentials and region to the SDK's standard AWS chain (AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY env vars or ~/.aws profile; AWS_REGION env, then boto3 session). `api_key` is ignored with a one-time warning; `base_url` still works as a full endpoint override (VPC endpoints, gateways). Bedrock speaks the same Messages API through the same `anthropic` SDK types (`anthropic.AnthropicBedrock`), so the adapter reuses the Anthropic adapter's translation layer — blocks, stop reasons, and the refusal -> LLMRefusalError guard — parameterizing only the warning attribution. Bedrock-specific deltas: - 403 AccessDenied -> LLMAuthError with a Bedrock "Model access" hint (the usual cause is a model not enabled for the account/region) - 400 "model identifier is invalid" -> LLMNotFoundError (Bedrock reports bad model IDs as ValidationException, not 404) - SDK's bare no-credentials RuntimeError -> LLMAuthError - throttling 429s report to the global RateLimiter like the reference adapter; the 529 branch is kept harmless for Anthropic-compat gateways behind base_url Model IDs are cross-region inference profiles. config/models.json gains six bedrock records (us./global. families for opus-4-8, sonnet-4-6, haiku-4-5) with IDs verified against a live Bedrock account via list-inference-profiles; prices mirror the direct Anthropic API records. Note opus-4-8 and sonnet-4-6 profiles carry no -v1:0 suffix. Tests: bedrock row in the contract-test ADAPTERS fixture with a scenario factory stubbing AnthropicBedrock, plus 15 Bedrock-specific tests (constructor plumbing, verbatim profile IDs, error mapping, rate-limiter coordination). Live-validated: messages.create round-trip confirmed against us.anthropic.claude-sonnet-4-6 and global.anthropic.claude-haiku-4-5-20251001-v1:0 in us-east-1. Go wizard support (setup.go) is deliberately out of scope; a hand-authored config works, as the README documents. Co-Authored-By: Claude Fable 5 --- README.md | 5 +- config/models.json | 36 ++ libs/openant-core/core/model_registry.py | 2 +- .../tests/_llm_factories/bedrock.py | 152 +++++++++ .../tests/test_llm_adapter_contract.py | 7 + .../tests/test_llm_bedrock_adapter.py | 310 ++++++++++++++++++ .../utilities/llm/providers/__init__.py | 8 +- .../utilities/llm/providers/anthropic.py | 17 +- .../utilities/llm/providers/bedrock.py | 299 +++++++++++++++++ 9 files changed, 825 insertions(+), 11 deletions(-) create mode 100644 libs/openant-core/tests/_llm_factories/bedrock.py create mode 100644 libs/openant-core/tests/test_llm_bedrock_adapter.py create mode 100644 libs/openant-core/utilities/llm/providers/bedrock.py diff --git a/README.md b/README.md index c1b76afa..92dbc99f 100644 --- a/README.md +++ b/README.md @@ -87,8 +87,9 @@ Wizard defaults reflect the project's per-phase recommendations (stronger reason | `anthropic` | [console.anthropic.com](https://console.anthropic.com/settings/keys) | Reference adapter. NOT included in Claude Pro / Max subscriptions — separate billing. | | `openai` | [platform.openai.com](https://platform.openai.com/api-keys) | NOT included in ChatGPT / Codex subscriptions — separate billing. | | `google` | [aistudio.google.com](https://aistudio.google.com/apikey) | NOT included in Gemini Advanced — separate billing. | +| `bedrock` | — (AWS credential chain) | Claude on AWS Bedrock. No `api_key`: credentials come from `AWS_ACCESS_KEY_ID`/`AWS_SECRET_ACCESS_KEY` env vars or a `~/.aws` profile, region from `AWS_REGION`. Model IDs are inference profiles (`us.anthropic.claude-sonnet-4-6`, `global.anthropic.claude-haiku-4-5-20251001-v1:0`, ...) — enable them under "Model access" in the Bedrock console and list them with `aws bedrock list-inference-profiles`. Not offered by the setup wizard yet; configure by hand (see below). | -All three support tool calling, so any of them can drive the `enhance` and `verify` phases that use the agentic tool-use loop. +All four support tool calling, so any of them can drive the `enhance` and `verify` phases that use the agentic tool-use loop. #### Quick path for Anthropic-only setups @@ -216,7 +217,7 @@ openant project switch # switch active project Things on the list, in no particular order: -- **More provider adapters.** Ollama (local models), vLLM, Cohere, Mistral, Groq, Amazon Bedrock, Azure OpenAI — each is a small Python adapter recipe (plus a few Go wizard/probe touch-points if you want it offered by `openant setup llm`) per the contributor guide. Lower the barrier to local / on-prem inference. +- **More provider adapters.** Ollama (local models), vLLM, Cohere, Mistral, Groq, Azure OpenAI — each is a small Python adapter recipe (plus a few Go wizard/probe touch-points if you want it offered by `openant setup llm`) per the contributor guide. Lower the barrier to local / on-prem inference. - **Subscription-based auth.** ChatGPT / Codex, Claude Pro / Max, and Gemini Advanced subscriptions don't currently grant API quota — users have to maintain a separate API-tier key per provider. OAuth-based adapters that ride the consumer subscription would close that gap. - **Cross-provider tool-call quirks.** All three shipped adapters support tool calling, but the long tail (parallel tool calls, strict-mode schema enforcement, retry semantics on partial JSON) behaves differently per provider. Real-world scans surface these — PRs welcome. - **More languages.** The supported-languages list above is current coverage. Rust, Java, C#, and Swift come up frequently. diff --git a/config/models.json b/config/models.json index f945be07..f9520db7 100644 --- a/config/models.json +++ b/config/models.json @@ -38,6 +38,42 @@ "source": "liveness contested and not verifiable in build env; asserted neither alive nor dead", "retrieved": "2026-07-23" }, + { + "id": "us.anthropic.claude-opus-4-8", "provider": "bedrock", "status": "current", + "price": {"input": 15.00, "output": 75.00}, + "source": "verified against a live Bedrock account via list-inference-profiles (us-east-1, status ACTIVE); price mirrors the anthropic claude-opus-4-8 record (Bedrock Claude token rates match the direct Anthropic API)", + "retrieved": "2026-07-31" + }, + { + "id": "global.anthropic.claude-opus-4-8", "provider": "bedrock", "status": "current", + "price": {"input": 15.00, "output": 75.00}, + "source": "verified against a live Bedrock account via list-inference-profiles (us-east-1, status ACTIVE); price mirrors the anthropic claude-opus-4-8 record (Bedrock Claude token rates match the direct Anthropic API)", + "retrieved": "2026-07-31" + }, + { + "id": "us.anthropic.claude-sonnet-4-6", "provider": "bedrock", "status": "current", + "price": {"input": 3.00, "output": 15.00}, + "source": "verified against a live Bedrock account via list-inference-profiles (us-east-1, status ACTIVE); price mirrors the anthropic claude-sonnet-4-6 record (Bedrock Claude token rates match the direct Anthropic API)", + "retrieved": "2026-07-31" + }, + { + "id": "global.anthropic.claude-sonnet-4-6", "provider": "bedrock", "status": "current", + "price": {"input": 3.00, "output": 15.00}, + "source": "verified against a live Bedrock account via list-inference-profiles (us-east-1, status ACTIVE); price mirrors the anthropic claude-sonnet-4-6 record (Bedrock Claude token rates match the direct Anthropic API)", + "retrieved": "2026-07-31" + }, + { + "id": "us.anthropic.claude-haiku-4-5-20251001-v1:0", "provider": "bedrock", "status": "current", + "price": {"input": 1.00, "output": 5.00}, + "source": "verified against a live Bedrock account via list-inference-profiles (us-east-1, status ACTIVE); price mirrors the anthropic claude-haiku-4-5-20251001 record (Bedrock Claude token rates match the direct Anthropic API)", + "retrieved": "2026-07-31" + }, + { + "id": "global.anthropic.claude-haiku-4-5-20251001-v1:0", "provider": "bedrock", "status": "current", + "price": {"input": 1.00, "output": 5.00}, + "source": "verified against a live Bedrock account via list-inference-profiles (us-east-1, status ACTIVE); price mirrors the anthropic claude-haiku-4-5-20251001 record (Bedrock Claude token rates match the direct Anthropic API)", + "retrieved": "2026-07-31" + }, { "id": "gpt-4o", "provider": "openai", "status": "current", "price": {"input": 2.50, "output": 10.00}, diff --git a/libs/openant-core/core/model_registry.py b/libs/openant-core/core/model_registry.py index e08558ad..32126e3e 100644 --- a/libs/openant-core/core/model_registry.py +++ b/libs/openant-core/core/model_registry.py @@ -39,7 +39,7 @@ _CONFIG_REL = Path("config") / "models.json" _SEARCH_LEVELS = 6 _VALID_STATUS = frozenset({"current", "retired", "unknown"}) -_VALID_PROVIDERS = frozenset({"anthropic", "openai", "google"}) +_VALID_PROVIDERS = frozenset({"anthropic", "openai", "google", "bedrock"}) def _search_upward(start: Path) -> Path | None: diff --git a/libs/openant-core/tests/_llm_factories/bedrock.py b/libs/openant-core/tests/_llm_factories/bedrock.py new file mode 100644 index 00000000..78bd2b65 --- /dev/null +++ b/libs/openant-core/tests/_llm_factories/bedrock.py @@ -0,0 +1,152 @@ +"""Scenario factory for the Bedrock adapter contract tests. + +Each scenario builds a fake ``anthropic.AnthropicBedrock`` client wired +with the right scripted behavior, then constructs a +:class:`BedrockAdapter` over that fake. Bedrock speaks the same +Messages API through the same ``anthropic`` SDK types and exception +classes as the direct API, so the scripted responses and raised +exceptions mirror the Anthropic factory — only the client spec and the +endpoint URL differ. + +See ``tests/test_llm_adapter_contract.py`` for the scenario catalogue. +""" + +from __future__ import annotations + +from types import SimpleNamespace +from typing import Any +from unittest.mock import MagicMock + +import anthropic +import httpx + +from utilities.llm import LLMAdapter +from utilities.llm.providers.bedrock import BedrockAdapter + + +_ENDPOINT = "https://bedrock-runtime.us-east-1.amazonaws.com/model/test/invoke" + + +def _text_block(text: str) -> SimpleNamespace: + return SimpleNamespace(type="text", text=text) + + +def _tool_use_block(*, id: str, name: str, input: dict) -> SimpleNamespace: + return SimpleNamespace(type="tool_use", id=id, name=name, input=input) + + +def _response( + *, content: list, input_tokens: int, output_tokens: int, stop_reason: str +) -> SimpleNamespace: + return SimpleNamespace( + content=content, + usage=SimpleNamespace( + input_tokens=input_tokens, output_tokens=output_tokens + ), + stop_reason=stop_reason, + ) + + +def _fake_httpx_response(status_code: int, *, retry_after: str | None = None) -> httpx.Response: + headers = {} + if retry_after is not None: + headers["retry-after"] = retry_after + return httpx.Response( + status_code=status_code, + headers=headers, + request=httpx.Request("POST", _ENDPOINT), + ) + + +def _script_text(call_args: dict) -> SimpleNamespace: + # The contract test asserts content=="hi there", usage 3/5, end_turn. + return _response( + content=[_text_block("hi there")], + input_tokens=3, + output_tokens=5, + stop_reason="end_turn", + ) + + +def _script_tool_use_round(call_args: dict) -> SimpleNamespace: + has_assistant = any(m.get("role") == "assistant" for m in call_args["messages"]) + if not has_assistant: + return _response( + content=[ + _tool_use_block( + id="toolu_test_1", + name="echo", + input={"text": "hello"}, + ) + ], + input_tokens=10, + output_tokens=8, + stop_reason="tool_use", + ) + return _response( + content=[_text_block("echoed: hello")], + input_tokens=20, + output_tokens=4, + stop_reason="end_turn", + ) + + +def _raise_auth(_call_args: dict): + raise anthropic.AuthenticationError( + message="invalid signature", + response=_fake_httpx_response(401), + body=None, + ) + + +def _raise_rate_limit(_call_args: dict): + # Bedrock throttling (ThrottlingException) surfaces as the SDK's + # standard 429 RateLimitError. + raise anthropic.RateLimitError( + message="too many requests, please wait before trying again", + response=_fake_httpx_response(429, retry_after="7"), + body=None, + ) + + +def _raise_connection(_call_args: dict): + raise anthropic.APIConnectionError( + request=httpx.Request("POST", _ENDPOINT), + ) + + +def _raise_not_found(_call_args: dict): + raise anthropic.NotFoundError( + message="model not found: ghost-model", + response=_fake_httpx_response(404), + body=None, + ) + + +_SCENARIO_HANDLERS = { + "text": _script_text, + "tool_use_round": _script_tool_use_round, + "auth_error": _raise_auth, + "rate_limit": _raise_rate_limit, + "connection_error": _raise_connection, + "model_not_found": _raise_not_found, + "validate_ok": _script_text, # any valid response satisfies validate + "validate_auth_fail": _raise_auth, # validate is a thin wrapper over create +} + + +def make_adapter(scenario: str) -> LLMAdapter: + """Build a BedrockAdapter whose SDK is scripted for ``scenario``.""" + if scenario not in _SCENARIO_HANDLERS: + raise KeyError(f"Unknown scenario: {scenario!r}") + + handler = _SCENARIO_HANDLERS[scenario] + + def side_effect(**kwargs: Any) -> Any: + return handler(kwargs) + + fake_client = MagicMock(spec=anthropic.AnthropicBedrock) + fake_client.messages = MagicMock() + fake_client.messages.create = MagicMock(side_effect=side_effect) + + return BedrockAdapter(_client=fake_client) diff --git a/libs/openant-core/tests/test_llm_adapter_contract.py b/libs/openant-core/tests/test_llm_adapter_contract.py index e518df1e..f3be4752 100644 --- a/libs/openant-core/tests/test_llm_adapter_contract.py +++ b/libs/openant-core/tests/test_llm_adapter_contract.py @@ -112,12 +112,19 @@ def _google_factory(): return make_adapter +def _bedrock_factory(): + from tests._llm_factories.bedrock import make_adapter + + return make_adapter + + # Each row: (display_name, scenario_factory_callable) # Add a row when registering a new adapter. ADAPTERS: list[tuple[str, Callable[[str], LLMAdapter]]] = [ ("anthropic", _anthropic_factory()), ("openai", _openai_factory()), ("google", _google_factory()), + ("bedrock", _bedrock_factory()), ] diff --git a/libs/openant-core/tests/test_llm_bedrock_adapter.py b/libs/openant-core/tests/test_llm_bedrock_adapter.py new file mode 100644 index 00000000..631ba25b --- /dev/null +++ b/libs/openant-core/tests/test_llm_bedrock_adapter.py @@ -0,0 +1,310 @@ +"""Bedrock-adapter-specific tests. + +The shared contract harness (``test_llm_adapter_contract.py``) covers +behaviors every adapter must satisfy, and the request/response +translation layer is the Anthropic adapter's (reused, covered by +``test_llm_anthropic_adapter.py``). This file covers the bits that are +specific to Bedrock: + +* constructor plumbing — ``base_url`` forwarded, ``api_key`` ignored + with a one-time warning, no aws_* kwargs passed (region and + credentials must resolve through the SDK's own AWS chain) +* inference-profile model IDs passed through verbatim +* AccessDenied (403) mapped to LLMAuthError with a "Model access" hint +* 400 "model identifier is invalid" mapped to LLMNotFoundError +* missing-credentials RuntimeError mapped to LLMAuthError +* Bedrock throttling (429) reports to the global rate limiter + +These tests stub the SDK boundary so nothing hits the network. +""" + +from __future__ import annotations + +from types import SimpleNamespace +from unittest.mock import MagicMock + +import anthropic +import httpx +import pytest + +from utilities.llm import ( + LLMAuthError, + LLMNotFoundError, + LLMRateLimitError, + LLMResponseError, + Message, + TextBlock, +) +from utilities.llm.providers import bedrock as bedrock_module +from utilities.llm.providers.bedrock import BedrockAdapter +from utilities.rate_limiter import get_rate_limiter, reset_rate_limiter + + +@pytest.fixture(autouse=True) +def _reset_state(): + reset_rate_limiter() + bedrock_module.reset_warnings() + yield + reset_rate_limiter() + bedrock_module.reset_warnings() + + +def _ok_response(*, text="hi", input_tokens=1, output_tokens=1, stop_reason="end_turn"): + return SimpleNamespace( + content=[SimpleNamespace(type="text", text=text)], + usage=SimpleNamespace(input_tokens=input_tokens, output_tokens=output_tokens), + stop_reason=stop_reason, + ) + + +def _stub_adapter(side_effect): + client = MagicMock(spec=anthropic.AnthropicBedrock) + client.messages = MagicMock() + client.messages.create = MagicMock(side_effect=side_effect) + return BedrockAdapter(_client=client), client + + +def _fake_http_resp(status, *, retry_after=None): + headers = {} + if retry_after is not None: + headers["retry-after"] = retry_after + return httpx.Response( + status_code=status, + headers=headers, + request=httpx.Request( + "POST", "https://bedrock-runtime.us-east-1.amazonaws.com/model/test/invoke" + ), + ) + + +def _complete(adapter, model="us.anthropic.claude-sonnet-4-6"): + return adapter.complete( + model=model, + system=None, + messages=[Message(role="user", content=[TextBlock("hi")])], + max_tokens=8, + ) + + +# --------------------------------------------------------------------------- +# Constructor plumbing +# --------------------------------------------------------------------------- + + +class TestConstructor: + def _patched(self, monkeypatch): + captured = {} + + class FakeBedrock: + def __init__(self, **kwargs): + captured.update(kwargs) + self.messages = MagicMock() + + monkeypatch.setattr( + "utilities.llm.providers.bedrock.anthropic.AnthropicBedrock", FakeBedrock + ) + return captured + + def test_passes_base_url_to_sdk(self, monkeypatch): + captured = self._patched(monkeypatch) + BedrockAdapter(base_url="https://vpce-123.bedrock-runtime.us-east-1.vpce.amazonaws.com") + assert captured["base_url"] == ( + "https://vpce-123.bedrock-runtime.us-east-1.vpce.amazonaws.com" + ) + assert captured["max_retries"] == 5 + + def test_no_aws_kwargs_passed(self, monkeypatch): + """Region and credentials must resolve through the SDK's own AWS + chain (AWS_REGION env, then ~/.aws via boto3) — the adapter + passes no aws_* kwargs, so `openant` behaves exactly like the + aws CLI on the same machine.""" + captured = self._patched(monkeypatch) + BedrockAdapter() + assert not any(key.startswith("aws_") for key in captured) + assert "base_url" not in captured + + def test_api_key_is_ignored_not_forwarded(self, monkeypatch, capsys): + """The registry constructs every adapter with api_key=...; Bedrock + has no API-key auth in the pinned SDK, so the kwarg must be + dropped (never forwarded) and warned about (never silent).""" + captured = self._patched(monkeypatch) + BedrockAdapter(api_key="sk-ant-should-be-ignored") + assert "api_key" not in captured + err = capsys.readouterr().err + assert "ignores `api_key`" in err + + def test_api_key_warning_is_once_per_process(self, monkeypatch, capsys): + self._patched(monkeypatch) + BedrockAdapter(api_key="k1") + BedrockAdapter(api_key="k2") + err = capsys.readouterr().err + assert err.count("ignores `api_key`") == 1 + + def test_no_warning_without_api_key(self, monkeypatch, capsys): + self._patched(monkeypatch) + BedrockAdapter() + assert "api_key" not in capsys.readouterr().err + + +# --------------------------------------------------------------------------- +# Inference-profile model IDs +# --------------------------------------------------------------------------- + + +class TestInferenceProfileIds: + def test_profile_id_passed_verbatim(self): + adapter, client = _stub_adapter(lambda **kw: _ok_response()) + _complete(adapter, model="global.anthropic.claude-opus-4-8") + assert client.messages.create.call_args.kwargs["model"] == ( + "global.anthropic.claude-opus-4-8" + ) + + def test_validate_probes_the_passed_profile(self): + adapter, client = _stub_adapter(lambda **kw: _ok_response()) + adapter.validate(model="us.anthropic.claude-haiku-4-5-20251001-v1:0") + kwargs = client.messages.create.call_args.kwargs + assert kwargs["model"] == "us.anthropic.claude-haiku-4-5-20251001-v1:0" + assert kwargs["max_tokens"] == 1 + + +# --------------------------------------------------------------------------- +# Bedrock-specific error mapping +# --------------------------------------------------------------------------- + + +class TestErrorMapping: + def test_access_denied_maps_to_auth_error_with_model_access_hint(self): + def respond(**kw): + raise anthropic.PermissionDeniedError( + message=( + "You don't have access to the model with the specified " + "model ID." + ), + response=_fake_http_resp(403), + body=None, + ) + + adapter, _ = _stub_adapter(respond) + with pytest.raises(LLMAuthError) as exc_info: + _complete(adapter) + assert "Model access" in str(exc_info.value) + + def test_validate_access_denied_carries_the_same_hint(self): + def respond(**kw): + raise anthropic.PermissionDeniedError( + message="AccessDeniedException", + response=_fake_http_resp(403), + body=None, + ) + + adapter, _ = _stub_adapter(respond) + with pytest.raises(LLMAuthError) as exc_info: + adapter.validate(model="us.anthropic.claude-sonnet-4-6") + assert "Model access" in str(exc_info.value) + + def test_invalid_model_identifier_400_maps_to_not_found(self): + """Bedrock reports a malformed/unknown model ID as a 400 + ValidationException, not a 404 — it must still fail like a + typo'd model so the registry's validate() catches it at init.""" + + def respond(**kw): + raise anthropic.BadRequestError( + message="The provided model identifier is invalid.", + response=_fake_http_resp(400), + body=None, + ) + + adapter, _ = _stub_adapter(respond) + with pytest.raises(LLMNotFoundError): + _complete(adapter, model="us.anthropic.claude-typo-v1:0") + + def test_other_400_is_a_response_error(self): + def respond(**kw): + raise anthropic.BadRequestError( + message="max_tokens must be positive", + response=_fake_http_resp(400), + body=None, + ) + + adapter, _ = _stub_adapter(respond) + with pytest.raises(LLMResponseError): + _complete(adapter) + + def test_missing_credentials_maps_to_auth_error(self): + """The SDK's SigV4 signer raises a bare RuntimeError when the AWS + chain resolves no credentials; it must surface typed, with a + pointer at the env vars, not as an unhandled RuntimeError.""" + + def respond(**kw): + raise RuntimeError("could not resolve credentials from session") + + adapter, _ = _stub_adapter(respond) + with pytest.raises(LLMAuthError) as exc_info: + _complete(adapter) + assert "AWS_ACCESS_KEY_ID" in str(exc_info.value) + + adapter, _ = _stub_adapter(respond) + with pytest.raises(LLMAuthError): + adapter.validate(model="us.anthropic.claude-sonnet-4-6") + + def test_unrelated_runtime_error_propagates(self): + """Only the SDK's no-credentials RuntimeError is auth-shaped; + anything else must not be swallowed into the taxonomy.""" + + def respond(**kw): + raise RuntimeError("something else entirely") + + adapter, _ = _stub_adapter(respond) + with pytest.raises(RuntimeError, match="something else"): + _complete(adapter) + + +# --------------------------------------------------------------------------- +# Rate-limiter coordination +# --------------------------------------------------------------------------- + + +class TestRateLimiterCoordination: + def test_throttling_429_reports_to_global_limiter(self): + def respond(**kw): + raise anthropic.RateLimitError( + message="too many requests", + response=_fake_http_resp(429, retry_after="3"), + body=None, + ) + + adapter, _ = _stub_adapter(respond) + with pytest.raises(LLMRateLimitError): + _complete(adapter) + assert get_rate_limiter().is_in_backoff() + + def test_529_from_a_compat_gateway_still_maps_to_rate_limit(self): + """Bedrock itself never sends 529, but base_url may point at an + Anthropic-compat gateway that does; the branch is kept and must + keep behaving like the reference adapter.""" + + def respond(**kw): + raise anthropic.APIStatusError( + message="overloaded", + response=_fake_http_resp(529, retry_after="5"), + body=None, + ) + + adapter, _ = _stub_adapter(respond) + with pytest.raises(LLMRateLimitError) as exc_info: + _complete(adapter) + assert exc_info.value.retry_after == 5 + assert get_rate_limiter().is_in_backoff() + + def test_other_api_status_errors_do_not_trigger_backoff(self): + def respond(**kw): + raise anthropic.APIStatusError( + message="internal error", + response=_fake_http_resp(500), + body=None, + ) + + adapter, _ = _stub_adapter(respond) + with pytest.raises(LLMResponseError): + _complete(adapter) + assert not get_rate_limiter().is_in_backoff() diff --git a/libs/openant-core/utilities/llm/providers/__init__.py b/libs/openant-core/utilities/llm/providers/__init__.py index 0c494527..1eafffbf 100644 --- a/libs/openant-core/utilities/llm/providers/__init__.py +++ b/libs/openant-core/utilities/llm/providers/__init__.py @@ -43,10 +43,14 @@ def get_adapter_class(provider_type: str) -> Type[LLMAdapter]: from .google import GoogleAdapter return GoogleAdapter + if provider_type == "bedrock": + from .bedrock import BedrockAdapter + + return BedrockAdapter raise ValueError( f"Unknown provider type: {provider_type!r}. " - f"Supported in this release: 'anthropic', 'openai', 'google'. " + f"Supported in this release: 'anthropic', 'openai', 'google', 'bedrock'. " f"To add a provider, see " f"docs/features/llm-providers/HOW_TO_ADD_AN_ADAPTER.md." ) @@ -58,4 +62,4 @@ def known_provider_types() -> list[str]: Used by the Go CLI's ``llm-provider set`` to validate the ``type`` field before writing config.json. """ - return ["anthropic", "openai", "google"] + return ["anthropic", "openai", "google", "bedrock"] diff --git a/libs/openant-core/utilities/llm/providers/anthropic.py b/libs/openant-core/utilities/llm/providers/anthropic.py index 3c0fc3f3..c6a288c9 100644 --- a/libs/openant-core/utilities/llm/providers/anthropic.py +++ b/libs/openant-core/utilities/llm/providers/anthropic.py @@ -85,7 +85,7 @@ _warned_block_kinds_lock = threading.Lock() -def _warn_unknown_block_kind(kind: str) -> None: +def _warn_unknown_block_kind(kind: str, *, adapter: str = "AnthropicAdapter") -> None: """One-time stderr warning when the response carries a content-block kind the adapter doesn't translate, so a dropped block isn't silent.""" should_warn = False @@ -95,7 +95,7 @@ def _warn_unknown_block_kind(kind: str) -> None: should_warn = True if should_warn: sys.stderr.write( - f"warning: AnthropicAdapter received unknown content block " + f"warning: {adapter} received unknown content block " f"kind {kind!r}; dropping it. If the pipeline should consume " f"this, add a ContentBlock kind in utilities/llm/adapter.py " f"and translate it here.\n" @@ -308,8 +308,13 @@ def _tool_to_anthropic(tool: ToolDef) -> dict[str, Any]: } -def _response_to_unified(response: Any) -> CompletionResult: - """Translate an anthropic SDK ``Message`` object into our types.""" +def _response_to_unified(response: Any, *, adapter: str = "AnthropicAdapter") -> CompletionResult: + """Translate an anthropic SDK ``Message`` object into our types. + + ``adapter`` names the calling adapter in one-time warnings — the + Bedrock adapter reuses this translation layer (same SDK, same wire + types) and its warnings should not blame AnthropicAdapter. + """ content_blocks: list[ContentBlock] = [] for block in response.content: kind = getattr(block, "type", None) @@ -330,7 +335,7 @@ def _response_to_unified(response: Any) -> CompletionResult: # symptom isn't silent. For a security tool, a silently # dropped "refusal" paired with a benign stop_reason could # read as an empty success. - _warn_unknown_block_kind(str(kind)) + _warn_unknown_block_kind(str(kind), adapter=adapter) # R4-5: a usage-less response (rare, but seen on some proxies and on # error-shaped 200s) must not AttributeError here — the downstream @@ -376,7 +381,7 @@ def _response_to_unified(response: Any) -> CompletionResult: should_warn = True if should_warn: sys.stderr.write( - f"warning: AnthropicAdapter received unknown stop_reason " + f"warning: {adapter} received unknown stop_reason " f"{raw_stop!r}; normalising to 'end_turn'. Add this value " f"to StopReason in utilities/llm/adapter.py and the " f"_ANTHROPIC_STOP_REASONS table if it's a new SDK addition.\n" diff --git a/libs/openant-core/utilities/llm/providers/bedrock.py b/libs/openant-core/utilities/llm/providers/bedrock.py new file mode 100644 index 00000000..f84bd661 --- /dev/null +++ b/libs/openant-core/utilities/llm/providers/bedrock.py @@ -0,0 +1,299 @@ +"""AWS Bedrock adapter — Claude models via ``anthropic.AnthropicBedrock``. + +Bedrock serves the same Claude models through the same Messages API and +the same ``anthropic`` SDK wire types as the direct Anthropic API, so +this adapter reuses the Anthropic adapter's translation layer wholesale +(content blocks, stop reasons, refusal handling — see +``providers/anthropic.py``). What differs is everything around the +transport: + +* **Client:** ``anthropic.AnthropicBedrock`` instead of + ``anthropic.Anthropic``. Same request/response types, same typed + exception classes. +* **Credentials:** native AWS SigV4 via the standard AWS credential + chain — ``AWS_ACCESS_KEY_ID``/``AWS_SECRET_ACCESS_KEY`` environment + variables, or a ``~/.aws`` profile. The registry constructs every + adapter as ``cls(api_key=..., base_url=...)`` (see + ``registry.build_adapter``), and Bedrock has no API-key equivalent in + the pinned SDK, so ``api_key`` is IGNORED here (with a one-time + warning when set). This keeps the config schema and the Go CLI + untouched. +* **Region:** resolved by the SDK — ``AWS_REGION`` env var, then the + boto3 session (``~/.aws/config``), then a warned ``us-east-1`` + fallback. ``base_url`` is still honoured as a full endpoint override + (VPC endpoints, internal gateways). +* **Model IDs are inference profiles:** prefixed ``us.`` / ``eu.`` / + ``global.`` — e.g. ``us.anthropic.claude-sonnet-4-6`` or + ``global.anthropic.claude-haiku-4-5-20251001-v1:0`` (the version + suffix varies by model) — not bare Anthropic model names. A model + that is not enabled for the + account/region surfaces as an AccessDenied-flavoured 403 — mapped to + ``LLMAuthError`` with a pointer at the Bedrock console's + "Model access" page. A malformed model ID surfaces as a 400 + ValidationException — mapped to ``LLMNotFoundError`` (best-effort + message sniff) so a typo'd profile ID fails validate() the same way + a typo'd model fails on the other adapters. +* **Missing credentials:** the SDK raises a bare ``RuntimeError`` from + its SigV4 signing path when the chain resolves nothing; mapped to + ``LLMAuthError`` so scans fail with a typed, actionable message. +* **No 529:** Bedrock throttling arrives as 429-equivalents through the + SDK's ``RateLimitError`` and is reported to the global rate limiter + exactly like the reference adapter. Bedrock does not emit Anthropic's + 529 "overloaded", but the 529 branch is kept (it is harmless and + ``base_url`` may point at an Anthropic-compat gateway that does). +""" + +from __future__ import annotations + +import sys +import threading +from typing import Any, Optional + +import anthropic + +from ._ratelimit import report_rate_limit, wait_for_rate_limit +from .anthropic import ( + _message_to_anthropic, + _response_to_unified, + _retry_after_from, + _tool_to_anthropic, +) +from .._pricing import LazyProviderPricing +from .._redact import redact_secrets, redacted_cause_from +from ..adapter import ( + CompletionResult, + LLMAuthError, + LLMConnectionError, + LLMNotFoundError, + LLMRateLimitError, + LLMResponseError, + Message, + ToolDef, +) + + +_MODEL_ACCESS_HINT = ( + " (Bedrock AccessDenied usually means this model is not enabled for " + "the account in this region — request it under 'Model access' in the " + "Bedrock console, and check AWS_REGION matches where access was granted)" +) + +_NO_CREDENTIALS_MARKER = "could not resolve credentials" + +# One-time warning when a config sets api_key on a bedrock provider. +# Silent-ignoring config a user explicitly wrote would violate the +# no-silent-drops rule the other adapters follow. +_api_key_warned = False +_api_key_warned_lock = threading.Lock() + + +def _warn_api_key_ignored() -> None: + global _api_key_warned + with _api_key_warned_lock: + if _api_key_warned: + return + _api_key_warned = True + sys.stderr.write( + "warning: BedrockAdapter ignores `api_key` — Bedrock authenticates " + "via the AWS credential chain (AWS_ACCESS_KEY_ID/" + "AWS_SECRET_ACCESS_KEY env vars or ~/.aws profile). Remove " + "`api_key` from the bedrock provider entry to silence this.\n" + ) + + +def reset_warnings() -> None: + """Clear this adapter's one-time-warning memory (for tests / new scans).""" + global _api_key_warned + with _api_key_warned_lock: + _api_key_warned = False + + +class BedrockAdapter: + """:class:`LLMAdapter` implementation backed by ``anthropic.AnthropicBedrock``.""" + + name = "bedrock" + supports_tools = True + + # Per-million-token rates, resolved lazily from config/models.json. + # Bedrock's Claude token prices match the direct Anthropic API; the + # registry keys them by inference-profile ID (us.anthropic..., + # eu.anthropic...) so cost tracking resolves without warnings. + pricing = LazyProviderPricing("bedrock") + + def __init__( + self, + *, + api_key: Optional[str] = None, + base_url: Optional[str] = None, + max_retries: int = 5, + _client: Optional[anthropic.AnthropicBedrock] = None, + ): + """Construct the adapter. + + Args: + api_key: IGNORED (one-time warning when set). Bedrock has no + API-key auth in the pinned SDK; credentials come from + the standard AWS chain. Accepted so the registry can + construct every adapter with the same two kwargs. + base_url: Full endpoint override (VPC endpoint, gateway). + ``None`` means the SDK's regional default + (bedrock-runtime..amazonaws.com). + max_retries: Forwarded to the SDK. The SDK's built-in retry + covers transient network blips; our rate limiter + handles 429-coordinated backoff on top. + _client: Injected SDK instance for testing. Production + callers should not pass this. + """ + if api_key is not None: + _warn_api_key_ignored() + if _client is not None: + self._client = _client + return + + kwargs: dict[str, Any] = {"max_retries": max_retries} + if base_url is not None: + kwargs["base_url"] = base_url + # Deliberately no aws_* kwargs: region and credentials resolve + # through the SDK's own chain (AWS_REGION env → boto3 session → + # us-east-1 fallback) so `openant` behaves exactly like the + # aws CLI on the same machine. + self._client = anthropic.AnthropicBedrock(**kwargs) + + # ------------------------------------------------------------------ + # Public API + # ------------------------------------------------------------------ + + def complete( + self, + *, + model: str, + system: Optional[str], + messages: list[Message], + max_tokens: int, + tools: Optional[list[ToolDef]] = None, + ) -> CompletionResult: + # supports_tools=True so we don't gate-check `tools` here — + # the contract allows tools through. + request: dict[str, Any] = { + "model": model, + "max_tokens": max_tokens, + "messages": [_message_to_anthropic(m) for m in messages], + } + if system is not None: + request["system"] = system + if tools: + request["tools"] = [_tool_to_anthropic(t) for t in tools] + + # Cooperate with the cross-worker backoff before issuing the + # call — same pattern as the other adapters (see _ratelimit.py). + wait_for_rate_limit() + + try: + response = self._client.messages.create(**request) + except anthropic.AuthenticationError as exc: + raise LLMAuthError(redact_secrets(str(exc))) from redacted_cause_from(exc) + except anthropic.PermissionDeniedError as exc: + # On Bedrock a 403 is most often AccessDenied for a model + # the account never enabled — auth-shaped, but the fix is + # in the Bedrock console, so say so. + raise LLMAuthError( + redact_secrets(str(exc)) + _MODEL_ACCESS_HINT + ) from redacted_cause_from(exc) + except anthropic.RateLimitError as exc: + # Bedrock throttling (ThrottlingException) rides the SDK's + # 429 mapping; report it to the global limiter exactly like + # the reference adapter so multi-worker scans coordinate. + retry_after = _retry_after_from(exc) + report_rate_limit(retry_after) + raise LLMRateLimitError(redact_secrets(str(exc)), retry_after=retry_after) from redacted_cause_from(exc) + except anthropic.NotFoundError as exc: + raise LLMNotFoundError(redact_secrets(str(exc))) from redacted_cause_from(exc) + except anthropic.APIConnectionError as exc: + raise LLMConnectionError(redact_secrets(str(exc))) from redacted_cause_from(exc) + except anthropic.APIStatusError as exc: + raise _classify_status_error(exc, report_429=True) from redacted_cause_from(exc) + except RuntimeError as exc: + # The SDK's SigV4 signer raises a bare RuntimeError when the + # AWS chain resolves no credentials at all. Surface it typed. + if _NO_CREDENTIALS_MARKER in str(exc): + raise LLMAuthError(_no_credentials_message(exc)) from redacted_cause_from(exc) + raise + + return _response_to_unified(response, adapter="BedrockAdapter") + + def validate(self, model: str) -> None: + # Cheapest valid request: 1-token cap, single "hi" message. + # Probing the actual configured inference-profile ID catches + # typo'd/unenabled profiles at init. Like the reference + # adapter, validate() does not wait on the cross-worker + # backoff — it is a one-shot probe at scan startup. + try: + self._client.messages.create( + model=model, + max_tokens=1, + messages=[{"role": "user", "content": "hi"}], + ) + except anthropic.AuthenticationError as exc: + raise LLMAuthError(redact_secrets(str(exc))) from redacted_cause_from(exc) + except anthropic.PermissionDeniedError as exc: + raise LLMAuthError( + redact_secrets(str(exc)) + _MODEL_ACCESS_HINT + ) from redacted_cause_from(exc) + except anthropic.RateLimitError as exc: + retry_after = _retry_after_from(exc) + raise LLMRateLimitError(redact_secrets(str(exc)), retry_after=retry_after) from redacted_cause_from(exc) + except anthropic.NotFoundError as exc: + raise LLMNotFoundError(redact_secrets(str(exc))) from redacted_cause_from(exc) + except anthropic.APIConnectionError as exc: + raise LLMConnectionError(redact_secrets(str(exc))) from redacted_cause_from(exc) + except anthropic.APIStatusError as exc: + raise _classify_status_error(exc, report_429=False) from redacted_cause_from(exc) + except RuntimeError as exc: + if _NO_CREDENTIALS_MARKER in str(exc): + raise LLMAuthError(_no_credentials_message(exc)) from redacted_cause_from(exc) + raise + + +# ---------------------------------------------------------------------- +# Bedrock-specific error classification +# ---------------------------------------------------------------------- + + +def _classify_status_error(exc: anthropic.APIStatusError, *, report_429: bool) -> Exception: + """Map a residual APIStatusError to the adapter taxonomy. + + Returns (not raises) the mapped exception so callers keep their + ``raise ... from`` chaining at the call site. + """ + status = getattr(exc, "status_code", None) + message = redact_secrets(str(exc)) + if status == 529: + # Bedrock itself never sends 529, but base_url may point at an + # Anthropic-compat gateway that does; classify it transient, + # same as the reference adapter. + retry_after = _retry_after_from(exc) + if report_429: + report_rate_limit(retry_after) + return LLMRateLimitError(message, retry_after=retry_after) + if status == 400 and "model identifier" in message.lower(): + # Bedrock reports a malformed/unknown model ID as a 400 + # ValidationException ("The provided model identifier is + # invalid."), not a 404. Best-effort sniff so a typo'd + # inference-profile ID fails like a typo'd model elsewhere. + return LLMNotFoundError( + message + " (Bedrock model IDs are inference profiles, e.g. " + "us.anthropic.claude-sonnet-4-6 — list what is enabled with " + "`aws bedrock list-inference-profiles`)" + ) + # Everything else (other 400s, 422, 500, ...) is a structural + # response problem from the pipeline's perspective. + return LLMResponseError(message) + + +def _no_credentials_message(exc: RuntimeError) -> str: + return ( + f"{redact_secrets(str(exc))} — the AWS credential chain resolved no " + f"credentials. Export AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY (and " + f"AWS_REGION), or configure a ~/.aws profile; the bedrock adapter " + f"does not use `api_key`." + ) From 733d62833fe93b5ab67b9503fbc7dd750cd86084 Mon Sep 17 00:00:00 2001 From: nahumkorda Date: Sat, 1 Aug 2026 13:15:58 +0300 Subject: [PATCH 2/2] docs: usage guide for the Bedrock adapter MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds utilities/llm/providers/BEDROCK.md — prerequisites (model access + minimal IAM policy), the credential/region resolution chain, full config.json example, the inference-profile ID conventions and their suffix inconsistency, cost accounting, an error/troubleshooting table, and current limitations — and links it from the README's shipped-adapters table. Placed next to the adapter code (the repo convention, cf. the parser PARSER_PIPELINE.md files) because docs/ is gitignored. Co-Authored-By: Claude Fable 5 --- README.md | 2 +- .../utilities/llm/providers/BEDROCK.md | 188 ++++++++++++++++++ 2 files changed, 189 insertions(+), 1 deletion(-) create mode 100644 libs/openant-core/utilities/llm/providers/BEDROCK.md diff --git a/README.md b/README.md index 92dbc99f..34b791f6 100644 --- a/README.md +++ b/README.md @@ -87,7 +87,7 @@ Wizard defaults reflect the project's per-phase recommendations (stronger reason | `anthropic` | [console.anthropic.com](https://console.anthropic.com/settings/keys) | Reference adapter. NOT included in Claude Pro / Max subscriptions — separate billing. | | `openai` | [platform.openai.com](https://platform.openai.com/api-keys) | NOT included in ChatGPT / Codex subscriptions — separate billing. | | `google` | [aistudio.google.com](https://aistudio.google.com/apikey) | NOT included in Gemini Advanced — separate billing. | -| `bedrock` | — (AWS credential chain) | Claude on AWS Bedrock. No `api_key`: credentials come from `AWS_ACCESS_KEY_ID`/`AWS_SECRET_ACCESS_KEY` env vars or a `~/.aws` profile, region from `AWS_REGION`. Model IDs are inference profiles (`us.anthropic.claude-sonnet-4-6`, `global.anthropic.claude-haiku-4-5-20251001-v1:0`, ...) — enable them under "Model access" in the Bedrock console and list them with `aws bedrock list-inference-profiles`. Not offered by the setup wizard yet; configure by hand (see below). | +| `bedrock` | — (AWS credential chain) | Claude on AWS Bedrock. No `api_key`: credentials come from `AWS_ACCESS_KEY_ID`/`AWS_SECRET_ACCESS_KEY` env vars or a `~/.aws` profile, region from `AWS_REGION`. Model IDs are inference profiles (`us.anthropic.claude-sonnet-4-6`, `global.anthropic.claude-haiku-4-5-20251001-v1:0`, ...) — enable them under "Model access" in the Bedrock console and list them with `aws bedrock list-inference-profiles`. Not offered by the setup wizard yet; configure by hand — full guide: [`utilities/llm/providers/BEDROCK.md`](libs/openant-core/utilities/llm/providers/BEDROCK.md). | All four support tool calling, so any of them can drive the `enhance` and `verify` phases that use the agentic tool-use loop. diff --git a/libs/openant-core/utilities/llm/providers/BEDROCK.md b/libs/openant-core/utilities/llm/providers/BEDROCK.md new file mode 100644 index 00000000..ebd70f13 --- /dev/null +++ b/libs/openant-core/utilities/llm/providers/BEDROCK.md @@ -0,0 +1,188 @@ +# Using the Bedrock adapter + +OpenAnt's `bedrock` provider type runs the pipeline's Claude phases +through [Amazon Bedrock](https://aws.amazon.com/bedrock/) instead of the +direct Anthropic API. Same models, same wire format (the adapter is +built on `anthropic.AnthropicBedrock`), but billed to your AWS account +and authenticated with AWS credentials instead of an API key. + +Reasons to use it instead of the `anthropic` adapter: + +- **AWS-consolidated billing and governance.** Token spend lands on the + AWS bill, inside existing budgets, Cost Explorer, and IAM controls — + no separate Anthropic billing account. +- **No long-lived API key.** Auth rides the standard AWS credential + chain, including short-lived STS/SSO credentials. +- **Data-locality options.** Regional inference profiles (`us.…`, + `eu.…`) keep traffic inside a geography; `global.…` profiles trade + that for better availability. + +## Prerequisites + +1. An AWS account with **model access enabled** for the Claude models + you plan to use: Bedrock console → *Model access* → request/enable + the Anthropic models. Without this, every call fails with an + AccessDenied 403 (see troubleshooting). +2. Credentials with permission to invoke them. A minimal IAM policy: + + ```json + { + "Version": "2012-10-17", + "Statement": [{ + "Effect": "Allow", + "Action": ["bedrock:InvokeModel"], + "Resource": [ + "arn:aws:bedrock:*::foundation-model/anthropic.*", + "arn:aws:bedrock:*:*:inference-profile/*" + ] + }] + } + ``` + + (Both resource types are needed: requests target an inference + profile, which fans out to regional foundation models.) Add + `bedrock:ListInferenceProfiles` if you want the listing command + below to work with the same credentials. +3. Credentials and region visible to the process — see next section. + +## Credentials and region + +The adapter deliberately takes **no** AWS-specific configuration. +Credentials and region resolve through the AWS SDK's standard chain, so +`openant` authenticates exactly like the `aws` CLI on the same machine: + +- **Credentials**, in the SDK's order: `AWS_ACCESS_KEY_ID` / + `AWS_SECRET_ACCESS_KEY` (+ `AWS_SESSION_TOKEN` for temporary + credentials) environment variables; then the `~/.aws/credentials` / + `~/.aws/config` profiles (honoring `AWS_PROFILE`), including SSO and + assumed roles. +- **Region**: the `AWS_REGION` environment variable, then the region of + the resolved AWS profile. If neither is set the SDK falls back to + `us-east-1` with a warning — set the region explicitly rather than + relying on that. + +Sanity-check both before a scan; if this works, OpenAnt will too: + +```bash +aws sts get-caller-identity +aws bedrock list-inference-profiles --query 'inferenceProfileSummaries[].inferenceProfileId' +``` + +## Configuration + +The `openant setup llm` wizard does not offer `bedrock` yet; add it to +`~/.config/openant/config.json` by hand. Note the provider entry has +**no `api_key`** — a complete single-provider example (all seven +pipeline phases are required): + +```json +{ + "$schema_version": 2, + "default_llm": "via-bedrock", + "llm_providers": { + "bedrock": {"type": "bedrock"} + }, + "llm_configs": { + "via-bedrock": { + "app_context": {"provider": "bedrock", "model": "us.anthropic.claude-haiku-4-5-20251001-v1:0"}, + "llm_reach": {"provider": "bedrock", "model": "us.anthropic.claude-sonnet-4-6"}, + "enhance": {"provider": "bedrock", "model": "us.anthropic.claude-haiku-4-5-20251001-v1:0"}, + "analyze": {"provider": "bedrock", "model": "us.anthropic.claude-sonnet-4-6"}, + "verify": {"provider": "bedrock", "model": "us.anthropic.claude-sonnet-4-6"}, + "dynamic_test": {"provider": "bedrock", "model": "us.anthropic.claude-haiku-4-5-20251001-v1:0"}, + "report": {"provider": "bedrock", "model": "us.anthropic.claude-haiku-4-5-20251001-v1:0"} + } + } +} +``` + +Run a scan against it: + +```bash +openant scan /path/to/repo --llm-config via-bedrock +``` + +Mixing is fine too — an `llm_configs` entry can point some phases at +`bedrock` and others at any other provider. + +If an `api_key` is present on the entry anyway (e.g. a config shared +across provider types), it is ignored with a one-time warning — Bedrock +has no API-key auth in the pinned SDK. + +### `base_url` override + +Only needed for a Bedrock VPC endpoint (PrivateLink) or an +Anthropic-compatible internal gateway: + +```json +"bedrock": {"type": "bedrock", "base_url": "https://vpce-….bedrock-runtime.us-east-1.vpce.amazonaws.com"} +``` + +## Model IDs are inference profiles + +Bedrock does not serve Claude under the direct-API model names. Requests +target **inference profiles**, whose IDs add a routing prefix — and, +inconsistently, a version suffix: + +| Direct Anthropic ID | Bedrock inference profile | +|---|---| +| `claude-opus-4-8` | `us.anthropic.claude-opus-4-8` / `global.anthropic.claude-opus-4-8` | +| `claude-sonnet-4-6` | `us.anthropic.claude-sonnet-4-6` / `global.anthropic.claude-sonnet-4-6` | +| `claude-haiku-4-5-20251001` | `us.anthropic.claude-haiku-4-5-20251001-v1:0` / `global.anthropic.claude-haiku-4-5-20251001-v1:0` | + +Do **not** guess IDs by convention — the `-v1:0` suffix exists on some +profiles and not others (the table above was verified against a live +account). List what your account actually has: + +```bash +aws bedrock list-inference-profiles --query 'inferenceProfileSummaries[].inferenceProfileId' +``` + +`us.` profiles route within US regions; `global.` profiles route +worldwide for better availability; `eu.` / `apac.` variants exist for +accounts homed in those geographies. IDs pass through to Bedrock +verbatim. + +## Cost accounting + +`config/models.json` ships pricing records for the profiles in the table +above (both `us.` and `global.` variants), mirroring the direct +Anthropic rates — Bedrock's Claude token pricing matches the direct API. +A profile outside that set still works but reports `$0` in cost +accounting with a one-time warning; add a record with +`"provider": "bedrock"` to price it. + +## Errors and troubleshooting + +| Symptom | Meaning | Fix | +|---|---|---| +| `LLMAuthError: … AWS_ACCESS_KEY_ID …` | The AWS chain resolved no credentials at all | Export the env vars or configure `~/.aws`; verify with `aws sts get-caller-identity` | +| `LLMAuthError: … 403 … Model access …` | Credentials are valid but the model isn't enabled (or IAM denies `bedrock:InvokeModel`) | Enable the model under *Model access* in the Bedrock console; check the IAM policy above | +| `LLMAuthError: … account is currently being verified …` | New-account holdback — AWS verifies fresh accounts before serving some models | Transient; typically clears within a couple of hours. Meanwhile `global.` profiles may already work | +| `The security token included in the request is invalid` | Expired/rotated or malformed credentials | Refresh them; if pasted by hand, check for copy artifacts (a real `AWS_ACCESS_KEY_ID` is exactly 20 characters) | +| `LLMNotFoundError: … model identifier is invalid …` | Typo'd or unavailable profile ID (Bedrock reports this as a 400, not a 404) | Use `list-inference-profiles` output verbatim; mind the `-v1:0` suffix inconsistency | +| `LLMRateLimitError` | Bedrock throttling (429) | Nothing to do — workers back off cooperatively via the global rate limiter | + +Two behaviors worth knowing because they differ from the `anthropic` +adapter: + +- **Model access is a second gate.** Valid AWS credentials are not + enough; each model must also be enabled for the account. That's why + 403s get the "Model access" hint. +- **Unknown models are 400s.** Bedrock reports a bad model ID as a 400 + ValidationException; the adapter still surfaces it as + `LLMNotFoundError`, so a typo'd profile fails fast at + config-validation time instead of mid-scan. + +## Current limitations + +- Not offered by the `openant setup llm` wizard yet (config by hand, as + above). Wizard support needs a few Go touch-points in + `apps/openant-cli/cmd/setup.go`. +- Only Claude models. Bedrock hosts other model families, but this + adapter speaks Anthropic's wire format; non-Claude Bedrock models + would need their own adapter. +- No per-provider region field — region comes from the environment or + AWS profile. To scan against two regions, run with different + `AWS_REGION` values (or `AWS_PROFILE`s) rather than two provider + entries.