feat(python): add openinference-instrumentation-typesafe

[axiomofjoy]

Adds openinference-instrumentation-typesafe, an instrumentor for the TypeSafe AI Python SDK (typesafe-sdk >= 0.6.0).

What it traces

TypeSafeClient.system_one and AsyncTypeSafeClient.system_one become OpenInference LLM spans.

A System One call isn't a chat exchange. You send a state plus a map of typed questions (Noul, Choice, Score) and get back one typed answer per question — there's no message list on either side. So the span records the request and response bodies as input.value and output.value and stops there; no llm.input_messages / llm.output_messages. The questions map rides in llm.invocation_parameters, where it plays the same role a JSON response schema plays for chat-completion APIs.

Beyond that: llm.provider (typesafe), llm.request.model_name and llm.response.model_name (e.g. jev-latest → jev-1.13.0), and llm.token_count.{prompt,completion,total}.

Everything is built from the shared helpers in openinference.instrumentation, and spans go through OITracer — so TraceConfig masking, context attributes (session, user, metadata, tags) and suppress_tracing all work. Errors record the exception and set ERROR status.

Notable choices

• Patched through the SDK's public surface. The wrappers target typesafe_sdk.TypeSafeClient / typesafe_sdk.AsyncTypeSafeClient rather than the typesafe_sdk._core.client.* modules that define them. wrapt patches the same class object either way, and the private path carries no compatibility promise.
• python/uv.toml gets a typesafe-sdk exemption. The instrumentor's floor, 0.6.0, was published more recently than CI's rolling 3-day exclude-newer cutoff, so resolution fails without it. It uses the far-future date the file already documents, deliberately: a per-package timestamp overrides the global cutoff, so a real date would become a permanent ceiling and quietly turn the typesafe-latest canary into a no-op.
• No VCR cassettes. The SDK speaks httpx2, which vcrpy doesn't patch, so the tests inject an httpx2.MockTransport that records request bodies and replays a canned System One response. Fully offline, no API key.

Testing

Covers sync and async, structured state, raw-dict questions, the client-level default model, extra_body, error status, suppress_tracing, context attributes, TraceConfig(hide_inputs, hide_outputs), uninstrument, and the entry point.

tox run-parallel -e py310-ci-typesafe,py310-ci-typesafe-latest,py314-ci-typesafe,py314-ci-typesafe-latest

All four pass: ruff format, ruff check, mypy clean, 10 tests.

Also verified end to end against a local Phoenix by running the three examples/*.py scripts against the live API — 4 traces, 4 LLM spans, all OK, with the request body, invocation params, token counts, metadata and session link rendering as expected.

Open questions for reviewers

1. Should typesafe become a well-known llm.provider value? Right now it's a literal in this package and is absent from OpenInferenceLLMProviderValues (Python and TS) and from spec/semantic_conventions.md. Precedent (#3348 for Ollama) says the enum entry lands in the same PR as the instrumentor; I left it out because it's a cross-language spec change and wanted a call on the spelling (typesafe vs typesafe-ai) first. Happy to add it here.
2. input.value is reconstructed, not captured. get_request_attributes rebuilds the request body from the bound call arguments, and the tests assert it equals what went over the wire. That equality is maintained by a second copy of the SDK's body-building rules, so an SDK change to defaults or the extra_body merge would make the span quietly disagree with the request. Wrapping typesafe_sdk._core.endpoints.prepare_system_one instead would make the captured body a fact rather than a reconstruction — worth doing now, or as a follow-up?
3. Per-answer confidence and probabilities currently live only inside the output.value JSON. Flattened attributes for them could be a follow-up.
4. The instrumentor class is TypeSafeAIInstrumentor (vendor's full name, cf. VertexAIInstrumentor) while the distribution is openinference-instrumentation-typesafe. Say the word if you'd rather have TypeSafeInstrumentor.

The SDK has no streaming and no tool calling, so there's no stream proxy.

---

Generated by Claude Code

[mikeldking]

Looks like this was kicked off #3769

[github-actions]

Code review

No issues found. Checked for bugs and CLAUDE.md compliance.

[caroger]

hide_inputs redacts input.value and leaves this llm.invocation_parameters copy of questions on the span. Chat instrumentors get the prompt stripped because it lives on llm.input_messages, which that flag does clear.

That is a masking hole, not a leftover field. TypeSafe instructions are the prompt. They often describe customer tickets. A user who sets hide_inputs=True because state is PII still exports the full questions map, and the README tells them hide_inputs / hide_outputs is enough. test_trace_config_hides_inputs_and_outputs never looks at LLM_INVOCATION_PARAMETERS, so this ships as "masking works."

Update the README masking bullet to TraceConfig(hide_inputs=True, hide_outputs=True, hide_llm_invocation_parameters=True). Change test_trace_config_hides_inputs_and_outputs to instrument with that config and assert SpanAttributes.LLM_INVOCATION_PARAMETERS is absent.

[mikeldking]

Fixed in 5a230f3, though by a different route than suggested, and the PII framing here doesn't hold for this API.

On the premise. A System One call has no input messages — the analogy to chat instrumentors getting the prompt stripped via llm.input_messages doesn't transfer. The customer ticket is the state, and state was already excluded from the invocation parameters (k != "state"), so it was never in the copy that survived hide_inputs. What survived was the developer-authored question schema: instructions plus criteria. Canary run on a local Phoenix, real API call, TraceConfig(hide_inputs=True, hide_outputs=True) under the old code:

input.value=__REDACTED__   output.value=__REDACTED__
PII-CANARY-STATE anywhere:    false     <- caller data was already covered
INSTRUCTIONS-CANARY anywhere: true      <- the question schema was not

On the real problem. Your mechanism was right: hide_inputs doesn't touch LLM_INVOCATION_PARAMETERS (only hide_llm_invocation_parameters does, config.py:346). And it was worse than one flag missing, because questions was recorded twice — in input.value and again in the invocation parameters. hide_inputs cleared the first copy, hide_llm_invocation_parameters the second, so neither flag alone removed the instructions and the split was the actual defect.

What we did instead of documenting a third flag. llm.invocation_parameters is now call configuration only: the model and any extra_body fields. The state and the questions are recorded once, in input.value. So TraceConfig(hide_inputs=True, hide_outputs=True) is the whole masking story for request and response content, with no extra flag for a user to know about:

typesafe-hide-inputs-v2   input.value=__REDACTED__  output.value=__REDACTED__
                          llm.invocation_parameters={"model": "jev-latest"}
                          PII-CANARY-STATE anywhere:    false
                          INSTRUCTIONS-CANARY anywhere: false
                          token_count.total=385

test_trace_config_hides_inputs_and_outputs now asserts both the state and the instructions text are absent under hide_inputs alone, and the README no longer advertises hide_llm_invocation_parameters as part of the masking story, since it no longer covers request content.

The trade-off, for the record: llm.invocation_parameters is now {"model": ...} for calls that pass no extra_body, which is thin in the Phoenix params panel. Happy to revisit if you'd rather see the schema surfaced there — but if so it should be under an attribute that hide_inputs covers, not one it doesn't.

[caroger]

state and questions go through _to_builtins. extra_body does not, so Mapping/Sequence values become repr strings on the span while the SDK encodes them as JSON. extra_body={"ids": range(3)} is [0,1,2] on the wire and "range(0, 3)" here.

The PR's contract is that input.value is the request body. Tests assert it equals the recorded HTTP JSON. extra_body is a public SDK field whose values are JSONValue, which includes Mapping and Sequence. Leaving it unconverted means the span reports a string the collector cannot query the same way as the wire, and the MappingProxyType fix you already landed for questions does not apply to the rest of the body.

After body.update(extra_body), set body = _to_builtins(body). Add a test next to test_abstract_mapping_questions_and_tuple_state that input.value equals the recorded request when extra_body holds a MappingProxyType.

[mikeldking]

Confirmed and fixed in 5fdfa2e. extra_body now goes through _to_builtins before the merge, so it gets the same treatment as state and questions.

The mechanism is safe_json_dumps, which is json.dumps(..., default=str) — so an abstract Mapping in extra_body was serialized as its repr, on both input.value and llm.invocation_parameters.

Proven against a local Phoenix with two builds of the same working tree, one call each to the real API, passing extra_body={"state": MappingProxyType({...})} (the SDK documents extra_body as last-write-wins over state):

jq -r '.[0].attributes["input.value"] | fromjson | .state | type'

before (fix reverted): string   -> "{'ticket': {'subject': 'Charged twice for one order', ...}}"
after:                 object   -> {"ticket": {"subject": "Charged twice for one order", ...}}

test_abstract_containers_in_extra_body covers it. One note for whoever reads that test next: the tuple assertion passes either way, since json.dumps already renders tuples as arrays — the MappingProxyType assertion is the one that catches the bug.