Version: opentelemetry-instrumentation-genai-langchain 1.1b1
(callback_handler.py:214-227 and :322-388), with langchain-ollama 1.1.0.
What happens.
on_chat_model_start resolves the request model from invocation_params and metadata under
the keys model_name, model_id, model, and returns without a span when none is present.
ChatOllama publishes the model only as the LangSmith key metadata["ls_model_name"], so no
inference span is ever produced for it (tool and chain spans are unaffected).
on_llm_end builds ToolCallRequest output parts only when the resolved finish_reason is
tool_calls or tool_use. ChatOllama reports done_reason in generation_info (no
finish_reason), so the resolved value is "unknown" and the message's tool_calls are
dropped from gen_ai.output.messages; downstream, tool spans cannot be linked to the
inference that requested them.
Reproduce. ChatOllama(model=...).bind_tools([tool]).invoke([...]) under the instrumentation.
Suggested fix. Fall back to metadata["ls_model_name"] (and ls_provider, which
normalize_provider already reads) when no model key is present; build tool-call parts whenever
message.tool_calls is non-empty, independent of the provider's finish-reason spelling.
Downstream workaround (forgehawk SDK). ForgehawkLangChainCallbackHandler copies ls_model_name into
metadata["model"] and passes the official on_llm_end a view with finish_reason="tool_calls"
when the message carries tool calls and the official resolution found nothing
(sdk/python/src/forgehawk/instrumentors/langchain.py).
Found while building the forgehawk Python SDK (ZioSec) on top of opentelemetry-util-genai; happy to open a PR for the suggested fix if maintainers agree with the direction.
Version:
opentelemetry-instrumentation-genai-langchain1.1b1(
callback_handler.py:214-227and:322-388), withlangchain-ollama1.1.0.What happens.
on_chat_model_startresolves the request model frominvocation_paramsandmetadataunderthe keys
model_name,model_id,model, and returns without a span when none is present.ChatOllamapublishes the model only as the LangSmith keymetadata["ls_model_name"], so noinference span is ever produced for it (tool and chain spans are unaffected).
on_llm_endbuildsToolCallRequestoutput parts only when the resolvedfinish_reasonistool_callsortool_use.ChatOllamareportsdone_reasoningeneration_info(nofinish_reason), so the resolved value is"unknown"and the message'stool_callsaredropped from
gen_ai.output.messages; downstream, tool spans cannot be linked to theinference that requested them.
Reproduce.
ChatOllama(model=...).bind_tools([tool]).invoke([...])under the instrumentation.Suggested fix. Fall back to
metadata["ls_model_name"](andls_provider, whichnormalize_provideralready reads) when no model key is present; build tool-call parts whenevermessage.tool_callsis non-empty, independent of the provider's finish-reason spelling.Downstream workaround (forgehawk SDK).
ForgehawkLangChainCallbackHandlercopiesls_model_nameintometadata["model"]and passes the officialon_llm_enda view withfinish_reason="tool_calls"when the message carries tool calls and the official resolution found nothing
(
sdk/python/src/forgehawk/instrumentors/langchain.py).Found while building the forgehawk Python SDK (ZioSec) on top of opentelemetry-util-genai; happy to open a PR for the suggested fix if maintainers agree with the direction.