Skip to content

instrumentation-genai-langchain: ChatOllama produces no inference span (model key is ls_model_name) and tool-call output parts are dropped when the provider reports done_reason #679

Description

@bytesrabbit

Version: opentelemetry-instrumentation-genai-langchain 1.1b1
(callback_handler.py:214-227 and :322-388), with langchain-ollama 1.1.0.

What happens.

  1. on_chat_model_start resolves the request model from invocation_params and metadata under
    the keys model_name, model_id, model, and returns without a span when none is present.
    ChatOllama publishes the model only as the LangSmith key metadata["ls_model_name"], so no
    inference span is ever produced for it (tool and chain spans are unaffected).
  2. on_llm_end builds ToolCallRequest output parts only when the resolved finish_reason is
    tool_calls or tool_use. ChatOllama reports done_reason in generation_info (no
    finish_reason), so the resolved value is "unknown" and the message's tool_calls are
    dropped from gen_ai.output.messages; downstream, tool spans cannot be linked to the
    inference that requested them.

Reproduce. ChatOllama(model=...).bind_tools([tool]).invoke([...]) under the instrumentation.

Suggested fix. Fall back to metadata["ls_model_name"] (and ls_provider, which
normalize_provider already reads) when no model key is present; build tool-call parts whenever
message.tool_calls is non-empty, independent of the provider's finish-reason spelling.

Downstream workaround (forgehawk SDK). ForgehawkLangChainCallbackHandler copies ls_model_name into
metadata["model"] and passes the official on_llm_end a view with finish_reason="tool_calls"
when the message carries tool calls and the official resolution found nothing
(sdk/python/src/forgehawk/instrumentors/langchain.py).


Found while building the forgehawk Python SDK (ZioSec) on top of opentelemetry-util-genai; happy to open a PR for the suggested fix if maintainers agree with the direction.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions