Skip to content

Redfish 4xx/5xx exception data recorded on the root operation span, not the bmc client span (contra telemetry-contract.md) #387

Description

@spyroot

Defect

A BMC HTTP error (e.g. 400 from a rejected PATCH) surfaces in Splunk APM as a redfish_ctl.redfish_exceptions.RedfishException with a full stack trace on the root operation span (e.g. dns-set), instead of as record_exception + error.type on the client (bmc) span. The root reads like a code bug and the exception data is on the wrong span; by contrast 404s show cleanly as http_404 on the client span.

Contract violated

spec/dmtf/redfish/2026.1/reference/telemetry-contract.md (Tracing): Redfish client spans include method, target host, status code, exception data, and error status for 4xx/5xx. So exception data + error status belong on the CLIENT span; the root should carry a clean operation STATUS.

Root cause

read_api_respond (redfish_manager_base.py:1373) raises RedfishException (400/401/403/404/5xx) during response parsing — after the client span (traced_request) has closed at the HTTP call. So it propagates past the client span, up through execute, and OTel auto-records it on the root operation_span with a stack trace.

Fix direction

  • record_exception the RedfishException on the client span (it already carries the structured envelope: status, error.code, @Message.ExtendedInfo) + error.type=http_<code> + http.response.status_code + ERROR status — at the traced_request/base_request_respond seam, while the span is still open.
  • Root operation span then shows a clean STATUS=ERROR.
  • Do NOT str()-flatten the envelope (error contract: "Do not route JSON or YAML through str(exception)").
  • Cover sync AND async paths; keep async context across the executor boundary (telemetry-contract.md "Async route").

Evidence

Live GB300 trace c3731 (dns-set, bad IP 999.999.999.999 -> HTTP 400): root redfish_ctl: dns-set shows RedfishException + stack trace + exception.message; children redfish.bmc.request show 400. Tag Spotlight: RedfishException (1) vs http_404 (3).

Note

PR #386 attempted this but is contract-misaligned (str()-flattened the envelope, targeted the root span, dead code for the 400 since base_patch raises). To be closed in favor of this issue.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions