Skip to content

[Operability] Preserve server diagnostics behind generic orchestrator 503 responses #361

Description

@seonghobae

Buyer and operator problem

Global Ask deliberately hides raw provider and orchestration failures behind a stable 503 so customer-facing responses do not leak provider traces. The current broad except Exception boundary also captures unexpected programming defects, but the cause is not guaranteed to reach structured server telemetry. This can turn a real regression into an opaque availability incident.

Required boundary

  • Keep the current generic customer response and exception-detail non-disclosure.
  • Classify known provider/transport/schema exceptions explicitly.
  • Emit structured server-side diagnostics for unexpected exceptions with operation code, correlation/request ID, exception class, and stack trace.
  • Exclude prompt text, response text, bearer tokens, provider keys, tenant PII, source-post bodies, and model credentials from logs and traces.
  • Preserve exception chaining.
  • Define the alert/metric distinction between known provider unavailability and internal programming faults.

Acceptance

  • known provider failure → generic 503 + non-sensitive provider-unavailable metric;
  • unexpected AttributeError/programming defect → generic 503 + structured internal-error log/trace and distinct metric;
  • no raw exception message is returned to the caller;
  • log-capture tests verify required fields and forbidden sensitive fields;
  • OpenTelemetry cardinality remains bounded;
  • full backend tests, security scan, SAST, and operability documentation pass;
  • no Global Ask feature or fail-closed behavior is removed.

Origin

Tracked from duplicate exact-head review observations on PR #342. This issue preserves the secure customer boundary while restoring operator diagnosability.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions