Skip to content

Show upstream network latency and prefill breakdown in request details - #79

Closed
chokhung588-cell wants to merge 1 commit into
AITNR:mainfrom
chokhung588-cell:feat/upstream-latency-column
Closed

chokhung588-cell wants to merge 1 commit into
AITNR:mainfrom
chokhung588-cell:feat/upstream-latency-column

Conversation

@chokhung588-cell

Copy link
Copy Markdown

Summary

  • Add an "Upstream latency" (上游额外延迟) column to the Request Details (请求明细) table in the embedded dashboard.
  • Display a detailed hover tooltip breaking down:
    • Upstream connection setup (DNS + TCP + TLS or Connection reused)
    • Upstream TTFB (Time To First Byte of HTTP response headers)
    • Upstream model prefill time (Prefill = TTFT - Upstream TTFB)
  • Persist column visibility preference in dashboard preferences.
  • Provide full localization across all 4 supported locales (en, zh-CN, zh-TW, ru).
  • Add export support for all 4 new metrics in CSV reports.
  • Seamless backward compatibility: requests from older core versions gracefully display — / 不可用 with zero schema migration required.

Why

In proxy architectures, users frequently ask why the client-side Time To First Token (TTFT) is significantly higher than expected. Without upstream breakdown, operators cannot determine whether the delay is caused by:

  1. The client ↔ proxy route,
  2. The proxy ↔ upstream connection handshake (e.g. cold TLS handshakes), or
  3. The upstream model's prompt prefill computation.

By recording upstream timings exposed by CLIProxyAPI, the dashboard now enables instant diagnosis.

Changes

  1. internal/plugin/usage.go:
    • Decode upstream_ttfb, conn_setup, first_packet, and conn_reused from usage payload with dual format tolerance (nanoseconds integer or duration string).
  2. internal/plugin/request_log.go:
    • Propagate upstream timings to RequestDetail struct.
    • Include 4 new columns in CSV export (Upstream TTFB (ns), Conn setup (ns), First packet (ns), Conn reused).
  3. internal/plugin/dashboard.go:
    • Render the new "Upstream latency" column with formatted seconds (s / ms).
    • Add rich hover tooltip calculating prefill duration and connection reuse status.
  4. internal/plugin/preferences.go:
    • Include upstream latency column in columns_detail preference configuration.
  5. internal/plugin/locales/{en,zh-CN,zh-TW,ru}.json:
    • Add localized labels and tooltip strings across all 4 locales.
  6. Tests (internal/plugin/upstream_latency_test.go):
    • TestDecodeUsageUpstreamTiming: validates JSON parsing of integers and strings.
    • TestDecodeUsageUpstreamTimingMissing: verifies graceful backward compatibility with missing fields.
    • TestRequestDetailForUsagePropagatesUpstreamTiming: verifies struct propagation.
    • TestRequestDetailJSONRoundTripKeepsUpstreamTiming: verifies BoltDB JSON serialization lossless round-trip.

Validation

  • gofmt -w . — clean.
  • go vet ./... — clean (no warnings).
  • go test -v ./... -count=1 — all passed (100% test suite pass).
  • Generated dashboard JavaScript syntax verified.
  • Locale JSON files verified valid UTF-8.
  • Production deployment verification: deployed compiled .so on live Linux VPS (Debian 11 / CPA v7.3.15), verified real streaming requests (gemini-3.8-flash-high) and tooltip accuracy.

@chokhung588-cell

Copy link
Copy Markdown
Author

Holding PR open until upstream CLIProxyAPI core PR (router-for-me/CLIProxyAPI#6087) is merged.

@chokhung588-cell

Copy link
Copy Markdown
Author

PR is ready and tested on live VPS (Debian 11 / CPA v7.3.15). Note: Fully backward-compatible with existing core; paired with router-for-me/CLIProxyAPI#6087.

@chokhung588-cell

Copy link
Copy Markdown
Author

Temporarily closed by author; will reopen once router-for-me/CLIProxyAPI#6087 is merged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant