Skip to content

jsonrpc: replace jhttp.Bridge with an in-tree bridge - #1037

Merged
cjonas9 merged 28 commits into
feature/full-historyfrom
bypass-jrpc2-marshal
Oct 12, 2026
Merged

cjonas9 merged 28 commits into
feature/full-historyfrom
bypass-jrpc2-marshal

Conversation

@cjonas9

@cjonas9 cjonas9 commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

Serves JSON-RPC over HTTP with an in-tree bridge (jsonrpc/bridge.go) instead of jrpc2's jhttp.Bridge. The bridge calls handlers directly, encodes each result once, and writes a json.RawMessage result as is, which #1060 needs for its memoized getLatestLedger body. jrpc2 stays at upstream v1.3.3; go.mod is unchanged.

jhttp.Bridge ran each request through an in-memory client and server and re-encoded the result on the way out. One POST over loopback with a 3.1 MB result (median of 8, M4 Pro, Go 1.26):

result jhttp.Bridge in-tree bridge
struct 38.1 ms, 28.5 MiB 2.57 ms, 7.4 MiB
json.RawMessage 44.0 ms, 24.3 MiB 0.35 ms, 11 KiB

Changes

  • Responses are byte-identical to jhttp.Bridge's (TestBridge_ParityWithJHTTP). As in jrpc2.Server, at most NumCPU handlers run at once.
  • Global duration limit: the bridge runs handlers under context.WithTimeout and answers 504 when they return. MakeHTTPRequestDurationLimiter and bufferedResponseWriter, with their per-request goroutine and response copy, are gone; their counters and log lines moved into the bridge.
  • Cancellation: handlers run on the request context, so a client disconnect, the global limit or Handler.Close cancels them. sendTransaction ignores that cancellation and is bounded only by its own limit. Calls that fail after their context ends are labeled canceled in the request metrics.
  • Handler.Close waits for every handler to return, including ones a per-method duration limit stopped waiting for, so none outlives the stores.
  • Panics in a handler are recovered and logged; the old bridge crashed the process for one outside the per-method limiter.
  • Logs: json_req is the caller's JSON-RPC ID; the old bridge's client renumbered requests.

Notes

  • 504 timing: sent when the handlers return, not at the deadline. Per-method limits (5–15 s) are under the 25 s global one, so this only shows for a batch that queues past it.
  • Trusted results: a non-empty json.RawMessage result is written without validation or HTML escaping. Only methods: memoize latest-ledger reads per ledger sequence #1060 returns one.
  • Merging: please squash-merge; the history has intermediate fork pins.

@socket-security

socket-security Bot commented Sep 24, 2026 •

Copy link
Copy Markdown

No dependency changes detected. Learn more about Socket for GitHub.

👍 No dependency changes detected in pull request

@cjonas9
cjonas9 added this pull request to stack #1038 September 24, 2026 15:24
@cjonas9
cjonas9 force-pushed the loosen-ingestion-lock-holding branch from 6f10c28 to a12f81b Compare September 25, 2026 17:59
@cjonas9
cjonas9 force-pushed the bypass-jrpc2-marshal branch from 8715914 to b6d65ee Compare September 25, 2026 17:59
@cjonas9 cjonas9 changed the title bypass jrpc2 marshal Bypass jrpc2 marshal to enable raw endpoint response passthrough Sep 25, 2026
@cjonas9
cjonas9 force-pushed the bypass-jrpc2-marshal branch 2 times, most recently from 670361a to 0bfa01d Compare September 25, 2026 20:03
@cjonas9 cjonas9 added this to the platform sprint 76 milestone Sep 29, 2026
@cjonas9
cjonas9 force-pushed the loosen-ingestion-lock-holding branch 2 times, most recently from 5748c3d to a6efc89 Compare September 30, 2026 20:24
@cjonas9
cjonas9 force-pushed the bypass-jrpc2-marshal branch from 0bfa01d to 6df8786 Compare September 30, 2026 20:24
Base automatically changed from loosen-ingestion-lock-holding to feature/full-history October 2, 2026 21:56
@cjonas9
cjonas9 marked this pull request as ready for review October 2, 2026 23:15
Copilot AI balanced review requested due to automatic review settings October 2, 2026 23:15
@cjonas9
cjonas9 marked this pull request as draft October 2, 2026 23:16

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Commit failures can leave the ledger cache inconsistent, and the dependency switch breaks the documented Handler.Close behavior.

Review effort: Balanced
Findings: 1 High severity · 1 Medium severity

Open (2)
What changed in this PR

Switches the JSON-RPC server to a fork that bypasses the in-memory client round trip, reducing response serialization overhead.

Changes:

  • Migrates server-side JSON-RPC usage to stellar-experimental/jrpc2.
  • Disables jrpc2’s semaphore in favor of existing request limiters.
  • Updates SQLite oldest-ledger cache publication during trimming.
File Description
go.mod Updates JSON-RPC dependencies.
go.sum Records dependency checksums.
cmd/​stellar-rpc/​internal/​jsonrpc/​jsonrpc.go Uses the forked HTTP bridge and disables its concurrency limit.
cmd/​stellar-rpc/​internal/​jsonrpc/​specs.go Migrates handler types to the fork.
cmd/​stellar-rpc/​internal/​jsonrpc/​specs_test.go Updates specification tests.
cmd/​stellar-rpc/​internal/​network/​backlogQ.go Migrates queue limiter types.
cmd/​stellar-rpc/​internal/​network/​backlogQ_test.go Updates queue limiter tests.
cmd/​stellar-rpc/​internal/​network/​requestdurationlimiter.go Migrates duration limiter types.
cmd/​stellar-rpc/​internal/​network/​requestdurationlimiter_test.go Updates duration limiter tests.
cmd/​stellar-rpc/​internal/​methods/​differential_test.go Updates differential test handlers.
cmd/​stellar-rpc/​internal/​methods/​get_events.go Migrates the events handler.
cmd/​stellar-rpc/​internal/​methods/​get_events_differential_test.go Updates events differential tests.
cmd/​stellar-rpc/​internal/​methods/​get_fee_stats.go Migrates the fee-statistics handler.
cmd/​stellar-rpc/​internal/​methods/​get_health.go Migrates the health handler.
cmd/​stellar-rpc/​internal/​methods/​get_latest_ledger.go Migrates the latest-ledger handler.
cmd/​stellar-rpc/​internal/​methods/​get_latest_ledger_test.go Updates latest-ledger tests.
cmd/​stellar-rpc/​internal/​methods/​get_ledger_entries.go Migrates the ledger-entry handler.
cmd/​stellar-rpc/​internal/​methods/​get_ledgers.go Migrates the ledger-range handler.
cmd/​stellar-rpc/​internal/​methods/​get_network.go Migrates the network handler.
cmd/​stellar-rpc/​internal/​methods/​get_transaction.go Migrates the transaction handler.
cmd/​stellar-rpc/​internal/​methods/​get_transactions.go Migrates the transaction-list handler.
cmd/​stellar-rpc/​internal/​methods/​get_transactions_test.go Updates transaction-list tests.
cmd/​stellar-rpc/​internal/​methods/​get_transactions_differential_test.go Updates transaction differential tests.
cmd/​stellar-rpc/​internal/​methods/​get_version_info.go Migrates the version handler.
cmd/​stellar-rpc/​internal/​methods/​handler.go Uses the forked handler adapter.
cmd/​stellar-rpc/​internal/​methods/​handler_test.go Updates handler-adapter tests.
cmd/​stellar-rpc/​internal/​methods/​send_transaction.go Migrates transaction submission.
cmd/​stellar-rpc/​internal/​methods/​simulate_transaction.go Migrates transaction simulation.
cmd/​stellar-rpc/​internal/​methods/​simulate_transaction_test.go Updates simulation tests.
cmd/​stellar-rpc/​internal/​rpcv1/​sqlitedb/​db.go Publishes oldest-ledger cache values during commits.
cmd/​stellar-rpc/​internal/​rpcv1/​sqlitedb/​ledger.go Extracts oldest-ledger lookup logic.
cmd/​stellar-rpc/​internal/​rpcv1/​sqlitedb/​ledger_test.go Tests trim-time cache publication.
cmd/​stellar-rpc/​internal/​rpcv2/​eventsapi/​get_events_v1.go Migrates the v1 events adapter.
cmd/​stellar-rpc/​internal/​rpcv2/​eventsapi/​query_events.go Migrates event queries.
cmd/​stellar-rpc/​internal/​rpcv2/​eventsapi/​query_events_test.go Updates event-query tests.
cmd/​stellar-rpc/​internal/​rpcv2/​eventsapi/​v1_parity_test.go Updates parity tests.
cmd/​stellar-rpc/​internal/​rpcv2/​jsonrpc.go Migrates RPC v2 handler types.
cmd/​stellar-rpc/​internal/​rpcv2/​jsonrpc_test.go Updates RPC v2 tests.
cmd/​stellar-rpc/​internal/​rpcv2/​ledger_reads_bench_test.go Updates ledger benchmarks.
cmd/​stellar-rpc/​internal/​rpcv2/​rpcv2test/​rpcv2test.go Migrates RPC v2 test infrastructure.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread cmd/stellar-rpc/internal/rpcv1/sqlitedb/db.go
Comment thread cmd/stellar-rpc/internal/jsonrpc/jsonrpc.go Outdated
@cjonas9
cjonas9 force-pushed the bypass-jrpc2-marshal branch from 6df8786 to 4490bc6 Compare October 2, 2026 23:22
@cjonas9
cjonas9 removed this pull request from stack #1038 October 2, 2026 23:23
@cjonas9
cjonas9 changed the base branch from feature/full-history to cache-getLatestLedger October 2, 2026 23:28
@cjonas9
cjonas9 added this pull request to stack #1061 October 2, 2026 23:29
@cjonas9
cjonas9 removed this pull request from stack #1061 October 5, 2026 22:57
@cjonas9 cjonas9 linked an issue Oct 5, 2026 that may be closed by this pull request
2 of 6 tasks
@cjonas9
cjonas9 changed the base branch from cache-getLatestLedger to feature/full-history October 5, 2026 22:58
Copilot AI balanced review requested due to automatic review settings October 9, 2026 19:12

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread cmd/stellar-rpc/internal/jsonrpc/bridge.go
Comment thread cmd/stellar-rpc/internal/jsonrpc/jsonrpc.go
Comment thread cmd/stellar-rpc/internal/methods/get_ledgers.go
Comment thread cmd/stellar-rpc/internal/methods/get_version_info.go Outdated
Comment thread cmd/stellar-rpc/internal/preflight/preflight.go Outdated
Base automatically changed from apt-get-core to feature/full-history October 9, 2026 19:25
Copilot AI balanced review requested due to automatic review settings October 9, 2026 20:47

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread cmd/stellar-rpc/internal/jsonrpc/jsonrpc.go
Copilot AI balanced review requested due to automatic review settings October 9, 2026 20:52

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Invalid-request IDs break claimed bridge parity, and completion logs still report the generated ID instead of the caller’s JSON-RPC ID.

6 open findings
Previously missed (1)

In code that hasn't changed since last review

Medium severity Preserve raw IDs for parser-generated errors

cmd/​stellar-rpc/​internal/​jsonrpc/​bridge.go:295

escapeID does not match jhttp.Bridge for statically invalid requests. Upstream's marshalError inserts ParsedRequest.ID directly, while only responses that pass through its client/server are JSON-marshaled and HTML-escaped. Therefore the added html id on invalid request parity case produces "<b>&" from the old bridge but "\u003cb\u003e\u0026" here. Track whether a response is a parser-generated error and preserve its raw ID in that path, while retaining escaping for handler responses.

🧠 Review effort: Balanced

Copilot AI balanced review requested due to automatic review settings October 9, 2026 20:58

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread cmd/stellar-rpc/internal/jsonrpc/bridge.go
Copilot AI balanced review requested due to automatic review settings October 9, 2026 21:05

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

A detached sendTransaction can remain active after rpcv1 closes its captive Core during shutdown.

2 open findings
1 resolved since last review

🧠 Review effort: Balanced

Comment thread cmd/stellar-rpc/internal/jsonrpc/jsonrpc.go
@cjonas9 cjonas9 changed the title Bypass jrpc2 marshal to enable raw endpoint response passthrough jsonrpc: replace jhttp.Bridge with an in-tree bridge (#1037) Oct 12, 2026
@cjonas9 cjonas9 changed the title jsonrpc: replace jhttp.Bridge with an in-tree bridge (#1037) jsonrpc: replace jhttp.Bridge with an in-tree bridge Oct 12, 2026
@cjonas9
cjonas9 merged commit 25a431b into feature/full-history Oct 12, 2026
27 of 35 checks passed
@cjonas9
cjonas9 deleted the bypass-jrpc2-marshal branch October 12, 2026 16:06
@github-project-automation github-project-automation Bot moved this from Blocked to Done in Platform Scrum Oct 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

Raw response passthrough: stop jrpc2 from re-marshaling and re-parsing handler results

3 participants