Skip to content

feat(mcp): stateless spec modern fanout routing and server discovery ( 28-07-2026 ) - #2545

Merged
johnugeorge merged 27 commits into
theagentrouter:mainfrom
Hritik003:mcp-new-spec-phase0/add-modern-fanout-logic
Sep 15, 2026
Merged

johnugeorge merged 27 commits into
theagentrouter:mainfrom
Hritik003:mcp-new-spec-phase0/add-modern-fanout-logic

Conversation

@Hritik003

Copy link
Copy Markdown
Contributor

Description

This PR adds phase0 support for modern fanout routing as per

  • docs: proposal for new MCP Release candidate ( 28-07-2026 ) #2431 that are required for the new MCP stateless spec. It introduces a new modern request handling path for all the fan out APIs;
    • server/discover,
    • tools/list,
    • resources/list,
    • resources/templates/list,
    • prompts/list,
      validates modern protocol headers, fans out requests across backends, and aggregates responses into a single JSON-RPC result. It also extracts capability aggregation into a shared helper (unionServerCapabilities) so both stateful and stateless paths use the same capability-union behavior, and bumps github.com/modelcontextprotocol/go-sdk to v1.7.0 for modern MCP support.

Related Issues/PRs

Dependent PR

Special notes for reviewers (if applicable)

  • Please focus review on fanout/merge behavior and protocol compliance in internal/mcpproxy/modern.go, especially:

    • modern header and version validation
    • partial-backend-failure handling during aggregation
    • merged discover/list response
  • this PR also has some common code with feat(mcp): stateless spec era detection and legacy split (28-07-2026) #2518 which will be adjusted and rebased once the first PR goes in.

Testing
Built and deployed the changes to local cluster, tested the functionality using MCP inspector.

Signed-off-by: Hritik003 <hritik.raj@nutanix.com>
Signed-off-by: Hritik003 <hritik.raj@nutanix.com>
@codecov-commenter

codecov-commenter commented Aug 17, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 89.72603% with 45 lines in your changes missing coverage. Please review.
✅ Project coverage is 86.39%. Comparing base (da1017d) to head (973eb71).

Files with missing lines Patch % Lines
internal/mcpproxy/modern.go 88.97% 25 Missing and 20 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2545      +/-   ##
==========================================
+ Coverage   86.33%   86.39%   +0.05%     
==========================================
  Files         183      184       +1     
  Lines       24509    24945     +436     
==========================================
+ Hits        21160    21550     +390     
- Misses       2168     2194      +26     
- Partials     1181     1201      +20     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@Hritik003
Hritik003 marked this pull request as ready for review August 17, 2026 10:36
@Hritik003
Hritik003 requested a review from a team as a code owner August 17, 2026 10:36
@dosubot dosubot Bot added the size:XL This PR changes 500-999 lines, ignoring generated files. label Aug 17, 2026
@Hritik003

Copy link
Copy Markdown
Contributor Author

/retest

Comment thread internal/mcpproxy/modern.go Outdated
Comment on lines +127 to +135
headerVersion := r.Header.Get(mcpProtocolVersionHeader)
if headerVersion == "" {
headerVersion = protocolVersion20260728
}
if !isSupportedVersion(headerVersion) {
errType = metrics.MCPErrorUnsupportedProtocolVersion
err = fmt.Errorf("unsupported protocol version: %s", headerVersion)
onErrorResponse(w, http.StatusBadRequest, fmt.Sprintf("unsupported protocol version: %s", headerVersion))
return

@swiftdiaries swiftdiaries Aug 29, 2026 •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this PR depends on #2518 , should this handler consume the validated result from detectClientEra instead of independently validating or inferring the version?
Inline with that, what's the intended request flow after rebase? Is it servePOST` → `detectClientEra` → reject or dispatch a validated request → `serveModernPOST?

Broader question: can we try and centralize validation to ensure it's consistent and there's one place to make changes/evolve it? Also, have cleaner separation of concerns.

@Hritik003 Hritik003 Sep 3, 2026 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this PR depends on #2518 , should this handler consume the validated result from detectClientEra instead of independently validating or inferring the version?

Right, I think we should avoid these version checks here, and detectClientEra function in the previous PR. Nevertheless, all the necessary checks are already present in #2518 itself.

Inline with that, what's the intended request flow after rebase? Is it servePOST→detectClientEra→ reject or dispatch a validated request →serveModernPOST?

yes

Broader question: can we try and centralize validation to ensure it's consistent and there's one place to make changes/evolve it? Also, have cleaner separation of concerns.

the detectClientEra function in the previous PR takes care of that

Comment on lines +212 to +215
var results []*mcp.DiscoverResult
for _, backend := range routeConfig.backends {
backendStartAt := time.Now()
result, err := m.discoverBackend(ctx, route, backend)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is it possible for a request to receive a backend (and it's tools/prompts/resources) that isn't authorized?
Can the backend selection be a separate step that produces the backend set consumed downstream (discovery, fanout)? Trying to see if we can clearly define the input interface here. I was thinking a validated request + authorized backend set for the request.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is it possible for a request to receive a backend (and it's tools/prompts/resources) that isn't authorized? Can the backend selection be a separate step that produces the backend set consumed downstream (discovery, fanout)? Trying to see if we can clearly define the input interface here. I was thinking a validated request + authorized backend set for the request.

Agree, earlier on initialize we used to call newSession() which used to set the selected authorized backends, and tools/list prompts/list.... used to iterate through the selected backends returned from newSession(). Since in modern, we dont have any initialize and every request is stateless, reevaluating every request is actually the right stateless behavior, a new token can change which backends are visible. Legacy cannot do that mid session because the set is fixed at initialize.

Comment thread internal/mcpproxy/modern.go Outdated
Comment on lines +217 to +237
if err != nil {
m.l.Warn("server/discover failed for backend",
slog.String("backend", backend.Name),
slog.String("error", err.Error()))
backendMetrics.RecordMethodErrorCount(ctx, req.Method, nil, metrics.MCPStatusError)
backendMetrics.RecordRequestErrorDuration(ctx, backendStartAt, errorType(err), nil)
continue
}
if span != nil {
span.RecordRouteToBackend(backend.Name, "", true)
}
backendMetrics.RecordMethodCount(ctx, req.Method, nil)
backendMetrics.RecordRequestDuration(ctx, backendStartAt, nil)
results = append(results, result)
}
if len(results) == 0 {
m.l.Error("server/discover failed for all backends", slog.String("route", route))
onErrorResponse(w, http.StatusInternalServerError, "failed to discover any backend")
return handlerResult{}, errors.New("failed to discover any backend")
}
merged := mergeDiscoverResults(results)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what happens when there are partial discovery results? when there are temporary failures on a backend?
could it create a different backend set for a similar request? trying to see what the best possible response should be when we have that ambiguity

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For initialize in the previous spec, this was the standard we followed during fanout;

Comment thread internal/mcpproxy/modern.go Outdated
}
for _, v := range r.SupportedVersions {
if !slices.Contains(versions, v) {
versions = append(versions, v)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we intersect against the gateway's supported versions here? to prevent accidentally advertising versions the gateway doesn't support

@Hritik003 Hritik003 Sep 3, 2026 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point.
planning to not have any hardcoded logic for showing supported versions. Was thinking to directly return the merged list of supported versions from all the backends beneath. This PR #2543 handles that logic, and would want to use the same for showing supported versions.

Comment thread internal/mcpproxy/modern.go Outdated
Comment on lines +526 to +537
var rpcResp map[string]json.RawMessage
if err := json.Unmarshal(jsonPayload, &rpcResp); err != nil {
return nil, fmt.Errorf("parse response: %w (body: %.200s)", err, string(jsonPayload))
}
if errField, ok := rpcResp["error"]; ok && string(errField) != "null" {
return nil, fmt.Errorf("backend error: %s", string(errField))
}
result, ok := rpcResp["result"]
if !ok || string(result) == "null" {
return nil, fmt.Errorf("backend returned no result")
}
return result, nil

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we add some validation logic here?
some to things to validate:

  • The payload is a valid JSON-RPC 2.0 response.
  • The response ID matches the upstream request ID.
  • Exactly one of result or error is present.
  • Required modern fields are present and valid, including resultType.
  • A malformed response is recorded as a backend failure and is not included in the aggregated result.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Out of all the validations suggested, i think we can up few,

but validations like valid JSON-RPC 2.0 does not make sense for a proxy. Because once we are able to pull up result from the response, these extra checks do not change aggregation behavior

Exactly one of result or error is present.

Also for this, we anyways inject the resultType in the ensureResultType function on the outbound response. resultType is only mattered during MRTR cases like input_required, but given that we are deferring those changes in this phase, lets keep it simple?

@swiftdiaries swiftdiaries left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the context and the work here! Areas I focused on for the review:

  • request validation at ingress
  • backend selection logic
  • discovery logic
  • response handling

P.S Tried my best with the small context I have of the project and the code. So feel free to disregard things if they are not relevant

Signed-off-by: Hritik003 <hritik.raj@nutanix.com>
@Hritik003
Hritik003 requested a review from a team as a code owner September 9, 2026 14:53
@netlify

netlify Bot commented Sep 9, 2026 •

Copy link
Copy Markdown

✅ Deploy Preview for theagentrouter ready!

Name Link
🔨 Latest commit e352c2d
🔍 Latest deploy log https://app.netlify.com/projects/theagentrouter/deploys/6aa8ca0b1db4c10008a2d419
😎 Deploy Preview https://deploy-preview-2545--theagentrouter.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@codecov

codecov Bot commented Sep 9, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 93.41865% with 36 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
internal/mcpproxy/modern.go 91.96% 34 Missing ⚠️
internal/mcpproxy/handlers.go 95.23% 1 Missing ⚠️
internal/mcpproxy/mcpproxy.go 98.76% 1 Missing ⚠️

📢 Thoughts on this report? Let us know!

Signed-off-by: Hritik Raj <hritik.raj@nutanix.com>
Signed-off-by: Hritik003 <hritik.raj@nutanix.com>
Signed-off-by: Hritik003 <hritik.raj@nutanix.com>
Comment on lines +164 to +168
_, selectedBackends, err := m.resolveModernRouteBackends(w, route)
if err != nil {
return handlerResult{}, err
}

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

since we are calling it at every handler, can we call it at some top level?

Signed-off-by: Hritik003 <hritik.raj@nutanix.com>
Signed-off-by: Hritik003 <hritik.raj@nutanix.com>
@Hritik003

Copy link
Copy Markdown
Contributor Author

/retest

@@ -648,9 +655,45 @@ func decodeCapabilityFlags(hex string) *mcpsdk.ServerCapabilities {
// If ANY backend supports a capability, the merged result includes it.
// Sub-fields like ListChanged and Subscribe are OR'd across all backends.
func (s *session) mergedCapabilities() *mcpsdk.ServerCapabilities {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do you need the separate helper method unionServerCapabilities for this?

}()

route := r.Header.Get(internalapi.MCPRouteHeader)
if route == "" {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we need this?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so in legacy, every session remembers which route it points to inside its session id. But its not the case with modern cause it being stateless, and if we skip this check, a missing header would look route <name> not found and return 404 masking the gateway bug

Comment thread internal/mcpproxy/modern.go Outdated
// Start a tracing span for the request, mirroring the legacy path. The span
// is closed in recordPOSTCompletion. Fan-out handlers additionally record
// per-backend routing on the span via RecordRouteToBackend.
if params := modernParamsForHeaderMetadata(req); params != nil {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

start span only for supportedMethod. Similar to legacy.go?

// against this request. The returned map is the only backend set discovery and
// list fan-out may talk to. Writes 404 (unknown route) or 403 (no matching
// backends) on failure.
func (m *mcpRequestContext) resolveModernRouteBackends(w http.ResponseWriter, route filterapi.MCPRouteName) (*mcpProxyConfigRoute, map[filterapi.MCPBackendName]filterapi.MCPBackend, error) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

forwardHeader extraction before authorizing against backend selector?

@aish1331 aish1331 Sep 10, 2026 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you also need to extract per backend forwardHeader. Can you please compare this with legacy flow to find what's missing and fix it?

m.l.Warn("server/discover failed for backend",
slog.String("backend", backend.Name),
slog.String("error", err.Error()))
backendMetrics.RecordMethodErrorCount(ctx, req.Method, nil, metrics.MCPStatusError)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this already being recorded in recordPOSTCompletion?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nope, we set perBackendMetrics=true which ends the span when it goes to defer block. So its not a duplicate

1. start a span only on supported methods like legacy
2. resolveModernRouteBackends now follow newSession: extract route level headers ---> apply auth ---> extract per backend frwd headers

Signed-off-by: Hritik003 <hritik.raj@nutanix.com>

// modernParamsForHeaderMetadata best-effort parses params for methods where
// addMCPHeaders can enrich upstream metadata (tool name/resource URI).
func modernParamsForHeaderMetadata(req *jsonrpc.Request) mcp.Params {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can this be reused from legacy?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

those helpers aren’t in legacy

return nil
}

func writeJSONRPCResult(w http.ResponseWriter, id jsonrpc.ID, result any) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same, can this be reused from legacy?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

those helpers aren’t in legacy

@mohitgurnani

Copy link
Copy Markdown
Contributor

Reviewed this alongside #2661/#2663 — really solid refactor overall (nice that forwardHeaders handling got extracted into shared helpers so it's reused as-is by the modern path, and the caching-hint merge logic being unified across legacy/modern is a good call).

One thing worth a look before this lands: sendToAllModernBackendsAndAggregateResponses silently drops any backend that fails (non-200, network error, bad response shape) — logs a Warn, excludes it, moves on. That's a reasonable default for partial failures, but handleServerDiscover is the only one of the five fan-out handlers that checks len(results) == 0 and fails the request (500) when literally every backend errors. handleModernToolsList, handleModernResourcesList, handleModernResourceTemplatesList, and handleModernPromptsList don't have that guard — if 100% of backends fail, they still return a normal 200 with an empty list, which is indistinguishable from a route that genuinely has zero tools/resources/prompts.

Two possible fixes for that part: add the same len(results) == 0 guard to those four handlers, or (probably better, so it can't drift again) push the check into sendToAllModernBackendsAndAggregateResponses itself so every current and future caller inherits it.

Separately, even short of total failure, it'd be worth surfacing which backends dropped out of a partial result and why — not just a server-side log line and a metric. Right now a caller who gets 8 tools back from a 10-backend route has no way to tell "that's everything" from "2 backends silently failed." sendToAllModernBackendsAndAggregateResponses already knows exactly which backend failed and the error for each one at the point it does continue — it wouldn't need to block the response (the merged partial result can still return 200), but riding that failure list along as response metadata (or a trace/log correlation id surfaced to the caller) would turn today's silent drops into something a client can actually act on, instead of only being visible if someone happens to be watching the logs. Given the discussion on #2661/#2663 has been about not letting failures get silently absorbed into a successful-looking response, seems worth closing both of these while this is fresh rather than after it ships.

@swiftdiaries

Copy link
Copy Markdown

+1 on the error when no backends are selected / all backends failed (+tests for both complete/partial failures)

@Hritik003

Hritik003 commented Sep 14, 2026 •

Copy link
Copy Markdown
Contributor Author

Separately, even short of total failure, it'd be worth surfacing which backends dropped out of a partial result and why — not just a server-side log line and a metric. Right now a caller who gets 8 tools back from a 10-backend route has no way to tell "that's everything" from "2 backends silently failed." sendToAllModernBackendsAndAggregateResponses already knows exactly which backend failed and the error for each one at the point it does continue — it wouldn't need to block the response (the merged partial result can still return 200), but riding that failure list along as response metadata (or a trace/log correlation id surfaced to the caller) would turn today's silent drops into something a client can actually act on, instead of only being visible if someone happens to be watching the logs. Given the discussion on #2661 has been about not letting failures get silently absorbed into a successful-looking response, seems worth closing both of these while this is fresh rather than after it ships.

Thats a good point and would help in debugging in great detail. But in my opinion that can be done in another PR because if we are doing that, legacy code also need to stay consistent of logging partial backend failures and it will need few refactorings out of the scope of this PR.

Signed-off-by: Hritik003 <hritik.raj@nutanix.com>
Signed-off-by: Hritik003 <hritik.raj@nutanix.com>

@aish1331 aish1331 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!!

@swiftdiaries

swiftdiaries commented Sep 14, 2026 •

Copy link
Copy Markdown

LGTM, this took a while thanks for being patient!

@johnugeorge
johnugeorge enabled auto-merge (squash) September 15, 2026 04:34
@johnugeorge
johnugeorge merged commit 12885b2 into theagentrouter:main Sep 15, 2026
63 of 65 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XL This PR changes 500-999 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants