Skip to content

fix: surface DPoP proof rejections instead of "unknown error" (CXP-1002) - #202

Open
agustin-conductor wants to merge 1 commit into
mainfrom
bugfix/dpop-nonce-retry
Open

fix: surface DPoP proof rejections instead of "unknown error" (CXP-1002)#202
agustin-conductor wants to merge 1 commit into
mainfrom
bugfix/dpop-nonce-retry

Conversation

@agustin-conductor

@agustin-conductor agustin-conductor commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

Fixes the the API returned an unknown error failures that were breaking full syncs on baton-okta connectors using OAuth 2.0 / private-key auth. CXP-1002

Rebased onto #201. That PR rewrote handleOktaResponseError, which overlapped with an earlier version of this branch — see Relationship to #200 below for what was dropped as redundant.

The problem

Okta answers a bad DPoP proof with a 400, a zero-length body, and the reason in a DPoP-scheme WWW-Authenticate header.

The vendored SDK is structurally blind to that. CheckResponseForError (requestExecutor.go:634-651) reads WWW-Authenticate only when the status is 401/403 and the header contains Bearer — it misses on both counts here. It then runs _ = json.NewDecoder(...).Decode(&e) on the empty body, discarding the decode error, and returns a zero-valued okta.Error whose Error() is the API returned an unknown error. One failed call cancels the sync, so a ~0.16% call-level failure rate stopped full syncs from ever completing.

Root cause, confirmed live

Probed against a DPoP-enabled tenant with hand-crafted proofs:

Proof defect Status Body WWW-Authenticate
valid (control) 200
unsolicited stale nonce 200
replayed proof (same jti) 400 0 bytes error="invalid_dpop_proof", "The DPoP proof JWT has already been used."
iat 10 min past 400 0 bytes "…issued more than five minutes in the past."
iat 10 min future 400 0 bytes "…issued in the future."

So the trigger is invalid_dpop_proof, not use_dpop_nonce as the ticket originally concluded. A likely replay path exists in our own stack: uhttp's transport retries on stale connections (transport.go:292) and retryableRequest returns the request with headers untouched — same proof, same jti — and it sits below this round-tripper, so no fresh proof is minted. Clock skew produces the identical signature.

The nonce theory is ruled out: 59/59 instrumented resource calls sent a proof with no nonce claim and got 200, Okta never returned a DPoP-Nonce on a resource response, and an unsolicited stale nonce is simply ignored. Okta's nonce requirement is authorization-server only, and token_source.go already handles that correctly.

Changes

pkg/oktaauth/round_tripper.go — the substance of this PR.

  • Restate the challenge as the error body — give an empty-bodied rejection the JSON the SDK expects, so the real reason reaches the caller:
    before: the API returned an unknown error
    after:  the API returned an error: The DPoP proof JWT has already been used.
    
  • Retry proof replay once with a fresh proof, gated on the "already been used" description. A generic invalid_dpop_proof is deliberately not retried — a skewed clock would only triple the traffic.
    The challenge is passed through whole rather than parsed into its auth-params. RFC 9110 allows several challenges in one header (Bearer error="invalid_token", DPoP error="use_dpop_nonce"), and attributing a param to the right scheme needs a real parser — quoting the header drops nothing (resource and scope survive too) and cannot misattribute one scheme's error to another. Bounded to client errors with an empty body, so a real body is never replaced.
  • Warn on any DPoP-related 4xx, logging the full challenge before the SDK discards it — logarithmically sampled (1, 10, 100, every 1000) with a total_occurrences field, per ci-review.md L1 and L7, since a skewed clock is not retried and would otherwise emit one line per request for a whole sync. This is what will identify replay vs. clock skew on the next occurrence.
  • Match a nonce challenge on 400 as well as 401, and in the JSON-body shape as well as the header, with a bounded retry loop. Defensive only — unreachable on the tenants observed, but correct per RFC 9449 and consistent with the token endpoint.

pkg/connector/helpers.gogetError now names the HTTP status and excerpts an undecodable body. Its five call sites (group.go:211, role.go:456/498, app.go:407/479) all return its error verbatim, so an empty body surfaced as a bare unexpected end of JSON input with no status and no prefix. #200 rewrote handleOktaResponseError but left getError untouched.

pkg/connector/event_log.go — the event feed no longer returns the SDK error bare, so it picks up a grpc code and the status line like every other call site. #200 did not touch this file.

Relationship to #200

#200's fallback is WrapErrorsWithRateLimitInfo(GrpcCodeFromHTTPStatus(status), ...), and that helper opens with status.New(preferredCode, resp.Status) — so the status line already becomes the error message for every non-2xx, with a meaningful grpc code (400→InvalidArgument, 403→PermissionDenied, 429→Unavailable) and rate-limit details attached.

Two changes from the pre-rebase version of this branch were therefore dropped as redundant:

  • a 429 → codes.Unavailable mapping — GrpcCodeFromHTTPStatus already does this, and also carries rate-limit info
  • an isOpaqueOktaError branch that named the status for zero-valued errors — Return GRPC statuses for more Okta errors. #200 does it for all non-2xx, more generally

Their helpers_test.go covers those paths (14 cases, including 429), so the corresponding tests here were dropped too. Nothing in this PR duplicates #200's handling.

Testing

  • pkg/oktaauth/nonce_challenge_test.go — body-sniff restoration (including a body twice the sniff limit, and that 200s are never buffered), replay-retry vs. clock-skew-no-retry, the auth-param boundary parser, bounded retries, and an end-to-end check that the real okta.CheckResponseForError now renders Okta's reason. Fixtures are the verbatim challenge strings captured from the live probe.
  • pkg/connector/get_error_test.go — status naming and body excerpting on an undecodable body, that the excerpt trims to a rune boundary (verified by mutation: removing the guard fails the test), and that a decodable body still parses.
  • Three clean full syncs against a live tenant: 119 actions, 0 errors, identical to the pre-change baseline.
  • go build, go vet, golangci-lint clean (only pre-existing goconst hits in actions.go).

Notes for review

  • The branch name says nonce-retry, which undersells it — the nonce widening is the defensive part; the proof-rejection handling is the actual fix.
  • The root cause differs from the ticket description. CXP-1002 has a comment with the full evidence; its Root cause section still describes the nonce mechanism and should be revised.
  • Whether Anduril's failures are replay or clock skew is not settled. The new warning answers it on the next occurrence.
  • An unrelated bug found along the way — event feeds can never work on OAuth auth because okta.logs.read is never requested — is split out as CXP-1021 and deliberately not in this PR, since it changes the scope set for every existing OAuth customer.

🤖 Generated with Claude Code

@linear-code

linear-code Bot commented Aug 26, 2026

Copy link
Copy Markdown

CXP-1002

// "the API returned an unknown error". Restating the header as the JSON the SDK
// expects turns that back into the reason Okta actually gave.
func annotateEmptyDPoPError(resp *http.Response, body []byte) {
if len(bytes.TrimSpace(body)) > 0 {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Suggestion: annotateEmptyDPoPError infers "body is empty" from the body argument, but sniffClientErrorBody only populates it for 4xx — it returns nil for 2xx/3xx/5xx. Since finish is the single exit for every response, a 200 or 5xx whose WWW-Authenticate carries error=/error_description= params (RFC 9110 permits the header on any status) would have its real body closed and replaced by the synthesized JSON. Worth guarding with the same range the sniffer uses, e.g. if resp.StatusCode < 400 || resp.StatusCode >= 500 { return }. (medium confidence — not observed on the probed tenants, but the invariant is only held by the caller today)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 25a1ddb — you were right that the invariant was only held by the caller. Added if resp.StatusCode < 400 || resp.StatusCode >= 500 { return } inside the function, so it no longer infers anything from an empty body argument outside the range where sniffClientErrorBody populates it. TestAnnotateEmptyDPoPError_OnlyTouchesClientErrors covers 200/302/500 with a real body plus a challenge header present.

if !strings.Contains(strings.ToLower(challenge), "dpop") && !strings.Contains(errCode, "dpop") {
return
}
ctxzap.Extract(ctx).Warn("oktaauth: dpop failure on resource request",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Suggestion: this Warn is unsampled and fires once per failing request. Since the PR deliberately does not retry a generic invalid_dpop_proof (clock skew), a skewed clock means every request in a sync emits this line with the full challenge string — thousands per sync, which is exactly the alert-noise case the repo criteria call out (L7). Consider logarithmic sampling (1, 10, 100, every 1000) with a total_occurrences field.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 25a1ddb with logarithmic sampling per L7 — 1, 10, 100, then every 1000, with a total_occurrences field. The counter lives on the dpopRoundTripper rather than in a package var so two connectors in one process don't share it.

(It briefly went to Debug instead; your follow-up on that thread was the better call and it's back at Warn.)

Comment thread pkg/oktaauth/round_tripper.go Outdated

// authParam returns the quoted value of one auth-param, matching only at a
// parameter boundary so "error" does not match inside "error_description".
func authParam(challenge, name string) string {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Suggestion: authParam scans the entire WWW-Authenticate value without scoping to the DPoP scheme. RFC 9110 §11.6.1 allows several challenges in one header (Bearer error="invalid_token", DPoP error="use_dpop_nonce"), and the first boundary match wins — so a Bearer challenge's error/error_description would be reported as the DPoP reason and synthesized into the response body. Scoping the scan to the substring starting at the DPoP scheme token would make it match the function's stated intent.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 25a1ddb, though not the way you suggested — the parser is gone entirely.

Scoping the scan fixed one ordering and left the reverse broken (you caught that in the follow-up), and correct scheme attribution needs a real RFC 9110 parser. The synthesis doesn't actually need attribution: it only has to hand the SDK the challenge the SDK is about to discard. So the whole header is now restated verbatim as errorSummary, and this class of bug dissolves rather than being fixed — nothing is attributed, so nothing can be misattributed. Net −121 lines.

Bonus: resource and scope now survive into the error too, which the parsed version dropped.

Comment thread pkg/connector/helpers.go Outdated
func bodyExcerpt(body []byte) string {
excerpt := strings.TrimSpace(string(body))
if len(excerpt) > errorBodyExcerptLimit {
return excerpt[:errorBodyExcerptLimit] + "..."

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Suggestion: slicing at a fixed byte offset can cut a multi-byte UTF-8 rune in half, so the excerpt embedded in the returned error (and any structured log field built from it) can carry invalid UTF-8. Backing up to a rune boundary — e.g. for !utf8.ValidString(excerpt[:n]) { n-- }, or strings.ToValidUTF8 on the result — keeps the message clean.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 25a1ddb. Went with the rune-boundary backup rather than strings.ToValidUTF8utf8.RuneStart walks back at most 3 bytes with no allocation, where ToValidUTF8 rescans and rebuilds the string. Extracted as truncateAtRuneBoundary and used by both truncation paths (see your follow-up on the challenge one).

@github-actions

github-actions Bot commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

Connector PR Review: fix: surface DPoP proof rejections instead of "unknown error" (CXP-1002)

Blocking Issues: 0 | Suggestions: 1 | Threads Resolved: 0
Criteria: Criteria status: loaded .claude/skills/ci-review.md from trusted base b63b23c3dd31.
Review mode: full
View review run

Review Summary

Scanned the full PR diff (round_tripper.go, helpers.go, event_log.go, plus two new test files) for security and correctness. Both prior findings are addressed: sniffClientErrorBody now returns the bytes that arrived instead of nil on a partial read (round_tripper.go:349-363, covered by TestRoundTripper_PartialBodyReadIsNotTreatedAsEmpty), and getError's errors now carry a gRPC code derived from the HTTP status via bodyReadError/uhttp.GrpcCodeFromHTTPStatus (helpers.go:134-140, covered by TestGetError_ClassifiesByHTTPStatus). The retry loop is bounded at maxDPoPRetrySends, drains and closes each response before resending, guards resp.Body == nil on every path, and gates annotateEmptyDPoPError on the 4xx range itself; log levels and the logarithmic sampling follow L1/L7. No new blocking issues found.

Security Issues

None found. The DPoP failure log emits only the WWW-Authenticate challenge, status, method and path — no tokens, proofs or PII (H5/T3), and the sniff buffer is capped at errorBodySniffLimit.

Correctness Issues

None found.

Suggestions

  • pkg/oktaauth/token_source.go:308-309formatTokenError still slices the body excerpt at a raw byte offset (excerpt[:errorBodyExcerptLimit]), the exact rune-splitting case this PR adds truncateAtRuneBoundary (and a test) for in the same package. Low confidence that it bites in practice, but the helper is now one call away.
Prompt for AI agents
Verify each finding against the current code and only fix it if needed.

## Suggestions

In `pkg/oktaauth/token_source.go`:
- Around line 308-309: `formatTokenError` truncates the raw-body excerpt with a
  plain byte slice, `excerpt = excerpt[:errorBodyExcerptLimit] + "..."`, which can
  cut a multi-byte rune in half and put invalid UTF-8 into the returned gRPC error
  message. This PR added `truncateAtRuneBoundary(s string, maxBytes int) string` to
  `pkg/oktaauth/round_tripper.go` in the same package for exactly this case.
  Replace the slice with
  `excerpt = truncateAtRuneBoundary(excerpt, errorBodyExcerptLimit) + "..."` so both
  excerpt paths in the package behave identically, and consider extending the
  existing `TestTruncateAtRuneBoundary` coverage to the token-error path.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No blocking issues found.

Comment thread pkg/connector/get_error_test.go Outdated
Comment on lines +68 to +69
// Three-byte runes tile the limit so it lands mid-rune.
body := strings.Repeat("é", errorBodyExcerptLimit)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Suggestion: this test doesn't exercise the bug it names. é (U+00E9) is two bytes, not three, so strings.Repeat("é", 200) is 400 bytes and excerpt[:200] lands exactly on a rune boundary (100 whole runes). utf8.ValidString would pass even with a plain excerpt[:errorBodyExcerptLimit] and no strings.ToValidUTF8, so the guard added in bodyExcerpt is untested. Use a genuine three-byte rune so 200 doesn't divide evenly — e.g. strings.Repeat("€", errorBodyExcerptLimit) (or "あ"), which puts the cut mid-rune at byte 200.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct, and it was my error — é is two bytes, so 200 divided evenly and the test proved nothing. Fixed in 25a1ddb by switching to (three bytes, so the cut lands mid-rune at byte 200).

Verified by mutation this time: with the guard removed the test fails with excerpt is not valid UTF-8: "€€€…\xe2\x82...", and passes with it restored.

Comment thread pkg/oktaauth/round_tripper.go Outdated
if !strings.Contains(strings.ToLower(challenge), "dpop") && !strings.Contains(errCode, "dpop") {
return
}
ctxzap.Extract(ctx).Debug("oktaauth: dpop failure on resource request",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Suggestion: this is the diagnostic the PR describes as "Warn on any DPoP-related 4xx … what will identify replay vs. clock skew on the next occurrence", but at Debug it won't be emitted in a production sync, so the next occurrence still gives you nothing. Repo criteria L1 puts upstream 4xx at Warn; L7's answer to a per-request warning is logarithmic sampling (1, 10, 100, every 1000) with a total_occurrences field, not silencing. Consider Warn + sampling here, or at minimum update the PR description so it matches the code.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, and reverted in 25a1ddb — back to Warn, with the logarithmic sampling from L7 (1, 10, 100, every 1000) and a total_occurrences field, so the noise concern is handled by rate-limiting rather than by silencing.

You were also right about the description mismatch; the PR body now states the sampling and cites L1/L7.

Comment thread pkg/oktaauth/round_tripper.go Outdated
Comment on lines +276 to +291
func dpopChallengeScope(challenge string) string {
for i := 0; i+len(dpopHdr) <= len(challenge); i++ {
if !strings.EqualFold(challenge[i:i+len(dpopHdr)], dpopHdr) {
continue
}
after := i + len(dpopHdr)
startsToken := i == 0 || challenge[i-1] == ' ' || challenge[i-1] == ','
// A scheme token is followed by whitespace or ends the value; anything else
// means this is a parameter name that merely begins with the same letters.
endsToken := after >= len(challenge) || challenge[after] == ' '
if startsToken && endsToken {
return challenge[i:]
}
}
return ""
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Suggestion: the scope is bounded at the start but not at the end — it returns challenge[i:] all the way to the end of the header. With the challenges in the other order (DPoP algs="RS256", Bearer error="invalid_token", error_description="wrong scheme"), authParam finds the Bearer params and they get reported as the DPoP reason and synthesized into the error body by annotateEmptyDPoPError. TestDPoPChallengeParams_ScopedToDPoPScheme only covers "bearer challenge first", so this order isn't caught. Terminate the slice at the next scheme token (a bare token followed by whitespace, not name="), and add a DPoP-first test case. Related: isResourceNonceChallenge (line 217) still does an unscoped strings.Contains on the whole header, so it doesn't get this scoping at all.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right on both counts — bounding only the start left the reverse ordering broken, and isResourceNonceChallenge wasn't scoped at all.

Rather than add end-bounding, 25a1ddb removes the parser: the challenge is restated verbatim, so no attribution happens and neither ordering can misattribute. isResourceNonceChallenge needs no scoping either, since use_dpop_nonce is a DPoP-specific code. Tests cover a Bearer-only header and a mixed header, asserting the neighbouring challenge is preserved rather than swapped in.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No blocking issues found.

Comment thread pkg/oktaauth/round_tripper.go Outdated
zap.Int("send", send),
zap.Bool("proof_replay", reason == dpopRetryProofReplay),
)
_, _ = io.Copy(io.Discard, resp.Body)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Suggestion: io.Copy(io.Discard, resp.Body) dereferences resp.Body unguarded, while both sniffClientErrorBody (line 328) and annotateEmptyDPoPError (line 266) explicitly nil-check it. A transport that returns a nil-bodied 4xx carrying a use_dpop_nonce challenge header reaches this line via dpopRetryNonce and panics. net/http's own transport always sets a body so this is unlikely in production, but the guard is inconsistent with the rest of the file — a if resp.Body != nil wrapper around the drain/close pair would settle it.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed a real panic and fixed in 25a1ddb. io.Discard implements ReaderFrom, so io.Copy calls Discard.ReadFrom(nil) which calls Read on a nil interface:

PANIC: runtime error: invalid memory address or nil pointer dereference

Your path analysis holds too — sniffClientErrorBody returns early on a nil body without replacing it, so nothing intervenes before the drain. Guarded the drain/close pair, and TestRoundTripper_NilResponseBodyDoesNotPanic uses a stub transport that returns a nil-bodied 4xx with a nonce challenge. Verified by mutation: without the guard the test panics.

Comment thread pkg/oktaauth/round_tripper.go Outdated
return
}
if len(challenge) > maxChallengeSummaryLen {
challenge = challenge[:maxChallengeSummaryLen] + "..."

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Suggestion: challenge[:maxChallengeSummaryLen] cuts at a byte offset and can split a multi-byte rune — the same hazard the sibling bodyExcerpt in pkg/connector/helpers.go:141 guards with strings.ToValidUTF8, and which TestBodyExcerpt_TruncatesOnRuneBoundary pins. Impact here is cosmetic (json.Marshal substitutes U+FFFD rather than erroring, and WWW-Authenticate is ASCII in practice), but applying the same strings.ToValidUTF8 keeps the two truncation paths consistent.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 25a1ddb — both paths now share truncateAtRuneBoundary, which backs up over continuation bytes with utf8.RuneStart.

Agreed the impact here is cosmetic: RFC 9110 §5.5 makes header field values US-ASCII, so a multi-byte rune in WWW-Authenticate is already out of spec. Consistency was the right reason to do it.

Comment on lines +206 to +209
// Both markers are DPoP-specific, so finding them anywhere in the header is
// unambiguous; no need to work out which challenge they belong to.
if strings.Contains(challenge, invalidDPoPProofErrorCode) && strings.Contains(challenge, proofReplayDesc) {
return dpopRetryProofReplay

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Suggestion: the comment says "both markers are DPoP-specific", but proofReplayDesc = "already been used" is plain English, not a DPoP token — the two strings.Contains checks match independently anywhere in the header. A multi-challenge WWW-Authenticate carrying a DPoP clock-skew rejection plus a Bearer error_description="...already been used" would be classified as a replay and resent. Bounded at maxDPoPRetrySends, so the cost is a couple of wasted requests, and TestRoundTripper_BearerChallengeDoesNotTriggerProofRetry covers only the Bearer-only header, not the mixed one. Worth either matching proofReplayDesc only within the DPoP challenge or softening the comment's claim.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right — the comment overclaimed. invalid_dpop_proof is DPoP-specific; "already been used" is plain English.

Fixed the code rather than softening the comment: 25a1ddb widens proofReplayDesc to "DPoP proof JWT has already been used", which is DPoP-specific, so the claim now holds and the cross-challenge false positive is gone. Fail-safe direction too — a reworded description means no retry, i.e. the pre-PR behavior.

Added TestRoundTripper_MixedChallengeDoesNotTriggerProofRetry for the case you named: DPoP rejecting for clock skew alongside a Bearer challenge mentioning reuse → 1 call, no retry.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No blocking issues found.

Comment thread pkg/oktaauth/round_tripper.go Outdated
Comment on lines +353 to +361
prefix, err := io.ReadAll(io.LimitReader(resp.Body, errorBodySniffLimit))
resp.Body = restoredBody{
Reader: io.MultiReader(bytes.NewReader(prefix), resp.Body),
Closer: resp.Body,
}
if err != nil {
return nil
}
return prefix

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Suggestion: on a partial-read error prefix has already been restored onto resp.Body, but returning nil tells the callers there was no body at all. finish then hands that nil to annotateEmptyDPoPError, which sees an empty body and replaces the restored partial payload with the challenge restatement (and retryableDPoPFailure loses a body-carried use_dpop_nonce). Returning prefix unconditionally keeps whatever did arrive; the restore path is already correct either way.

Suggested change
prefix, err := io.ReadAll(io.LimitReader(resp.Body, errorBodySniffLimit))
resp.Body = restoredBody{
Reader: io.MultiReader(bytes.NewReader(prefix), resp.Body),
Closer: resp.Body,
}
if err != nil {
return nil
}
return prefix
prefix, err := io.ReadAll(io.LimitReader(resp.Body, errorBodySniffLimit))
resp.Body = restoredBody{
Reader: io.MultiReader(bytes.NewReader(prefix), resp.Body),
Closer: resp.Body,
}
// Even on a read error, prefix holds the bytes that did arrive and has been
// restored onto the body -- report them rather than claiming an empty body,
// which would let annotateEmptyDPoPError overwrite real content.
return prefix
}

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in fffa61c — accepted, prefix is now returned unconditionally. Your trace was right about the consequence; the mutation test shows it directly, with the partial payload replaced by the challenge restatement when the old return nil is restored.

One note on the snippet: it keeps prefix, err := and then never uses err, which does not compile. Used prefix, _ := with a comment explaining the read error is dropped because the caller meets it again on its own read.

Worth adding that this path is not hypothetical for this PR — a body dying mid-read is the same stale-connection scenario the proof-replay hypothesis rests on.

Comment thread pkg/connector/helpers.go
Comment on lines 113 to 124
bytes, err := io.ReadAll(response.Body)
if err != nil {
return okta.Error{}, err
return okta.Error{}, fmt.Errorf("okta-connectorv2: %s: read error body: %w", response.Status, err)
}

// An empty or non-JSON body used to surface as a bare "unexpected end of JSON
// input" with no status code, which made an empty 400 indistinguishable from
// an empty 403 in logs.
err = json.Unmarshal(bytes, &errOkta)
if err != nil {
return okta.Error{}, err
return okta.Error{}, fmt.Errorf("okta-connectorv2: %s: unparseable error body %q: %w", response.Status, bodyExcerpt(bytes), err)
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Suggestion: the new messages add the status text and the prefix, but still carry no gRPC code. All five call sites (group.go:211, role.go:456/498, app.go:407/479) return this verbatim, so an unparseable 403 body surfaces as codes.Unknown, while a parseable one reaches handleOktaResponseErrorWrapErrorsWithRateLimitInfo(GrpcCodeFromHTTPStatus(...)) and gets PermissionDenied. Same upstream failure, two different codes depending only on whether Okta sent a body. Consider wrapping with uhttp.WrapErrors(uhttp.GrpcCodeFromHTTPStatus(response.StatusCode), ...) here so the retry/surface decision is driven by the status either way.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in fffa61c — the asymmetry was real: an unparseable 403 reached the sync as codes.Unknown while a parseable one became PermissionDenied for the same upstream failure, and a 429 was retryable only if Okta happened to send a body.

getError now routes both failure paths through uhttp.WrapErrors(uhttp.GrpcCodeFromHTTPStatus(response.StatusCode), ...) in a small bodyReadError helper, keeping the connector prefix, the status text, and the body excerpt in the message. TestGetError_ClassifiesByHTTPStatus covers 403/429/404/400, asserting both the code and that the message still carries prefix and status.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No blocking issues found.

Okta answers a bad DPoP proof with a 400, a zero-length body, and the reason
in a DPoP-scheme WWW-Authenticate header. The vendored SDK's
CheckResponseForError reads that header only for 401/403 responses whose
scheme is Bearer, then discards the decode failure on the empty body, so every
such rejection reached the caller as "the API returned an unknown error" and
failed the whole sync with no way to tell what happened.

Confirmed live against a DPoP-enabled tenant. A replayed proof jti, an iat
more than five minutes old, and an iat in the future all produce that exact
shape; an unsolicited nonce is accepted, and no resource response ever offers
a DPoP-Nonce, so the resource server issues no nonce challenges at all.

oktaauth:
- restate the WWW-Authenticate challenge as the error body the SDK expects, so
  it renders what Okta actually said. The challenge is passed through whole
  rather than parsed into auth-params: RFC 9110 allows several challenges in
  one header, attributing a param to the right scheme needs a real parser, and
  quoting the header drops nothing and cannot misattribute one scheme's error
  to another. Bounded to client errors with a genuinely empty body, so a real
  body is never replaced -- including a body that failed partway through being
  read, whose bytes are reported rather than mistaken for an empty body.
- retry once with a fresh proof when Okta reports a replayed proof jti; uhttp's
  transport retries a failed request with its headers untouched, so a
  stale-connection retry resends the same proof. Matched on the full
  DPoP-specific phrase, not on "already been used" alone, which another
  scheme's challenge in the same header could carry. A generic
  invalid_dpop_proof is not retried, since a skewed clock would only triple
  the traffic.
- log any DPoP-related 4xx at Warn with the full challenge, before the SDK
  discards it, logarithmically sampled with a total_occurrences field: a
  skewed clock is not retried, so an unsampled line would fire once per
  request for a whole sync
- guard the retry drain against a nil response body. net/http always sets one,
  but io.Copy on a nil ReadCloser panics and the two neighbouring helpers
  already check it.
- match a nonce challenge on 400 as well as 401, and in the JSON-body shape as
  well as the header, with a bounded retry loop. Defensive: unreachable on the
  tenants observed, but correct per RFC 9449 and consistent with the token
  endpoint's existing handling

connector:
- getError now names the HTTP status, excerpts an undecodable body, and carries
  a grpc code derived from that status. Its five call sites return its error
  verbatim, so an empty body surfaced as a bare "unexpected end of JSON input"
  with no status and no prefix, and an unparseable body reached the sync as
  codes.Unknown while a parseable one became PermissionDenied for the same
  upstream failure. #200 rewrote handleOktaResponseError but left getError
  untouched.
- the event feed no longer returns the SDK error bare, so it picks up a grpc
  code and the status line like every other call site. #200 did not touch
  event_log.go.

Both truncation paths back up to a rune boundary rather than cutting at a byte
offset, so an excerpt reaching an error message or a log field stays valid UTF-8.

Status codes and grpc codes for everything routed through
handleOktaResponseError, including 429, come from #200's
GrpcCodeFromHTTPStatus fallback; no duplicate handling is added here.

Validated with four clean full syncs against a live tenant (119 actions, no
errors) plus unit coverage of each path using the captured challenge strings.
The panic guard, the partial-read fix, and both rune-boundary cuts are verified
by mutation.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No blocking issues found.

Comment thread pkg/oktaauth/round_tripper.go

@mateoHernandez123 mateoHernandez123 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good — the empty-body DPoP 400 now surfaces Okta's reason and the replay retry is gated on the live challenge string.


// truncateAtRuneBoundary cuts s to at most maxBytes without splitting a rune, so
// the result stays valid UTF-8. A raw byte slice can leave a partial rune behind.
func truncateAtRuneBoundary(s string, maxBytes int) string {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a verbatim duplicate of truncateAtRuneBoundary added in pkg/connector/helpers.go:160 by this same PR. Also, pkg/oktaauth/token_source.go:309's pre-existing formatTokenError has the exact bug this function guards against (excerpt[:errorBodyExcerptLimit] can split a UTF-8 rune) and isn't updated to use it. Suggest consolidating to one rune-safe truncation helper in this package and pointing formatTokenError at it too, rather than shipping the fix twice while leaving a live instance of the bug unfixed next to it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants