Skip to content

fix(gmail): avoid duplicate sanitized content - #992

Merged
steipete merged 1 commit into
mainfrom
fix/gmail-sanitize-json-duplicate
Aug 14, 2026
Merged

fix(gmail): avoid duplicate sanitized content#992
steipete merged 1 commit into
mainfrom
fix/gmail-sanitize-json-duplicate

Conversation

@steipete

Copy link
Copy Markdown
Collaborator

Summary

  • stop copying sanitized Gmail headers and bodies beside the existing message result
  • preserve the default message envelope while allowing --results-only to unwrap it
  • document the JSON shape and add fixture-backed regression coverage

Reporter credit is recorded in the changelog. Thanks @ronny-rentner.

Verification

  • go test ./internal/cmd -run '^(TestGmailGetCmd_SanitizeContent_JSONUsesSafeEnvelope|TestGmailGetCmd_JSON_ResultsOnlyPreservesCompleteAttachmentResult|TestGmailThreadGet_SanitizeContent_JSONUsesSafeEnvelope)$' -count=1
  • make ci
  • built Linux CLI exercised against a local TLS Gmail fixture, with no live mailbox access:
    • default sanitized JSON: top-level keys message; no URL or raw payload
    • --results-only: direct sanitized message; no wrapper, URL, or raw payload
  • autoreview clean with no accepted/actionable findings

Closes #986

@clawsweeper

clawsweeper Bot commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

@clawsweeper clawsweeper Bot added merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. P1 Urgent regression or broken agent/channel workflow affecting real users now. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. labels Aug 14, 2026
@clawsweeper

clawsweeper Bot commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

Codex review: found issues before merge. Reviewed August 14, 2026, 3:24 PM ET / 19:24 UTC.

ClawSweeper review

What this changes

The PR removes duplicate top-level sanitized headers and body from gmail get --json --sanitize-content, while documenting and testing direct output through --results-only.

Merge readiness

⚠️ Ready for maintainer review - 4 items remain

Keep this collaborator-authored PR open for a maintainer compatibility decision. It fixes the reported duplication, but it silently removes existing top-level JSON fields from the default sanitized Gmail response.

Priority: P1
Reviewed head: e7a3cd22c3cb778f0bde2f9c8401eda43bd28733
Owner decision: Required. See Decision needed.

Review scores

Measure Result What it means
Overall readiness 🦐 gold shrimp (3/6) The patch is focused and has useful fixture-backed validation, but the unresolved default JSON compatibility break blocks a higher rating.
Proof confidence 🌊 off-meta tidepool Not applicable: This collaborator PR is not subject to the external contributor proof gate; its body nevertheless includes a real built-CLI run against a local TLS Gmail fixture.
Patch quality 🦐 gold shrimp (3/6) 1 actionable review finding remain.

Verification

Check Result Evidence
Real behavior Not applicable Not applicable: This collaborator PR is not subject to the external contributor proof gate; its body nevertheless includes a real built-CLI run against a local TLS Gmail fixture.
Evidence reviewed 6 items Current default contract: Current main emits sanitized message, headers, and—when present—body at the top level, so automation can currently read .headers and .body.
Proposed breaking shape change: The branch replaces the multi-field sanitized payload with only message, removing the existing default top-level fields.
Existing opt-in mechanism: The root flag already promises primary-result-only JSON, and the formatter unwraps a map with one candidate; this supports a compatibility-preserving --results-only path without changing the default response.
Findings 1 actionable finding [P1] Preserve the default sanitized JSON contract
Security None None.

How this fits together

Gmail read commands fetch a message, optionally sanitize unsafe content, and serialize the result for CLI JSON consumers. The JSON formatter then optionally unwraps a single primary result for automation users who request --results-only.

flowchart LR
  A[Gmail message] --> B[Gmail read command]
  B --> C[Content sanitization]
  C --> D[JSON envelope selection]
  D --> E[Automation output]
Loading

Decision needed

Question Recommendation
Should the default sanitized Gmail JSON response remain backward-compatible, with compact output only under --results-only, or should this release intentionally remove the top-level headers and body fields? Preserve the default envelope: Keep the current default fields and make --results-only return the direct sanitized message, with tests for both contracts.

Why: Both choices are technically feasible, but selecting a breaking default JSON contract requires maintainer ownership of the upgrade policy.

Before merge

  • Preserve the default sanitized JSON contract (P1) - This replaces the current default top-level headers and body with only message. Existing scripts that read .headers or .body will receive no value after upgrade. Keep the default envelope and make the direct compact message available through --results-only, unless maintainers explicitly accept the breaking contract change.
  • Resolve merge risk (P1) - Merging changes the default sanitized JSON schema: existing scripts reading top-level .headers or .body will lose those fields after upgrade.
  • Complete next step (P1) - A narrow repair is possible, but maintainers must first choose whether the default JSON compatibility break is intentional.

Findings

  • [P1] Preserve the default sanitized JSON contract — internal/cmd/gmail_get.go:77
Agent review details

Security

None.

Review metrics

Metric Value Why it matters
Production and regression delta production -7, tests +22, docs/changelog +3 The focused implementation reduction is paired with targeted coverage, but it still changes an established default response contract.

Root-cause cluster

Relationship: fixed_by_candidate
Canonical: #986
Summary: This PR is the explicit candidate fix for the open report about duplicate sanitized Gmail content.

Members:

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge-risk options

Maintainer options:

  1. Preserve the default JSON schema (recommended)
    Restore the existing default sanitized envelope and use --results-only as the opt-in direct-message shape before merging.
  2. Approve an intentional breaking response change
    Land the compact default only after maintainers explicitly accept that existing JSON consumers must update their field paths.

Technical review

Best possible solution:

Preserve the established default sanitized envelope and make the compact sanitized message available only through the already-documented --results-only path, with regression coverage for both shapes.

Do we have a high-confidence way to reproduce the issue?

Yes—current main constructs top-level sanitized headers and body, while the branch returns only message; a consumer reading either removed path will reproduce the compatibility break from source.

Is this the best way to solve the issue?

No. The change fixes duplication, but changing the default schema is not the narrowest safe path; preserve it and make --results-only the compact opt-in response.

Full review comments:

  • [P1] Preserve the default sanitized JSON contract — internal/cmd/gmail_get.go:77
    This replaces the current default top-level headers and body with only message. Existing scripts that read .headers or .body will receive no value after upgrade. Keep the default envelope and make the direct compact message available through --results-only, unless maintainers explicitly accept the breaking contract change.
    Confidence: 0.99

Overall correctness: patch is incorrect
Overall confidence: 0.99

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 35f2bb286662.

Labels

Label changes:

  • add P1: If merged as written, existing Gmail automation that consumes the default top-level sanitized fields can fail immediately after upgrade.
  • add merge-risk: 🚨 compatibility: The branch removes currently emitted top-level JSON fields from the default output shape.
  • add rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🌊 off-meta tidepool and patch quality is 🦐 gold shrimp.
  • add status: ⏳ waiting on author: ClawSweeper has contributor-facing work open and is waiting for author action. Not applicable: This collaborator PR is not subject to the external contributor proof gate; its body nevertheless includes a real built-CLI run against a local TLS Gmail fixture.

Label justifications:

  • P1: If merged as written, existing Gmail automation that consumes the default top-level sanitized fields can fail immediately after upgrade.
  • merge-risk: 🚨 compatibility: The branch removes currently emitted top-level JSON fields from the default output shape.
  • rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🌊 off-meta tidepool and patch quality is 🦐 gold shrimp.
  • status: ⏳ waiting on author: ClawSweeper has contributor-facing work open and is waiting for author action. Not applicable: This collaborator PR is not subject to the external contributor proof gate; its body nevertheless includes a real built-CLI run against a local TLS Gmail fixture.

Evidence

What I checked:

  • Current default contract: Current main emits sanitized message, headers, and—when present—body at the top level, so automation can currently read .headers and .body. (internal/cmd/gmail_get.go:77, 35f2bb286662)
  • Proposed breaking shape change: The branch replaces the multi-field sanitized payload with only message, removing the existing default top-level fields. (internal/cmd/gmail_get.go:77, e7a3cd22c3cb)
  • Existing opt-in mechanism: The root flag already promises primary-result-only JSON, and the formatter unwraps a map with one candidate; this supports a compatibility-preserving --results-only path without changing the default response. (internal/outfmt/outfmt.go:101, 35f2bb286662)
  • Feature provenance: Blame attributes the current sanitized JSON payload to the v0.36.0 release commit and the replacement line to this PR's commit, both authored by Peter Steinberger. (internal/cmd/gmail_get.go:77, eaa5d6316890)
  • Submitted real-command proof: The PR body records a built Linux CLI run against a local TLS Gmail fixture, covering both default sanitized JSON and --results-only; no live mailbox data was used. (e7a3cd22c3cb)
  • Focused patch scope: The diff is clean and changes one implementation file, one regression test file, one workflow document, and the changelog. (internal/cmd/gmail_get.go:77, e7a3cd22c3cb)

Likely related people:

  • Peter Steinberger: Current-main blame attributes the sanitized payload to Peter's v0.36.0 release commit, and he authored the proposed focused repair. (role: current Gmail JSON-path author and recent contributor; confidence: medium; commits: eaa5d6316890, e7a3cd22c3cb; files: internal/cmd/gmail_get.go, internal/cmd/gmail_sanitize_test.go, docs/gmail-workflows.md)

Rank-up moves

Optional improvements that raise the rating; they are not merge blockers.

  • Resolve the default-versus---results-only compatibility decision and update the implementation and regression tests accordingly.

Rating scale

Score Internal tier Crab rank Meaning
6/6 S 🦀 challenger crab Exceptional readiness
5/6 A 🦞 diamond lobster Very strong readiness
4/6 B 🐚 platinum hermit Good normal PR; ordinary maintainer review
3/6 C 🦐 gold shrimp Useful, but confidence is limited
2/6 D 🦪 silver shellfish Proof or implementation needs work
1/6 F 🧂 unranked krab Not merge-ready
N/A NA 🌊 off-meta tidepool Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@steipete
steipete merged commit e5c4424 into main Aug 14, 2026
15 checks passed
chrischall added a commit to chrischall/gogcli-mcp that referenced this pull request Aug 14, 2026
…nitized get (#272)

Raises `MIN_GOG_VERSION` **0.35.0 → 0.37.0** (and
`fly-gog-runner/Dockerfile`'s pin with it) and takes up what 0.36.0 /
0.37.0 added.

## New tools

**Gmail — draft-side reply/forward** (gog 0.36.0, openclaw/gogcli#977)

| Tool | |
|---|---|
| `gog_gmail_drafts_reply` | Save a reply as a draft — inherited
recipients, subject and quote; never sends |
| `gog_gmail_drafts_reply_all` | Same, to every participant |
| `gog_gmail_drafts_forward` | Save a forward as a draft; `to` is
optional, so it can be staged with no recipients at all |

They take the same flag set as the send commands and share gog's
composition path, so `replySchema` / `appendReplyFlags` are reused
verbatim rather than re-declared. Staging a reply previously meant
`gog_gmail_drafts_create` + `replyToThreadId`, which threads the draft
but inherits neither the original's recipients nor its quoted body —
both had to be rebuilt by hand, and a missed Cc is invisible until the
draft goes out.

**Sheets — Connected Sheets reads** (gog 0.37.0, openclaw/gogcli#938)

`gog_sheets_datasource_list` / `_describe`, and
`gog_sheets_datasource_table_list` / `_describe` / `_read`. Read-only by
construction (gog exposes no create/update/refresh/delete here). A
data-source table has no id of its own in the Sheets API — its
definition lives on its top-left cell — so extracts are addressed by a
sheet-qualified A1 anchor like `Extracts!B3`.

## Behaviour these depend on

- **`gog_gmail_get` gains `sanitizeContent`.** The flag predates 0.37.0
but emitted the headers and body **twice** in JSON
(openclaw/gogcli#992), so the flag meant to shrink the payload enlarged
it. Verified against a live 0.37.0 build: the sanitized message now
arrives under a single `message` key.
- **`gog_calendar_events` gains `days`**, and it and
`gog_calendar_search` now describe the window rules gog 0.36.0 enforces
(openclaw/gogcli#981). `--days` is a window *length* anchored at
`--from`; it used to discard `--from` silently and answer for today, at
exit 0, in a well-formed table. Fixed presets no longer combine with
`from`/`to`/`days`, and `days` no longer combines with `to`. The old
descriptions invited exactly the combinations that now fail, and the
base calendar test asserted an arg array gog refuses to run.
- **The auth tools gain `extraScopes`** (plus `--force-consent` on the
interactive one, since Google re-prompts for a *new* scope only when
consent is forced). Nothing else can request `bigquery.readonly`, which
Google demands whenever a Sheets response *contains* BigQuery Connected
Sheets data — without it the new sheets tools cannot be authorized
through the wrapper at all. Documented in `docs/auth-scopes.md`.

## Not adopted: `gmail search --count`

gog 0.36.0 (openclaw/gogcli#985) upstreamed this wrapper's match-count
probe, down to the page size and the exact/lower-bound split. The local
probe stays, and the stale comment claiming gog *cannot* supply the
count is corrected to say why:

- it is spent only on a result set already known to be truncated, where
`--count` is decided before the search runs and would cost every search
an extra Gmail request;
- it is best-effort, where gog returns the probe's error from the whole
command — a failed count would turn a search that *did* succeed into an
error.

## Verification

Beyond the unit tests (the mocked suites only assert arg arrays), every
new command was run against the real gog v0.37.0 binary with fake ids,
which parses flags before any API call:

- `gmail drafts reply` / `reply-all` / `forward` — full flag sets parse;
failures are the API's 400 on the fake id (and a send-as validation for
`--from`), not unknown flags.
- `sheets datasource list` / `describe` / `table list` / `table
describe` / `table read` — all reach the API and 404 on the fake
spreadsheet.
- `calendar events --from 2026-09-25 --days 5` returns the anchored
window; `--today --from …` and `--days … --to …` are both rejected with
the messages the tool descriptions now quote.
- `gmail get --sanitize-content --json` on a real message: top-level
keys are exactly `["message"]`.

`npm test` (all workspaces + fly-gog-runner), `npm run typecheck` and
`npm run build` are green; the 100% coverage gate holds.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_01EjH6C4jVKavpHiFwBW574N

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. P1 Urgent regression or broken agent/channel workflow affecting real users now. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

gmail get --json --sanitize-content emits the message content twice

1 participant