Skip to content

✨ Inbox: read PDFs in a sandboxed hive-pdftext process - #18426

Merged
kubestellar-prow[bot] merged 2 commits into
v5from
feat/inbox-upload-pdf
Nov 16, 2026
Merged

kubestellar-prow[bot] merged 2 commits into
v5from
feat/inbox-upload-pdf

Conversation

@clubanderson

Copy link
Copy Markdown
Member

Fourth PR of drag-and-drop ingest: PDFs, parsed out of process.

What it does

  • A file starting with %PDF- and named .pdf (or with no extension), or a link serving application/pdf, becomes a pdf item: one ## Page N section per page with text, a page-count summary, and the Info title. A PDF with no text layer is refused with "no text layer (scanned PDF); OCR not supported yet".
  • Library: github.com/ledongthuc/pdf (BSD-3-Clause, pure Go). It is already in go.mod and NOTICE (the knowledge document parser uses it), so no new dependency. Poppler pdftotext (GPL) was avoided.
  • Sandbox: the new hive-pdftext command (built and shipped in the hive image next to hive-whisper) is the only place an uploaded PDF is parsed. The hive runs it:
    • as the existing low-privilege hive-transcribe uid via su-exec, with an environment of only PATH and LC_ALL;
    • under prlimit (1 GiB address space, CPU = timeout + 10 s, file size = the PDF's size, 32 open files, no core dumps) and nice -n 10;
    • in an empty network namespace (unshare --user --net) when the kernel allows it (probed once);
    • with the PDF on stdin, copied into a private 0700 work dir (file 0600) removed on exit, and only capped JSON on stdout;
    • with a 60 s wall clock after which the whole process group is SIGKILLed.
  • Caps: 200 pages, 1 MiB of text; library panics are parse failures. HIVE_PDF_SANDBOX, HIVE_PDF_RUN_AS, HIVE_PDF_MEMORY_MB override path, uid and memory. Without the sandbox the upload is refused with a clear message.

Tests

  • pkg/inbox/pdfbox with hand-built fixtures: a two-page text PDF with an Info title, a scanned-like PDF without text, page/char/byte caps, empty and malformed input, and the private work dir (0700, removed after the run).
  • cmd/hive-pdftext: flags, version, exit codes.
  • Hive side with fake sandboxes: argv (su-exec, prlimit, caps), stdin delivery, scanned refusal, exit codes, unreadable and oversize output, missing sandbox or user, and a hanging sandbox (with a child) killed at the timeout.

🤖 Generated with Claude Code

PDF text is extracted out of process by the new hive-pdftext command
(pure Go, github.com/ledongthuc/pdf, BSD-3-Clause, already in go.mod): as
the hive-transcribe uid via su-exec, empty environment, prlimit + nice,
empty network namespace where available, private 0700 work dir, page and
character caps, and a wall-clock timeout that kills the process group.
Scanned PDFs without a text layer are refused with a clear message.

Signed-off-by: Andy Anderson <andy@clubanderson.com>
@kubestellar-prow kubestellar-prow Bot added dco-signoff: yes Indicates the PR's author has signed the DCO. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Nov 16, 2026
@github-actions

github-actions Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

Open PR overlap check

Base branch: v5

Conflicts are checked with every PR merged onto the current tip of v5, so a PR that only needs a rebase is not reported as conflicting with its siblings.

PR Title Relation Files
#18417 ✨ meetings: Settings tab "Meeting attendee" with Kokoro persona presets (Betty E) same files dashboard/openapi.json
src/docs/api-reference/knowledge.md
#18419 ⚠️ Retire the inception API, watcher and brainstorm inception mode same files dashboard/openapi.json
#18424 ✨ meetings: Betty drafts issues, RFCs, Spek sessions and knowledge notes for approval (Betty F) same files dashboard/openapi.json
src/docs/api-reference/knowledge.md
src/docs/env-vars.md

@github-actions

Copy link
Copy Markdown
Contributor

Mutation testing

gremlins found 14 surviving mutant(s) in changed files: the code ran under tests, but changing it did not fail any test.

File Line Mutation
src/pkg/inbox/pdfbox/pdfbox.go 170 CONDITIONALS_BOUNDARY
src/pkg/inbox/pdfbox/pdfbox.go 205 CONDITIONALS_BOUNDARY
src/pkg/inbox/pdfbox/pdfbox.go 206 ARITHMETIC_BASE
src/pkg/inbox/pdfbox/pdfbox.go 206 INVERT_NEGATIVES
src/pkg/inbox/upload_pdf.go 76 CONDITIONALS_BOUNDARY
src/pkg/inbox/upload_pdf.go 167 CONDITIONALS_BOUNDARY
src/pkg/inbox/upload_pdf.go 195 CONDITIONALS_BOUNDARY
src/pkg/inbox/upload_pdf.go 207 ARITHMETIC_BASE
src/pkg/inbox/upload_pdf.go 212 CONDITIONALS_NEGATION
src/pkg/inbox/upload_pdf.go 216 ARITHMETIC_BASE
src/pkg/inbox/upload_pdf.go 216 INVERT_NEGATIVES
src/pkg/inbox/upload_pdf.go 229 CONDITIONALS_NEGATION
src/pkg/inbox/upload_pdf.go 242 CONDITIONALS_BOUNDARY
src/pkg/inbox/upload_pdf.go 242 CONDITIONALS_NEGATION

9 mutant(s) in changed files are not reached by any test (NOT COVERED). A maintainer may apply the mutation/acknowledged label to accept this.

File Line Mutation
src/pkg/inbox/pdfbox/pdfbox.go 45 ARITHMETIC_BASE
src/pkg/inbox/upload_pdf.go 33 ARITHMETIC_BASE
src/pkg/inbox/upload_pdf.go 100 CONDITIONALS_NEGATION
src/pkg/inbox/upload_pdf.go 101 CONDITIONALS_NEGATION
src/pkg/inbox/upload_pdf.go 117 CONDITIONALS_NEGATION
src/pkg/inbox/upload_pdf.go 135 CONDITIONALS_NEGATION
src/pkg/inbox/upload_pdf.go 182 ARITHMETIC_BASE
src/pkg/inbox/upload_pdf.go 182 CONDITIONALS_BOUNDARY
src/pkg/inbox/upload_pdf.go 182 CONDITIONALS_NEGATION

Verdict: lived (#17349).

Signed-off-by: Andy Anderson <andy@clubanderson.com>
@hivecommons-hive

hivecommons-hive Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

Hive status — updated in place; edits do not notify. Full history is in the Activity Monitor.

@clubanderson this PR is held at stage/071-needs-author and the hive will not merge it yet.

Why:

  • the review swarm returned requires_human: it has a question for you, not a fix for the hive
  • the reviewer asked you: clubanderson should confirm that a silently-disabled network namespace (fail-open) is acceptable for the PDF sandbox, since the PR also touches the Dockerfile and the security model.

Next: the hive does not push to your branch. Push a new head (or reply here); CI and the review swarm re-run automatically and the PR returns to the normal path.

@hivecommons-hive

Copy link
Copy Markdown
Contributor
605f7d7

ℹ️ Sentinel notice — informational; author is trusted (owner), merge is not blocked

Hive flagged this PR (author @clubanderson, head 605f7d7983c4) because it matches behaviors that can override security controls, escalate privileges or damage the codebase. This is a heuristic, not an accusation — a maintainer should confirm the change is intended before it merges.

  • sensitive_path — changes 1 sensitive path(s) (Touches a sensitive path (OWNERS, workflows, policies, hive config, security docs, …))
    • src/Dockerfile

Hive did not add the sentinel-alert label because this author is trusted. The finding remains recorded for audit and the dashboard Security tab; set sentinel.trusted_authors_block: true to require the blocking label for trusted authors too.

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Decision needed: @clubanderson should confirm that a silently-disabled network namespace (fail-open) is acceptable for the PDF sandbox, since the PR also touches the Dockerfile and the security model.

Read upload_pdf.go, upload_source.go, upload_types.go and the Dockerfile hunk in the diff. The design (su-exec to hive-transcribe, empty env, prlimit, process-group kill, capped stdout) looks sound to me. One verified concern:

Network isolation can be disabled for the life of the process by one cancelled request (low–medium) — src/pkg/inbox/upload_pdf.go, isolation() (the isoOnce.Do block). The probe runs exec.CommandContext(ctx, …) with the first caller's request context, and r.iso = probe.Run() == nil. If that context is cancelled or times out (client disconnect, or the 60 s Run timeout) before the probe finishes, Run() errors, iso is stored as false, and sync.Once never retries. Every later PDF then runs without unshare --user --net, with no log line saying so. The sandbox still has uid/prlimit/env limits, but the "empty network namespace" the PR body and security-model.md promise would silently not apply. Suggest probing with context.Background() plus a short timeout, and logging when isolation is unavailable.

Minor, not a defect: Run reports any ctx.Err() != nil as "took longer than %s", so a cancelled client shows the timeout message.

Because this adds a privileged-uid exec path and edits src/Dockerfile and security-model.md, it needs a maintainer's sign-off on the fail-open behaviour. I did not review the pdfbox parser internals or the test files in depth. Several CI jobs were still in progress when I read it.

— hive: agent=reviewer-2 backend=claude model=claude-sonnet-5-5 claude=2.1.325

@clubanderson clubanderson added lgtm Indicates that a PR is ready to be merged. approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Nov 16, 2026
@kubestellar-prow

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

Approval requirements bypassed by manually added approval.

This pull-request has been approved by:

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kubestellar-prow
kubestellar-prow Bot merged commit ec0d5c7 into v5 Nov 16, 2026
94 of 96 checks passed
@kubestellar-prow
kubestellar-prow Bot deleted the feat/inbox-upload-pdf branch November 16, 2026 14:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. dco-signoff: yes Indicates the PR's author has signed the DCO. lgtm Indicates that a PR is ready to be merged. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant