Skip to content

✨ Inbox: drag and drop documents into the Inbox - #18411

Merged
hivecommons-hive[bot] merged 3 commits into
v5from
feat/inbox-upload-documents
Nov 16, 2026
Merged

hivecommons-hive[bot] merged 3 commits into
v5from
feat/inbox-upload-documents

Conversation

@clubanderson

Copy link
Copy Markdown
Member

First PR of drag-and-drop ingest for the Inbox: text documents, the drop/Add UI and the new upload source. URL fetch (SSRF guard), docx, sandboxed PDF and audio/video drops follow in separate PRs; audio/video will plug into the in-hive recording pipeline once it lands.

What it does

  • UI. The whole Inbox section is a drop zone (overlay while dragging), plus a + Add button with a keyboard-operable file picker, and Ctrl/Cmd+V paste of files while the Inbox has focus. Multi-file drops show one row per file with upload progress (XHR per file) and per-file errors (too large, unsupported type, failed parse, too many files). Status is announced in a polite live region. Owner and read-write only; read sessions see none of it. Existing tokens only.
  • Backend. New builtin source upload and POST /api/inbox/uploads (multipart, one or more file fields). Items show upload · <filename>, who added them, and work with the Share menu, fact status and route history like any other item. Audited per user as inbox_upload (outcome or refusal code, size; never content or file name).
  • Type detection. Magic bytes first (PE, ELF, Mach-O, wasm, zip, gzip, 7z, rar, PDF, OLE/.doc, RTF, audio/video, images are refused whatever they are named), then UTF-8 validation (no NUL, low control-char ratio, UTF-16 refused), then a parse check for the format the extension asks for. Files without an extension: JSON if it parses, else text.
  • Extraction (in-process, 30 s timeout, previews capped at 256 KiB, item text at 1 MiB, original stored whole):
    • md / txt: as is, BOM dropped, newlines normalised; md title from the first # heading.
    • json: validated (syntax errors name the line), pretty-printed in a fenced block with a top-level keys / array length summary.
    • jsonl: every line validated; record count and the first 20 records.
    • yaml: validated with gopkg.in/yaml.v3 (already a dependency, MIT/Apache-2.0); fenced block with a summary.
    • csv / tsv: encoding/csv; markdown table preview of the first 20 rows (pipes escaped, 20 columns max) with row and column counts.
    • Fences are always longer than any backtick run in the content.
    • .doc, .rtf, .odt, .pptx, .xlsx are refused with a message saying what to save it as.
  • Storage. connector-state/uploads/<id>/ (0700) with original, extracted.md, meta.json (0600, exclusive create). Ids are validated by regexp, so request paths never leave the store.
  • Knowledge. A fact at connectors/personal/upload-<id>.md (source: upload, attr_upload_content: ready, attr_added_by), with the text inside the WrapUntrusted envelope; the vault is connected and reindexed at once. Destinations get the text via Content.Markdown (labelled ## Document), and the agent prompt wraps it again.
  • Limits (inbox.uploads.*, env overrides HIVE_INBOX_UPLOADS_*): max_file_bytes 25 MiB, max_files_per_drop 20, quota_bytes 2 GiB, document_retention_days 0 (keep), media_retention_days 5. An hourly retention sweep deletes only originals; text, meta, item and fact stay.
  • Cross-site requests (Sec-Fetch-Site: cross-site) are refused on top of the SameSite=Lax session cookie.

Tests

  • Go: sniffing table (disguised PE/ELF/Mach-O/zip/PDF/OLE/media, unknown and unsupported extensions, UTF-16, NUL, Latin-1, control chars, empty), per-type extraction with fixtures (md, json, malformed json, jsonl, yaml, csv, tsv, wide and long tables, fence escaping, caps, cancellation), store perms, quota, retention, id validation, fact contents, WrapUntrusted before the agent prompt with injected markers, SetSources keeping the source, config defaults/env/validation, and the HTTP role matrix (read 403, unverified owner 403, cross-site 403), per-file statuses, mixed drops and audit entries.
  • JS (node): role gating, drop overlay, multi-file rows with progress and per-file errors, network errors, paste focus rules, the Add panel.

Docs

New docs/inbox-uploads.md (types, limits, retention, privacy, API), API reference row, dashboard/openapi.json, security model paragraph, env vars, documentation map, changelog fragment. static/index.html regenerated.

No new dependencies.

🤖 Generated with Claude Code

@kubestellar-prow kubestellar-prow Bot added the dco-signoff: yes Indicates the PR's author has signed the DCO. label Nov 16, 2026
@kubestellar-prow

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign hanthor for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kubestellar-prow kubestellar-prow Bot added the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label Nov 16, 2026
@github-actions

github-actions Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

Open PR overlap check

Base branch: v5

Conflicts are checked with every PR merged onto the current tip of v5, so a PR that only needs a rebase is not reported as conflicting with its siblings.

PR Title Relation Files
#18405 ✨ Inbox: send items into an Inception (spek) session same files dashboard/openapi.json
src/docs/env-vars.md
src/docs/inbox-voice-sources.md
src/pkg/dashboard/static-src/js/inbox.js
src/pkg/dashboard/static/index.html
src/pkg/inbox/inbox.go
#18415 ✨ Inbox: pills record where an item was shared, by whom and when CONFLICT dashboard/test/inbox-item.test.js (conflict)
src/docs/inbox-voice-sources.md
src/pkg/dashboard/api_inbox.go
src/pkg/dashboard/static-src/css/inbox.css
src/pkg/dashboard/static-src/js/inbox.js
src/pkg/dashboard/static/index.html
src/pkg/inbox/inbox.go
src/pkg/inbox/service.go
#18407 ⚠️ Retire /api/campaigns, campaign Jam and Recheck same files dashboard/openapi.json
src/docs/api-reference/knowledge.md
src/pkg/dashboard/static/index.html
#18413 ✨ meetings: Betty answers from the web, speaks when asked and digs into hard questions (phase B) same files src/docs/env-vars.md
#18408 ✨ Inbox: record recordings directly in hive (backend) CONFLICT dashboard/openapi.json (conflict)
src/cmd/hive/boot_inbox.go (conflict)
src/docs/api-reference/knowledge.md (conflict)
src/docs/documentation-map.md
src/pkg/config/inbox_config.go (conflict)
src/pkg/dashboard/api_inbox.go
src/pkg/inbox/inbox.go (conflict)
src/pkg/inbox/service.go (conflict)

@github-actions github-actions Bot added the conflicts-with-open-pr Open PR has a merge conflict with another open PR label Nov 16, 2026
@hivecommons-hive

hivecommons-hive Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

Hive status — updated in place; edits do not notify. Full history is in the Activity Monitor.

@clubanderson this PR is held at stage/071-needs-author and the hive will not merge it yet.

Why:

  • the branch conflicts with the base and needs a rebase

Next: the hive does not push to your branch. Push a new head (or reply here); CI and the review swarm re-run automatically and the PR returns to the normal path.

Evidence bundle `#184115116c99` sha256:69e51a0bf4afb078eb57bbbeede2b2bb266bf0552811523f8623ad31ff238459 — download from the dashboard or `hivectl review evidence #18411`.

Drop files onto the Inbox, use the new Add button, or paste them with
Ctrl/Cmd+V: Markdown, text, JSON, JSON Lines, YAML, CSV and TSV become
upload items with extracted markdown and a Knowledge fact.

- Content sniffing (magic bytes, UTF-8, parse check), never the name or
  client MIME alone; executables, archives, PDF, Office and media refused.
- Per-file progress rows and errors; owner and read-write only.
- Storage under connector-state/uploads/<id>/ (0700/0600), per-hive
  quota, files-per-drop and per-file caps (inbox.uploads.*, env
  overrides), retention sweep for originals.
- Extracted text fenced with WrapUntrusted in the fact and before any
  agent prompt; audited per user as inbox_upload.

Signed-off-by: Andy Anderson <andy@clubanderson.com>
… items

Signed-off-by: Andy Anderson <andy@clubanderson.com>
…imit

Signed-off-by: Andy Anderson <andy@clubanderson.com>
@clubanderson
clubanderson force-pushed the feat/inbox-upload-documents branch from 92772aa to 5116c99 Compare November 16, 2026 13:27
@github-actions github-actions Bot added the conflicts-with-open-pr Open PR has a merge conflict with another open PR label Nov 16, 2026
@hivecommons-hive
hivecommons-hive Bot merged commit c91544f into v5 Nov 16, 2026
83 of 84 checks passed
@kubestellar-prow
kubestellar-prow Bot deleted the feat/inbox-upload-documents branch November 16, 2026 13:39
@hivecommons-hive

Copy link
Copy Markdown
Contributor

merged by Hive trusted-author auto-merge: author @clubanderson holds role owner and GitHub permission admin on repo; CI green at 5116c99.

@github-actions

Copy link
Copy Markdown
Contributor

Mutation testing

⏱️ The 900s mutation budget ran out, so 1 package(s) were not fully mutated and the results below do not cover them:

  • pkg/inbox: skipped (not started)

❌ gremlins did not produce a report for 1 package(s), so no verdict can be derived from this run (see the job log):

  • pkg/dashboard: gremlins exited 137

6 mutant(s) in changed files are not reached by any test (NOT COVERED). A maintainer may apply the mutation/acknowledged label to accept this.

File Line Mutation
src/cmd/hive/boot_inbox.go 99 CONDITIONALS_NEGATION
src/cmd/hive/boot_inbox.go 102 CONDITIONALS_NEGATION
src/cmd/hive/boot_inbox.go 103 ARITHMETIC_BASE
src/cmd/hive/boot_inbox.go 103 CONDITIONALS_NEGATION
src/cmd/hive/boot_inbox.go 103 CONDITIONALS_NEGATION
src/cmd/hive/boot_inbox.go 108 CONDITIONALS_NEGATION

Verdict: error (#17349).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

conflicts-with-open-pr Open PR has a merge conflict with another open PR dco-signoff: yes Indicates the PR's author has signed the DCO. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant