Skip to content

feat: add local Browser Use, model discovery, and automation notifications - #60

Closed
guygubaby wants to merge 106 commits into
mainfrom
feat
Closed

guygubaby wants to merge 106 commits into
mainfrom
feat

Conversation

@guygubaby

@guygubaby guygubaby commented Oct 26, 2026 •

Copy link
Copy Markdown
Collaborator

Summary / 变更说明

Enable tasks to use authorized local Chrome/Edge pages, configure model channels through protocol and model discovery, and deliver monitored automation results to IM.

  • Task references: add authorized @ task mentions and direct reads of the latest visible assistant reply, with stable continuation and legacy-history support.
  • Automations: add durable IM result notifications, bounded monitoring, and per-run conversation selection. Conversation-managed creation preserves notification and monitoring settings while pinning the selected task in the same transaction.
  • Local Browser Use: add an opt-in Chrome/Edge extension with pairing, explicitly shared tabs, task ownership, cancellation/revocation, snapshots and control, installation guidance, and draggable chat previews. Cloud browsing remains independent; local Plan actions are read-only.
  • Model setup: detect supported protocols, adapt missing /v1 endpoint variants while retaining proxy prefixes, discover image-generation models, and preserve the selected channel after catalog updates.
  • Web and deployment: refresh responsive layouts, segmented controls, cards, attribution, dialog focus visibility, and new-task composer focus; include isolated Core UAT deployment tooling.

Fixes #59. Supersedes #57 and #58.

Compatibility and rollout

Apply 20261022090000_add_automation_notifications and 20261023100000_add_automation_monitoring before updating the API. These migrations add nullable configuration columns and a delivery table without deleting historical rows. Existing automations retain their conversation behavior, with notifications disabled and monitoring unlimited when the new metadata is absent.

Deploy compatible API, Runner/Worker, and Web versions together, updating API first. Existing task snapshots refresh their built-in publication; old Workers must not remain paired with the new publication. Before rollout, check for a user-created capability named linksense-local-browser, which is now reserved. Older model settings and browser credentials without approval metadata remain supported. Codex's native lifecycle, approval, cancellation, and retry semantics remain authoritative.

The extension is distributed as an unpacked Chrome/Edge 127+ download. Extension 0.4.5 accepts same-origin HTTP and HTTPS, including LAN IPs and domain names. HTTP uses WS and is unencrypted; existing pairing, origin checks, rate limits, and tab authorization remain enforced. Reload the updated extension and LinkSense page to enable HTTP on older installations. It operates authorized browser tabs and does not provide desktop control or local-file transfer. Page content and screenshots are processed by LinkSense and the configured model.

Keep the pinned dependency security overrides, including the Zod, undici, body-parser, and MSAL Node fixes.

Verification / 验证

Validated locally after merge and CI repairs on c55d0f9:

  • pnpm test:packages: Shared 665, API 4,461, Runner 1,064, and Docs 23 passed; Runner retains one existing skip.
  • Affected Web component/node tests: 145 passed. Application integration tests for automation notifications, projects, new-task focus, and Browser Use: 67 passed.
  • Browser regression suite: 22 passed, covering model setup, narrow/dark layouts, setup guidance, and the real extension's pairing, reconnect, control, previews, and revocation.
  • pnpm typecheck, pnpm lint, and pnpm build passed. Lint retains the existing TanStack Virtual/React Compiler warning.

Post-merge CI repairs add Redis-backed pairing rate limits (10 exchanges per minute, shared across API instances), isolate browser-intent module mocks from the shared test graph, normalize long fractional prices in linear time, and make test escaping literal. No security gate or timeout threshold was relaxed.

HTTP follow-up validation (171fd51): 93 focused API/Web unit tests, API/Web type checks, affected-file lint, and API/Web production builds passed. Real Chrome completed pairing, reconnect, control, and revocation through a non-localhost HTTP name with window.isSecureContext === false; no secure-context bypass was used.

Regressions cover preserving monitoring/notification settings through conversation creation and transactional pinning, retaining both automation and local-browser activity summaries, forwarding both sets of scoped runtime endpoints/tokens, and keeping combined startup instructions within the existing 1,000-token budget.

Linux CI on c55d0f9: passed; the HTTP follow-up 171fd51 is pending. The complete verify job passed frozen install, production dependency audit, types/lint, all package/deployment tests, database migration checks, privileged Runner filesystem checks, and production builds. Code scanning, dependency review, and secret scanning also pass for this head. Docker-backed CI checks were not rerun on the local Mac. The existing Dev Server was updated from feat, with API/Web now on the HTTP follow-up 171fd51 and unchanged Runner/Worker code retained at c55d0f9. Its configuration and data volumes are preserved, and its downloadable extension is 0.4.5. Post-deployment checks passed for service readiness, matching API/Web build IDs, gateway WebSocket forwarding, migration state, and unchanged user/conversation/automation counts. Database and configuration backups were retained. Live model/IM integrations, Edge/Windows, and enterprise browser policies still require acceptance testing.

Checklist / 检查项

  • Behavior changes and merge interactions include regression tests; user-visible resources cover all six supported locales.
  • Data compatibility, migrations, deployment order, and the new reserved capability name are documented.
  • Package tests, affected Web/browser tests, type checks, lint, and production builds pass locally.
  • HTTP follow-up head Linux CI passes (previous head c55d0f9 passed).
  • Live model, IM delivery, and target-browser acceptance is complete.

@guygubaby

Copy link
Copy Markdown
Collaborator Author

Latest synchronization follow-up: feat now includes main 088cf2d. Commit 1f1cf48 updates the five UAT source-build Dockerfile paths to deploy/docker/ after main's reorganization, without changing build targets, source roots, credentials or volume identities. Added the missing-path regression first (failed on the old paths), then verified all five UAT deployment/upgrade tests pass after the fix. No force push, main merge, deployment or database mutation was performed; new exact-head CI is separate from cancelled earlier runs.

@guygubaby guygubaby closed this Nov 4, 2026
@guygubaby
guygubaby deleted the feat branch November 4, 2026 01:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

3 participants