Multi-toggle for what App Context sends (Off / App summary / Screenshot LLM) + per-app "send screenshot" allow-list#244
Open
pianoandglass wants to merge 3 commits into
Conversation
Contributor
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (5)
🚧 Files skipped from review as they are similar to previous changes (4)
📝 WalkthroughWalkthroughThis PR adds App Context mode selection, metadata context resolution, screenshot-scope allowlisting, and source-specific runtime behavior. Settings now configure these modes, while tests can explicitly force the screenshot path. ChangesApp Context mode and scope
Estimated code review effort: 4 (Complex) | ~60 minutes Sequence Diagram(s)sequenceDiagram
participant PromptsSettingsView
participant AppContextService
participant AppContextSource
participant AccessibilityAPI
participant ScreenshotCapture
PromptsSettingsView->>AppContextService: run context prompt test
AppContextService->>AppContextSource: resolve effective source
AppContextSource->>AccessibilityAPI: read focused app context
AccessibilityAPI-->>AppContextSource: metadata and focus data
AppContextSource-->>AppContextService: off, metadata, or screenshot
alt screenshot source
AppContextService->>ScreenshotCapture: capture active window
ScreenshotCapture-->>AppContextService: screenshot data
else metadata or off source
AppContextService-->>PromptsSettingsView: metadata summary or empty result
end
Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
pianoandglass
marked this pull request as ready for review
June 25, 2026 10:35
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 3
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
Sources/AppContextService.swift (1)
159-169: 🔒 Security & Privacy | 🟡 MinorOff mode does not leak to prompts, but retains metadata in history
The initial concern regarding prompt construction is resolved. Downstream services (
PostProcessingService) ingest onlycontext.contextSummary. Verification confirms that for the.offeffective source,AppContextServiceexplicitly returns an empty string for this field (line 323), ensuring no app title or selection data is sent to the LLM.However,
AppState.swiftpersists the individual fields (selectedText,windowTitle,appName) directly into thePipelineHistoryItemhistory (lines 2814, 2830-2832) regardless of the source mode. While this data is not transmitted to the model, it remains stored locally in the history log, which may still constitute a privacy concern depending on requirements.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@Sources/AppContextService.swift` around lines 159 - 169, The off-mode prompt issue is already resolved, but `AppState` still persists `selectedText`, `windowTitle`, and `appName` into `PipelineHistoryItem` even when the effective source is `.off`. Update the history-building path in `AppState` so those fields are omitted, cleared, or conditionally suppressed when off mode is active, and keep the behavior aligned with how `AppContextService` already returns an empty `contextSummary` for `.off`.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@Sources/AppContextService.swift`:
- Around line 326-333: The metadata fallback in AppContextService uses
AppContextSource.webContext(), which re-queries the frontmost app and can mix
data from different apps. Update the fallback path to thread the existing
appElement through the web-context lookup instead of calling webContext()
directly, and keep the metadataSummary assembly in sync with the captured
application context.
In `@Sources/AppContextSource.swift`:
- Around line 195-198: webContext() is re-reading the current frontmost app
instead of using the app already captured by collectContext(), which can mix
metadata from different focus states. Update the AppContextSource flow so
collectContext() passes the existing NSRunningApplication and/or AXUIElement
into webContext(), and have webContext() derive host, focus, and address-bar
state from that provided context rather than calling
NSWorkspace.shared.frontmostApplication again.
- Around line 105-116: In AppContextSource’s context-building logic,
addressBarFocused should be checked before any webHost-based phrasing so the
returned message uses the address-bar wording instead of the current page host.
Update the conditional flow around the host/title handling to prioritize the
address bar case first, then fall back to the existing hostMatchesApp and
webHost branches, keeping the existing app/title formatting intact.
---
Outside diff comments:
In `@Sources/AppContextService.swift`:
- Around line 159-169: The off-mode prompt issue is already resolved, but
`AppState` still persists `selectedText`, `windowTitle`, and `appName` into
`PipelineHistoryItem` even when the effective source is `.off`. Update the
history-building path in `AppState` so those fields are omitted, cleared, or
conditionally suppressed when off mode is active, and keep the behavior aligned
with how `AppContextService` already returns an empty `contextSummary` for
`.off`.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 27a20bdd-aa3a-4006-aa7d-7dcf694f6e74
📒 Files selected for processing (4)
Sources/AppContextScopeStore.swiftSources/AppContextService.swiftSources/AppContextSource.swiftSources/SettingsView.swift
pianoandglass
pushed a commit
to pianoandglass/freeflow
that referenced
this pull request
Jun 25, 2026
…d + address-bar precedence - webContext(expectedBundleID:) bails when the frontmost app changed since collectContext captured it, so the App-summary line never mixes one app's page host with another app's name/title (CodeRabbit Major zachlatta#244). Kept collectContext byte-identical: the leaf passes the already-captured bundleIdentifier instead of threading the AXUIElement (which would edit collectContext and break cross-feature coexistence). - metadataSummary checks addressBarFocused before the page host, so typing in the omnibox reads 'in the <app> address bar' even when a page is loaded (CodeRabbit Minor zachlatta#244). Files: Sources/AppContextSource.swift, Sources/AppContextService.swift.
…reenshot LLM) + per-app "send screenshot" allow-list Modified files: - Sources/AppContextSource.swift (new) - Sources/AppContextScopeStore.swift (new) - Sources/AppContextService.swift - Sources/SettingsView.swift
…d + address-bar precedence - webContext(expectedBundleID:) bails when the frontmost app changed since collectContext captured it, so the App-summary line never mixes one app's page host with another app's name/title (CodeRabbit Major zachlatta#244). Kept collectContext byte-identical: the leaf passes the already-captured bundleIdentifier instead of threading the AXUIElement (which would edit collectContext and break cross-feature coexistence). - metadataSummary checks addressBarFocused before the page host, so typing in the omnibox reads 'in the <app> address bar' even when a page is loaded (CodeRabbit Minor zachlatta#244). Files: Sources/AppContextSource.swift, Sources/AppContextService.swift.
The upstream test target hardcodes its source list; this branch's AppContextService.swift now references AppContextSource, so the test runner failed to compile. Declared as a separate prerequisite line (prerequisites accumulate in make) and switched the recipe to $^ so the compile list always mirrors the declared prerequisites. Verified: make test builds and passes. Modified files: - Makefile
pianoandglass
force-pushed
the
app-context-multiselector-pr
branch
from
July 19, 2026 08:16
4bcb861 to
af7b93d
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This reworks the existing App Context settings (formerly "Context Prompt") in the prompts tab into a single control over current activity and what it sends to the AI. Alongside the existing screenshot path, it adds on-device context detection: FreeFlow can describe the user app, window, and current web page locally and send a one-line text description — no image, no vision model. A single multi-toggle selects one of three context sources.
Screenshot sending can now be disabled or limited to a chosen list of apps.
With the default settings, behavior is identical to before this PR.
Context source
Control with three options:
Per-app "send screenshot" allow-list
Shown in Screenshot mode. Two choices:
/Applications) and listed as removable chips with their icon and name.This lets you keep the screenshot only where it helps — for example an email client, where the model benefits from seeing recipients and the subject — while everything else stays fast and text-only.
What "App summary" sends
The one-line summary is built from window/app metadata read through the Accessibility API:
User is dictating on github.com (Pull requests) in Safari.User is dictating in ChatGPT).User is dictating in the <app> address bar.User is dictating in <app>.Vision Prompt, Test, and Resolution
The controls that already existed are kept and grouped under Screenshot mode, since they only apply when the vision model runs:
When the source is Off or App summary, this whole block is disabled and dimmed, and the app icons in the allow-list chips are greyed so they read as inactive.
Defaults and scope
UserDefaults;AppState, the post-processing/cleanup logic, and the run-history database are not touched.Changes by file
Sources/AppContextSource.swift(new, ~300 lines) — the settings model and the Accessibility reads.UserDefaultskeys (appContextMode,appContextScreenshotScope,appContextScreenshotApps) and theMode/Scope/EffectiveSourceenums, with reads that default to the upstream behavior.effectiveSource(forBundleID:)— resolves the source for one dictation, including the per-app rule (apps outside the allow-list fall back to App summary).metadataSummary(...)— a pure function that composes the App-summary line (host + title, redundancy collapse, address-bar wording, title cleanup), plus the small string helpers it uses.webContext()(one read; opts the app into Accessibility and skips when permission is missing), a breadth-firstAXWebAreasearch, address-bar detection, and the low-level attribute helpers.Sources/AppContextScopeStore.swift(new, ~195 lines) — the allow-list store and the scope UI.AppContextAllowlist— an observable store that loads/persists the bundle-id list toUserDefaultsand presents the "Add app…" file picker.AppContextAppInfo— resolves and caches each app's display name and icon.ScreenshotScopeSection— the two radio rows, the app chips (icon, name, remove button), the dashed "Add app…" button, the empty state, accessibility labels, and the icon-greying when the block is disabled.Sources/AppContextService.swift(modified, +35 lines) — the existing context service.forceScreenshotForTestflag used only by the Test button.collectContext(...)is unchanged (byte-identical tomain).Sources/SettingsView.swift(modified) — the Prompts tab.appContextModesetting and pointed the Test button at the per-instance flag.Summary by CodeRabbit
Summary by CodeRabbit
New Features
Bug Fixes