Skip to content

omamobile-shell: add Ask Agent conversations to native Search - #614

Open
marselnenaj wants to merge 4 commits into
derekross:mainfrom
marselnenaj:codex/native-search-agent-chat
Open

marselnenaj wants to merge 4 commits into
derekross:mainfrom
marselnenaj:codex/native-search-agent-chat

Conversation

@marselnenaj

@marselnenaj marselnenaj commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

Scope review pending: further adapter development is paused at the maintainer’s request. The CI layout correction is tracked separately below.

Add the phone's Ask Agent flow to native Search: tap the Ask card, edit the draft and Send, read replies, Stop a request or start a new conversation. Back returns to Search; Home/close dismiss it and cancel active work. The conversation stays in memory while the local bridge is idle.

The UI follows the configured default agent. This first native adapter supports an already installed Codex CLI; other agents receive an explicit terminal handoff through the existing Omarchy route. It reuses normal login/account configuration and explicitly requests the workspace-write sandbox. No client is installed. This non-interactive bridge has no approval UI; preserving the environment does not establish an interactive approval flow. Real-client acceptance and tool-activity visibility remain unresolved review requirements. This PR adds no microphone/login facade or personal configuration.

Request IDs, acknowledged cancellation/reset and exact owned thread identities prevent stale responses from reaching later requests. Completed context survives Stop and failed follow-ups. Account-home changes start fresh context. A stopped local bridge can recover through explicit re-entry or New chat. Generated task files remain in private session workspaces; only empty folders are removed at shutdown. Non-empty folders remain under ${XDG_STATE_HOME:-~/.local/state}/omamobile/agent-chat/; users may want to review and clear retained task files themselves.

Depends on #612. Review the agent addition separately from the Search dependency: review only the addition. After #612 lands, this can be rebased onto main. The maintainer will allocate the package version at merge; the branch number is not reserved.

Interface

Actual rendered QML with genuine Omarchy modules, neutral themes and fictional app/conversation fixtures:

Ask the configured agent from Search Native conversation in a dark theme Native conversation in a light theme

Recorded Search-to-conversation flow

The recording uses 106 actual captures spanning 3.977 seconds, encoded losslessly with original intervals and no interpolated frames. Its reply events are inert fixtures rendered by the real controller and view. The 420 × 580 portrait previews show a reduced available viewport without drawing a keyboard. They do not measure phone, compositor blur, layer-shell focus or frame pacing.

Compact available viewport:

Conversation controls contained in a compact viewport

Validation

  • 24 backend subprocess fixtures pass, covering literal argv/stdin, exact resume, account changes, cancellation, reset, timeout/output limits, Unicode framing and artifact preservation.
  • 32 Qt cases pass, retaining all 19 app-search lifecycle cases and covering editable draft/explicit submission, Back, lock, stale callbacks, compact viewports, Stop acknowledgement and helper recovery.
  • The actual Quickshell controller connects to the real Python bridge and an inert CLI. Eight recorded calls verify failed-follow-up continuity, account changes, reset, restart and unsupported-provider handling. No cloud request is made by tests.
  • Genuine Omarchy 4.5 light/dark/landscape and 4.0.6 light/dark previews pass. Palette, fonts, spacing, corners and motion remain configurable. Independent source review and the commit-range policy gate pass.

The full final package suite passes via tests/packages omamobile-shell (398.85 seconds), including the actual UI/Python/inert-CLI integration. Final staging bytes and source pins match. Compact controls remain within the viewport even when platform button metrics prefer a wider size; a new regression exercises that case without weakening the original compact-layout check.

CI now installs Quickshell to run the actual UI/bridge test rather than silently omitting it; it is also a check dependency. Staging includes the executable, its namespaced module and both new QML components. Exact-package phone acceptance for real account/client, touch keyboard, gestures and lock transitions remains outstanding. Existing fullscreen-strip and default-output limits from #612 remain.

Hosted checks for d444ca7 now pass, including the full package suite: GitHub Actions run. The previous head failed its compact-layout test; the unchanged test and additional preferred-width regression pass on this head.

marselnenaj and others added 3 commits November 10, 2026 15:40
Separate explicit dock app search from the swipe-driven App Library while retaining the existing catalog and launch path. Allow the previous tap behavior through the nativeSearch setting.

Own initial input focus through the attributed Omarchy focus prime, cancel late requests when closing or when a known lock arrives, and close Search through native Back and Home without per-gesture helper processes.

Use theme palette, typography, corners and motion tokens. Include neutral light/dark previews and an animated component capture. The full package suite and actual Qt lifecycle tests pass; final dock wording was checked separately. Physical layer-shell and on-screen keyboard acceptance remain outstanding.

Co-Authored-By: Codex <codex@openai.com>
Keep the phone search layout when extracting the public app-search view:
place the query at the top and use compact icon/name/description rows rather
than enclosing each result in a separate bordered card. Show three initial
suggestions and retain the existing bounded matching catalog while typing.

Paint a shared translucent theme tint once, keep typography and corners
configurable, and expose clamped theme alpha overrides. Document optional
namespace-scoped compositor blur without changing global blur settings.

Replace placeholder-only media with actual light/dark QML captures using
original licensed neutral SVG icons and test scenery. Preserve native focus,
known-lock cancellation and Back/Home routing. Component tests pass; exact
package phone acceptance and physical frame pacing remain outstanding.

Co-Authored-By: Codex <codex@openai.com>
Restore the phone's Ask Agent card and editable in-place conversation.
Only explicit Send submits a task. Back returns to Search, Home closes it,
and closing or locking cancels active work without reviving stale input.

Use the configured provider and its existing account environment. The first
native adapter supports Codex with workspace-write; other providers retain
an explicit terminal handoff. Keep credentials with the CLI, bind requests
and completed conversation identities, acknowledge cancellation/reset and
preserve task artifacts. Never enable approval or sandbox bypass flags.

Exercise the actual QML/Python transport against inert CLIs, including
failed-turn continuity, account changes and local transport recovery. Add
Quickshell to CI/check dependencies and include neutral themed screenshots
and an actual component recording. Phone acceptance remains outstanding.

Depends on the native Search core in derekross#612; allocate the version at merge.

Co-Authored-By: Codex <codex@openai.com>
@marselnenaj
marselnenaj force-pushed the codex/native-search-agent-chat branch from 43bfdab to b05222c Compare November 10, 2026 14:50
The hosted compact-layout check found a composer extending beyond a
320-pixel viewport. A wider platform font makes the terminal handoff
button's preferred width propagate through the action row and composer.

Allow actions to shrink between their touch minimum and natural width,
and bound/elide their labels while retaining the full accessible name.
This leaves unconstrained layouts at their natural button sizes.

Reproduce the overflow with an oversized action preference in a temporary
conversation, alongside the unchanged compact viewport test. The old
implementation fails the new regression; the correction passes all 32 Qt
cases. Refresh dark/light previews and include a compact example.

Co-Authored-By: Codex <codex@openai.com>
@derekross

Copy link
Copy Markdown
Owner

Thank you, @marselnenaj. I read the bridge closely, because this is the first part of the shell that would talk to an agent on the user's behalf, and a lot of it is done the way I'd want: it only starts on an explicit Send, it runs the client the user already installed and never a wrapper that could install one, it keeps no credentials or transcript of its own, a lock or a dismissal stops the request, everything is bounded, and replies are plain text. The private per-session folder, opened without following links, is a nice touch.

I can't merge it yet, and part of that is a decision I have to make rather than something for you to fix:

  1. Scope. The project installs no proprietary agent, and this doesn't either. But it would put code for one vendor's client, and its event format, into omamobile-shell for us to maintain. I need to decide whether the shell should have a native chat for that, or keep to Omarchy's own route, which treats every agent alike. I'll answer here once I've thought it through; please don't put more work into the adapter until then.
  2. CI is red on this head. test_conversation_compact_viewports_keep_editable_draft_and_terminal_action_inside fails with 'agentDraft stays inside 320,240 at 36,144 316x44' returned FALSE: the draft is 316 px wide at x 36 in a 320 px viewport. The description says the suite passes, so it may depend on the fonts of the machine; either way it has to pass in the CI container.
  3. It hasn't met the real client. All the tests use a stand-in that accepts any arguments. Has codex exec --sandbox workspace-write resume <id> --json --skip-git-repo-check - run against a real Codex CLI, and which version? I'd like one recorded run: a first message, a follow-up, Stop, New chat, a signed-out client, and a lock during a request. Also, the README's protocol link (learn.chatgpt.com/docs/non-interactive-mode): is that the right address? I couldn't confirm it.
  4. What the agent does is hidden. In the terminal the user watches the agent's commands. Here tool events are dropped and only the final text shows, and as far as I understand that client's exec mode nothing asks before a command runs, so one typed sentence can have the agent read anything the user can read on the phone and send it on. The voice bar already says " runs with full permissions on this phone" before it hands over. This view needs at least that line, and I'd want either a visible trace of what the agent did (even "ran 3 commands"), or the read-only sandbox for this surface. Could you also check the README's "approval policy is retained" against the client's documentation for exec?
  5. Order. This sits on omamobile-shell: open native app search from the Home dock #612, so everything I asked there applies (the phone run, the icon font, the theme keys, and docs/review-evidence/ out of the commit; this PR adds a second folder of images). omamobile-shell: open native app search from the Home dock #612 lands as a squash, so afterwards git rebase --onto origin/main <#612's last head> <your branch> keeps just your commit. I'll set the version at merge; 0.2.115 is already spoken for.

Smaller things: the workflow change (Quickshell in the CI container, and as a check dependency) deserves a line of its own in the description, and please confirm the aarch64 package build finds it. Session folders that the agent wrote into are never removed; that's fine for now but should be said in the README as a thing the user may want to clear.

@marselnenaj

Copy link
Copy Markdown
Contributor Author

Thank you for the detailed review. I will pause further adapter work until you decide the scope, and leave the default/opt-in decision and version allocation to you.

The compact-layout failure was real. I traced the overflow to the action control’s preferred width propagating into the composer, added shrink bounds and label elision, and kept the original failing assertion unchanged. Commit d444ca7 also adds an oversized-control regression. The 32 Qt cases and complete package suite pass locally; I am checking the hosted run separately before calling CI fixed.

I have not exercised the adapter against a real Codex client on the phone. The fixtures do not establish real-client acceptance, and I will not present them as that. Your point about visibility and non-interactive approvals is valid: retaining the environment is not evidence of an interactive approval flow, so the README’s approval claim needs correction before acceptance. I will wait for your scope decision before changing the adapter or choosing between read-only execution and visible tool activity.

The documentation URL currently resolves for me: https://developers.openai.com/codex/noninteractive redirects to https://learn.chatgpt.com/docs/non-interactive-mode. That page documents JSONL events and session resume; it does not replace the requested exact-version client run.

I am handling the Search-specific theme/icon and dependency questions in #612. I will also remove committed review media in favour of PR attachments, and make the Quickshell CI/check dependency and retained non-empty session folders explicit in the description. No new adapter features are being added while the scope decision is pending.

@derekross

Copy link
Copy Markdown
Owner

Thank you, @marselnenaj, for pausing as I asked, and for the plain answers. The compact-layout fix is right: the original assertion is untouched, the new case fails on the old code, and CI is green on this head. I also appreciate that you won't present the fixtures as a real-client run, and that you agree the README's approval sentence has to change.

I've made the scope decision, and it's no to a vendor's adapter inside omamobile-shell. This is about what the shell should own, not about the quality of what you wrote:

  • The shell would carry code written to one client's command line and event format, which can change in any release of that client, with nothing on our side that would notice. Today's route through Omarchy's own agent prompt treats every agent alike, and I want the shell to stay that way: the owner picks the agent, and we don't make one of them first-class.
  • In the terminal the user sees every command the agent runs. Here, as you confirmed, nothing asks and nothing shows, with write access to a workspace and read access to everything the user can read. I'm not comfortable shipping that as a built-in surface.

What I would look at, if you want to take this further, is a different shape, and I'd ask for a short design issue before any code: the conversation view and the bridge's JSON-lines protocol in the shell with no vendor in them; adapters as separate drop-in files that the shell doesn't ship (a package of your own, or the user's); read-only as the default for this surface, the "runs with full permissions on this phone" line the voice bar already shows, and a visible trace of what the agent did; and a recorded run with the real client, with its version, before any adapter is accepted anywhere. Much of your bridge would carry over: explicit Send only, bounded output, a lock cancelling the request, plain-text replies.

So please don't rebase this one. #612 is the PR to finish; I've left notes there on building its Back handling on today's main. I'll close this once #612 is settled, and the branch stays yours to draw from.

@marselnenaj

Copy link
Copy Markdown
Contributor Author

Thanks for the scope decision. The part I wanted to contribute is the mobile flow: find the action from Search, compose and review a prompt comfortably, then send it explicitly. Could a smaller follow-up keep that UI and hand the prompt to the existing Omarchy agent route? The user's configured agent and the normal picker would work the same way as in Omarchy, and the terminal would show the agent's activity. This would use the existing LauncherModel.promptArgv and picker path, with no Codex adapter or model execution owned by the Search surface.

If the in-shell conversation itself is worth pursuing separately, I can write the short design issue you requested first. It would cover a vendor-neutral bridge, separately installed adapters, read-only by default, clear permissions and visible activity, explicit Send and lock cancellation. I would qualify each adapter against a named real client version before proposing it. I'll leave #614 as it is and finish #612 first.

@derekross

Copy link
Copy Markdown
Owner

Thank you, Marsel. Yes to the smaller follow-up, in its smallest form.

The App Library already does the hand-over you describe: when the text reads like a sentence or matches no app, it offers "Ask " and askAgent() runs LauncherModel.promptArgv, or the picker when no agent is chosen. With nativeSearch on, Search loses that, so an "Ask " row in Search under the same rule, calling the same function, would be welcome: every agent treated alike, the agent's work visible in the terminal, nothing for the shell to run or keep. I'd rather the Search field itself be the draft than add a separate compose sheet; if you think the review step needs its own surface, tell me what the field can't do and I'll think about it.

Please open it as its own pull request once #612 is merged and has run on a phone, on top of main, with nothing from this branch's bridge or conversation view in it.

For the in-shell conversation, the design issue first, as you offer. The list in your comment is the right one.

I'll close this pull request when #612 is settled, as I said. The branch stays yours.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants