Repository navigation
core, desktop, android: cross-platform Linux preparation & USB session reliability - #23
Conversation
thtbee
left a comment
There was a problem hiding this comment.
Hi @jochenkirstaetter, thank you for splitting this out so quickly. It's much easier to review on its own, and several points from the last round are sorted: the key file is now plain with 0600 and named for what it is, Windows keeps its own title-bar glyphs, .sh launch tiles are refused, and the clipboard garbage is fixed where it starts in PhoneClipboard.kt.
I went through every file and checked each change against how the code behaves today. Here's what needs to change before this can go in.
Must change before merge
-
The PC's folders get shared without the PC's sharing setting (
desktop/app/src/pc_storage.rs).ensure_enablednow also allows access when any paired phone has the per-devicestoragetoggle on. On a PC that toggle means "browse that phone here" (the phone's storage in File Explorer), not "let it browse this PC". So someone who only mounts their phone's storage would open their PC's folders to every paired phone, with Your PC in your pocket off. The PC's folders need to stay behindshare_pc_storagealone; that split was made on purpose in ffad687. -
Turning on PC sharing switches on phone storage for every device, at every start (
desktop/app/src/settings.rs,set_share_pc_storage, and the new call incore_host.rs). It now sets the per-devicestoragetoggle on all paired devices, and runs on each launch throughapply_runtimeandcore_host::run. That turns on a separate, sensitive permission as a side effect, and turns it back on after someone switched it off for a device. Please remove it; the two settings are independent. -
The control stream's write timeout can drop healthy sessions (
core/nectarlink-core/src/session.rs,writer). Each write now getsREQUEST_TIMEOUT(10 s), and the stream closes if it takes longer. Control messages can be up to 1 MiB: an MMS part in a reply is up to 900 KB, and photo and SMS lists are filled to the frame limit. Over Connect over mobile data at around 1 Mbit/s one of those takes 8 to 16 s, so the session would be torn down mid-reply, over and over. A dead connection is already caught by QUIC's idle timeout and the heartbeat. If the aim is to catch a stalled peer, please base it on progress (nothing written for a while) rather than a fixed time per frame, with a test for a large frame on a slow link. -
The duplicate-session rule needs to come out the same on both devices (
core/nectarlink-core/src/node.rs,register_session). In the last review I listed preferring USB here as good. Looking at it more closely, both devices only agree on which connection is USB once both have registered the tunnel (usb_bridge_portson the phone,usb_peerson the PC). In the moment where one side has and the other hasn't, each can keep a different connection and close the other's, which is the connection churn fixed in 0.1.4. The new test brings USB up before any LAN session exists, so it doesn't cover that moment. Please add a two-device test that brings the tunnel up while a LAN session is live, or keep the existing rule inregister_sessionand leave the USB preference tomaintain, whichdial_with_usb_interruptalready handles. -
A maximized window doesn't come to the front any more (
desktop/app/qml/MainWindow.qml,restoreAndBringToFront). For a maximized window it callsshowMaximized(),raise()andrequestActivate()instead ofbringToFront(). On Windows,NativeWindow::bringToFrontis what gets past the foreground lock, so a maximized window now stays behind other apps when opened from the tray or after an update. Please callbringToFront()on that path too. -
Timeline icons go back to matching English text (
desktop/app/qml/pages/TimelinePage.qml,iconForRow). It looks for "Copied image", "Copied file " and "Link · " in the title. The titles are translated now, so this quietly falls back in Simplified Chinese; 054bafb moved the timeline off English text for this reason. Please base the icon on the entry's kind and file name, as the file-extension part already does. -
A new test fails on CI (
desktop/app/src/settings.rs,desktop_i18n_catalog_is_complete_preserves_placeholders_and_compiles_to_qm). It readsnectarlink_zh_CN.qmfromOUT_DIR, which isn't there in the CI build, so "Windows app (x64)" fails. The catalog is already checked in CI bycargo xtask strings, and the Android strings by Android lint, so I'd remove this test andandroid_i18n_catalog_matches_all_translatable_keys_and_placeholders(the desktop crate shouldn't read Android resources) rather than fix them. -
The description doesn't match the code. It mentions replacing an
.expectinupgrade_to, a oneshot inspawn_sessionand a 15 s timeout onSession::rpc. None of those exist onmainor in this branch. Please describe what the PR actually changes; it's the record of why each change was made.
Worth changing
- Scope. A few parts aren't Linux preparation and change Windows or Android: remembering the window's position and size, the Android clipboard fix, thumbnails for any image file in the Timeline, and
ThemeAndLocalizationTest. Some are good (the clipboard one fixes a real bug), but in their own PRs, each with its own description and tests, they're easier to review and to trace if something regresses. The window geometry also still needs the two-monitor check from last time. - Older Qt in shared QML.
if ("topLeftRadius" in this), theQt.ColorSchemeguards,variableAxesset from handlers,preferredRendererType = 2and theTheme.errorContainer || ...fallbacks all make the shared QML run on Qt older than the 6.12 we build with. That's a decision for the Linux port as a whole: ship the pinned Qt (an AppImage or Flatpak), or support distribution Qt. Please decide and document it first. If it's distribution Qt, I'd rather keep the version checks in one place than spread through the components. file_url(desktop/app/src/icons.rs) now trims every leading/, which turns a Windows network path (\\server\share\a.png) intofile:///server/share/a.png. Please keep the Windows behavior and only normalize Unix paths.CloseOnDropcloses a cancelled dial withCLOSE_DUPLICATE;CLOSE_NORMAL, or a code of its own, would describe it better.EVENT_CAPACITYfrom 512 to 2048: fine together with the resync, but a comment on why would help.is_valid_app_pathaccepts any binary directly in/usr/binand friends, shells included. Fine as a guard since tiles are set up on the PC, butsh,bashand the like could be refused for the same reason as.sh.
Good as they are
session.rsrequest: the timeout now covers a full outbox too, so a call can't hang.node.rs: cancelled dials closed, loopback addresses kept out of the stored ones,publish_onlinechecking the session is alive,connected()and USB tunnels waking the supervisor instead of dialing in parallel.- FFI: sending state again after the event channel lags.
try_send_realtimefor PC audio frames.handoff.rs: the multi-byte text fixes, with a test.features.rs: Linux as a desktop OS, with a test that the matrix matches Windows.identity.rs:0600on the key file, with a test.
How to move this forward
Once the eight points above are sorted, I'll do a quick second pass. Happy to help with 3 and 4 in particular, since they touch code that's had a lot of tuning. #12 can then go on top as planned.
|
Hi @thtbee Again, thanks for the great, detailed feedback. That's highly appreciated. Apologies for any inconvenience caused. Also, I'm going to isolate those non-Linux preparations into their own PR stacked on top of this branch, or probably will rebase them to the future Cheers |
f24da46 to
2ddd348
Compare
2ddd348 to
c0e456a
Compare
|
Hi @thtbee Branch has been rebased onto latest Hope this helps. Cheers |
thtbee
left a comment
There was a problem hiding this comment.
Hi @jochenkirstaetter, thanks for turning this around so quickly, and no apology needed. Most of what I raised came from history that isn't written down anywhere, so it was hard to know from the outside.
This version is much easier to follow. The PC storage changes, the write timeout, the USB preference in register_session, the window and Timeline changes and the i18n tests are all out, file_url keeps network paths working, and cancelled dials close with CLOSE_NORMAL. Splitting the window, clipboard and thumbnail work into #26, #27 and #28 is a good call too.
I went through the new diff file by file. A few things are left.
Must change before merge
-
connected()waking the supervisor gets around the reconnect backoff (core/nectarlink-core/src/node.rs).Node::connectedis called from about 50 places, many on events: each phone notification forwarded to a PC, each clipboard copy, battery and media updates. When a paired device is off for a few hours, every one of those calls now wakesmaintainout of its backoff sleep and starts a dial, so the backoff that grows toBACKOFF_MAXstops applying while anything is happening. On a phone with a PC that's switched off, that's a dial on almost every notification, which costs battery. Discovery and the USB tunnel already wake the supervisor when the device actually shows up. If there's a case where neither does, could you tell me which one? I'd rather wake from that event than from every call. -
Waking during
DEFER_DIALbrings back the dial race it exists to avoid (maintain). When two devices come up together, the higher ID waits 750 ms so the lower one's dial usually wins. LAN discovery wakes the supervisor at exactly that moment, so withwake.notified()in thatselect!both now dial at once, and one connection gets dropped. Skipping the wait when a USB tunnel is ready makes sense; please leave the wake out of the wait itself. -
The description should match the code, since it becomes the record of the change. Some items in it don't appear in the diff:
endpoint.network_change()on a USB tunnel to a paired peer is already onmain.- Serializing
RemoteStorageEntry::modifiedas au64isn't in this branch.
And some changes aren't mentioned:
- the FFI resync after a lagged event stream;
register_sessionskipping connections that are already closed;publish_onlinecheckingis_alive();on_usb_tunnel_acceptedstarting and waking the supervisor instead of dialing itself;- the UTF-8 boundary fixes in
handoff.rs.
Worth changing
- Code that only builds with #12.
settings.rscallscrate::pc_notifications::start_listener()on Linux, which doesn't exist in this branch, andtry_data_dirgets#[allow(dead_code)]although Windows uses it. Both are only needed by the Linux backend, so I'd move them to #12. If one has to stay here,#[cfg_attr(not(windows), allow(dead_code))]keeps the warning on Windows. try_send_realtimereturnsOk(())when it drops a frame. That's right for audio, but the doc comment should say so, so nobody uses it later for something that must arrive.- The new test checks that a ring works during a burst of 128 frames. With the writer draining as it goes, 128 frames stay at or under the drop threshold, so the test passes with or without
try_send_realtime. A burst larger thanOUTBOX_CAPACITYwould show that frames get dropped and the ring still gets through.
Good as they are
CloseOnDropindial, anddial_with_usb_interruptleaving a slow LAN dial when a tunnel comes up.- Keeping loopback addresses out of
last_addrs, and keeping the earlier LAN addresses when a session has only USB ones. - Timing out
Session::requestacross both enqueue and reply. - The Linux role,
.desktopand AppImage tiles with shells refused, the0600key file, andfile_url. - Vector caption glyphs off Windows, with Snap Layouts input kept on Windows.
Once the first two are sorted, this looks ready to me.
|
Hi @thtbee I'm learning, I'm learning... Reviewing your feedback and getting those changes done ASAP. This is really a solid base for all the other stacked, pending branches.I really like the dynamics in this project and how "parallel" development across multiple OS is coming together nicely. Be right back with mitigated changes. Cheers |
c0e456a to
322b261
Compare
thtbee
left a comment
There was a problem hiding this comment.
Thanks, Jochen, this addresses everything: connected() no longer wakes the supervisor, DEFER_DIAL keeps its wait, the Linux-only bits moved to #12, and the test now shows frames being dropped while the ring gets through. No need to apologize for the rounds either; this is how it should go, and the result is a solid base for the other platforms.
One small thing in the description: it says launch tiles also accept binaries under /usr/games/ and /opt/ and refuse .bash, .zsh, .fish and .py files, but is_valid_app_path doesn't do either (only .AppImage files can be in /opt). Please update that line so the record matches the code.
322b261 to
82c2500
Compare
|
Hi @thtbee Minor change has been applied to I'm wondering whether it might not be better to evaluate $PATH environment var instead of providing a list of discrete folder paths. Checking things and reverting back. Cheers |
8db66c0 to
c6848c8
Compare
|
Hi @thtbee OK, that looks way better now. Crossing fingers this is ready now. Cheers |
c6848c8 to
81b4797
Compare
thtbee
left a comment
There was a problem hiding this comment.
Thanks for trying the PATH idea. I think it causes a problem though, so I've moved this back to changes requested.
Saved tiles are checked again every time the Deck loads: DeckConfig::load runs sanitized(), which drops any tile whose action fails is_valid(), and the next save writes the file without it. With the check now depending on PATH, whether a tile survives depends on how NectarLink was started. Autostart, a systemd user unit or a Flatpak usually get a shorter PATH than a terminal, often without ~/.local/bin. So a tile for ~/.local/bin/obsidian could disappear for good after a reboot, with no message.
Tiles are only ever set up on the PC itself, and the phone only gets the wire layout without paths, so this check is a sanity guard rather than a security boundary. I'd keep it to the shape of the path and leave the rest to launch time:
- absolute, with no
..segments and no control characters; - not a shell (
sh,bashand the others) or a.shscript; .desktopand.AppImageas before.
If the app is gone when the tile is pressed, launch_app fails and the PC can say so, which is easier to understand than a tile that vanished.
One smaller thing: the Windows asserts in the test now only run on Windows, but .exe and .lnk don't depend on the OS. With the check above, all of them can stay unconditional again.
|
Hi @thtbee Checking the CI issue(s). No worries, gonna sort it out. Cheers |
81b4797 to
cfead7c
Compare
|
Hi @thtbee I'm not too sure about your findings and conclusion about the As for the asserts, I introduced them specifically because those Unix paths, e.g. /usr/bin/firefox, and others, do not exist under Windows and would cause havoc, or? No issues to revert things, but I'm under the impression that it might go the wrong way. What's your POV? Cheers |
|
Fair questions, and I'm not sure I explained the problem well. It's less about Saved tiles are checked again every time the Deck loads:
On the asserts: So my suggestion is still to check the shape (absolute, no Happy to be convinced otherwise if you've seen a case where shape alone lets through something it shouldn't. |
|
Hi @thtbee Got it, and already have the "comprise solution" coming up. I was also digging deeper into the The latest push shall cover both ends nicely now. I also added the Let me know if that's suitable now. Cheers |
Core (`core/nectarlink-core`): - `src/features.rs`: Add `DeviceFacts::for_os(os)` and map `"linux"` with `DeviceKind::Unknown` to `Role::Desktop` alongside `"windows"` and `"macos"`. - `src/deck.rs`: Extend `is_valid_app_path` to accept `.desktop` and `.appimage` launchers as well as direct binary names under `/usr/bin/`, `/usr/local/bin/`, `/snap/bin/`, and `/var/lib/flatpak/exports/bin/`, while rejecting `.sh` scripts, shell binaries (`sh`, `bash`, `dash`, `zsh`, `fish`, `csh`, `tcsh`, `ksh`), and `..` traversal. - `src/identity.rs`: Set `0o600` permissions on `identity.key.tmp` before atomic rename on `#[cfg(unix)]`. - `src/handoff.rs`: Use char-boundary-safe `str::get(..)` slicing in `find_url_or_geo_token` and `extract_http_map_query` so multi-byte UTF-8 input at byte index 4 or after a URL marker cannot panic. - `src/node.rs`: - Increase `EVENT_CAPACITY` from `512` to `2048` so initial notification and capability bursts do not lag `NodeEvent` subscribers. - Check `current.is_alive()` in `Shared::publish_online` before emitting `LinkState::Online`. - Start and wake the peer supervisor in `Shared::on_usb_tunnel_accepted` (and wake the supervisor in its post-grace fallback instead of dialing outside the supervisor). - Skip already-closed connections (`conn.close_reason().is_some()`) at the start of `Shared::register_session`. - Filter loopback addresses (`127.0.0.1` / `::1` from transient USB tunnels) out of persisted `last_addrs` in `register_session`, `purge_non_lan_addrs`, `Node::start`, and `Node::set_away_mode` (preserving existing LAN `last_addrs` when a session connects over loopback USB). - Add `CloseOnDrop` in `Shared::dial` so an in-flight QUIC connection is closed if the dial future is dropped before registration. - Add `dial_with_usb_interrupt` in `maintain` and skip `DEFER_DIAL` when an outbound USB tunnel address is already present so an active USB tunnel preempts a slow LAN dial without letting LAN discovery cancel `DEFER_DIAL`. - Route `Node::send_pc_audio_pcm` stream fallback (`> 960` bytes) through `Session::try_send_realtime`. - `src/session.rs`: - Add `Session::try_send_realtime` to drop loss-tolerant real-time frames (returning `Ok(())`) when the session outbox is at or past half capacity (`OUTBOX_CAPACITY / 2`), keeping outbox capacity reserved for control RPCs and heartbeats. - Wrap `Session::request` in a single `tokio::time::timeout` covering both `self.send(...)` and `rx.await`. - `tests/two_devices.rs`: Add `usb_upgrade_reconnect_and_realtime_pcm_do_not_starve_control_rpcs` integration test covering LAN-to-USB upgrade, dropping excess real-time PCM frames past half outbox capacity while `ring` succeeds, and LAN reconnect after USB tunnel teardown. FFI (`core/nectarlink-ffi`): - `src/lib.rs`: Resynchronize paired-device link, wake, peer-info, capability, and queue state when the `NodeEvent` broadcast receiver reports `RecvError::Lagged`. Desktop (`desktop/app`): - `src/icons.rs`: Format Unix absolute paths (`/home/...`), Windows drive paths (`C:\...`), and Windows UNC paths (`\\server\share\...`) into valid `file://` URLs without quadruple slashes. - `src/local_api.rs`: Use `crate::win::secure_random` in `generate_token()` instead of calling `windows::Security::Cryptography::CryptographicBuffer` directly. - `qml/MainWindow.qml`, `qml/components/CaptionButtons.qml`, `qml/pages/DeckPage.qml`, `qml/pages/WalkthroughPage.qml`, `qml/setup/SetupWindow.qml`: Gate Windows DWM backdrop (`wantBackdrop`) and `Segoe Fluent Icons` glyphs on `Qt.platform.os === "windows"` with `QtQuick.Shapes` vector icons on non-Windows, and show platform-appropriate application launcher labels/filters on Linux. Signed-off-by: Jochen Kirstätter <7329802+jochenkirstaetter@users.noreply.github.com>
Evaluate the PATH environment variable for direct Unix binary paths in is_valid_app_path instead of a hardcoded directory list, rejecting relative, root, traversal, and world-writable temporary directories (/tmp, /var/tmp, /dev/shm) as well as shell binaries and .sh scripts. Gate OS-specific LaunchApp path assertions in unit tests by target OS. Signed-off-by: Jochen Kirstätter <7329802+jochenkirstaetter@users.noreply.github.com>
…nd support tilde paths Signed-off-by: Jochen Kirstätter <7329802+jochenkirstaetter@users.noreply.github.com>
b69c871 to
70c0219
Compare
thtbee
left a comment
There was a problem hiding this comment.
Thanks for digging into it, and for being open about it. This version does exactly what I hoped: the check is about the shape of the path, so a saved tile gives the same answer however and wherever NectarLink starts, and all the asserts run on every OS again. Supporting ~/ and expanding it in execute_action is a nice touch, and keeping /etc, /tmp, /dev, /proc and /sys out makes sense as a sanity guard.
It should also fix #12's Windows failure in builds_and_validates_editor_actions, since /usr/bin/firefox no longer depends on Windows' PATH.
One small thing: the deck.rs line in the description still describes the PATH version. Could you update it to the shape check, since the description ends up as the record of the change?
Prepares
nectarlink-coreanddesktop/appfor non-Windows desktop targets while tightening session and reconnect reliability across all platforms.core/nectarlink-core&core/nectarlink-ffifeatures.rs: Map(DeviceKind::Desktop, "linux")toRole::Desktopalongside"windows"and"macos", with a feature-matrix parity test against Windows.identity.rs: Createidentity.keywith0o600permissions on Unix (#[cfg(unix)]).deck.rs: Extendis_valid_app_pathwith a deterministic shape check that accepts.desktopand.appimagelaunchers as well as absolute (/...) and home-relative (~/..., expanded inexecute_action) Unix paths across all OSes, while rejecting.shscripts, shell binaries (sh,bash,dash,zsh,fish,csh,tcsh,ksh),..traversal, and non-executable system/temp trees (/etc,/tmp,/var/tmp,/dev,/proc,/sys).handoff.rs: Clampcursorandselection_endto valid UTF-8 character boundaries before slicing.session.rs: ApplyREQUEST_TIMEOUTacross both outbox enqueue and reply wait inSession::request. Addtry_send_realtime(withOUTBOX_REALTIME_MAX_FILL = 96) for droppable real-time PC audio frames so control RPCs (FindDeviceRing, etc.) are never starved by audio backpressure.node.rs:CloseOnDropguard indialanddial_with_usb_interruptso cancelled or raced LAN dials close cleanly withCLOSE_NORMALand interrupt promptly when a USB tunnel arrives.last_addrsand preserve prior LAN addresses when a session only reports USB loopback addresses.session.is_alive()inpublish_onlineand skip already-closed connections inregister_session.on_usb_tunnel_acceptedstart and wake the peer supervisor instead of dialing in parallel, and skipDEFER_DIALwhen a USB tunnel is ready while keeping the 750 ms wait intact for LAN discovery.EVENT_CAPACITYfrom 512 to 2048 to absorb startup/reconnect event bursts.core/nectarlink-ffi/src/lib.rs: Re-emit a full state snapshot (DevicesChanged,PendingPairingsChanged,TransfersChanged,NearbyChanged,TrustedNetworksChanged) when the broadcast event stream lags.desktop/appicons.rs: Normalize Unix absolute paths (/path/to/icon.png->file:///path/to/icon.png) infile_urlwhile keeping Windows drive and UNC (\\server\share\...) path formatting unchanged.qml/MainWindow.qml: Render caption bar minimize/maximize/restore/close icons with crisp vectorShapePathglyphs on non-Windows platforms while keeping the native Windows font glyphs and Snap Layouts hit-testing on Windows.