Arduino IDE parity: event-driven board detection, serial monitor fixe…

docs/parity-implementation-notes.md

@@ -0,0 +1,93 @@
+# Arduino IDE Parity — Implementation Notes (July 2026)
+
+Companion to `arduino-ide-comparison.md`. This documents what was implemented
+from that report's §7 recommendations, across **tinyStudio** (branch
+`development`) and **tinyService** (branch `claude-development`, shared
+`1.2.0` / service `1.1.0`). Scope notes honored: the Visual view stands in
+for the Serial Plotter, the Library Manager UX was left as-is, and everything
+works in both the desktop and browser builds (LSP is desktop-only — the
+language server needs the sketch on disk).
+
+## What changed
+
+**1. Event-driven board detection (report §3, Bugs 4–5).** tinyService now
+runs one long-lived `arduino-cli board list --watch --format jsonmini`
+process (`board-watch.service.ts`) and broadcasts a `board-events` push to
+every client on plug/unplug (~150 ms debounce). New clients get a snapshot on
+connect; `list-boards` is served instantly from the watcher's memory. The
+frontend subscribes (`useArduino.ts`) instead of polling — the 8 s
+poll-until-first-board and the 5 s refresh cache are gone. Unplugging the
+selected board now clears the selection with a toast; a replacement port is
+adopted automatically.
+
+**2. Serial monitor lifecycle (Bugs 1–3, 13).** `serial.handler.ts` was
+rebuilt around per-session objects: `opened` is only reported once confirmed
+(banner, first sketch output, or survive-the-confirm-timer for arduino-cli
+≥1.x's quiet piped mode); port-open failures are surfaced as `error` messages
+carrying arduino-cli's own words (no longer filtered away); stale
+child-process handlers can no longer delete or "close" their replacement
+session (the Windows taskkill race); output is forwarded verbatim (partial
+lines flushed after 120 ms) instead of trimmed/filtered.
+
+**3. Failed builds stay visible + inline errors (Bug 6, rec 3).** The
+monitor panel only snaps back to Serial on success, and never overrides a tab
+the user picked mid-build. Compiler output is parsed into `file:line:col`
+diagnostics (`lib/compileErrors.ts`) and rendered as Monaco markers in the
+matching file (`arduino-compile` owner).
+
+**4. Request ids (Bug 7).** The shared protocol carries an optional `id` on
+every request; the service echoes it on all replies; `waitForResponse`
+matches on it. Concurrent same-action requests no longer cross-talk. Fully
+backward compatible (id-less messages still work).
+
+**5. Board settings gear (rec 5, Bug 8).** New `BoardOptionsMenu` next to
+the port pill: FQBN config options (PSRAM, partition scheme, CPU freq, …) via
+the new `board-details` action, encoded into the FQBN like the Arduino IDE;
+plus "Change board…" — a searchable picker over installed board definitions
+for wrong VID/PID guesses (guessed boards are labeled). The
+every-tinyCore-FQBN-collapses-to-one-variant behavior was removed; VID
+`303A` still defaults to tinyCore (tradeoff kept) but is flagged as a guess
+and overridable.
+
+**6. Monitor UX parity (rec 6).** Full baud list (300 → 2 000 000, incl.
+74880 for ESP boot messages), line-ending selector (None/NL/CR/Both — sends
+are raw now, the backend appends nothing), timestamps toggle, and per-port
+persistence of baud + line ending (localStorage).
+
+**7. Language server (rec 7).** tinyService bridges
+`arduino-language-server` + clangd over WebSocket at `/lsp?fqbn=…`
+(`lsp.service.ts`; stdio framing handled server-side). The renderer has a
+dependency-free LSP client (`lib/lsp/monacoLsp.ts`) wiring completion, hover,
+signature help, and live diagnostics into Monaco. Desktop-only; degrades
+silently when binaries are missing. **Fetch binaries with:**
+`npm run fetch:language-server -- current` (they land in
+`vendor/language-server/` and are picked up automatically; packaged builds
+bundle them via electron-builder.yml).
+
+**8. Quality of life (rec 8, Bugs 9, 10, 12).** Real upload progress parsed
+from esptool/avrdude output (the fake 10%-per-200ms timer is gone); esptool
+success markers recognized in the timeout fallback; tinyService binds the
+first free port from 3000 (the renderer asks the main process for the real
+URL — web builds keep the `tinyservice.url` localStorage override); stale
+React closures in the compile/upload timeout-recovery paths fixed.
+
+## Not done (deliberately)
+
+Programmer selection / Upload Using Programmer / Burn Bootloader (needs new
+service actions — small follow-up now that `board-details` returns
+programmers), Include Library / Add .ZIP, sketch archive/save-as, network
+(mDNS) upload, and Library Manager UX changes (kept per preference).
+
+## To ship
+
+1. Publish `@mister-industries/shared@1.2.0` and
+   `@mister-industries/tinyservice@1.1.0` from the tinyService repo, then
+   bump both deps in tinyStudio's package.json. (Until then, the freshly
+   built dists were copied into `node_modules/@mister-industries/*/dist` for
+   local testing — `npm install` will overwrite them.)
+2. `npm run fetch:language-server -- current` for LSP in dev.
+3. Test on real hardware: plug/unplug detection, upload with the monitor
+   open, monitor across baud changes, a failing sketch (inline errors), the
+   board options gear on a tinyCore, and — if binaries fetched — completion
+   and hover in the editor. The LSP client is new wiring and has not run
+   against real hardware/binaries yet; treat it as experimental.

electron-builder.yml

@@ -38,6 +38,13 @@ extraResources:
     to: arduino-cli/win32-x64/arduino-cli.exe
     filter:
       - '**/*'
+  # arduino-language-server + clangd (optional; fetched by
+  # scripts/fetch-language-server.mjs). Powers editor code intelligence via
+  # tinyService's /lsp bridge; the app degrades gracefully when absent.
+  - from: vendor/language-server
+    to: language-server
+    filter:
+      - '**/*'
 win:
   executableName: tinystudio
 nsis:

package.json

@@ -21,6 +21,7 @@
     "typecheck": "npm run typecheck:node && npm run typecheck:web",
     "start": "electron-vite preview",
     "fetch:arduino-cli": "node scripts/fetch-arduino-cli.mjs",
+    "fetch:language-server": "node scripts/fetch-language-server.mjs",
     "prebuild": "node scripts/fetch-arduino-cli.mjs",
     "predev": "node scripts/fetch-arduino-cli.mjs current",
     "dev": "electron-vite dev",

scripts/fetch-language-server.mjs

@@ -0,0 +1,161 @@
+// scripts/fetch-language-server.mjs
+//
+// Downloads the Arduino Language Server + clangd binaries for each platform
+// tinyStudio ships and drops them under vendor/language-server/<platform>/.
+// These power the editor's code intelligence (completion, hover, live
+// diagnostics) via tinyService's /lsp WebSocket bridge. Entirely optional:
+// when the binaries are missing the app runs exactly as before, minus LSP.
+//
+// Uses the same artifact hosting the Arduino IDE uses:
+//   https://downloads.arduino.cc/arduino-language-server/nightly/arduino-language-server_<SUFFIX>
+//   https://downloads.arduino.cc/tools/clangd_<VERSION>_<SUFFIX>.tar.bz2
+//
+// Idempotent: skips any platform whose binaries already exist.
+//
+//   node scripts/fetch-language-server.mjs               # all platforms
+//   node scripts/fetch-language-server.mjs current       # just this machine
+
+import { execFileSync } from 'node:child_process'
+import {
+  chmodSync,
+  copyFileSync,
+  existsSync,
+  mkdirSync,
+  mkdtempSync,
+  readdirSync,
+  rmSync,
+  statSync,
+  writeFileSync
+} from 'node:fs'
+import { tmpdir } from 'node:os'
+import { dirname, join } from 'node:path'
+import { fileURLToPath } from 'node:url'
+
+// clangd version published on downloads.arduino.cc/tools (same one the
+// Arduino IDE 2.x bundles). Bump deliberately.
+const CLANGD_VERSION = '14.0.0'
+const LS_BASE = 'https://downloads.arduino.cc/arduino-language-server'
+const TOOLS_BASE = 'https://downloads.arduino.cc/tools'
+
+const __dirname = dirname(fileURLToPath(import.meta.url))
+const repoRoot = join(__dirname, '..')
+const vendorRoot = join(repoRoot, 'vendor', 'language-server')
+
+function tarExe() {
+  if (process.platform === 'win32') {
+    const sys = join(process.env.SystemRoot || 'C:\\Windows', 'System32', 'tar.exe')
+    if (existsSync(sys)) return sys
+  }
+  return 'tar'
+}
+
+// platform dir -> download suffixes + binary names.
+const TARGETS = {
+  'windows-x64': { suffix: 'Windows_64bit', archive: 'zip', exe: '.exe' },
+  'macos-x64': { suffix: 'macOS_64bit', archive: 'tar.gz', exe: '' },
+  'macos-arm64': { suffix: 'macOS_ARM64', archive: 'tar.gz', exe: '' },
+  'linux-x64': { suffix: 'Linux_64bit', archive: 'tar.gz', exe: '' },
+  'linux-arm64': { suffix: 'Linux_ARM64', archive: 'tar.gz', exe: '' }
+}
+
+async function download(url, dest) {
+  const res = await fetch(url)
+  if (!res.ok) throw new Error(`Download failed (${res.status}) for ${url}`)
+  writeFileSync(dest, Buffer.from(await res.arrayBuffer()))
+}
+
+/** Recursively find a file by name in a directory tree. */
+function findFile(root, name) {
+  for (const entry of readdirSync(root)) {
+    const p = join(root, entry)
+    if (statSync(p).isDirectory()) {
+      const found = findFile(p, name)
+      if (found) return found
+    } else if (entry === name) {
+      return p
+    }
+  }
+  return null
+}
+
+async function fetchOne(platform) {
+  const target = TARGETS[platform]
+  if (!target) {
+    throw new Error(`Unknown platform "${platform}". Valid: ${Object.keys(TARGETS).join(', ')}`)
+  }
+
+  const outDir = join(vendorRoot, platform)
+  const lsBin = `arduino-language-server${target.exe}`
+  const clangdBin = `clangd${target.exe}`
+  const outLs = join(outDir, lsBin)
+  const outClangd = join(outDir, clangdBin)
+  if (existsSync(outLs) && existsSync(outClangd)) {
+    console.log(`✓ ${platform}: already present`)
+    return
+  }
+  mkdirSync(outDir, { recursive: true })
+
+  const tmp = mkdtempSync(join(tmpdir(), 'arduino-ls-'))
+  try {
+    // ── arduino-language-server (nightly channel, like fresh IDE builds) ──
+    if (!existsSync(outLs)) {
+      const asset = `arduino-language-server_${target.suffix}.${target.archive}`
+      console.log(`↓ ${platform}: downloading ${asset} ...`)
+      await download(`${LS_BASE}/nightly/${asset}`, join(tmp, asset))
+      execFileSync(tarExe(), ['-xf', asset], { cwd: tmp, stdio: 'inherit' })
+      const extracted = findFile(tmp, lsBin)
+      if (!extracted) throw new Error(`${lsBin} not found inside ${asset}`)
+      copyFileSync(extracted, outLs)
+      if (!platform.startsWith('windows')) chmodSafe(outLs)
+      console.log(`✓ ${platform}: ${outLs}`)
+    }
+
+    // ── clangd (bzip2 tarball; bsdtar reads .tar.bz2 natively) ──
+    if (!existsSync(outClangd)) {
+      const asset = `clangd_${CLANGD_VERSION}_${target.suffix}.tar.bz2`
+      console.log(`↓ ${platform}: downloading ${asset} ...`)
+      await download(`${TOOLS_BASE}/${asset}`, join(tmp, asset))
+      execFileSync(tarExe(), ['-xf', asset], { cwd: tmp, stdio: 'inherit' })
+      const extracted = findFile(tmp, clangdBin)
+      if (!extracted) throw new Error(`${clangdBin} not found inside ${asset}`)
+      copyFileSync(extracted, outClangd)
+      if (!platform.startsWith('windows')) chmodSafe(outClangd)
+      console.log(`✓ ${platform}: ${outClangd}`)
+    }
+  } finally {
+    rmSync(tmp, { recursive: true, force: true })
+  }
+}
+
+function chmodSafe(p) {
+  try {
+    chmodSync(p, 0o755)
+  } catch {
+    /* best effort on non-posix hosts */
+  }
+}
+
+function hostPlatform() {
+  const arm = process.arch === 'arm64'
+  if (process.platform === 'win32') return 'windows-x64'
+  if (process.platform === 'darwin') return arm ? 'macos-arm64' : 'macos-x64'
+  if (process.platform === 'linux') return arm ? 'linux-arm64' : 'linux-x64'
+  throw new Error(`Unsupported host platform: ${process.platform}/${process.arch}`)
+}
+
+async function main() {
+  const requested = process.argv
+    .slice(2)
+    .map((a) => (a === 'current' || a === '--current' ? hostPlatform() : a))
+  const platforms = requested.length ? requested : Object.keys(TARGETS)
+  console.log(`Fetching arduino-language-server (nightly) + clangd ${CLANGD_VERSION} for: ${platforms.join(', ')}`)
+  for (const p of platforms) {
+    await fetchOne(p)
+  }
+  console.log('Done.')
+}
+
+main().catch((err) => {
+  console.error('\nfetch-language-server failed:', err.message)
+  process.exit(1)
+})