Skip to content

Latest commit

 

History

History
150 lines (127 loc) · 11 KB

startup.md

File metadata and controls

150 lines (127 loc) · 11 KB

Startup and stop

Modules: daemon, leasing, core. Entry: src/daemon/main.ts startDaemon → DaemonServer.start (src/daemon/server.ts).

Both modes, before the branch

  1. prepareDataDirectory creates the data directory 0700 when missing; an existing one keeps its mode, and one group or others can reach is a warn line once the logger exists. loadConfig reads config.json; its warnings are logged once the logger exists.
  2. Logger → daemon.log. At log.level: debug the ProcessRunner is wrapped.
  3. Host info read once (sw_vers on macOS); bounded, never throws.
  4. EventBus → the event history is created (createEventHistory). It touches no file yet. refuseWhenDaemonAnswers (src/daemon/main.ts) first connects to the socket: a daemon that answers makes this one fail with DaemonAlreadyRunningError, before any state.db is opened. openStateDatabase then opens the role's state.db (both roles) in this order, each failure being the daemon's start error: an old file of the role's former state refuses, then node:sqlite missing, the open and its pragmas, quick_check, and in one BEGIN IMMEDIATE transaction each owner's version (a newer one, or a migration that throws, refuses). createAuth then builds the token store and the admin secret (hash in memory), once, for both modes.
  5. Started with --gateway (role gateway, data directory <root>/gateway) → the gateway section. Otherwise go on.

Worker: build (before the claim)

  1. Registry.load — the registry and device-data tables of state.db.
  2. Instance identity — loadInstanceId, instance.json. Written once per home, never regenerated.
  3. Worker lock — only when the drivers are discovered (no drivers option, no SIMLOCK_DRIVERS_MODULE): NodeWorkerLock.take before discoverDrivers. A live holder in another folder fails the start with its details; a stale or unreadable lock is taken over; a live holder in the same folder is left alone and decided by the socket claim (ADR 0059). Root validation — discoverDrivers: each driver checks its owned root against the instance id. A refused root (OwnedRootError), or on Android an adb server it cannot start, drops that platform as a DriverRejection; a missing SDK drops it silently; any other error fails the start. iOS only on macOS. The Android driver starts its adb server and reattaches running emulators here.
  4. Host facts read in the background; host load sampler built.
  5. ComponentInstaller, createCore, the optional-module host (createModuleHost; it loads each enabled module's worker half, agent-device's supervisor among them), createLeasing, core.connect(leasing.corePorts), CleanupReaper, Nuke, GatewayUplink when gateway.url and gateway.token are set, DaemonServer.

Worker: claim and converge (DaemonServer.start)

  1. Claim the socket — DaemonEndpointHost.start. A live daemon → DaemonAlreadyRunningError; the start rejects, the entry logs Daemon already running and exits 0. A stale socket file is removed. Right after the claim the winner makes sure it holds the worker lock (claimed): it writes itself as the owner, and a live holder in another folder fails the start (the entry awaits this after daemon.start() settles, then stops the daemon).
  2. Host load sampling starts.
  3. admin.token written (0600, atomic).
  4. Converge starts. Health starting. From here hello and status.get answer; every other request parks.
  5. onSocketClaimed, nothing awaited: the event history starts (opens history.db, sets a damaged one aside, migrates the projections, writes the ring, prunes, rebuilds a projection whose check failed; a newer stored version or a failed migration stops the daemon, exit 1); agent-device starts; the uplink dials once the first host load reading is in; HTTP starts when http.enabled.

Converge (src/daemon/worker-composition.ts convergeStartup), one step after another. Once a stop reaches its settle step, the steps after the read are skipped. Before step 16 it removes the .<name>.<pid>.<uuid>.tmp files and .<root>.<pid>.<id>.staging directories of dead writers beside each device root and in the data directory (sweepWriteLeftovers).

Before step 16, a driver with deviceTcp sweeps the reverse rules into 47000-47999 a stopped daemon's tunnels left, on its idle owned emulators; on a leased emulator it only reads the rules and reserves their ports, which its destroy frees; on a phone it only reads them and reserves their ports, which its reclaim frees once no rule points to them (Android: AndroidDriver manual); a failure is a warn.

  1. Settle open requests — leasing.settleRequests: every open request fails INTERNAL; lease.rejected with reason daemon-restarted.
  2. Startup read — core.readStartup: one listManaged per driver, side by side, each bounded by STARTUP_READ_LIMIT_MS (60 s, not a config key). A throw or a timeout makes that platform unreadable. It never fails startup.
  3. Doctor reconcile — core.doctor.reconcile({ read }) on that read; no second listing.
  4. Lease reconciler — LeaseReconciler.run(read), table below. Leases on physical devices are skipped.
  5. Restore TTL timers for the leases left. One whose deadline passed expires through the ordinary expiry path. The physical watcher notes enrolled devices left in reclaiming.
  6. Core converge — StartupConverger.converge(read), readable platforms only, unleased and unclaimed virtual devices only:
    1. Quarantine re-arm: retry timers of quarantined devices.
    2. Interrupted reclaims (reclaiming): with deferredReclaimLeaseId → full reclaim in the background under a reclaim claim; else shut down, device.shutdown (initiator startup-interrupted-reclaim).
    3. Spent deletes: shutdown devices that may not be granted again are deleted inline. A failed delete is logged; the device stays shutdown.
    4. Idle devices of a virtualDevices: false platform: a ready one is shut down, then each is deleted (initiator virtual-devices-off). Claimed and leased ones are skipped.
    5. Baselines checkStale (a failure is logged), so a baseline the driver calls stale is known before the first lease.
    6. Warm pool and capacity observer start. The pool closes the lease rows of its learned-kind record whose lease the registry no longer holds, and its first pass boots the recorded kinds.
  7. queue.changed with the first depth. Converge done → health running.

Arm (skipped when a stop already ran):

  1. Owner-routed push subscription.
  2. daemon.started (redacted config), then one event per driver rejection.
  3. Health monitor and physical device watcher start.

Converge throws → health failed; parked requests get DAEMON_STARTUP_FAILED; stop convergence-failed; the start rejects. The claim (step 11) or the admin.token write (step 13) throws → parked requests get DAEMON_STARTUP_FAILED; the start runs Stop's steps 4 to 6 and closes connections and a claimed socket, with no daemon.stopping and no admin.token removal, then rejects; the entry logs it and exits. A stop already running (a daemon.stop accepted after the claim) is the one teardown instead: the start rejects at once with its own error, since that stop's stopAuxiliary waits for the start to settle, and startDaemon waits for the stop before it rejects. A DaemonAlreadyRunningError skips steps 5 and 6: the live daemon may share what the drivers hold (Simlock's adb server, adopted through its record). HTTP fails to bind → the converge hook waits for the listener before step 22's running, so health failed, no daemon.started, stop http-start-failed; the start rejects.

Lease reconciler outcomes (ADR 0019)

Device in the read Lease Device
running kept; timer restored unchanged
stopped or transitioning ended background reclaim
absent, platform readable ended marked missing in the same write (device.deleted, initiator doctor); no reclaim
platform unreadable ended stays reclaiming with deferredReclaimLeaseId; no reclaim, no claim

An ended lease emits lease.released reason device-lost, or lease.expired when its deadline passed while no daemon ran. A later start that reads the platform runs the deferred reclaim (step 21.2).

Gateway

  1. Instance identity — instance.json.
  2. FleetLeaseIndex, routing policy, GatewayService (drained workers in state.db), FleetLeaseCoordinator, DaemonServer with GatewayDispatcher. No registry, drivers, reaper or health monitor.
  3. DaemonServer.start: claim → host load → admin.token → converge starts → the event history starts → HTTP starts (uplink listener on the same server).
  4. Converge — GatewayService.start, then FleetLeaseCoordinator.start. A worker that dials before it ends is refused and redials.
  5. daemon.started. The gateway's dispatcher never parks a request.

Stop

DaemonServer.stop(reason) — one stop per process; a second call gets the same promise.

  1. Mark stopping; close the warm pool (core.closeWarmPool). New requests get DAEMON_STOPPING; daemon.stop still answers.
  2. Stop auxiliary fronts: agent-device edge (cancel its waiting allocates), HTTP (after its start settles), the uplink and its redial loop.
  3. Log, emit daemon.stopping.
  4. Unsubscribe pushes; dispose host load, dispatcher, reaper, health monitor, then wait for every cleanup run in flight (scheduled, dry-run or rule-scoped) and every health tick or recovery reboot in flight to end; dispose physical watcher.
  5. Settle — later converge steps are skipped; leasing drains the physical watcher and background reclaims; core drains the warm pool. Then core shuts down every ready device with no lease (core.shutdownIdle, initiator daemon-stop; leased devices keep running), before any driver is disposed.
  6. Dispose — core and leasing timers, ComponentInstaller.close, then leasing stops planning and waits for every lease request already provisioning or booting (a failed boot destroys its device; a queued request starts nothing and ends with the daemon), then the modules' stop (beforeDriversDispose: agent-device stop), then every driver's dispose (all tried; failures logged), then the worker lock is released (after the drivers, so no other worker sweeps while this one still acts on devices), then the event history stops (cancels its prune timer, closes history.db), so the stop's own events are written. A start that fails in the build releases the lock in startDaemon.
  7. Close connections, remove admin.token, release the socket (released even when the removal fails).

A gateway's stop: agent-device edge and HTTP, then FleetLeaseCoordinator and GatewayService are disposed, then the event history stops and state.db closes.

A stop releases nothing. Leases stay in state.db with their deadlines; the next start re-arms or ends them (steps 19–20).