Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 36 additions & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,10 @@ jobs:

connect-lifecycle-e2e:
name: connect lifecycle e2e (linux/wsl mode)
needs: ut-it
# 只依赖 verify-build:ut-it 在 macOS/Windows 存在与常驻链路无关的存量
# 红(test_skill_tools_atom 路径解析等),连接类 e2e 是持久化改动的关键
# 验证,不应被无关红灯连坐跳过。
needs: verify-build
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
Expand All @@ -102,6 +105,38 @@ jobs:
run: pip install -e .[dev]
- name: Connect lifecycle E2E (local stub, no network)
run: pytest tests/e2e/test_connect_lifecycle_e2e.py -v
- name: Supervised self-heal E2E (watchdog, no-systemd path)
run: pytest tests/e2e/test_supervised_selfheal_e2e.py -v

connect-e2e-windows:
name: connect e2e (windows, real schtasks)
needs: verify-build
runs-on: windows-latest
env:
XSKILL_WIN_E2E: "1" # 一次性 runner,允许写真实用户 Profile
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.11"
cache: pip
- name: Install xskill (with dev extras)
run: pip install -e .[dev]
- name: Windows connect lifecycle E2E (real Task Scheduler)
run: pytest tests/e2e/test_windows_connect_e2e.py -v

platform-matrix-e2e:
# 多发行版容器矩阵(ubuntu/debian/openEuler/鸿蒙模拟)验证 supervised
# 常驻链路(自愈 + cron 自启 + stop 清理)。拉镜像+装依赖较重,只在
# nightly / 手动触发跑;本地随时可 bash tests/docker_e2e/platform_matrix/run.sh all
name: platform matrix e2e (docker, nightly)
if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'
needs: verify-build
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Run distro matrix (supervised self-heal in containers)
run: bash tests/docker_e2e/platform_matrix/run.sh all

real-llm-e2e:
name: real-llm-e2e (linux)
Expand Down
2 changes: 2 additions & 0 deletions openspec/changes/cross-platform-persistence/.openspec.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
schema: spec-driven
created: 2026-07-12
84 changes: 84 additions & 0 deletions openspec/changes/cross-platform-persistence/design.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
# design — cross-platform-persistence

## 1. 能力探测层(service.py)

| 探测函数 | 判定 | 备注 |
|---|---|---|
| `_is_wsl()` | 现有实现不变 | env + /proc osrelease |
| `_is_harmony()` | os-release `ID`/`ID_LIKE` ∈ {harmonyos, openharmony, ohos} 或 `uname -r` 含 `ohos` | 只读文件,无子进程 |
| `_linux_flavor()` | `"wsl" \| "harmony" \| "linux"` | wsl 优先(WSL 里跑鸿蒙容器不现实) |
| `_systemd_user_available()` | 现有实现不变 | `systemctl --user show-environment` |
| `_crontab_available()` | `which crontab` 且 `crontab -l` 退出码 ∈ {0,1}(1=「no crontab for user」) | timeout 5s |
| `_wsl_interop_available()` | `which wsl.exe` 且 `which schtasks.exe`(WSL interop 开启时 Windows PATH 自动追加) | 不主动执行 .exe |

所有探测失败 = 能力缺失,不抛异常。探测结果进 status 的 `degraded` 说明。

## 2. SupervisedProcessBackend(新增,method="supervised")

- `install_and_start()`:
1. 已 running 则幂等返回;
2. detach 拉起 watchdog:`<python> -m xskill connect --supervise`(`start_new_session=True`,
日志 `~/.xskill/logs/connect-supervisor.log`);
3. 按 flavor 挂开机自启(见 §3),失败仅记 `degraded`;
4. 等待 state 文件出现 child_pid(最多 10s)后返回 status。
- watchdog 主体在新模块 `supervisor.py::run_supervisor()`:
- 循环 spawn `<python> -m xskill connect --foreground`(env 加 `XSKILL_SUPERVISED=1`),
每次 spawn 后把 `child_pid` 写回 daemon state;
- 退避:初值 1s ×2 递增、封顶 300s;子进程存活 ≥600s 则退避归零;
- **停止语义**:收到 SIGTERM → 给 child SIGTERM(5s 后 SIGKILL)→ 清 state → 退出;
- **防双跑**:启动时若 state 里 watchdog_pid 存活则直接退出(幂等)。
- `stop()`:SIGTERM watchdog(兜底再杀 child)+ 卸开机自启挂载 + 清 state。
- `status()`:`running` = watchdog 活 && child 活;单侧死亡分别汇报
(`watchdog_alive` / `child_alive`),便于诊断。

## 3. 开机自启挂载(可插拔,随 supervised/systemd 后端组合)

| flavor | 机制 | 装 | 卸 |
|---|---|---|---|
| linux/harmony | crontab `@reboot <python> -m xskill start --quiet # xskill-connect` | 读现 crontab,去重后追加 marker 行 | 按 marker 过滤重写 |
| wsl | `schtasks.exe /Create /TN Xskill_WSL_Boot /SC ONLOGON /TR "wsl.exe -d <distro> -u <user> -- <python> -m xskill start --quiet"` | interop 调用 | `schtasks.exe /Delete` |
| 任一失败 | 记 `boot_autostart: "none"` + `degraded` 警告 | — | — |

`xskill start --quiet`:已 running 时静默退出 0(幂等),使自启触发器可无脑重复执行。
WSL + systemd 组合同样挂 Windows 任务(systemd 只解决「VM 内」自启;VM 本身要 Windows 拉)。

## 4. LinuxServiceBackend 选择链(重写)

```
install: systemd? ──yes─► SystemdUserBackend(linger 失败→警告不阻断)
└─no──► SupervisedProcessBackend
override: XSKILL_CONNECT_BACKEND ∈ {systemd, supervised, detached}
stop/status: 先按 state.method 路由(接管旧安装态),state 缺失再按探测链
```

`WSLSystemdRequiredBackend` 删除;`DetachedProcessBackend` 保留但仅显式 override 可达。

## 5. updater 加固

- `_health_check(python) -> bool`:`subprocess.run([python, "-m", "xskill", "--version"],
timeout=60)`,returncode==0。
- `_install` 成功后:health check 失败 → `pip install xskill==<升级前版本>` 回滚(同一
`_PIP_TIMEOUT` 约束)→ journal 记 `bad_versions[target] = {ts, reason}`;回滚也失败则
critical 日志 + 不重启(宁可跑旧代码在内存里,也不重启进坏版本)。
- journal:`~/.xskill/update_journal.json`,损坏容忍(解析失败视为空);`_check_and_update`
对 PyPI/server 候选版本先过 `bad_versions` 滤网。健康升级成功记 `last_good`。
- `_restart()`:`XSKILL_SUPERVISED=1` 时全平台统一 `os._exit(1)` 交给 watchdog;
其余路径维持现状(execv / schtasks exit-1 / startup_folder spawn)。

## 6. 兼容与迁移

- state 文件新增键(watchdog_pid/child_pid/boot_autostart/flavor)全部增量;旧 state 的
`method ∈ {systemd-user, detached, schtasks, startup_folder}` 继续被识别。
- 旧 detached 安装态在下次 `xskill start` 时被停掉并迁移到新链路(沿用现有迁移逻辑)。
- Windows startup_folder 的 .vbs 内容从 `connect --foreground` 换成 `connect --supervise`;
旧 .vbs 无需主动迁移,下次 start 重写。

## 7. 测试设计

- **单测**(全 OS 可跑,无真进程):探测矩阵、选择链矩阵、cron marker 幂等、interop 命令
拼装、journal 读写与滤网、_restart 分支路由。
- **进程级 e2e**(Linux):supervised 全生命周期 + kill child 自愈 + stop 全清理;
伪 crontab(PATH shim)验证 @reboot 装卸。
- **windows e2e**(CI windows-latest):schtasks 真装卸 + status。
- **docker 矩阵**:ubuntu:24.04 / debian:12 / openEuler(+os-release 覆写模拟鸿蒙),
容器内无 systemd → 必然落 supervised 链,跑同一套 e2e 用例。
88 changes: 88 additions & 0 deletions openspec/changes/cross-platform-persistence/proposal.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
# 跨平台常驻兼容性升级:能力探测式 fallback + 产品级自愈与更新回滚

## Why

`xskill connect` 的常驻能力目前按「平台名」硬编码策略,存在四个武断点,导致
Windows / WSL / Ubuntu / 鸿蒙 / 其他 Linux 上体验参差:

1. **WSL 无 systemd 直接硬失败**(`WSLSystemdRequiredBackend`)。这个策略双重错误:
- 过苛:用户明明可以以「会话内常驻 + 崩溃自愈」的降级模式运行,却被一刀切拒绝;
- 没解决真问题:即使 systemd + linger 齐备,**Windows 重启后 WSL VM 也不会自动拉起**
——WSL 的开机自启只能靠 Windows 侧触发器(计划任务/启动项经 interop 调 `wsl.exe`),
现行实现完全没有这一层。
2. **detached 降级无崩溃自愈**(`restart_policy: none`)。无 systemd 的 Linux(精简容器、
老发行版、鸿蒙)上进程一崩就死透,与「常驻」的承诺不符。
3. **鸿蒙(HarmonyOS/OpenHarmony)零适配**。鸿蒙终端是 Linux 内核 + 无 systemd 用户态,
现行代码把它当普通 Linux,直接掉进无自愈的 detached。
4. **自动更新无健康检查、无回滚**。pip 装上一个坏 wheel(半残依赖、二进制不兼容
——鸿蒙/老 glibc 上很现实)后直接重启,进程起不来 → systemd/schtasks 无限重启循环,
且 updater 永远不会重试回好版本。

## What Changes

### 1. 设计原则:按「能力探测」选择后端,平台名只用于提示与遥测

后端选择不再 `if 平台名 == X 则拒绝/允许`,而是逐项探测能力(systemd --user 可用?
crontab 可用?WSL interop 可用?),按优先级取第一个可用项;每一级降级都在
`xskill status` 里如实汇报(新增 `crash_recovery` / `boot_autostart` / `degraded` 字段),
**绝不伪装成完整常驻,也绝不因为不完美而拒绝服务**。

### 2. Linux 族(linux / wsl / harmony 统一链路)

```
systemd --user 可用 ──► SystemdUserBackend(自愈=systemd,自启=linger)
│ linger 失败 → 降级警告,不再硬失败
▼ 不可用
SupervisedProcessBackend(新增)
watchdog 进程托管 connect --foreground,指数退避自动重启(自愈=watchdog)
开机自启按 flavor 补挂:
wsl → Windows 计划任务经 interop 调 `wsl.exe -d <distro> … xskill start`
linux/harmony → crontab @reboot(marker 管理,幂等装卸)
探测不可用 → status.degraded 明示「不随开机自启」,仍正常常驻
```

- WSL + systemd 场景同样补挂 Windows 侧计划任务(否则 Windows 重启后 unit 不会跑)。
- 鸿蒙识别:`/etc/os-release` 的 `ID/ID_LIKE ∈ {harmonyos, openharmony, ohos}` 或
`uname -r` 带 `-ohos`;识别结果仅影响提示文案与自启挂载方式,主链路与 Linux 一致。
- `XSKILL_CONNECT_BACKEND` 支持 `systemd|supervised|detached` 显式覆盖(原有 `detached`
语义保留:裸 detached 仍可选,但不再是默认降级)。

### 3. Windows 原生路径加固

schtasks 主路径不变;Group Policy 拒绝后的「启动文件夹」降级从裸 `connect --foreground`
改为拉 supervisor —— 降级路径同样获得崩溃自愈。

### 4. 自动更新产品级加固(updater)

- **升级后健康检查**:pip 安装成功后、重启前,用子进程跑 `<python> -m xskill --version`
验证新版本可导入可执行;失败即 **pip 回滚到升级前版本**。
- **坏版本拉黑**:健康检查失败/回滚的版本记入 `~/.xskill/update_journal.json`,
后续检查跳过该版本,杜绝「升级→崩→回滚→再升级」死循环。
- **supervisor 感知的重启**:被 watchdog 托管时(`XSKILL_SUPERVISED=1`)统一以非零退出码
重启,由 watchdog 用新版本拉起,不再自行 spawn 孤儿进程。

### 5. 多端测试

- 单测:平台×能力矩阵(wsl/harmony/linux × systemd/cron/interop/裸),全部 monkeypatch,
三大 OS 的 CI ut-it 矩阵均可跑;重写 `test_wsl_persistence_policy.py` 以匹配新策略。
- e2e(Linux CI + 本地):supervised 链路「start → 杀 connect 子进程 → watchdog 自动拉起
→ stop 全清理」真实进程验证;updater 回滚用假 PyPI + 坏包验证。
- **docker 发行版矩阵**(`tests/docker_e2e/platform_matrix/`):ubuntu:24.04、debian:12、
openEuler(鸿蒙用户态最近似,另以覆写 os-release 模拟 harmony 识别),容器内跑同一套
lifecycle + 自愈 e2e。
- CI:connect-lifecycle e2e 扩到 windows-latest(真 schtasks);docker 矩阵挂
nightly/workflow_dispatch。

## Impact

- 受影响模块:`team/client/service.py`(重构选择链)、新增 `team/client/supervisor.py`、
`team/client/updater.py`(健康检查/回滚/journal)、`cli.py`(`--supervise` 隐藏 flag、
status 新字段渲染)、CI workflow。
- 行为变化(面向用户):
- WSL 无 systemd:从报错拒绝 → 正常常驻(自愈 by watchdog)+ 尽力挂 Windows 自启;
- 无 systemd Linux/鸿蒙:崩溃自愈从无到有;
- `xskill status` 多出 `flavor/crash_recovery/boot_autostart/degraded` 字段(增量,
不破坏既有字段);
- 坏版本更新不再导致服务瘫痪。
- 兼容性:既有 systemd/schtasks 安装态可原位接管(state 文件 `method` 向后兼容);
`XSKILL_CONNECT_BACKEND=detached` 行为不变。
14 changes: 14 additions & 0 deletions openspec/changes/cross-platform-persistence/tasks.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# tasks — cross-platform-persistence

- [x] 1. 能力探测层:`_is_harmony` / `_linux_flavor` / `_crontab_available` / `_wsl_interop_available`
- [x] 2. `supervisor.py`:watchdog 循环(退避、child_pid 回写、SIGTERM 级联、防双跑)
- [x] 3. `SupervisedProcessBackend`(装/停/看 + 幂等)
- [x] 4. 开机自启挂载:cron @reboot marker 管理 + WSL interop schtasks 任务
- [x] 5. `LinuxServiceBackend` 选择链重写;删 `WSLSystemdRequiredBackend`;linger 失败降级为警告
- [x] 6. Windows startup_folder 降级改走 supervisor
- [x] 7. `cli.py`:`connect --supervise` 隐藏 flag、`start --quiet`、status 渲染新字段
- [x] 8. updater:health check + 回滚 + update_journal 拉黑 + supervisor 感知 `_restart`
- [x] 9. 单测:探测/选择链/cron/interop/journal 矩阵;重写 `test_wsl_persistence_policy.py`
- [x] 10. e2e:supervised 自愈生命周期(Linux);windows-latest lifecycle
- [x] 11. docker 发行版矩阵(ubuntu/debian/openEuler+鸿蒙模拟)脚本与文档
- [x] 12. CI 接线:windows lifecycle job、platform-matrix nightly job
38 changes: 28 additions & 10 deletions skills/using-xskill/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
name: using-xskill
description: Use when installing, configuring, or operating xskill (the `xskill` CLI / `pip install xskill`) — starting the daemon, registering trajectory dirs, joining a team server, understanding how trajectories become Skills, or rebuilding/re-distilling the skill library after a model change.
description: Use when installing, configuring, or operating xskill (the `xskill` CLI / `pip install xskill`) — starting the daemon, keeping `connect` resident on Windows/WSL/Linux/HarmonyOS, registering trajectory dirs, joining a team server, understanding how trajectories become Skills, or rebuilding/re-distilling the skill library after a model change.
---

# Using xskill
Expand All @@ -18,13 +18,27 @@ skill versions only replace old ones when real traffic shows they serve users be
A/B → installed into every agent's skill dir. You operate the daemon; the daemon does
the distilling.

## When to Use
## Step 1 — Detect the platform

- Installing xskill or filling in `~/.xskill/config.yaml` (LLM + embedding endpoints)
- Starting/keeping the daemon running (`xskill serve`), or backfilling old trajectories
- Joining or hosting a team server (`xskill serve --server` / `xskill connect`)
- Understanding the agent pipeline, atoms, canary/UX scoring, or deployment modes
- **Re-distilling the whole skill library** (e.g. after switching to a stronger model)
Anything involving the resident `connect` client (`xskill start/stop/status`, boot
autostart, crash recovery) is platform-dependent. Detect first, then read **only** the
matching reference:

1. **Windows** — you are in PowerShell/cmd (`sys.platform == "win32"`).
2. **WSL** — on Linux: `$WSL_DISTRO_NAME`/`$WSL_INTEROP` set, or
`grep -qi microsoft /proc/sys/kernel/osrelease`. (Checked before HarmonyOS.)
3. **HarmonyOS/OpenHarmony** — `ID`/`ID_LIKE` in `/etc/os-release` contains
`harmonyos`/`openharmony`/`ohos`, or `uname -r` contains `ohos`.
4. **Linux + systemd** — `systemctl --user show-environment` exits 0.
5. **Linux without systemd** — the probe above fails (containers, minimal distros).

| Detected | Read |
|----------|------|
| Windows | `references/platform-windows.md` |
| WSL (with or without systemd) | `references/platform-wsl.md` |
| Linux with systemd --user | `references/platform-linux-systemd.md` |
| Linux without systemd / container / HarmonyOS | `references/platform-linux-nosystemd.md` |
| macOS | no native backend yet — `xskill start` errors; host `xskill connect --foreground` under a launchd LaunchAgent (KeepAlive=true) yourself |

## Quick Reference

Expand All @@ -33,10 +47,10 @@ the distilling.
| `pip install xskill` | Install (Python 3.9+) |
| `xskill serve` | Standalone daemon: FastAPI + watcher; first run writes `~/.xskill/config.yaml` then exits |
| `xskill serve --server` | Team server: owns all LLM calls + git; prints a join token |
| `xskill connect <host:port> --token <t>` | Join a team server as a thin client |
| `xskill connect <host:port> --token <t>` | Join a team server; then hands the daemon to the OS persistence backend |
| `xskill start` / `stop` / `status` | Install / remove / inspect the resident `connect` task (`--quiet` on start = idempotent, for boot triggers) |
| `xskill registry add <path>` | Backfill / watch an extra trajectory directory |
| `xskill search traj\|skill <query>` | Search trajectories or skills |
| `xskill read <path> --eco <eco>` | Batch-ingest db trajectories (ngagent/opencode) |
| `xskill rebuild [--force]` | Re-distill from existing raw trajectories (see reference) |
| `xskill stats` | Token usage & estimated cost |

Expand All @@ -46,14 +60,18 @@ distilled unless `xskill serve` (or the team server) is running.
## Progressive Disclosure — read on demand

- **Install & configure** (config.yaml fields, per-agent collect/install paths, team
client setup): `references/installation.md`
client setup, auto-update/rollback): `references/installation.md`
- **Platform persistence** (per the routing table in Step 1):
`references/platform-{windows,wsl,linux-systemd,linux-nosystemd}.md`
- **How it works** (TaskAgent → TaskClusterAgent → SkillEditAgent, atoms, canary/UX
scoring, standalone vs team mode): `references/mechanisms.md`
- **Rebuild the skill library** (a ready-to-run prompt that walks a model through
re-distilling correctly): `references/rebuilding-skill-library.md`

## Common Mistakes

- **Hand-rolling systemd units / Task Scheduler entries.** `xskill start` probes
capabilities and installs the right persistence itself; manual setup is only for macOS.
- **Running `rebuild` with no daemon up.** `rebuild` only resets DB state; the watcher
in `serve` does the actual re-split/re-cluster every 30s. No daemon = nothing happens.
- **Deleting raw `~/.xskill/*_sessions/*.md`.** Those are the *input* to distillation —
Expand Down
Loading