Add Xarena spreadsheet benchmark package

benchmark/spreadsheet_xarena/README.md

@@ -0,0 +1,69 @@
+# Spreadsheet Xarena Benchmark
+
+This directory packages the xskill SpreadsheetBench submission for Xarena.
+
+## Layout
+
+```text
+benchmark/spreadsheet_xarena/
+  algo_app/                  # submitter-container image source
+  dataset/train_split/        # lightweight train/val/test task split files
+  dataset/prepare_data_root.sh # downloads or copies workbook data before build
+  third_party/SkillOpt/       # rollout harness used during training
+```
+
+`algo_app` is the algorithm image. It trains xskill inside the Xarena job and writes the skill package to the shared volume:
+
+```text
+/shared/skill/ALGO
+/shared/skill/DONE
+/shared/skill/skills/<skill-name>/SKILL.md
+```
+
+The evaluator image is still owned by the leaderboard board. `third_party/SkillOpt` is only used as the rollout harness for training trajectories; it is not the evaluator container.
+
+## Build
+
+From this directory:
+
+```bash
+cd algo_app
+TAG=main-xarena PUSH=1 LOAD_KIND=lb bash build.sh
+```
+
+By default the image name is:
+
+```text
+localhost:5000/p_user1/algo-xskill:<TAG>
+```
+
+`build.sh` copies the current repository checkout into the Docker build context, so changes on this MR branch are included in the image. It stages `third_party/SkillOpt` and a prepared `dataset/data_root` into `_ctx`.
+
+The workbook data is not committed. If `dataset/data_root` is missing, `build.sh` calls `dataset/prepare_data_root.sh`, which downloads `https://xskill.wiki/zip/xskill-compete.zip` and extracts the 100-task SpreadsheetBench package. To use an existing local dataset instead:
+
+```bash
+DATA_ROOT=/path/to/data_root TAG=main-xarena bash algo_app/build.sh
+```
+
+## Submit
+
+Submit the built image to the Spreadsheet leaderboard board with Xarena env vars similar to:
+
+```text
+EVAL_MODEL=deepseek-v4-flash
+XSKILL_WORKERS=3
+XSKILL_MAX_TURNS=5
+XSKILL_EPOCHS=4
+XSKILL_VAL_BLOCK=true
+XSKILL_VAL_BLOCK_TIMEOUT=1800
+OUTPUT_DIR=/shared/out
+```
+
+The API keys should come from the leaderboard Kubernetes secret:
+
+```text
+DEEPSEEK_API_KEY
+DASHSCOPE_API_KEY
+```
+
+Do not put API keys into this repository or into `env_text`.

benchmark/spreadsheet_xarena/algo_app/.gitignore

@@ -0,0 +1,2 @@
+/_ctx/
+!config.yaml

benchmark/spreadsheet_xarena/algo_app/Dockerfile

@@ -0,0 +1,54 @@
+FROM python:3.11-slim
+ENV PYTHONUNBUFFERED=1 \
+    PIP_INDEX_URL=https://pypi.tuna.tsinghua.edu.cn/simple \
+    PIP_TRUSTED_HOST=pypi.tuna.tsinghua.edu.cn
+
+# ── 系统依赖 + Node 20（claude CLI 运行时）────────────────────────────────
+# git/bash 给脚本与 setuptools-scm 用；dulwich 让 xskill 不依赖系统 git，但留着无害。
+RUN apt-get update && apt-get install -y --no-install-recommends \
+        git bash curl ca-certificates \
+    && curl -fsSL https://deb.nodesource.com/setup_20.x | bash - \
+    && apt-get install -y --no-install-recommends nodejs \
+    && rm -rf /var/lib/apt/lists/*
+
+# ── claude CLI（绝对路径在 entrypoint 用 $(command -v claude)）─────────────
+RUN npm install -g @anthropic-ai/claude-code \
+    && claude --version
+
+# ── Python 第三方依赖 ─────────────────────────────────────────────────────
+COPY requirements.txt /app/requirements.txt
+RUN pip install --no-cache-dir -r /app/requirements.txt
+
+# ── xskill 本体（从烘焙源码 editable 装）──────────────────────────────────
+# 源码烘焙后无 .git，setuptools-scm 无法推断版本 -> 用 PRETEND_VERSION 兜底。
+COPY _ctx/xskill /app/xskill
+ENV SETUPTOOLS_SCM_PRETEND_VERSION=0.6.1
+RUN pip install --no-cache-dir -e /app/xskill \
+    && xskill serve --help >/dev/null && echo "xskill serve OK"
+
+# ── SkillOpt 本体（rollout 用 eval_only.py）───────────────────────────────
+COPY _ctx/SkillOpt /app/SkillOpt
+RUN pip install --no-cache-dir -e /app/SkillOpt || true
+ENV PYTHONPATH=/app/SkillOpt
+
+# ── 数据（26M）、配置、训练 split、sync 脚本、空 skill ────────────────────
+COPY _ctx/data_root /data
+WORKDIR /app
+COPY config.yaml /app/config.yaml
+COPY train_split/ /app/train_split/
+COPY _ctx/sync_skills_to.sh /app/sync_skills_to.sh
+COPY entrypoint_train.sh /app/entrypoint_train.sh
+COPY multi_turn_rollout.py /app/multi_turn_rollout.py
+RUN chmod +x /app/entrypoint_train.sh /app/sync_skills_to.sh \
+    && : > /app/empty_skill.md
+
+# IS_SANDBOX=1：容器以 root 运行，claude CLI 默认拒绝 root 下的
+# --dangerously-skip-permissions；设此变量让其放行（容器本身即沙箱）。
+# rollout 子壳里也会再设一次，这里兜底任何镜像内 claude 调用。
+ENV SKILL_DIR=/shared/skill \
+    EVAL_MODEL=deepseek-v4-flash \
+    DATA_ROOT=/data \
+    TRAIN_SPLIT=/app/train_split \
+    XSKILL_PROMO_THRESHOLD=5 \
+    IS_SANDBOX=1
+CMD ["bash","/app/entrypoint_train.sh"]

benchmark/spreadsheet_xarena/algo_app/README.md

@@ -0,0 +1,31 @@
+# xskill Xarena Algorithm Image
+
+This is the submitter-container image for the SpreadsheetBench Xarena board.
+
+It starts xskill, runs SpreadsheetBench training rollouts, collects graduated skills, and writes the result to the Xarena shared volume:
+
+```text
+/shared/skill/ALGO
+/shared/skill/DONE
+/shared/skill/skills/<name>/SKILL.md
+```
+
+Build from this directory:
+
+```bash
+TAG=main-xarena PUSH=1 LOAD_KIND=lb bash build.sh
+```
+
+Useful runtime variables:
+
+```text
+EVAL_MODEL=deepseek-v4-flash
+XSKILL_WORKERS=3
+XSKILL_MAX_TURNS=5
+XSKILL_EPOCHS=4
+XSKILL_VAL_BLOCK=true
+XSKILL_VAL_BLOCK_TIMEOUT=1800
+OUTPUT_DIR=/shared/out
+```
+
+`DEEPSEEK_API_KEY` and `DASHSCOPE_API_KEY` must be injected by the leaderboard job secret.

benchmark/spreadsheet_xarena/algo_app/build.sh

@@ -0,0 +1,78 @@
+#!/usr/bin/env bash
+# Build the Xarena-compatible xskill SpreadsheetBench algorithm image.
+set -euo pipefail
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+BENCH_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
+REPO_ROOT="$(cd "$SCRIPT_DIR/../../.." && pwd)"
+
+REG="${REG:-localhost:5000}"
+IMAGE_REPO="${IMAGE_REPO:-p_user1/algo-xskill}"
+TAG="${TAG:-main-xarena}"
+IMG="$REG/$IMAGE_REPO:$TAG"
+
+DATA_ROOT="${DATA_ROOT:-$BENCH_DIR/dataset/data_root}"
+SKILLOPT_SRC="${SKILLOPT_SRC:-$BENCH_DIR/third_party/SkillOpt}"
+PUSH="${PUSH:-0}"
+LOAD_KIND="${LOAD_KIND:-}"
+PREPARE_DATA="${PREPARE_DATA:-1}"
+
+require_dir() {
+  local path=$1
+  local label=$2
+  if [ ! -d "$path" ]; then
+    echo "missing $label: $path" >&2
+    exit 1
+  fi
+}
+
+if [ ! -d "$DATA_ROOT" ] && [ "$PREPARE_DATA" = "1" ]; then
+  bash "$BENCH_DIR/dataset/prepare_data_root.sh"
+fi
+
+require_dir "$DATA_ROOT" "SpreadsheetBench dataset"
+require_dir "$SKILLOPT_SRC" "SkillOpt rollout harness"
+
+cd "$SCRIPT_DIR"
+rm -rf _ctx
+mkdir -p _ctx
+
+rsync -a --delete \
+  --exclude .git \
+  --exclude .venv \
+  --exclude __pycache__ \
+  --exclude '*.pyc' \
+  --exclude .pytest_cache \
+  --exclude .mypy_cache \
+  --exclude .ruff_cache \
+  --exclude .key \
+  --exclude '.env' \
+  --exclude '*.pem' \
+  --exclude '*.key' \
+  --exclude 'benchmark/spreadsheet_xarena/algo_app/_ctx' \
+  --exclude 'benchmark/spreadsheet_xarena/dataset' \
+  --exclude 'benchmark/spreadsheet_xarena/third_party' \
+  "$REPO_ROOT/" _ctx/xskill/
+
+rsync -a --delete \
+  --exclude .git \
+  --exclude __pycache__ \
+  --exclude '*.pyc' \
+  --exclude .pytest_cache \
+  --exclude '.env' \
+  "$SKILLOPT_SRC/" _ctx/SkillOpt/
+
+rsync -a --delete "$DATA_ROOT/" _ctx/data_root/
+cp "$SCRIPT_DIR/sync_skills_to.sh" _ctx/sync_skills_to.sh
+
+docker build -t "$IMG" .
+
+if [ "$PUSH" = "1" ]; then
+  docker push "$IMG"
+fi
+
+if [ -n "$LOAD_KIND" ]; then
+  kind load docker-image "$IMG" --name "$LOAD_KIND"
+fi
+
+echo "built $IMG"

benchmark/spreadsheet_xarena/algo_app/config.yaml

@@ -0,0 +1,94 @@
+# xskill config TEMPLATE — 打榜镜像·xskill (reduced-scale real training).
+# xskill 不读环境变量/key 文件，所以 api_key 用占位符；entrypoint_train.sh 在
+# 运行时把 __DEEPSEEK_API_KEY__ / __DASHSCOPE_API_KEY__ 替换成真实 key 后落到
+# $XSKILL_HOME/config.yaml。
+#
+# 关键：xskill 的 baby->main "毕业"门槛是 candidates.py 里硬编码的
+# ATOM_PROMOTION_THRESHOLD（runner 构造 SkillEditAgent 时不传 threshold，所以
+# config 的 candidates.threshold 改不动它）。reduced 规模下必须在 entrypoint 里
+# 直接 patch 该常量调低，否则 4 条轨迹攒不满默认 10 分 -> 零毕业 -> 全是 baby
+# stub -> 没有真正蒸馏出的 SKILL.md。详见 entrypoint_train.sh。
+
+# ===== Skill repository (candidate + main 都落这里) =====
+skill_dir: __XSKILL_HOME__/skill
+
+# ===== LLM (generation / scoring / cluster / SkillEdit) =====
+llm:
+  base_url: https://api.deepseek.com
+  model:    deepseek-v4-flash
+  api_key:  __DEEPSEEK_API_KEY__
+  max_tokens: 10000
+  request_timeout: 120
+  connect_timeout: 15
+
+# ===== Embedding (atom 入库 + AtomTaskSearch 向量检索) =====
+# DeepSeek 无 embeddings API；用 DashScope text-embedding-v4。
+embedding:
+  base_url: https://dashscope.aliyuncs.com/compatible-mode/v1
+  model:    text-embedding-v4
+  api_key:  __DASHSCOPE_API_KEY__
+  dim:      0
+
+# ===== 候选攒分阈值 (v1 路径 + stale 归档用；v2 毕业门槛见 entrypoint patch) =====
+candidates:
+  threshold:        2     # v1 ready_for_promotion / stale 用；调低无副作用
+  stale_days:       3650  # 别在短跑里把候选归档成 stale
+  min_source_trajs: 1
+
+# ===== SkillEditAgent（把 candidates 整理成正文 SKILL.md 的写作 agent）=====
+skill_edit_agent:
+  tool_call_limit:     24
+  timeout_seconds:     600
+  read_file_max_bytes: 15000
+
+# ===== Canary（main->staging 灰度）：team-CS 在线进化模式开启并调低门槛。 =====
+# epoch1.. team-CS：worker 作为 distinct client 上传解题轨迹，server 端 CS 归因
+# 给 atom 按 side 打 ux_score；main 有分后 SkillEdit 把已有技能的更新路由到
+# commit_to_staging 开灰度分支，check_and_decide 在样本够时 promote/discard。
+# 小数据下必须调低（否则永远凑不够样本）：probability=0.5 让 ~50% 流量进 staging，
+# min_samples/total_samples=2 让 2 条样本就能决策，scope_top_n=1。
+# 注：entrypoint_train.sh 渲染时还会再覆盖这段（双保险），见 1a) 段。
+canary:
+  enabled:       true
+  probability:   0.5
+  min_samples:   2
+  max_days_hold: 14
+  rotate_interval: 60
+  scope_top_n:   1
+  total_samples: 2
+
+# ===== description 触发优化：commit 时跑一次 hill-climb 调 frontmatter。短跑
+# 可保留（失败不阻塞 commit），但把预算压小省时间/省 token。 =====
+skill_opt:
+  enabled:       true
+  n_cases:       6
+  runs_per_case: 1
+  max_iters:     2
+  max_llm_calls: 60
+  train_frac:    0.6
+  seed:          42
+  catalog_max_skills: 12
+  catalog_desc_cap:   256
+  probe_case_timeout: 45
+  rerun_enabled:      false
+
+# ===== Watcher（serve 里的目录轮询）=====
+watcher:
+  poll_interval:  10
+  max_concurrent: 2
+
+# ===== Ingest（claude_code session jsonl -> traj_*.md 桥接）=====
+ingest:
+  # 评测场景：脚本批量产 session、写完即定稿，settle 调小到 5s 便于尽快入库。
+  settle_seconds: 5
+  mask_patterns:
+    # 剥掉 SkillOpt codex harness 每题固定的 turn-0 提示词外壳，防聚类被任务外壳吸住。
+    - '(?s)Use the workspace files to solve the task\..*?summarize the approach\.'
+
+# ===== Dashboard：关掉（打榜镜像无需 web 控制台）=====
+dashboard:
+  enabled:  false
+  public:   false
+  password: ""
+  default_harness: claude_code
+  default_model:   deepseek-v4-flash