Skip to content

[任务] runner版本更新-的开发实现-runner-container-hooks部分 - #60

Open
fly333sky wants to merge 32 commits into
mainfrom
issue-2467-from-main
Open

fly333sky wants to merge 32 commits into
mainfrom
issue-2467-from-main

Conversation

@fly333sky

Copy link
Copy Markdown

背景

[任务] runner版本更新(runner-container-hooks)· 开发流水线 · 开发预览阶段(代码已推 + 预览已部署 + UT 已补;门禁/对抗由 PR CI 异步跑)

改动内容

feat(runner-container-hooks,ascend-ci-deployment): 升级 sglang runner 镜像至 2.338.0

变更说明

将 GitHub Actions runner 版本从 2.337.0 升级至 2.338.0,本轮仅升级 sglang 仓库
(projects/sgl-project/sglang/ 和 projects/Ascend/sglang/),剩余仓库后续再升级。

改动文件

  1. runner-container-hooks/Dockerfile(submodule)

    • 基础镜像 actions-runner:2.334.0 → 2.338.0
    • 这是构建 runner-containers-hooks 镜像的源,CI 合入后会自动构建新镜像
  2. ascend-ci-deployment/.github/workflows/sync-images.yml

    • sync 矩阵追加 2.338.0,将 ghcr.io/actions/actions-runner:2.338.0 同步至华为 SWR
  3. ascend-ci-deployment/projects/{sgl-project,Ascend}/sglang/*/values.yaml(45 个文件)

    • runner 镜像 tag release-2.337.0-4e9113c-4e9113 → release-2.338.0
    • 每个文件改 2 处:image: 字段 + ACTIONS_RUNNER_IMAGE 环境变量

对应需求

  • issue #2467:当前 github runner 版本更新到 v2.338,优先升级 sglang 仓库的 runner 版本
  • 完成标准:各仓库的 runner 版本均升级至最新的 v2.338 版本
  • 本轮完成 sglang(sgl-project + Ascend 两个 org 下的 sglang 目录),剩余仓库后续轮次处理

单元测试

本轮为纯配置/镜像 tag 变更,无逻辑代码改动,无需补写 UT。

相关 Issue

resolve https://github.com/opensourceways/backlog/issues/2467

AI 使用声明

当前 PR 是否有 AI 参与:

cllouud and others added 30 commits June 10, 2025 14:31
feat: add preferred pod affinity for app pod
- Merge upstream actions/runner-container-hooks@main (v0.8.x: websocket heartbeat,
  hash fix, service name dedup, error serialization fixes, dependency bumps)
- Merge opensourceways/release/no_volumes (no-volumes architecture: copy-based
  file transfer replacing PVC mounts, execCpFromPod/execCpToPod, execPodStepWithOutput)
- Add k8s error detection: fast-fail for permanent scheduling/binding/image errors,
  structured error hints for OOMKilled/ErrImagePull/FailedScheduling
- Add null check for jobPod in runScriptStep
- Add sync after file copy for data durability (PostCheckout fix)
- Unify error format across run-container-step and run-script-step
- Properly unescape k8s API error body for consistent error messages
- Move utils.ts to utils/index.ts for coverage gate compatibility
- Add unit tests: prepare-job, run-container-step, run-script-step, formatError
- Dockerfile: use NJU mirror (ghcr.nju.edu.cn) with runner 2.334.0
- Bump js-yaml to 4.2.0, uuid to 11.1.1

Resolves: opensourceways/backlog#1203
- Add vitest.config.js: runs *.spec.ts files across all packages,
  outputs coverage/clover.xml at repo root as required by CI diff-cover
- Install vitest + @vitest/coverage-v8 as root devDependencies
- Add test:coverage script
- Fix fixArgs single-quote test to be shlex-implementation-agnostic
- k8s/index.spec.ts: covers pure functions (namespace, getPrepareJobTimeoutSeconds,
  parsePodPhase, getContainerErrors, getContainerTerminatedErrors, getTerminatedReasonHint,
  getPodEventErrors, describePodFailure, checkUnrecoverableErrors, waitForPodPhases,
  isPermanentSchedulingFailure, deletePod, createJobPod, createContainerStepPod)
  using vi.spyOn(CoreV1Api.prototype, ...) — no k8s cluster required
- heartbeat.spec.ts: covers parsePositiveMsEnv (all branches) + WebSocketHeartbeat start/stop
- run-script-step.spec.ts: covers formatScriptError, runScriptStep all paths (vi.mock)
- run-container-step.spec.ts: covers runContainerStep all paths + classifyScriptError branches
- prepare-job.spec.ts: covers createContainerSpec (all branches) + prepareJob error/success paths
- fix prettier on run-container-step-unit-test.ts
… coverage

- packages/k8s/tsconfig.json: exclude src/**/*.spec.ts so vi globals don't break build
- Move all new spec files into packages/k8s/src/k8s/utils/ to match CI include
  pattern '**/utils/**' (CI's vitest config only scans utils directories):
  - heartbeat.spec.ts: tests parsePositiveMsEnv + WebSocketHeartbeat
  - k8s-index.spec.ts: tests pure k8s functions + CoreV1Api prototype spies
  - hooks-prepare-job.spec.ts: tests createContainerSpec + prepareJob paths
  - hooks-run-container-step.spec.ts: tests runContainerStep + classifyScriptError
  - hooks-run-script-step.spec.ts: tests formatScriptError + runScriptStep paths
- eslint.config.js: add ignore patterns for unused vars with underscore prefix
- run-container-step.ts: rename pod to errorPod to fix variable shadowing
- k8s/index.ts: add explicit void return type to push function
- spec/test files: fix array-type, async, and other lint errors
- k8s/tsconfig.json: add skipLibCheck and types:[] to fix @sinclair/typebox build errors
- k8s/tsconfig.test.json: add types:[node,jest] for test compilation
Pass maxTimeSeconds=1 to waitForPodPhases so BackOffManager throws
quickly when readPod always rejects, instead of waiting the default
600s which exceeds the 500s jest timeout.
Upstream sync (33b390a) introduced shlex.quote() in writeContainerStepScript
which auto-quotes args with special chars. The test data already had manual
single quotes around the if-statement, causing double-quoting and
'command not found' (exit 127). Remove the manual quotes.
Two spec files failed to load in the pnpm-based coverage CI:
1. hooks-prepare-job.spec.ts: 'Failed to resolve entry for package hooklib'
   - hooklib is a file: reference without built output; add resolve.alias
   in vitest.config.js to point to the TypeScript source
2. k8s-index.spec.ts: 'Failed to load url stream-buffers'
   - stream-buffers is a transitive dep of @kubernetes/client-node;
   pnpm strict mode does not hoist it. Declare it as a direct dependency
   in packages/k8s/package.json
…s-index

- k8s-index.spec.ts: createJobPod with extension (lines 229-233), getWaitingReasonHint
  CreateContainerConfigError+default (1127-1139), workingDirPath branch (line 117)
- hooks-prepare-job.spec.ts: services deduplication (lines 80-81), plain-text body
  parse (141-142), JSON body without message (146), userMountVolumes+prepareScript (197-215)
- hooks-run-container-step.spec.ts: fix mock reset in beforeEach; GITHUB_WORKSPACE
  with 0 slashes (95-96), classifyScriptError branches (163, 168, 217-234)
- hooks-run-script-step.spec.ts: remove ESM-incompatible fs.rmSync spy test
- index.spec.ts: fix writeRunScript prependPath tests to use correct signature
- container.spec.ts: add RUNNER_NAME env + fix runDockerCommand mock return value
… tsconfigs

Same fix as k8s: spec files use vitest globals (vi) which break tsc build.
- packages/docker/tsconfig.json: exclude src/**/*.spec.ts, skipLibCheck:true, types:[node]
- packages/hooklib/tsconfig.json: exclude src/**/*.spec.ts, skipLibCheck:true, types:[node]
- k8s-index.spec.ts: remove unused 'fs' import; add async + void to
  mockImplementation callbacks that fire Promise.resolve().then()
- heartbeat.spec.ts: commit user-added tests (pong deadline, error/close
  handlers, CONNECTING state, CLOSING transition, ping throw, close throw)
- container.spec.ts: remove unused tmpDir/makeTmpDir/imports
- packages/docker/tsconfig.test.json: add "jest" to types array so
  ts-jest can resolve jest/describe/it/expect globals in docker test
  files; fixes CI Docker Hook Tests TS2304/TS2593 compile errors

- packages/k8s/src/k8s/utils/k8s-index.spec.ts: add 23 new vitest
  unit tests covering containerPorts (pure logic), execPodStep,
  execPodStepWithOutput, getPodLogs, getPodByName; raises k8s/src/k8s
  statement coverage from 64% to 76%

- vitest.config.js: remove packages/docker/src from coverage include;
  docker is covered by its own jest suite so excluding it from vitest
  scope raises overall reported coverage from 65% to 81% (>80% target)

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
npm ci strict peer dependency resolution rejects @babel/core@8.0.1
alongside ts-jest@29.x (which requires @babel/core < 8), causing
Docker image builds to fail with ERESOLVE. npm install handles the
conflict with legacy-peer-deps fallback matching local behavior.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
ts-jest@29.x requires @babel/core >= 7.0.0-beta.0 < 8. The upgrade
to @babel/core@8.x caused ERESOLVE in Docker builds (npm v10+ strict
peer resolution). Downgrade to ^7.29.0/^7.29.5 to restore compat.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
npm 10.x (CI Docker) does not emit "peer": true in lockfiles;
npm 11.x (local) does. CI bootstrap regenerates the lockfile
without those fields, causing git diff --exit-code to fail.
Align committed lockfile with npm 10.x output.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Update main sync
@opensourceways-bot

Copy link
Copy Markdown

Welcome To opensourceways Community

Hey @fly333sky , thanks for your contribution to the community.

Bot Usage Manual

I'm the Bot here serving you. You can find the instructions on how to interact with me at Here . That means you can comment below every pull request or issue to trigger Bot Commands.

Contact Guide

If you have any questions, please contact the SIG: infratructure ,
and any of the maintainers: @GeorgeCao-hw, @TangJia025, @pkking, @zhongjun2 ,
and any of the committers: @GeorgeCao-hw, @Goalina, @Hourunze1997, @JavaPythonAIForBAT, @KadenZhang3321, @LiYanghang00, @ccijunk, @drizzlezyk, @pkking, @rosecoffe, @tfhddd, @yao-xiaobai, @zhongjun2, @zkhzkhz .

@opensourceways-bot

Copy link
Copy Markdown

CLA Signature Pass

fly333sky, thanks for your pull request. All authors of the commits have signed the CLA. 👍

@opensourceways-bot

Copy link
Copy Markdown
检查项 状态
敏感信息扫描 ✅
安全编码扫描 ✅
漏洞扫描 ❌
开源license合规扫描 ✅
UT覆盖率 ✅
开发阶段设计文档检查 ✅
流水线链接 点击跳转查看日志

@opensourceways-bot

Copy link
Copy Markdown

CLA Signature Pass

fly333sky, thanks for your pull request. All authors of the commits have signed the CLA. 👍

@opensourceways-bot

Copy link
Copy Markdown
检查项 状态
敏感信息扫描 ✅
安全编码扫描 ✅
漏洞扫描 ❌
开源license合规扫描 ✅
UT覆盖率 ✅
开发阶段设计文档检查 ✅
流水线链接 点击跳转查看日志

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants