feat: unify LLM evaluation jobs under traits.eval - #58
Merged
Merged
Conversation
PixelCores
marked this pull request as ready for review
September 20, 2026 02:27
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
变更
LLM 评测统一使用
type: job+traits.eval。独立 Job 与 Application/Workflow Job 共用规格、校验和 Harbor Runner;model是被测对象,agent是执行 harness,env: ack描述执行环境。框架版本和 Runner 镜像由平台管理,不要求调用方提交 Harbor JobConfig。taskPackageId、attempts、concurrency、timeoutSeconds和resultPolicy位于traits.eval;Runner 与 trial 资源分别由traits.resources和sandboxResources控制。properties.labels与归属标签,Runner 专用标签仍由平台决定。eval。迁移与风险
这是公开请求契约的严格切换:旧
traits.evaluation、type: eval、评测spec、framework 字段及顶层resultPolicy不再接受。升级前应排空或取消旧评测、确认 Kubernetes Job 已停止,并导出所需历史结果。普通command请求不变。gRPC 的
JobTraits.eval和AppSpecTraits.eval保留原字段号 7 和 15,二进制字段号不变;ProtoJSON 名称和生成的 Go 字段改为eval。当前仍限 Harbor 0.22.0 与已有任务包边界;应用内结果请求需要executionKey。验证
go test ./... -count=1通过;相关包的go vet通过。eval编解码和旧键拒绝;JSON 示例及 Python 压测脚本语法检查通过。grpc-gen-check未在本机运行:缺少仓库固定的protoc 35.0;生成描述符已通过 Go/ProtoJSON 测试。