Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -27,3 +27,11 @@ ANTHROPIC_API_KEY=
# AI_MODEL=gpt-4o
# 限制发送给 AI 的 diff 最大字符数(防止超大 PR 超出上下文)
MAX_DIFF_LENGTH=40000

# ==== 可观测性(JSON-lines 结构化日志)====
# 详细事件日志总开关(默认 true;false 时仅输出 warn/error 与调用摘要)
# HEIMDALL_LOG_ENABLED=true
# 每次审查固定一行调用摘要(默认 true,独立于 HEIMDALL_LOG_ENABLED)
# HEIMDALL_INVOCATION_LOGS=true
# 日志级别过滤(默认 info):error | warn | info | debug
# HEIMDALL_LOG_LEVEL=info
11 changes: 11 additions & 0 deletions .mcp.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
{
"mcpServers": {
"github": {
"command": "npx",
"args": ["-y", "@modelcontextprotocol/server-github"],
"env": {
"GITHUB_PERSONAL_ACCESS_TOKEN": "${GITHUB_MCP_TOKEN}"
}
}
}
}
13 changes: 8 additions & 5 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ Core selling point: **model freedom** — supports Claude / GPT / Gemini / local

| Path | Role |
| --- | --- |
| `src/observability.ts` | **JSON-lines observability module** (structured logs; Worker imports it; mirrored to `scripts/observability.js`) |
| `src/review/prompt.ts` | **Heimdall persona prompt (single source)** — quality core; Worker imports it directly; changes must sync the `scripts/heimdall-review.js` copy |
| `src/review/parse.ts` | LLM JSON parse + report render + loose-JSON tolerance + dedup + **labels (en/zh/bilingual)** |
| `src/review/providers.ts` | AI providers (anthropic / openai / gemini + local) |
Expand All @@ -25,6 +26,7 @@ Core selling point: **model freedom** — supports Claude / GPT / Gemini / local
| `src/app.ts` | Probot event subscriptions (PR events + `@CoderHeimdall`) |
| `worker/index.ts` | Cloudflare Worker (webhook, signature, dedup, review, status marks) |
| `scripts/heimdall-review.js` | Actions-mode script (zero-dep, copied to target repo) |
| `scripts/observability.js` | Actions-mode observability mirror (CommonJS, copied with heimdall-review.js) |
| `template/heimdall-review.yml` | Actions-mode workflow (copied to target repo) |
| `test/` | Unit tests (node:test) |

Expand All @@ -41,11 +43,12 @@ npm run worker:deploy # deploy Worker
## Core Conventions (read before changing code)

1. **Three modes share one core**: changes under `src/review/` apply to the Worker automatically (imported modules); `scripts/heimdall-review.js` is a **separate copy** — sync prompt/parse/render changes.
2. **Prompt is the single source**: `src/review/prompt.ts`. Quality is driven by it; change carefully + add tests.
3. **Default on-demand**: unset `auto_review` = no auto review; `@CoderHeimdall` only.
4. **Triple dedup**: `hasExistingReview` (review query) + `heimdall/reviewed` commit status (needs App `statuses` perm) + Worker module cache.
5. **Cloudflare gotchas**: explicit `import { Buffer }`; GitHub API needs `User-Agent`; free `waitUntil` 30s (use `thinking: { type: "disabled" }`).
6. **Report language**: `REVIEW_LANGUAGE` = `en` (default) / `zh` / `bilingual`; labels in `src/review/parse.ts`.
2. **Observability mirror**: `src/observability.ts` (TS) and `scripts/observability.js` (CommonJS) are the same logic — keep them in sync when changing the observer API/events.
3. **Prompt is the single source**: `src/review/prompt.ts`. Quality is driven by it; change carefully + add tests.
4. **Default on-demand**: unset `auto_review` = no auto review; `@CoderHeimdall` only.
5. **Triple dedup**: `hasExistingReview` (review query) + `heimdall/reviewed` commit status (needs App `statuses` perm) + Worker module cache.
6. **Cloudflare gotchas**: explicit `import { Buffer }`; GitHub API needs `User-Agent`; free `waitUntil` 30s (use `thinking: { type: "disabled" }`).
7. **Report language**: `REVIEW_LANGUAGE` = `en` (default) / `zh` / `bilingual`; labels in `src/review/parse.ts`.

## Report Structure (rendered by parse.ts)

Expand Down
46 changes: 46 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,7 @@ Open a PR and comment `@CoderHeimdall` (or `@heimdall`) to see the review. For a
mkdir -p <target>/.github/workflows <target>/scripts
cp template/heimdall-review.yml <target>/.github/workflows/
cp scripts/heimdall-review.js <target>/scripts/
cp scripts/observability.js <target>/scripts/
```

### 2. Configure AI
Expand Down Expand Up @@ -211,6 +212,10 @@ AI_MODEL=claude-sonnet-5
| Per-provider base | `ANTHROPIC_BASE_URL` / `OPENAI_BASE_URL` / `GEMINI_BASE_URL` | env / Variable |
| **Report language** | `REVIEW_LANGUAGE = en \| zh \| bilingual` (**default en**) | env / Variable |
| Diff length cap | `MAX_DIFF_LENGTH` (default 40000) | env / `wrangler.toml [vars]` |
| Observability: detailed logs | `HEIMDALL_LOG_ENABLED` (**default true**) | env / Variable / `wrangler.toml [vars]` |
| Observability: per-review summary | `HEIMDALL_INVOCATION_LOGS` (**default true**) | env / Variable / `wrangler.toml [vars]` |
| Observability: log level | `HEIMDALL_LOG_LEVEL = error \| warn \| info \| debug` (**default info**) | env / Variable / `wrangler.toml [vars]` |
| Observability: per-repo override | `observability.logs.enabled / invocation_logs` | `.github/heimdall.yml` |
| Only review some files | `include: ["*.ts", ...]` | `.github/heimdall.yml` |
| Exclude files | `exclude: [...]` | `.github/heimdall.yml` |
| Min severity shown | `min_severity: important` | `.github/heimdall.yml` |
Expand All @@ -232,10 +237,51 @@ manual_reviewers:
- octocat
block_on_critical: true
auto_review: true # default is on-demand only

# Per-repo observability override (defaults come from env, see §Observability)
observability:
logs:
enabled: true
invocation_logs: true
```

---

## Observability

Heimdall emits **JSON-lines** structured logs to stdout/console — GitHub Actions workflow logs, Cloudflare Workers Logs, or self-hosted stdout — one line per event, tied together by a per-review `reviewId`.

**Toggles (operator default via env, per-repo override via `.github/heimdall.yml`):**

| Env | Default | Meaning |
| --- | --- | --- |
| `HEIMDALL_LOG_ENABLED` | `true` | Master switch for detailed stage logs (`review.*`, `llm.*`) |
| `HEIMDALL_INVOCATION_LOGS` | `true` | Always-on one-line summary per review (`review.invocation`) |
| `HEIMDALL_LOG_LEVEL` | `info` | Filter: `error \| warn \| info \| debug` (affects detailed logs only) |

**Per-repo override** (in the target repo's `.github/heimdall.yml`):

```yaml
observability:
logs:
enabled: false # turn off detailed logs for this repo
invocation_logs: true
```

`warn`/`error` are **always emitted** (a failure is never hidden); `enabled: false` silences only the info/debug detail.

**Key events** — diagnose "why was this PR skipped/failed":
- `review.skip` with `reason`: `draft_pr` · `bot_pr` · `not_auto_review` · `reviewer_not_whitelisted` · `dup_review` · `dup_cache` · `dup_status` · `missing_api_key` · `empty_diff` · `non_pr_event` · `no_trigger_comment`
- `review.error` with `reason`: `llm_error` · `parse_failed` · `post_inline_failed`
- Stage events: `review.start` → `review.config` → `review.diff` (debug) → `llm.start`/`llm.done` → `review.parse` → `review.post` → `review.complete`
- `review.invocation` — one summary line per review (outcome, `durationMs`, issue counts)

Example line:

```json
{"ts":"2026-08-16T02:40:00.000Z","level":"info","event":"review.skip","mode":"worker","repo":"octocat/hello-world","pr":12,"sha":"abc1234","reviewId":"h-x1y2z3","reason":"not_auto_review","msg":"默认仅按需审查,跳过自动审查"}
```

## Report Style

```markdown
Expand Down
48 changes: 47 additions & 1 deletion README.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ cp scripts/heimdall-review.js scripts/
| 定位 | 单仓库自用、快速接入 | 团队多仓库 / 产品化分发 |
| 需要注册 GitHub App | 否 | 是 |
| 需要服务器 | 否 | 否(Cloudflare 边缘) |
| 安装方式 | 复制 2 个文件到目标仓库 | 安装 GitHub App |
| 安装方式 | 复制 3 个文件到目标仓库 | 安装 GitHub App |
| 部署成本 | 免费 | 免费额度内(大 diff 建议 Pro) |

- **只想给自己的仓库加个 AI reviewer** → 模式 A,2 分钟
Expand All @@ -73,6 +73,7 @@ cp scripts/heimdall-review.js scripts/
mkdir -p <目标仓库>/.github/workflows <目标仓库>/scripts
cp template/heimdall-review.yml <目标仓库>/.github/workflows/
cp scripts/heimdall-review.js <目标仓库>/scripts/
cp scripts/observability.js <目标仓库>/scripts/
```

### 2. 配置 AI
Expand Down Expand Up @@ -216,6 +217,10 @@ AI_MODEL=claude-sonnet-5 # 你的网关支持的模型 ID
| 走代理网关 / 本地模型 | `AI_BASE_URL = https://<网关>` | 环境变量 / Actions Variable |
| 提供方专属 base_url | `ANTHROPIC_BASE_URL` / `OPENAI_BASE_URL` / `GEMINI_BASE_URL` | 环境变量 / Actions Variable |
| diff 长度上限 | `MAX_DIFF_LENGTH`(默认 40000)| 环境变量 / `wrangler.toml [vars]` |
| 可观测:详细日志总开关 | `HEIMDALL_LOG_ENABLED`(**默认 true**)| 环境变量 / Actions Variable / `wrangler.toml [vars]` |
| 可观测:每次审查调用摘要 | `HEIMDALL_INVOCATION_LOGS`(**默认 true**)| 环境变量 / Actions Variable / `wrangler.toml [vars]` |
| 可观测:日志级别 | `HEIMDALL_LOG_LEVEL = error \| warn \| info \| debug`(**默认 info**)| 环境变量 / Actions Variable / `wrangler.toml [vars]` |
| 可观测:仓库级覆盖 | `observability.logs.enabled / invocation_logs` | `.github/heimdall.yml` |
| 只审查某些文件 | `include: ["*.ts", ...]` | `.github/heimdall.yml` |
| 排除某些文件 | `exclude: ["**/generated/**", ...]` | `.github/heimdall.yml` |
| 只显示 ≥ 某严重度 | `min_severity: important` | `.github/heimdall.yml` |
Expand Down Expand Up @@ -258,10 +263,51 @@ block_on_critical: true

# 设为 true 时开启自动审查;默认不配置 = 仅手动触发(@CoderHeimdall)
auto_review: true

# 仓库级可观测性覆盖(默认来自环境变量,见下方「可观测性」小节)
observability:
logs:
enabled: true
invocation_logs: true
```

---

## 可观测性

海姆达尔向 stdout/console 输出 **JSON-lines** 结构化日志(GitHub Actions workflow 日志、Cloudflare Workers Logs、或自托管 stdout),一行一个事件,用每次审查的 `reviewId` 关联。

**开关(环境变量设运维默认,`.github/heimdall.yml` 可逐仓库覆盖):**

| 环境变量 | 默认 | 含义 |
| --- | --- | --- |
| `HEIMDALL_LOG_ENABLED` | `true` | 详细阶段日志总开关(`review.*`、`llm.*`)|
| `HEIMDALL_INVOCATION_LOGS` | `true` | 每次审查固定一行调用摘要(`review.invocation`)|
| `HEIMDALL_LOG_LEVEL` | `info` | 级别过滤:`error \| warn \| info \| debug`(只影响详细日志)|

**仓库级覆盖**(在被审查仓库的 `.github/heimdall.yml`):

```yaml
observability:
logs:
enabled: false # 本仓库关掉详细日志
invocation_logs: true
```

`warn`/`error` **始终输出**(失败永不隐藏);`enabled: false` 只关掉 info/debug 细节。

**关键事件** —— 诊断「这个 PR 为什么跳过 / 失败」:
- `review.skip` + `reason`:`draft_pr` · `bot_pr` · `not_auto_review` · `reviewer_not_whitelisted` · `dup_review` · `dup_cache` · `dup_status` · `missing_api_key` · `empty_diff` · `non_pr_event` · `no_trigger_comment`
- `review.error` + `reason`:`llm_error` · `parse_failed` · `post_inline_failed`
- 阶段事件:`review.start` → `review.config` → `review.diff`(debug)→ `llm.start`/`llm.done` → `review.parse` → `review.post` → `review.complete`
- `review.invocation` —— 每次审查一行摘要(outcome、`durationMs`、各级别问题数)

示例行:

```json
{"ts":"2026-08-16T02:40:00.000Z","level":"info","event":"review.skip","mode":"worker","repo":"octocat/hello-world","pr":12,"sha":"abc1234","reviewId":"h-x1y2z3","reason":"not_auto_review","msg":"默认仅按需审查,跳过自动审查"}
```

## 审查报告样式

```markdown
Expand Down
161 changes: 161 additions & 0 deletions docs/superpowers/specs/2026-08-16-observability-design.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,161 @@
# Heimdall 可观测性设计(Observability Design)

| 项 | 值 |
| --- | --- |
| 日期 | 2026-08-16 |
| 状态 | 已批准(待实现) |
| 目标 | 诊断审查失败与跳过(diagnose failures & skips) |
| 范围 | 三种运行时(Probot / Cloudflare Workers / GitHub Actions) |

---

## 1. 背景与目标

当前 Heimdall 的可观测性极弱:三种运行时里散落着中文 `console.log` / `console.error`,没有时间戳、没有关联 ID、没有耗时、没有结构化数据。GitHub 状态检查(`heimdall/reviewed`、`heimdall/critical`)是唯一的机器可读信号。

**核心目标**:让每个「为什么跳过 / 为什么失败」都能被一条结构化日志回答。次目标:阶段耗时可见(辅助诊断慢审查)。

**非目标**(用户明确排除):聚合指标(usage/quality metrics)、外部导出(OTel/Datadog/webhook)、按仓库统计成本。

## 2. 关键决策记录

| 决策点 | 结论 |
| --- | --- |
| 日志格式 | **JSON-lines**,一行一个事件 |
| 覆盖模式 | **全部三种**运行时 |
| 级别控制 | 环境变量 `HEIMDALL_LOG_LEVEL`(`error\|warn\|info\|debug`,默认 `info`) |
| 实现方案 | **方案 A**:共享零依赖 observability 模块 |
| Actions 镜像 | **方案 A**:独立 `scripts/observability.js`,`heimdall-review.js` require 它(复制文件数 2 → 3) |
| 配置位置 | **两者结合**:环境变量设默认,`.github/heimdall.yml` 可逐仓库覆盖 |
| `enabled` 语义 | **详细日志默认开**;`invocation_logs` 为始终开启的调用摘要 |

## 3. 配置模型

**优先级(高 → 低)**:仓库级 `.github/heimdall.yml` > 环境变量 > 默认值。

### 3.1 环境变量(运维默认)

| 变量 | 默认 | 作用 |
| --- | --- | --- |
| `HEIMDALL_LOG_ENABLED` | `true` | 详细事件日志总开关 |
| `HEIMDALL_INVOCATION_LOGS` | `true` | 每条审查固定一行调用摘要 |
| `HEIMDALL_LOG_LEVEL` | `info` | 详细日志过滤级别:`error\|warn\|info\|debug` |

### 3.2 仓库级覆盖(`.github/heimdall.yml`)

```yaml
observability:
logs:
enabled: false # 本仓库关掉详细日志
invocation_logs: true
```

### 3.3 语义

- `logs.enabled = true`(默认)→ 输出各阶段 info/debug 事件
- `logs.enabled = false` → 只保留 `warn`/`error`(失败永远可见)和调用摘要
- `invocation_logs = true`(默认)→ 每次审查固定一行 `review.invocation`(repo/pr/sha/耗时/结果/问题数),与 `enabled` 无关
- `HEIMDALL_LOG_LEVEL` 只过滤详细日志,不影响摘要

**解析器约束**:手写 YAML 解析器(`src/review/repo-config.ts` 及 Actions 副本)当前不支持嵌套 map,需扩展 `observability:` 块的递归解析;`RepoConfig` 新增 `observability` 字段。Worker 的 `Env` 接口新增三个变量。

## 4. 共享模块与输出格式

**`src/observability.ts`**(零依赖,镜像到 `scripts/observability.js`):

```ts
createObserver({ mode, enabled, invocationLogs, level }) → Observer
.info/.warn/.error/.debug(event, msg, fields?)
.start() → Span // finish(event, fields?) 自动带 durationMs
.child({ repo, pr, sha }) → Observer // 上下文绑定
```

每行 JSON 格式:

```json
{"ts":"2026-08-16T02:40:00.000Z","level":"info","event":"review.start","mode":"worker","repo":"octocat/hello-world","pr":12,"sha":"abc1234","msg":"开始审查"}
```

底层走 `console.log` / `console.error`:Probot 落 stdout,Worker 落 Workers Logs,Actions 落 workflow 日志。不引入任何依赖。

## 5. 事件目录

### 5.1 阶段事件

`review.start` → `review.config` → `review.diff` → `llm.start` → `llm.done` → `review.parse` → `review.post` → `review.complete`

阶段事件携带:`durationMs`、`provider`、`model`、问题数(critical/important/normal)、文件数、diff 字节数(字节数细节放 debug)。

### 5.2 跳过事件 `review.skip`(机器可读 `reason`)

| reason | 含义 |
| --- | --- |
| `draft_pr` | 草稿 PR |
| `bot_pr` | 机器人发起的 PR/评论 |
| `not_auto_review` | 默认仅按需审查,非自动触发 |
| `reviewer_not_whitelisted` | 触发者不在 manual_reviewers |
| `dup_review` | 同 commit 已有 review(hasExistingReview) |
| `dup_cache` | Worker 模块级缓存命中 |
| `dup_status` | 已有 heimdall/reviewed 成功状态 |
| `missing_api_key` | 未配置 AI 密钥 |
| `empty_diff` | 无可审查变更 |
| `non_pr_event` | 非 PR 事件 |
| `no_trigger_comment` | 评论未匹配触发词 |

### 5.3 失败事件 `review.error`(`reason`)

| reason | 含义 |
| --- | --- |
| `llm_error` | LLM 调用失败(HTTP 状态、缺 key、超时) |
| `parse_failed` | 结构化解析失败,降级为整体报告 |
| `post_inline_failed` | 行内评论发布失败,降级为整体报告 |

### 5.4 调用摘要 `review.invocation`

受 `invocation_logs` 控制,独立于 `enabled` 与级别。一行包含:repo / pr / sha / trigger / durationMs / outcome(posted / skipped / empty / failed / parse_fallback)/ 问题数。

## 6. 各运行时接线

| 运行时 | 接线 |
| --- | --- |
| Probot | `src/review/index.ts` 在 `runReview` 内创建 observer;`src/app.ts` 中 `runReview` **之前**的跳过也用 observer(草稿/机器人/非自动/白名单) |
| Worker | `import "../src/observability"`;`Env` 接口加三变量;替换现有 console 调用 |
| Actions | `scripts/observability.js` 镜像,`heimdall-review.js` `require`;复制文件数 2 → 3,README 快速开始同步 |

**顺序细节**:draft/bot 等跳过发生在读取 `.github/heimdall.yml` 之前,此时仅环境变量默认生效;仓库级覆盖在配置加载后、后续事件前应用。`warn`/`error` 始终输出。

## 7. 测试

`test/observability.test.js`(node:test,跑编译后的 `lib/`):

- JSON 行格式(可解析、含 ts/level/event/msg)
- 级别过滤(info 时 debug 被过滤;error 恒输出)
- `Span.finish` 自动带 `durationMs`
- `enabled=false` 时只出调用摘要 + warn/error
- `child()` 上下文继承
- YAML 解析器对 `observability:` 嵌套块的支持
- 现有 `npm test` 全部保持通过

## 8. 文档清单

- `.env.example`:加三个变量及注释
- `README.md` / `README.zh-CN.md`:新增 Observability 小节(事件/reason 码、`observability` 块、三变量)
- `AGENTS.md`:observability 模块与 `scripts/observability.js` 镜像同步约定
- 本文档提交至 `docs/superpowers/specs/2026-08-16-observability-design.md`

## 9. 文件改动清单

**新增**:
- `src/observability.ts`
- `scripts/observability.js`
- `test/observability.test.js`
- `docs/superpowers/specs/2026-08-16-observability-design.md`

**修改**:
- `src/review/index.ts`(接线 + 迁移 console)
- `src/app.ts`(跳过事件)
- `worker/index.ts`(接线 + Env + 迁移)
- `scripts/heimdall-review.js`(require 镜像 + 迁移)
- `template/heimdall-review.yml`(注释提及第三个文件)
- `src/review/repo-config.ts` + Actions 副本(YAML 嵌套解析 + `observability` 字段)
- `README.md` / `README.zh-CN.md` / `.env.example` / `AGENTS.md`
Loading
Loading