Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
3be61a8
refactor(errors): move WizardError next to the error codes
gewenyu99 Sep 22, 2026
9ce7899
refactor(agent): own the progress payload types
gewenyu99 Sep 22, 2026
f15dc7a
refactor(agent): own the outro, question and task-notice shapes
gewenyu99 Sep 22, 2026
53bddf8
refactor(programs): split ProgramRun into the agent definition and th…
gewenyu99 Sep 22, 2026
54f8e4b
refactor(programs): move authenticate and the token refresh to programs
gewenyu99 Sep 22, 2026
85ed50e
refactor(shared): name program documents and the audit ledger outside…
gewenyu99 Sep 22, 2026
a88c581
refactor(shared): move the Node package-manager detector next to its …
gewenyu99 Sep 22, 2026
0b98bd3
refactor(middleware): report benchmark lines as progress events
gewenyu99 Sep 22, 2026
856b613
refactor(yara): return the scan report line instead of printing it
gewenyu99 Sep 22, 2026
13fa0a2
refactor(agent): publish the handoff as a progress event
gewenyu99 Sep 22, 2026
5cffae6
build(lint): fence the agent's imports
gewenyu99 Sep 22, 2026
0cf68a2
Merge posthog/functional-a1-min into posthog/functional-a2a-edges
gewenyu99 Sep 23, 2026
112fce5
Merge posthog/functional-a1-min into posthog/functional-a2a-edges
gewenyu99 Sep 23, 2026
dcf418a
fix(lint): fence directory imports out of the agent too
gewenyu99 Sep 23, 2026
3c7dfb5
test(agent): cover the handoff, benchmark and scan-summary wiring
gewenyu99 Sep 23, 2026
c26e22d
Merge posthog/functional-a1-min into posthog/functional-a2a-edges
gewenyu99 Sep 23, 2026
d81afb9
Merge posthog/functional-a1-min into posthog/functional-a2a-edges
gewenyu99 Sep 23, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
75 changes: 75 additions & 0 deletions .eslintrc.cjs

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will thrash a lot, so consider this a transitional state

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is: A2b narrows this fence to src/agent/**, and C2d deletes it for per-layer TypeScript configs, where the agent's paths map only @env, @shared and @utils:

"paths": {
"@env": [
"../../.tsbuild/env/env.d.ts"
],
"@shared/*": [
"../../.tsbuild/shared/*"
],
"@utils/*": [
"../../.tsbuild/shared/utils/*"
],

Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,81 @@ module.exports = {
'prettier',
],
overrides: [
{
// The agent surface. It takes resolved data in, reports through
// progress events and asks through an injected answerer, so nothing
// here may import a UI, the session, detection, the CLI or a program.
// Only direct static imports are checked. Program types stay importable
// until B1 moves PROGRAM_BINDINGS to programs. Today's paths; A2b
// collapses them to src/agent/**.
files: [
'src/lib/agent/**/*.ts',
'src/lib/middleware/**/*.ts',
'src/lib/wizard-tools/**/*.ts',
'src/lib/gateway-session.ts',
'src/lib/safe-tools.ts',
'src/lib/wizard-ask-bridge.ts',
'src/lib/yara-hooks.ts',
'src/lib/yara-policy.ts',
],
excludedFiles: ['**/__tests__/**'],
rules: {
'@typescript-eslint/no-restricted-imports': [

@gewenyu99 gewenyu99 Sep 22, 2026 •

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's the potential issue: The import rule blocks static UI imports but allows the same dependency through a dynamic import.

agent loads the UI with a dynamic import -> rule doesn't inspect it -> forbidden UI dependency passes lint

Suggested fix: Apply the same dependency restrictions to static and dynamic imports.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left for C2d: ESLint's import rule can't see import(), and the per-layer TypeScript configs that replace this fence reject dynamic imports too; the fence comment now says only direct static imports are checked (https://github.com/PostHog/wizard/blob/3c7dfb5a/.eslintrc.cjs).

'error',
{
paths: [
{
name: '@utils/wizard-abort',
importNames: ['wizardAbort'],
message:
'The agent never exits the process: return a failure in RunResult.',
},
],
patterns: [
{
group: [
'@ui',
'@ui/**',
'**/ui',
'**/ui/**',
'@lib/wizard-session',
'**/wizard-session',
'@lib/detection',
'@lib/detection/**',

@gewenyu99 gewenyu99 Sep 22, 2026 •

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's the potential issue: The import rule is meant to keep the agent separate from the UI and other layers, but directory imports can bypass it.

agent imports the UI directory itself -> rule only matches files inside that directory -> forbidden UI dependency passes lint

Suggested fix: Cover directory imports as well as files inside them, for both aliases and relative imports.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed: the fence now also matches the directories themselves and relative steps paths, so ../../ui and @lib/detection fail lint:

wizard/.eslintrc.cjs

Lines 65 to 88 in 3c7dfb5

group: [
'@ui',
'@ui/**',
'**/ui',
'**/ui/**',
'@lib/wizard-session',
'**/wizard-session',
'@lib/detection',
'@lib/detection/**',
'**/detection',
'**/detection/**',
'@lib/registry',
'**/lib/registry',
'@lib/runners',
'@lib/runners/**',
'**/runners',
'**/runners/**',
'**/commands/**',
'@steps',
'@steps/**',
'**/steps',
'**/steps/**',
'@frameworks/**',
'**/frameworks/**',

'**/detection',
'**/detection/**',
'@lib/registry',
'**/lib/registry',
'@lib/runners',
'@lib/runners/**',
'**/runners',
'**/runners/**',
'**/commands/**',
'@steps',
'@steps/**',
'**/steps',
'**/steps/**',
'@frameworks/**',
'**/frameworks/**',
'@utils/setup-utils',
'**/setup-utils',
'@utils/oauth',
'**/utils/oauth',
],
message:
'The agent reports through progress events and asks through AgentInteraction; it takes everything else through RunConfig and RunInput.',
},
{
group: ['@lib/programs/**', '**/programs/**'],
allowTypeImports: true,
message:
'The agent takes program data through RunConfig. Types only, until B1 moves PROGRAM_BINDINGS to programs.',
},
],
},
],
},
},
{
files: [
'*.test.js',
Expand Down
2 changes: 1 addition & 1 deletion scripts/tui-host.no-jest.ts
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ import { configureGatewayFromCIEnvironment } from '@lib/gateway-session';
import { runProgramAgent } from '@lib/programs/run-agent-legacy';
import { TaskStreamPush, createFileDestination } from '@lib/task-stream/index';
import { getAuditChecks } from '@lib/programs/audit/types';
import { authenticate } from '@lib/agent/runner/shared/authenticate';
import { authenticate } from '@lib/programs/authenticate';
import { getOrAskForProjectData } from '@utils/setup-utils';
import { logToFile } from '@utils/debug';
import { join } from 'path';
Expand Down
15 changes: 5 additions & 10 deletions src/__tests__/architecture/known-violations.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,25 +2,19 @@
"violations": [
"src/commands/factories/family-picker.tsx -> src/commands/command.ts",
"src/env.ts -> src/lib/headless-mode.ts",
"src/lib/agent/mcp-prompt-streaming.ts -> src/ui/tui/services/mcp-suggested-prompts-services.ts",
"src/lib/detection/agentic.ts -> src/lib/agent/agent-interface.ts",
"src/lib/detection/project-scope.ts -> src/lib/agent/runner/shared/authenticate.ts",
"src/lib/errors/agent-map.ts -> src/lib/agent/signals.ts",
"src/lib/programs/agent-skill/index.ts -> src/lib/agent/agent-runner.ts",
"src/lib/programs/agent-skill/index.ts -> src/lib/programs/agent-skill/content/index.tsx",
"src/lib/programs/ai-observability/index.ts -> src/lib/programs/agent-skill/content/index.tsx",
"src/lib/programs/audit/detect.ts -> src/lib/agent/agent-runner.ts",
"src/lib/programs/audit/index.ts -> src/lib/agent/agent-runner.ts",
"src/lib/programs/dispatch-family.ts -> src/commands/command.ts",
"src/lib/programs/dispatch-family.ts -> src/commands/factories/shared.ts",
"src/lib/programs/error-tracking-upload-source-maps/detect.ts -> src/lib/agent/agent-runner.ts",
"src/lib/programs/error-tracking-upload-source-maps/index.ts -> src/lib/agent/agent-runner.ts",
"src/lib/programs/error-tracking-upload-source-maps/index.ts -> src/lib/programs/error-tracking-upload-source-maps/content/index.tsx",
"src/lib/programs/error-tracking-upload-source-maps/prompt.ts -> src/lib/agent/agent-interface.ts",
"src/lib/programs/error-tracking/index.ts -> src/lib/agent/runner/shared/types.ts",
"src/lib/programs/error-tracking/index.ts -> src/lib/programs/error-tracking/content/index.tsx",
"src/lib/programs/error-tracking/index.ts -> src/lib/programs/error-tracking/content/tips.ts",
"src/lib/programs/events-audit/index.ts -> src/lib/agent/agent-runner.ts",
"src/lib/programs/mcp-analytics/index.ts -> src/lib/agent/agent-runner.ts",
"src/lib/programs/metrics/index.ts -> src/lib/programs/agent-skill/content/index.tsx",
"src/lib/programs/migration/index.ts -> src/lib/agent/agent-runner.ts",
Expand All @@ -29,7 +23,7 @@
"src/lib/programs/posthog-integration/index.ts -> src/lib/agent/agent-runner.ts",
"src/lib/programs/posthog-integration/index.ts -> src/lib/programs/posthog-integration/content/index.tsx",
"src/lib/programs/program-registry.ts -> src/lib/programs/agent-skill/content/index.tsx",
"src/lib/programs/program-step.ts -> src/lib/agent/agent-runner.ts",
"src/lib/programs/program-run.ts -> src/lib/agent/runner/index.ts",
"src/lib/programs/program-step.ts -> src/ui/tui/components/TipsCard.tsx",
"src/lib/programs/program-step.ts -> src/ui/tui/primitives/index.ts",
"src/lib/programs/program-step.ts -> src/ui/tui/store.ts",
Expand All @@ -39,23 +33,23 @@
"src/lib/programs/run-agent-legacy.ts -> src/lib/agent/agent-interface.ts",
"src/lib/programs/run-agent-legacy.ts -> src/lib/agent/claude-settings.ts",
"src/lib/programs/run-agent-legacy.ts -> src/lib/agent/runner/index.ts",
"src/lib/programs/run-agent-legacy.ts -> src/lib/agent/runner/shared/authenticate.ts",
"src/lib/programs/run-agent-legacy.ts -> src/lib/agent/runner/switchboard/index.ts",
"src/lib/programs/run-agent-legacy.ts -> src/lib/yara-hooks.ts",
"src/lib/programs/self-driving/detect.ts -> src/lib/agent/agent-runner.ts",
"src/lib/programs/self-driving/index.ts -> src/lib/agent/agent-runner.ts",
"src/lib/programs/self-driving/index.ts -> src/lib/programs/self-driving/content/index.tsx",
"src/lib/programs/self-driving/index.ts -> src/lib/programs/self-driving/content/pricing.ts",
"src/lib/programs/self-driving/index.ts -> src/lib/programs/self-driving/content/tips.ts",
"src/lib/programs/self-driving/prompt.ts -> src/lib/agent/agent-interface.ts",
"src/lib/programs/self-driving/prompt.ts -> src/lib/agent/agent-runner.ts",
"src/lib/programs/warehouse-source/detect.ts -> src/lib/agent/agent-runner.ts",
"src/lib/programs/warehouse-source/index.ts -> src/lib/agent/agent-runner.ts",
"src/lib/programs/warehouse-source/index.ts -> src/lib/programs/warehouse-source/content/index.tsx",
"src/lib/programs/web-analytics-doctor/detect.ts -> src/lib/agent/agent-runner.ts",
"src/lib/task-stream/event-plan-watcher.ts -> src/ui/tui/store.ts",
"src/lib/task-stream/task-stream-push.ts -> src/ui/tui/store.ts",
"src/lib/wizard-ask-bridge.ts -> src/lib/agent/progress.ts",
"src/lib/wizard-session.ts -> src/lib/agent/claude-settings.ts",
"src/lib/wizard-session.ts -> src/lib/agent/progress.ts",
"src/lib/wizard-tools/handoff.ts -> src/lib/agent/progress.ts",
"src/lib/wizard-tools/index.ts -> src/lib/wizard-tools/mcp.ts",
"src/lib/wizard-tools/tools.ts -> src/lib/yara-hooks.ts",
"src/steps/add-mcp-server-to-clients/index.ts -> src/telemetry.ts",
Expand All @@ -75,6 +69,7 @@
"src/ui/tui/store.ts -> src/lib/agent/claude-settings.ts",
"src/ui/tui/store.ts -> src/lib/agent/token-pricing.ts",
"src/ui/wizard-ui.ts -> src/lib/agent/claude-settings.ts",
"src/ui/wizard-ui.ts -> src/lib/agent/progress.ts",
"src/utils/package-manager.ts -> src/telemetry.ts",
"src/utils/setup-utils.ts -> src/telemetry.ts",
"src/utils/wizard-abort.ts -> src/ui/logging-ui.ts"
Expand Down
58 changes: 58 additions & 0 deletions src/lib/__tests__/yara-flush-report.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
import fs from 'fs';
import { WIZARD_YARA_REPORT_FILE } from '@utils/paths';

// The flush runs inside the agent, which has no UI: the report line is
// returned to the caller, who decides where it goes.
vi.mock('@ui', () => ({
getUI: () => {
throw new Error('agent code reached the UI');
},
}));
vi.mock('@utils/debug');
vi.mock('@utils/analytics', () => ({
analytics: { wizardCapture: vi.fn(), captureException: vi.fn() },
}));
vi.mock('fs', async (importOriginal) => {
const actual = await importOriginal<typeof import('fs')>();
return {
...actual,
default: { ...actual, writeFileSync: vi.fn() },
writeFileSync: vi.fn(),
};
});

import {
flushScanReport,
recordExternalScan,
resetScanReport,
} from '@lib/yara-hooks';

describe('flushScanReport', () => {
beforeEach(() => {
resetScanReport();
vi.mocked(fs.writeFileSync).mockClear();
});

it('returns the report line once when a report was requested', () => {
recordExternalScan('PostToolUse', 'Write', [], 'warned');

const line = flushScanReport({ yaraReport: true });

expect(line).toContain(`YARA scan report: ${WIZARD_YARA_REPORT_FILE}`);
expect(line).toContain('1 tool calls scanned, 0 violations detected');
expect(fs.writeFileSync).toHaveBeenCalledWith(
WIZARD_YARA_REPORT_FILE,
expect.stringContaining('"totalScans": 1'),
);
// Idempotent: the first flush zeroed the scan state.
expect(flushScanReport({ yaraReport: true })).toBeUndefined();
});

it('returns nothing without --yara-report, but still flushes the state', () => {
recordExternalScan('PostToolUse', 'Write', [], 'warned');

expect(flushScanReport({ yaraReport: false })).toBeUndefined();
expect(fs.writeFileSync).not.toHaveBeenCalled();
expect(flushScanReport({ yaraReport: true })).toBeUndefined();
});
});
6 changes: 4 additions & 2 deletions src/lib/agent/__tests__/agent-prompt.test.ts
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
import { assemblePrompt, type PromptContext } from '@lib/agent/agent-prompt';
import type { ProgramRun } from '@lib/agent/agent-runner';
import type { AgentRunDefinition } from '@lib/agent/runner';
import { HostResolution } from '@lib/host-resolution';

function makeRunDef(overrides: Partial<ProgramRun> = {}): ProgramRun {
function makeRunDef(
overrides: Partial<AgentRunDefinition> = {},
): AgentRunDefinition {
return {
integrationLabel: 'test',
spinnerMessage: 'Working...',
Expand Down
8 changes: 8 additions & 0 deletions src/lib/agent/__tests__/progress-collector.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -37,3 +37,11 @@ it('isolates nested completion data from a mutating observer', () => {
nextSteps: { heading: 'Next', items: ['Keep the report'] },
});
});

it('keeps the published handoff text in the snapshot', () => {
const collector = createProgressCollector();
expect(collector.snapshot().handoffText).toBeUndefined();
collector.emit({ kind: 'handoff', text: '# Report' });
collector.emit({ kind: 'handoff', text: '# Report, revised' });
expect(collector.snapshot().handoffText).toBe('# Report, revised');
});
88 changes: 88 additions & 0 deletions src/lib/agent/__tests__/run-agent-standalone.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -131,6 +131,15 @@ vi.mock('@lib/agent/runner/switchboard/harness', () => {
},
});
await askIfRequested(inputs);
// A real harness feeds the benchmark middleware every SDK message.
inputs.middleware?.onMessage({
type: 'assistant',
message: { role: 'assistant', content: [{ type: 'text', text: 'hi' }] },
});
inputs.middleware?.finalize(
{ type: 'result', modelUsage: {}, num_turns: 1 },
10,
);
if (harnessState.throws) throw harnessState.throws;
spinner.stop('Done');
return harnessState.result as never;
Expand Down Expand Up @@ -443,6 +452,7 @@ describe('runAgent standalone', () => {
});
vi.mocked(flushScanReport).mockImplementationOnce(() => {
order.push('scan-flush');
return undefined;
});
const result = await runAgent(
config({
Expand Down Expand Up @@ -684,4 +694,82 @@ describe('runAgent standalone', () => {
// What was reported before the crash survives in the snapshot.
expect(result.snapshot.statusMessages).toContain('Installing the SDK');
});

it('sends benchmark output to onProgress when benchmarking', async () => {
const benchmarkPath = path.join(tmp, 'benchmark.json');
const configPath = path.join(tmp, '.benchmark-config.json');
fs.writeFileSync(
configPath,
JSON.stringify({ output: { benchmarkPath, logEnabled: false } }),
);
vi.stubEnv('POSTHOG_WIZARD_BENCHMARK_CONFIG', configPath);
vi.stubEnv('POSTHOG_WIZARD_BENCHMARK_FILE', benchmarkPath);
vi.stubEnv('POSTHOG_WIZARD_LOG_DIR', tmp);
const runInput = input();
runInput.flags.benchmark = true;
const events: AgentProgress[] = [];
try {
const result = await runAgent(config(), runInput, {
onProgress: (e) => events.push(e),
});

expect(result.outcome).toBe('success');
const logs = events.flatMap((e) => (e.kind === 'log' ? [e.message] : []));
expect(logs).toContainEqual(
expect.stringContaining(
`Benchmark data will be written to: ${benchmarkPath}`,
),
);
expect(logs).toContainEqual(
expect.stringContaining(`Results written to ${benchmarkPath}`),
);
expect(fs.existsSync(benchmarkPath)).toBe(true);
} finally {
vi.unstubAllEnvs();
}
});

it.each<[string, () => void, string]>([
['completes', () => undefined, 'success'],
[
'aborts',
() => {
harnessState.result = {
error: AgentErrorType.ABORT,
message: 'No Stripe found',
};
},
'aborted',
],
[
'crashes',
() => {
harnessState.throws = new Error('SDK exploded');
},
'crashed',
],
])(
'sends the scan summary to onProgress when the run %s',
async (_ending, arrange, outcome) => {
const summary =
'YARA scan report: /tmp/yara.json\n— YARA Scanner Summary —';
vi.mocked(flushScanReport).mockReturnValueOnce(summary);
arrange();
const runInput = input();
runInput.flags.yaraReport = true;
const events: AgentProgress[] = [];

const result = await runAgent(config(), runInput, {
onProgress: (e) => events.push(e),
});

expect(result.outcome).toBe(outcome);
expect(flushScanReport).toHaveBeenCalledWith({ yaraReport: true });
expect(events).toContainEqual({
kind: 'log',
level: 'info',
message: summary,
});
},
);
});
Loading
Loading