Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 16 additions & 8 deletions docs/operate/private-repositories.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,24 +87,32 @@ Every other channel follows the same four rules:
- **Which issue:** found by the marker label, or by its exact title when someone removed the label. Either way it must be one of the action's own reports: the body carries the report heading. A same-titled issue you opened by hand is left alone, unless its body pastes a report verbatim.
- **Whose issue:** the creator does not matter, so a rotated PAT reuses the issue instead of opening a second one. With several reports, an open one wins, then the newest.
- **Each run:** the body is replaced; the issue opens when the target fails or drifts and closes when it is healthy.
- **A long report is cut to fit:** GitHub keeps at most 65,536 characters in an issue body. A longer report keeps every section's status and leaves lines out: from the transcript first, then from the sections with the most detail. It opens by saying how many lines were left out and marks each cut in place, while the `artifact` channel delivers the whole report.
- **What the log says:** every write is announced with the placeholder, never the slug: `report: updated issue #7 in private repository #1` (or `created`), and `report: created label "settings-as-code-report" in private repository #1` when the delivery creates the marker label. A write that landed before a later step failed is announced the same way, ahead of the warning.
- **Permission:** the PAT needs `"Issues"` (read and write) on every target repository, on top of the section permissions.

The first delivery creates the marker label, `settings-as-code-report`. Each run reads it once per target, before that target's settings are checked or applied. Where the settings declare `labels` without naming the marker, the answer decides what that section does with it:

| The read answers | The marker is | Check | Apply |
| ------------------------------ | ------------- | ------------------------------------------------------ | ----------------------------------------------- |
| 404: no such label | left out | says nothing about it | does not create it |
| 200: the label exists | declared | compares it like any declared label | keeps it |
| anything else: the read failed | declared | reports `drift` where the repository has no such label | creates the label where the repository has none |

A failed read counts as a label that exists, so no label is deleted unread. A settings file that names the marker itself is outside the table: its own entry manages the label, and a rename away from the marker's name is dropped unless the read answers 404.

`private-report: issue-on-failure` is the quiet variant of `issue`. When a target fails (or drifts in check mode) it behaves identically: the reused, marker-labelled report issue is written and opened.

On a healthy run it only looks up an open report issue:
On a healthy run it reads the marker label, then looks for an open report issue carrying it:

- **One is open,** left over from an earlier failure: it is updated with the healthy report and closed.
- **None is open:** nothing is written, no issue, no label, zero notification noise, and the log says so: `report: nothing to deliver for private repository #1`.

A repository that never fails never sees an issue, at the cost that healthy runs' reports are not delivered anywhere. Choose `issue` or `artifact` when you need the report mirror on every run.
- **None is open:** the delivery writes nothing, no issue, no label, zero notification noise, and the log says so: `report: nothing to deliver for private repository #1`.

Two caveats on the quiet variant:
A repository that never fails never sees an issue, and no delivery creates the marker label there. The cost is that healthy runs' reports are not delivered anywhere. Choose `issue` or `artifact` when you need the report mirror on every run.

- **A declared `labels` section still gets the marker.** The `settings-as-code-report` marker label is injected into it, so the apply creates the label even on healthy repositories (a label, not a notification).
- **A removed marker label hides an open issue.** If someone removes the label while a report issue is open, the healthy path cannot find the issue, so it stays open until the next failing run recreates the label and reclaims it.
One caveat on the quiet variant: a removed marker label hides an open issue. If someone removes the label while a report issue is open, the healthy path cannot find the issue, so it stays open until the next failing run recreates the label and reclaims it.

Everything else matches `issue`: the same delivery gate, the same Issues grant (the healthy lookup is a read; the open and close are writes), and delivery still runs in `mode: check`.
Everything else matches `issue`: the same delivery gate, the same Issues grant (with no open issue, a healthy run's delivery only reads; the open and the close are writes), and delivery still runs in `mode: check`.

`private-report: artifact` concatenates the report for every proven-private target into one document, encrypts it to an age recipient, and uploads it as the workflow artifact `settings-as-code-private-report` (file `private-report.md.age`). Use it when the people who need the report cannot be given repository access, since the archive travels with the run rather than living in the target repo.

Expand Down
10 changes: 6 additions & 4 deletions src/flows/deliver.ts
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,6 @@ import { describeProblem, type Problem } from "../problem.js";
import type { ArtifactUploader } from "../report/artifact-report.js";
import {
type ClosedOutcome,
isIssueChannel,
openReportChannel,
type PrivateReportChannel,
type RedactedDetail,
Expand Down Expand Up @@ -137,8 +136,9 @@ export interface OpenedTarget {

export interface Delivery {
/**
* `work` is told whether to inject the issue channel's marker label. On close, a redacted target proven private or
* internal gets its report, an unproven one the withheld notice, and its public line is closed-value only.
* `work` is told whether to inject the issue channel's marker label, which the channel answers for a target it will
* report on. On close, a redacted target proven private or internal gets its report, an unproven one the withheld
* notice, and its public line is closed-value only.
*/
target(
opened: OpenedTarget,
Expand Down Expand Up @@ -192,7 +192,9 @@ export async function withDelivery<T>(
reports !== null && exposure.kind === "redacted" && isPrivateVisibility(exposure.visibility);
const delivery: Delivery = {
async target({ repo, channel, exposure }, work) {
const outcome = await work(deliverable(exposure) && isIssueChannel(cfg.privateReport));
const declaresMarker =
reports !== null && deliverable(exposure) && (await reports.declaresMarker(repo));
const outcome = await work(declaresMarker);
return closeTarget(io, channel, outcome, async (detail) => {
if (reports === null) {
return;
Expand Down
119 changes: 107 additions & 12 deletions src/report/composer.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
/**
* One markdown document per redacted target, rendered identically for both channels. Pure, and structurally typed on
* purpose, so it depends on no action-layer types.
* One markdown document per redacted target, rendered identically for both channels, except that a channel whose
* document has a length limit gets its lines cut to fit. Pure, and structurally typed on purpose, so it depends on no
* action-layer types.
*/

import type { AnnotationLevel } from "../io.js";
Expand Down Expand Up @@ -29,6 +30,9 @@ export interface ReportInput {
transcript: TranscriptLine[];
}

/** Joins the lines of one detail cell: a table row is a single line, so the cell breaks its lines in HTML. */
const BREAK = "<br>";

/** A fence longer than any backtick run inside the content, so a transcript line can never terminate the block early. */
function fenceFor(content: string): string {
const longest = content.match(/`+/g)?.reduce((max, run) => Math.max(max, run.length), 0) ?? 0;
Expand All @@ -42,12 +46,34 @@ function transcriptLine(entry: TranscriptLine): string {
/** The report's first line. src/report/issue-report.ts reads it back as the proof that an issue body is one of these reports. */
export const REPORT_HEADING = "# settings-as-code private report:";

export function composeReport(input: ReportInput): string {
/** How many lines of each list a document carries: one count per section's detail, in outcome order, and the transcript's. */
interface Kept {
detail: number[];
transcript: number;
}

/** Stands where a list was cut, in a detail cell and under the transcript alike. */
function leftOutMark(count: number): string {
return `(lines left out: ${count})`;
}

/** The document carrying `kept` lines of each list. A list cut short ends in its mark, and any cut is announced under the heading. */
function render(input: ReportInput, kept: Kept, limit: number): string {
const total = input.outcomes.reduce((sum, o) => sum + o.detail.length, input.transcript.length);
const carried = kept.detail.reduce((sum, count) => sum + count, kept.transcript);
const lines: string[] = [
`${REPORT_HEADING} ${input.target}`,
"",
"Full, unredacted report for this target. The public run redacts it; this document is its private mirror.",
"",
];
if (carried < total) {
lines.push(
`**Cut to fit ${limit} characters.** Lines left out: ${total - carried} of ${total}; each cut is marked where it was made.`,
"",
);
}
lines.push(
renderTable("| | |\n|---|---|", [
["Target", markdownCell(input.target)],
["Admin repository", markdownCell(input.adminRepo)],
Expand All @@ -59,29 +85,98 @@ export function composeReport(input: ReportInput): string {
"",
"## Sections",
"",
];
);
if (input.outcomes.length === 0) {
lines.push("No sections ran for this target.");
} else {
lines.push(
renderTable(
"| Section | Status | Detail |\n|---|---|---|",
input.outcomes.map((outcome) => [
markdownCell(outcome.key),
markdownCell(outcome.status),
outcome.detail.map(markdownCell).join("<br>"),
]),
input.outcomes.map((outcome, row) => {
const shown = outcome.detail.slice(0, kept.detail[row]).map(markdownCell);
const cut = outcome.detail.length - shown.length;
return [
markdownCell(outcome.key),
markdownCell(outcome.status),
(cut > 0 ? [...shown, leftOutMark(cut)] : shown).join(BREAK),
];
}),
),
);
}
lines.push("", "## Transcript", "");
if (input.transcript.length === 0) {
lines.push("No output was captured for this target.");
} else {
const body = input.transcript.map(transcriptLine).join("\n");
const fence = fenceFor(body);
lines.push(fence, body, fence);
const shown = input.transcript.slice(0, kept.transcript).map(transcriptLine);
if (shown.length > 0) {
const body = shown.join("\n");
const fence = fenceFor(body);
lines.push(fence, body, fence);
}
if (shown.length < input.transcript.length) {
lines.push(
...(shown.length > 0 ? [""] : []),
leftOutMark(input.transcript.length - shown.length),
);
}
}
lines.push("");
return lines.join("\n");
}

/**
* The lines the document carries under `limit`: each section's detail first, one line per section in turn, then the
* transcript from its start, each list ending at its first line that does not fit. These lines never push the
* document past the limit. The rest (heading, run table, section statuses, marks) is never cut, so a limit below it
* is not met.
*/
function fit(input: ReportInput, limit: number): Kept {
const kept: Kept = { detail: input.outcomes.map(() => 0), transcript: 0 };
// What a first transcript line adds beyond itself: the two fences, their line ends, and the blank line above the mark.
const fences = 2 * fenceFor(input.transcript.map(transcriptLine).join("\n")).length + 3;
let room = limit - render(input, kept, limit).length - fences;
Comment thread
Vivswan marked this conversation as resolved.
Resolved
const take = (cost: number): boolean => {
if (cost > room) {
return false;
}
room -= cost;
return true;
};
const open = new Set(input.outcomes.keys());
for (let depth = 0; open.size > 0; depth++) {
for (const row of open) {
const line = input.outcomes[row]?.detail[depth];
if (line !== undefined && take(markdownCell(line).length + BREAK.length)) {
kept.detail[row] = depth + 1;
} else {
open.delete(row);
}
}
}
for (const entry of input.transcript) {
if (!take(transcriptLine(entry).length + 1)) {
break;
}
kept.transcript += 1;
}
return kept;
}

function everyLine(input: ReportInput): Kept {
return {
detail: input.outcomes.map((outcome) => outcome.detail.length),
transcript: input.transcript.length,
};
}

/** The whole report, whatever its length. */
export function composeReport(input: ReportInput): string {
return render(input, everyLine(input), Number.POSITIVE_INFINITY);
}

/** The report for a channel whose document holds `limit` characters: over it, lines are left out and the document counts them. */
export function composeReportWithin(input: ReportInput, limit: number): string {
const whole = render(input, everyLine(input), limit);
return whole.length <= limit ? whole : render(input, fit(input, limit), limit);
}
33 changes: 24 additions & 9 deletions src/report/delivery.ts
Original file line number Diff line number Diff line change
Expand Up @@ -20,13 +20,15 @@ import { revealPrivate } from "../private-open.js";
import { describeProblem } from "../problem.js";
import type { SectionKey, SettingsFile } from "../schema.js";
import { type ArtifactUploader, deliverArtifactReport } from "./artifact-report.js";
import { composeReport } from "./composer.js";
import { composeReport, composeReportWithin, type ReportInput } from "./composer.js";
import {
deliverIssueReport,
ISSUE_BODY_LIMIT,
type IssueReportMode,
injectMarkerLabel,
type LandedWrites,
MARKER_LABEL,
mayHoldMarkerLabel,
} from "./issue-report.js";

/** The channels, in the words of the `private-report` input; the type below says what each delivers. */
Expand Down Expand Up @@ -90,9 +92,10 @@ export interface ReportRunMeta {
}

/**
* `on` is false when the channel is off or the target is not redacted. The notice is returned rather than emitted, so
* the caller can route it through the target's capturing sink. An injected document is a new document, so it earns
* its brand where every document does: through validateSettingsDoc, the brand's one mint.
* `on` is what the target's channel answered (ReportChannel.declaresMarker): false when no issue channel reports on
* the target, or when the target holds no marker label to keep. The notice is returned rather than emitted, so the
* caller can route it through the target's capturing sink. An injected document is a new document, so it earns its
* brand where every document does: through validateSettingsDoc, the brand's one mint.
*/
export function applyMarkerInjection(
settings: ValidatedSettings,
Expand Down Expand Up @@ -168,7 +171,16 @@ interface ReportTarget {
* content) and never changes any target's result.
*/
export interface ReportChannel {
/**
* Whether the target's settings must declare the marker label (applyMarkerInjection), asked before the target's
* work. Only the issue channel has a label to keep, and it says no for a target it reads as holding none.
*/
declaresMarker(repo: RepoRef | null): Promise<boolean>;

/** Delivers one target's report once the target's work is done. */
deliver(target: ReportTarget): Promise<void>;

/** Ends the run's deliveries; a channel that gathers its reports sends them here. */
flush(): Promise<void>;
}

Expand Down Expand Up @@ -202,9 +214,9 @@ export function openReportChannel(
}

/** The seal opens in full here: the readers are the target repository's own, or hold the artifact's decryption key. */
function composeTargetReport(meta: ReportRunMeta, target: ReportTarget): string {
function reportOf(meta: ReportRunMeta, target: ReportTarget): ReportInput {
const { slug, outcomes, transcript } = revealPrivate(target.detail);
return composeReport({
return {
target: slug,
adminRepo: meta.adminRepo,
runUrl: meta.runUrl,
Expand All @@ -213,7 +225,7 @@ function composeTargetReport(meta: ReportRunMeta, target: ReportTarget): string
timestamp: meta.timestamp,
outcomes: outcomes.map((o) => ({ key: o.key, status: o.status, detail: o.detail })),
transcript,
});
};
}

function issueChannel(
Expand All @@ -223,6 +235,8 @@ function issueChannel(
io: Io,
): ReportChannel {
return {
// An unparseable slug names no repository to read, and no settings are applied to it either.
declaresMarker: async (repo) => repo !== null && (await mayHoldMarkerLabel(api, repo)),
async deliver(target) {
if (target.repo === null) {
// The issue channel posts INTO the target repository, and an unparseable slug names none; the loss must not be silent.
Expand All @@ -232,7 +246,7 @@ function issueChannel(
);
return;
}
const body = composeTargetReport(meta, target);
const body = composeReportWithin(reportOf(meta, target), ISSUE_BODY_LIMIT);
const delivery = await deliverIssueReport(
api,
target.repo,
Expand Down Expand Up @@ -279,8 +293,9 @@ function artifactChannel(
): ReportChannel {
const reports: string[] = [];
return {
declaresMarker: async () => false,
async deliver(target) {
reports.push(`<!-- ${target.display} -->\n\n${composeTargetReport(meta, target)}`);
reports.push(`<!-- ${target.display} -->\n\n${composeReport(reportOf(meta, target))}`);
},
async flush() {
if (reports.length === 0) {
Expand Down
26 changes: 26 additions & 0 deletions src/report/issue-report.ts
Original file line number Diff line number Diff line change
Expand Up @@ -56,8 +56,15 @@ export const ISSUE_REPORT_ENDPOINTS = {
route: "POST /repos/{owner}/{repo}/labels",
statuses: { 201: "marker label created", 422: "the marker label already exists" },
},
readLabel: {
route: "GET /repos/{owner}/{repo}/labels/{name}",
statuses: { 200: "the target holds the marker label", 404: "the target holds no marker label" },
},
} as const satisfies Record<string, EndpointDecl>;

/** GitHub's cap on an issue body, from the 422 a longer one earns every time ("body is too long (maximum is 65536 characters)"); its REST reference states none. A string's length in UTF-16 units is never below that count. */
export const ISSUE_BODY_LIMIT = 65536;

export type IssueReportMode = "always" | "on-failure";

/**
Expand Down Expand Up @@ -433,6 +440,25 @@ export async function deliverIssueReport(
}
}

/**
* False only when the target answers 404 for the marker label. Declared there, the label would be created by the
* labels section, and a check would call a repository no report ever touched drifted. Any other answer, a failed read
* included, declares it without printing anything, so no label is deleted unread and the delivery that follows
* reports a failure that persists.
*/
export async function mayHoldMarkerLabel(api: GitHubClient, repo: RepoRef): Promise<boolean> {
try {
const label = await api.tryRequest(
"GET",
expand(ISSUE_REPORT_ENDPOINTS.readLabel, { repo }, { name: MARKER_LABEL }),
);
return !("error" in label && label.error.status === 404);
} catch {
// A client that throws breaks the GitHubClient contract; its message embeds the request path (the private slug), so nothing from it may escape.
return true;
}
}

/**
* A declared `labels` section under the delete policy would DELETE the marker label an earlier run's report delivery
* created, so the marker joins the declared set unless an entry already manages it. An entry renaming the marker AWAY
Expand Down
Loading
Loading