Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,9 +52,9 @@ Configuration lives at exactly one of two levels: in the package (`eslint.config
| [`graph-projection`](packages/core/graph/projection/README.md) | Projections of the model into a flowchart, swimlane, state machine and sequence diagram, plus derived hand-offs, and the inverses that turn an edited view into a diff against the model. |
| [`graph-domain`](packages/core/graph/domain/README.md) | Reasoning over the model and a run overlay: control graph, illegal cycles, three-valued status, ready set, blocked-by, impact, what-if and critical path. |
| [`graph-layout`](packages/core/graph/layout/README.md) | Layered layout of a projected flowchart, swimlane or state machine into positions in the model's `views` shape. |
| [`graph-source-contract`](packages/core/graph/source-contract/README.md) | The `GraphSource` port as an oRPC contract: versioned, paged and streamed sync of bound activities, participants, transitions and run state, resumable by cursor, with an in-memory fake and a conformance kit. |
| [`graph-source-github-issues`](packages/adapters/graph/source-github-issues/README.md) | The GitHub issues adapter for the `GraphSource` port: issues as bound activities, `blockedBy` as transitions, sub-issues, repositories, organisations and assignees, across organisations and repositories. |
| [`graph-store-contract`](packages/core/graph/store-contract/README.md) | The persistence port as an oRPC contract: save, load and delete models, set run state, upsert items from a source, structural ready, blocked-chain and impact queries, and a change feed; with an in-memory fake and a conformance kit. |
| [`graph-source-contract`](packages/core/graph/source-contract/README.md) | The `GraphSource` port as an oRPC contract: versioned, paged and streamed sync of bound activities, participants, transitions and run state, resumable by cursor, typed filters and scope listings, with an in-memory fake and a conformance kit. |
| [`graph-source-github-issues`](packages/adapters/graph/source-github-issues/README.md) | The GitHub issues adapter for the `GraphSource` port: issues, `blockedBy` and `blocking`, sub-issues and assignees across organisations, repositories and Projects v2 boards, with server-side filters, a scope listing and per-owner tokens. |
| [`graph-store-contract`](packages/core/graph/store-contract/README.md) | The persistence port as an oRPC contract: save, load and delete models, set run state, upsert items and relations from a source, ready, blocked-chain, impact and hierarchy queries, a change feed; with an in-memory fake and a conformance kit. |
| [`graph-store-sqlite`](packages/adapters/graph/store-sqlite/README.md) | The store port on SQLite through Drizzle: one table definition, committed drizzle-kit migrations applied the same way in every runtime, recursive SQL for the structural queries, against Drizzle's async SQLite database so any driver plugs in. |
| [`graph-sqlite-node`](packages/adapters/graph/sqlite-node/README.md) | The SQLite store on Node's built-in `node:sqlite` through `drizzle-orm/node-sqlite`, for the command line and servers. |
| [`graph-sqlite-wasm`](packages/adapters/graph/sqlite-wasm/README.md) | The SQLite store in the browser: SQLite wasm in a dedicated worker on OPFS `opfs-sahpool`, shared between tabs under a Web Lock with no special headers, served to the page over oRPC. |
Expand Down
47 changes: 29 additions & 18 deletions packages/adapters/graph/source-github-issues/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

[![GitHub](https://img.shields.io/badge/GitHub-181717?logo=github&logoColor=white)](https://github.com/ExaDev/process-graph) [![npm](https://img.shields.io/badge/npm-CB3837?logo=npm&logoColor=white)](https://www.npmjs.com/package/@exadev/process-graph-source-github-issues) [![Release](https://img.shields.io/github/v/release/ExaDev/process-graph)](https://github.com/ExaDev/process-graph/releases/latest) [![CI](https://img.shields.io/github/actions/workflow/status/ExaDev/process-graph/ci.yml?branch=main)](https://github.com/ExaDev/process-graph/actions)

The GitHub issues adapter for the `GraphSource` port in `@exadev/process-graph-source-contract`. It reads issues, their blockers and sub-issues through GitHub's GraphQL API across any number of organisations and repositories, and returns them as pages of the process model. It passes the port's conformance kit. The main entry point imports nothing Node-only; `./node` adds a token helper for Node.
The GitHub issues adapter for the `GraphSource` port in `@exadev/process-graph-source-contract`. It reads issues, what blocks them, what they block and their sub-issues through GitHub's GraphQL API across any number of organisations, repositories and Projects v2 boards, and returns them as pages of the process model. It also lists the scopes a picker offers. It passes the port's conformance kit. The main entry point imports nothing Node-only; `./node` adds a token helper for Node.

```ts
import { createRouterClient } from "@orpc/server";
Expand Down Expand Up @@ -30,30 +30,39 @@ const { model, overlay, relations, gaps } = assembleSync(pages);

## Scopes

| Scope | Reads |
| ------------------- | -------------------------------------------------------------------- |
| `github:owner/repo` | One repository's issues. |
| `github:owner` | The issues of every repository the account owns, by repository name. |
| Scope | Reads |
| ------------------------------ | ------------------------------------------------------------------------------------------------------- |
| `github:owner/repo` | One repository's issues. |
| `github:owner` | The issues of every repository the account owns, by repository name. |
| `github:owner/projects/number` | The issues on an organisation's or user's Projects v2 board, in the board's order; other items skipped. |

Scopes are read in the order given, each once. A repository or account that does not exist, or that the token cannot see, is a gap on that scope and the sync carries on; anything else that is not one of the two forms is a `BAD_REQUEST` with reason `invalid-scope`.
Scopes are read in the order given, each once. A repository, account or board that does not exist, or that the token cannot see, is a gap on that scope and the sync carries on; anything else that is not one of the forms is a `BAD_REQUEST` with reason `invalid-scope`. A board's pull requests and draft issues are not issues, so a board sync passes over them.

## Filters

A sync's `filter` becomes the arguments of GitHub's issue listing, so the narrowing happens on GitHub's side: `states` `open` and `done` are `OPEN` and `CLOSED`, `labels` keeps issues with any of the labels, and `order` is GitHub's `IssueOrder` (`CREATED_AT`, `UPDATED_AT` or `COMMENTS`, ascending or descending). With no filter every issue is listed, oldest first. A board has no such listing, so a filter on a sync that includes a board scope is a `BAD_REQUEST` with reason `invalid-scope`. A blocker, blocked issue or sub-issue reached from a listed issue is read whatever the filter, since the listed work depends on it.

## Listing scopes

`scopes.roots` lists the viewer's own account and then the organisations it belongs to; `scopes.children` lists the repositories an owner has, by name (a repository or a board has nothing inside it). Each entry carries the scope identifier to sync, its name, its description and URL, and whether GitHub has archived it. An owner that does not exist is a gap on an empty page. The viewer's listing asks the token port for a token with no owner (see Tokens).

## The model it produces

| GitHub | Model |
| ---------------------------- | ----------------------------------------------------------------------------------------- |
| Issue `owner/repo#n` | A bound activity `github:owner/repo#n`, labelled with its title, its actor the repository |
| Open or closed | `open` or `done` in the overlay |
| `blockedBy` | A transition from the blocker to the issue it blocks, `<blocker>-><issue>` |
| `blockedBy` and `blocking` | A transition from the blocker to the issue it blocks, `<blocker>-><issue>` |
| Sub-issue | A `child-of` relation from the sub-issue to its parent |
| Assignee | A `role` participant `github:@login` and an `assigned-to` relation from the issue |
| Repository | A `system` participant `github:owner/repo`, `part-of` its owner |
| Organisation or user account | A participant `github:@login`: an organisation is a `system`, a user a `role` |

Blockers and sub-issues in other repositories or organisations are included with their own repository and owner, so a dependency across organisations is a transition between their activities, and a hand-off between actors in the model. Every connection is read to its end: when an issue has more assignees, blockers or sub-issues than one request returns, the rest are read on their own.
Blockers, blocked issues and sub-issues in other repositories or organisations are included with their own repository and owner, so a dependency across organisations is a transition between their activities, and a hand-off between actors in the model. A dependency is read from both of its ends, so work blocked in a repository outside the requested scopes is still found from its blocker, and the two reads give one transition. Every connection is read to its end: when an issue has more assignees, blockers, blocked issues or sub-issues than one request returns, the rest are read on their own.

## Paging and resuming

A page is one page of one repository's issues, oldest first. The cursor in a page's `next` records the scope, the position in an owner's repository listing and GitHub's own cursor into the issues, as plain data, so a sync resumes in another process. Issues are listed by creation time and repositories by name, so a position stays valid as new issues are opened. If an owner has fewer repositories than when the cursor was issued, the sync carries on after that listing page.
A page is one page of one repository's issues, or of one board's items. The cursor in a page's `next` records the scope, the position in an owner's repository listing, GitHub's own cursor into the issues or items, and the filter, as plain data, so a sync resumes in another process and a cursor is refused under another filter. Without a filter, issues are listed by creation time and repositories by name, so a position stays valid as new issues are opened; ordering by `updated` or `comments` lets an issue move while a sync is under way. If an owner has fewer repositories than when the cursor was issued, the sync carries on after that listing page.

## Errors and rate limits

Expand All @@ -68,24 +77,26 @@ A rate-limited sync resumes from the last page's `next` once the limit resets.

## Tokens

The token comes from a `GitHubTokenPort`, `(owner, signal) => Promise<string>`, asked for the owner of each repository read, so different organisations can use different credentials. The token is sent only in the `Authorization` header and never logged or put in an error.
The token comes from a `GitHubTokenPort`, `(owner, signal) => Promise<string>`, asked for the owner of each repository or board read, so different organisations can use different credentials, and with `owner` undefined for the viewer's own listing, which no owner names. The token is sent only in the `Authorization` header and never logged or put in an error.

`storedTokenPort({ list, used })` chooses among tokens a host keeps (a browser's token store, say), each scoped to some owners or to any: a token naming the owner wins, the one naming the fewest owners first and then the most recently used; otherwise the most recently used token for any owner; a request with no owner takes only a token for any owner. Logins match without regard to case. `list` is read on every request, `used` is told which token was chosen so the host can keep its recency, and no fitting token is `unauthorised`. `selectStoredToken` is the choice on its own.

`githubTokenFromEnvironment()` from `./node` uses `GITHUB_TOKEN` when it is set and not empty, and otherwise asks the GitHub CLI once (`gh auth token`). A failure is reported as `unauthorised` without the CLI's output.

## Options

| Option | Default | Meaning |
| ---------- | ----------------------------- | -------------------------------------------------- |
| `token` | required | The token port. |
| `fetch` | the global `fetch` | How requests reach GitHub. |
| `now` | `Date.now` | The clock, for `retry-after`. |
| `pageSize` | half of GitHub's limit of 100 | Issues per page, and connection items per request. |
| Option | Default | Meaning |
| ---------- | ----------------------------- | ------------------------------------------------------------------- |
| `token` | required | The token port. |
| `fetch` | the global `fetch` | How requests reach GitHub. |
| `now` | `Date.now` | The clock, for `retry-after`. |
| `pageSize` | half of GitHub's limit of 100 | Issues, items or scopes per page, and connection items per request. |

## Tests

The offline tests replay GraphQL exchanges recorded from public repositories (`src/test-support/fixtures/github.json`): GitHub's demo organisation as an owner scope, and two repositories whose issues block each other across repositories and organisations. The fixture holds request documents, variables, statuses and bodies, never headers or tokens. Re-record it after a build with `node scripts/record-fixtures.ts`, with a token in `GITHUB_TOKEN` or the GitHub CLI logged in.
The offline tests replay GraphQL exchanges recorded from public repositories (`src/test-support/fixtures/github.json`): GitHub's demo organisation as an owner scope and as a listing, two repositories whose issues block each other across repositories and organisations, one of them also synced under a filter, and a public Projects v2 board of GitHub's own. The fixture holds request documents, variables, statuses and bodies, never headers or tokens. The viewer's listing is never recorded, since it would name the recording account's organisations; it is tested over written responses and live. Re-record it after a build with `node scripts/record-fixtures.ts`, with a token in `GITHUB_TOKEN` or the GitHub CLI logged in.

The live tests run the conformance kit and a cross-organisation sync against the real API when `PROCESS_GRAPH_LIVE_GITHUB=1` is set, and are reported as skipped otherwise.
The live tests run the conformance kit (scope listing included), a cross-organisation sync and a board sync against the real API when `PROCESS_GRAPH_LIVE_GITHUB=1` is set, and are reported as skipped otherwise.

```sh
pnpm build # tsc -b -> dist/ (ESM + .d.ts)
Expand Down
2 changes: 1 addition & 1 deletion packages/adapters/graph/source-github-issues/package.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "@exadev/process-graph-source-github-issues",
"version": "1.0.1",
"description": "The GitHub issues GraphSource adapter: issues as bound activities, blockedBy as transitions, sub-issues, repositories, organisations and assignees, read through the GraphQL API with pagination, rate-limit handling and typed errors across organisations and repositories.",
"description": "The GitHub issues GraphSource adapter: issues as bound activities, blockedBy and blocking as transitions, sub-issues, repositories, organisations and assignees, read through the GraphQL API with server-side filters, pagination, rate-limit handling and typed errors across organisations, repositories and Projects v2 boards, with a scope listing for pickers.",
"type": "module",
"repository": {
"type": "git",
Expand Down
Loading
Loading