Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions .github/workflows/reviewer.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
name: Reviewer

# Advisory PR review (umbra-reviewer): surfaces architecture + security issues
# Advisory PR review (signetry-reviewer): surfaces architecture + security issues
# and posts one recommendation comment. Advisory only — it never merges and never
# fails the PR. Changes to security-sensitive surfaces (workflows, packaging) are
# escalated to a human. See https://github.com/Signetry/reviewer
Expand All @@ -22,9 +22,9 @@ jobs:
- uses: actions/setup-python@v7
with:
python-version: "3.12"
- name: Install umbra-reviewer
- name: Install signetry-reviewer
# source-available (All Rights Reserved); install from source, not PyPI.
run: pip install "umbra-reviewer @ git+https://github.com/Signetry/reviewer@v0.1.1"
run: pip install "signetry-reviewer @ git+https://github.com/Signetry/reviewer@v0.1.2"
- name: Compute the PR diff
env:
BASE_SHA: ${{ github.event.pull_request.base.sha }}
Expand All @@ -40,7 +40,7 @@ jobs:
PR: ${{ github.event.pull_request.number }}
run: |
set -uo pipefail
umbra-reviewer review \
signetry-reviewer review \
--diff "$OUT/pr.diff" \
--repo "$GITHUB_REPOSITORY" \
--pr "$PR" \
Expand All @@ -58,7 +58,7 @@ jobs:
const out = process.env.OUT || '';
let body = 'Signetry Reviewer: no review was produced.';
try { body = fs.readFileSync(path.join(out, 'comment.md'), 'utf8'); } catch (e) {}
const marker = '<!-- umbra-reviewer -->';
const marker = '<!-- signetry-reviewer -->';
body = marker + '\n' + body;
const { owner, repo } = context.repo;
const issue_number = context.issue.number;
Expand Down
28 changes: 14 additions & 14 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,33 +7,33 @@ Until `1.0.0` the public API may change between minor versions.

### Changed — Signetry rename (breaking)

- Renamed the distribution `umbra-eval` → `signetry-eval` and the import package
`umbra_eval` → `signetry_eval`. The console command is now `signetry-eval`.
- All `UMBRA_*` environment variables are now `SIGNETRY_*`; the config directory
is `.signetry/` (was `.umbra/`). Product/brand prose updated to **Signetry**.
- Core dependency renamed and pinned: `signetry-core @ git+https://github.com/Signetry/core@v0.6.0`
(was `umbra-core @ ...@v0.5.4`). Imports moved `umbra_core` → `signetry_core`.
- Distribution `signetry-eval` and import package `signetry_eval`. The console
command is `signetry-eval`.
- Environment variables use `SIGNETRY_*`; the config directory is `.signetry/`.
Product/brand prose updated to **Signetry**.
- Core dependency pinned: `signetry-core @ git+https://github.com/Signetry/core@v0.6.0`
(was `signetry-core @ ...@v0.5.4`). Imports use `signetry_core`.
- No backward-compatibility fallbacks are provided.

## [0.2.2] — 2026-07-30

### Changed

- Install `umbra-core` from its **source repository** (`git+https://github.com/Signetry/core@v0.5.4`)
instead of PyPI — umbra-core is All Rights Reserved and no longer distributed on
- Install `signetry-core` from its **source repository** (`git+https://github.com/Signetry/core@v0.5.4`)
instead of PyPI — signetry-core is All Rights Reserved and no longer distributed on
PyPI. Enables `tool.hatch.metadata.allow-direct-references`.

## [0.2.1] — 2026-07-30

### Fixed

- Dependency floor raised to `umbra-core>=0.5.0` (was `0.3.0`) so a fresh install
- Dependency floor raised to `signetry-core>=0.5.0` (was `0.3.0`) so a fresh install
always has the detection engine (`scan_repository`) the corpus benchmark needs.

### Docs

- README now documents the **detection head-to-head benchmark** (52 cases, 7
languages; umbra-core 100% recall / 0 false positives vs Claude Opus 4.8 90%),
languages; signetry-core 100% recall / 0 false positives vs Claude Opus 4.8 90%),
the `corpus` / `realrepo` commands, and the honest note that any false positive
comes from the optional Semgrep layer, not the deterministic engine.

Expand All @@ -46,13 +46,13 @@ Until `1.0.0` the public API may change between minor versions.
academic/CWE, crafted, hard (cross-file taint, framework sinks, true-negative
traps), multilang, and cross-file-lang — with cited provenance per case and safe
decoys for false-positive measurement.
- `umbra-eval corpus` scores Umbra (live, via `umbra-core`) against competitor
- `signetry-eval corpus` scores Signetry (live, via `signetry-core`) against competitor
scanners (replayed from committed captures), reporting recall, false positives,
and a by-language breakdown. `--min-recall` / `--max-fp` gate a regression;
`--semgrep` enables the optional layer.
- `umbra-eval benchmark` (14-vuln fixture head-to-head) and `umbra-eval realrepo`
- `signetry-eval benchmark` (14-vuln fixture head-to-head) and `signetry-eval realrepo`
(live scan of real vulnerable repos).
- Committed head-to-head result: umbra-core **100% recall / 0 false positives** vs
- Committed head-to-head result: signetry-core **100% recall / 0 false positives** vs
claude-code-security-review (Claude Opus 4.8) 90% — deterministic, offline, free.

### CI
Expand All @@ -64,4 +64,4 @@ Until `1.0.0` the public API may change between minor versions.

- Initial adversarial evaluation suite: ASR (ungoverned vs governed) + utility-
under-defense across IPI, skill/MCP poisoning, and memory-injection scenarios,
run against the real `umbra-core` admission pipeline.
run against the real `signetry-core` admission pipeline.