Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,14 @@ them (`v0.1.0` … `v1.19.0`); the 1.x ranges below are derived from that
history, not written from memory. Rust alpha and beta tags identify previews;
the stable 2.0 release has not been cut.

## 2.0.0-beta.56 — 2026-10-19

**Skill evaluation review**

- Pin **LORE 0.62.24** with opt-in diagnostic traces and human-run coding-skill experiments against frozen task fixtures, separate verifier checks and one-use final tests.
- Display advisory results in **`PendingReview::display_text`**, including explicit stale evidence; require reading the evidence section before arming proposal approval.
- Preserve exact raw proposal hashes and human approval. No automatic skill installation, factual-memory scoring or model experiment runs are added.

## 2.0.0-beta.55 — 2026-10-19

**Background review history**
Expand Down
44 changes: 22 additions & 22 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

19 changes: 19 additions & 0 deletions docs/manual.md
Original file line number Diff line number Diff line change
Expand Up @@ -1979,3 +1979,22 @@ daemon sockets and the peer registry, kept out of the home directory
because home directories can be network-mounted (Unix sockets misbehave
there). The LORE store is neither — it stays `lore_core`'s own path,
shared with the Claude Code LORE plugin on purpose.

### Evaluating pending coding skills

The LORE proposal review includes advisory skill-evaluation evidence when
provided by the native carrier. Changed proposal or installed baseline bytes
are labeled stale; unavailable evidence is shown explicitly. Read the full
proposal and evidence section before arming approval. The evidence does not
authorize installation; the existing exact-byte human approval applies.

Run `lore skills evaluate PROPOSAL_ID --suite /absolute/corpus/suite.json`
from a human terminal to compare frozen baseline/candidate skills on separate
train, replay, validation and one-use final-test fixtures. This is opt-in;
DOXA does not launch experiments from the proposal screen. Automatic native
diagnostic capture is also opt-in through `LORE_DIAGNOSTIC_TRACES=1`.

See the [LORE evaluation workflow](https://github.com/docwilde/LORE/blob/v0.62.24/docs/skill-evaluation.md)
for the corpus format, immutable evidence, answer/growth checks and limits.
The included deterministic fixture validates the mechanism; a real task
corpus and model adapter are needed to measure skill quality.
2 changes: 1 addition & 1 deletion rust/doxa-claude/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "doxa-claude"
version = "2.0.0-beta.55"
version = "2.0.0-beta.56"
edition = "2021"
publish = false
license = "AGPL-3.0-only"
Expand Down
2 changes: 1 addition & 1 deletion rust/doxa-codegraph/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "doxa-codegraph"
version = "2.0.0-beta.55"
version = "2.0.0-beta.56"
edition = "2021"
publish = false
license = "AGPL-3.0-only"
Expand Down
2 changes: 1 addition & 1 deletion rust/doxa-daemon/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "doxa-daemon"
version = "2.0.0-beta.55"
version = "2.0.0-beta.56"
edition = "2021"
publish = false
license = "AGPL-3.0-only"
Expand Down
2 changes: 1 addition & 1 deletion rust/doxa-engines/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "doxa-engines"
version = "2.0.0-beta.55"
version = "2.0.0-beta.56"
edition = "2021"
publish = false
license = "AGPL-3.0-only"
Expand Down
2 changes: 1 addition & 1 deletion rust/doxa-fleet/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "doxa-fleet"
version = "2.0.0-beta.55"
version = "2.0.0-beta.56"
edition = "2021"
publish = false
license = "AGPL-3.0-only"
Expand Down
2 changes: 1 addition & 1 deletion rust/doxa-hub/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "doxa-hub"
version = "2.0.0-beta.55"
version = "2.0.0-beta.56"
edition = "2021"
publish = false
license = "AGPL-3.0-only"
Expand Down
2 changes: 1 addition & 1 deletion rust/doxa-isolation/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "doxa-isolation"
version = "2.0.0-beta.55"
version = "2.0.0-beta.56"
edition = "2021"
publish = false
license = "AGPL-3.0-only"
Expand Down
4 changes: 2 additions & 2 deletions rust/doxa-lore/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
[package]
name = "doxa-lore"
version = "2.0.0-beta.55"
version = "2.0.0-beta.56"
edition = "2021"
publish = false
license = "AGPL-3.0-only"

[dependencies]
lore-core = { git = "https://github.com/docwilde/LORE", rev = "c2b2da4dbcd80c4f2d478f92c29e9dff060ad103" }
lore-core = { git = "https://github.com/docwilde/LORE", rev = "d0dcbcc9a06690a1c340c858c5590d23fe814b77" }
doxa-codegraph = { path = "../doxa-codegraph" }
doxa-state = { path = "../doxa-state" }
doxa-vendors = { path = "../doxa-vendors" }
Expand Down
30 changes: 30 additions & 0 deletions rust/doxa-lore/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -515,6 +515,7 @@ pub struct PendingReview {
raw: String,
sha256: String,
inode: u64,
skill_evaluation: Option<Value>,
}

/// One exact active belief selected for review. The sidecar checks the raw
Expand Down Expand Up @@ -603,6 +604,25 @@ impl PendingReview {
pub fn inode(&self) -> u64 {
self.inode
}
pub fn skill_evaluation(&self) -> Option<&Value> {
self.skill_evaluation.as_ref()
}
/// Evaluation evidence is advisory and separate from the proposal bytes
/// whose digest authorizes resolution. Older carriers omit this section.
pub fn display_text(&self) -> String {
match &self.skill_evaluation {
Some(evidence) => {
let freshness = match evidence["current"].as_bool() {
Some(false) => "Stale: the proposal or installed baseline changed.\n",
Some(true) => "Matches this proposal and installed baseline.\n",
None => "",
};
format!("{}\n\nSkill evaluation (advisory; human approval required)\n{}{}",
self.raw, freshness, serde_json::to_string_pretty(evidence).unwrap_or_default())
},
None => self.raw.clone(),
}
}
}

#[derive(Debug, Clone, PartialEq)]
Expand Down Expand Up @@ -1171,11 +1191,21 @@ impl LoreClient {
if previous.is_some_and(|old| old.sha256() != sha256 || old.inode() != inode) {
return Err(LoreError::Remote("pending_changed"));
}
let skill_evaluation = match value.get("skill_evaluation") {
None => None,
Some(evidence) if evidence.is_object()
&& serde_json::to_vec(evidence).is_ok_and(|b| b.len() <= 8192)
&& evidence["status"].as_str().is_some_and(|s| matches!(s,
"not_evaluated" | "evidence_unavailable" | "quality_failed" |
"performance_failed" | "validation_passed" | "passed" | "final_test_failed")) => Some(evidence.clone()),
_ => return Err(LoreError::InvalidFrame),
};
Ok(PendingReview {
pid: pid.to_owned(),
raw: raw.to_owned(),
sha256: sha256.to_owned(),
inode,
skill_evaluation,
})
}

Expand Down
19 changes: 19 additions & 0 deletions rust/doxa-lore/tests/native.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,25 @@ use doxa_lore::{CodegraphSnapshot, LoreClient, LoreError};
use sha2::{Digest, Sha256};
use std::time::Duration;

#[test]
fn native_skill_review_keeps_evaluation_separate_from_exact_proposal_bytes() {
let dir = tempfile::tempdir().unwrap();
let cfg = lore_core::config::Config::for_root(dir.path().join("store"));
let cwd = dir.path().join("project");
let authority = lore_core::gate::Authority::Derived { agent: "fixture".into(), engine: "codex".into() };
let pid = lore_core::gate::stage(&cfg, &serde_json::json!({
"kind":"skill", "name":"fixture-skill", "body":"Run the fixed task checks.",
"project":lore_core::config::project_slug(&cwd)
}), &authority).unwrap();
let mut client = LoreClient::open_config(cfg.clone(), Duration::from_secs(2)).unwrap();
let review = client.pending_review(cwd.to_str().unwrap(), &pid).unwrap();
assert_eq!(review.skill_evaluation().unwrap()["status"], "not_evaluated");
assert!(review.display_text().contains("Skill evaluation (advisory"));
let raw = std::fs::read_to_string(cfg.root.join("pending").join(format!("{pid}.json"))).unwrap();
assert_eq!(review.raw(), raw);
assert_eq!(review.sha256(), format!("{:x}", Sha256::digest(raw.as_bytes())));
}

#[test]
fn reviewed_codegraph_read_is_scoped_and_rejects_stale_source() {
let owned = tempfile::tempdir().unwrap();
Expand Down
Loading
Loading