diff --git a/changelog.d/added-10683.md b/changelog.d/added-10683.md new file mode 100644 index 0000000000..2e9ce7bfd3 --- /dev/null +++ b/changelog.d/added-10683.md @@ -0,0 +1 @@ +- add Spek recheck outward discovery sources (#10683) diff --git a/src/changelog.d/added-spek-recheck-discovery.md b/src/changelog.d/added-spek-recheck-discovery.md new file mode 100644 index 0000000000..e585cd6e22 --- /dev/null +++ b/src/changelog.d/added-spek-recheck-discovery.md @@ -0,0 +1 @@ +- Add opt-in outward discovery sources for Spektacular recheck revisions, surfacing bounded external evidence in drift metadata and the Campaigns dashboard. diff --git a/src/docs/adr/0020-spek-continuous-convergence.md b/src/docs/adr/0020-spek-continuous-convergence.md index 1bee28fe14..7300275629 100644 --- a/src/docs/adr/0020-spek-continuous-convergence.md +++ b/src/docs/adr/0020-spek-continuous-convergence.md @@ -35,10 +35,22 @@ Each revision records drift evidence: - `prior_revision` - `delta_count` - `recheck_reason` (`cadence` or `manual`) - -The evidence source interface is `RecheckEvidenceSource`. This ADR accepts only -the codebase-head source for now; external upstream/competitor/standards -discovery remains a follow-up. +- optional `external[]`, `external_count`, and `sources_failed[]` discovery + evidence when outward discovery is enabled + +The evidence source interface is `RecheckEvidenceSource`. + +### Discovery sources + +The outward-looking half is opt-in under +`runs.spektacular.recheck.discovery`. Operators declare bounded sources of kind +`upstream_release`, `repo_activity`, `standards_feed`, or `landscape`; Hive +rejects undeclared kinds and hosts outside `variables.security.http_allowlist`. +Discovery runs before the recheck `spec` stage generator, records evidence on +the linked revision's `drift.external` block, and includes that evidence in the +spec-stage prompt as context. It is read-only and non-fatal: source failures are +recorded in `sources_failed` and never block the recheck. Evidence is never +auto-applied; the existing spec and plan human checkpoints remain authoritative. ## Consequences diff --git a/src/docs/api-reference.md b/src/docs/api-reference.md index e8e84fd384..702e8e7ea6 100644 --- a/src/docs/api-reference.md +++ b/src/docs/api-reference.md @@ -37,8 +37,8 @@ Auth levels are derived from dashboard middleware (`isPublicPath`, dashboard tok | `POST` | `/api/runs/{key}/checkpoint` | Owner only | Resolve a compact checkpoint decision. Body `{"action":"approve"|"reject","gen":}`; refuses stale generations with 409 and non-owner callers with 403 before mutating the plan. | `pkg/dashboard/api.go:83` | | `GET` | `/api/runs/{key}` | Dashboard auth/session | One run by URL-escaped canonical run key (`owner/repo%23number`); early lease-shaped keys (`repo!run-key:stage`) and queued Wavefront run-stage keys (`owner/repo!graph:node`) are also accepted. Includes timeline-derived stage history in the same `Run` shape as `/api/runs` when a lease/timeline exists. History entries that came from an owner reset carry the owner's `reason`. When the key belongs to a wired convergence campaign, the detail response also includes an optional `burndown` block with `source`, `satisfied`, `remaining`, `unknown`, and `scope`; the list endpoint omits this block. | `pkg/dashboard/api.go:84` | | `POST` | `/api/runs/{key}/reset` | Owner only | Move a run's lease back to an earlier stage (for example `implement` to `plan` after a rejected plan). Body `{"to": "", "reason": ""}`, both required. Mints a new generation so the relay holding the old one can no longer resume, persists before answering, and records the reason on the agent audit sink, the lifecycle timeline, and the `stage_completed` hook payload (`attrs.reset = "true"`). Returns `{ok, key, stage_from, stage, gen, reason}`. 400 for the same or a later stage or a missing reason, 404 when no live staged lease holds the key, 409 when the lease expired, 500 when the registry could not be written. | `pkg/dashboard/api.go:85` | -| `GET` | `/api/campaigns` | Dashboard auth/session | List resumable campaigns from archived Inception sessions and retained Spektacular (Spek) runs. Query `search`/`q`, `repo`, `stage`, `status`, and `owner`; each campaign is keyed by stable spec/session id and includes title, source, repos, current stage/step, artifact links, linked issues/PRs, contributors, last activity, status, engine, type, run link when applicable, and recheck status (`enabled`, `interval`, `last_at`, `next_at`, `in_flight`, `last_delta_count`). | `pkg/dashboard/api.go:86` | -| `GET` | `/api/campaigns/{id}` | Dashboard auth/session | Return one campaign by URL-escaped stable campaign id or run key, using the same campaign shape as `/api/campaigns`; recheck revisions may include a `drift` block with `codebase_changed`, `prior_revision`, `delta_count`, and `recheck_reason`. | `pkg/dashboard/api.go:87` | +| `GET` | `/api/campaigns` | Dashboard auth/session | List resumable campaigns from archived Inception sessions and retained Spektacular (Spek) runs. Query `search`/`q`, `repo`, `stage`, `status`, and `owner`; each campaign is keyed by stable spec/session id and includes title, source, repos, current stage/step, artifact links, linked issues/PRs, contributors, last activity, status, engine, type, run link when applicable, and recheck status (`enabled`, `interval`, `last_at`, `next_at`, `in_flight`, `last_delta_count`, `external_count`, `sources_failed`). | `pkg/dashboard/api.go:86` | +| `GET` | `/api/campaigns/{id}` | Dashboard auth/session | Return one campaign by URL-escaped stable campaign id or run key, using the same campaign shape as `/api/campaigns`; recheck revisions may include a `drift` block with `codebase_changed`, `prior_revision`, `delta_count`, `recheck_reason`, `external[]` (`source`, `kind`, `title`, `url`, `published_at`, `summary`), `external_count`, and `sources_failed`. | `pkg/dashboard/api.go:87` | | `POST` | `/api/campaigns/{id}/resume` | Owner only | Resume or pick up a campaign. Archived Inception campaigns take a durable campaign lease, restore their saved state and wiki working files; Spek campaigns return the retained run detail plus a CLI status command that reloads state from Spek artifacts. Body may include `{ "surface": "chat"|"contributor"|"cli"|"dashboard" }`; 409 means another user holds the lease. | `pkg/dashboard/api.go:88` | | `POST` | `/api/campaigns/{id}/release` | Owner only | Release the caller's Inception campaign lease and write a `campaign_release` audit entry so another user can continue it. Run-backed Spektacular campaigns return `409 Conflict`; this endpoint never revokes their run-lifecycle stage leases. | `pkg/dashboard/api.go:89` | | `POST` | `/api/campaigns/{id}/revise` | Owner only | Create a linked revision campaign (`revision_of`, `revision`) for revisiting shipped or parked Inception/Spek work without overwriting the previous campaign. The new revision is leased to the caller and audit-logged as `campaign_revise`. | `pkg/dashboard/api.go:90` | diff --git a/src/docs/spektacular.md b/src/docs/spektacular.md index f3083e33cd..bf968b1c42 100644 --- a/src/docs/spektacular.md +++ b/src/docs/spektacular.md @@ -274,14 +274,64 @@ imports the first capped set and records `campaign_recheck_delta_capped`. Each revision carries a `drift` evidence block with `codebase_changed`, `prior_revision`, `delta_count`, and `recheck_reason` (`cadence` or `manual`). The codebase signal compares the prior recorded head SHA with the current Git -HEAD when available. Outward-looking external-source discovery (upstreams, -competitors, standards trackers, landscape entries) is deliberately not in -this PR; the Go extension point is `RecheckEvidenceSource`, with only the -codebase-head source implemented. Follow-up: [#10667](https://github.com/hivecommons/hive/issues/10667). +HEAD when available. + +### Outward-looking discovery + +Outward discovery is opt-in under `runs.spektacular.recheck.discovery`. Hive +only reads operator-declared sources, bounded by `timeout` (default `30s`), +each source's `max_items` (default `20`), and `max_total_items` (default +`100`). It never searches the web, never crawls arbitrary or competitor +websites, never applies findings automatically, and rejects any configured +host that is not listed in `variables.security.http_allowlist`. GitHub-backed +sources use the existing authenticated GitHub client; feed sources use GET +through the normal proxy/egress path. + +Supported source kinds are: + +- `upstream_release`: GitHub releases for a named `owner/repo` since the prior + revision timestamp. +- `repo_activity`: merged PRs and tagged releases for a named `owner/repo` + since the prior timestamp. +- `standards_feed`: an Atom or RSS URL such as a spec repo's `releases.atom` + or a standards-body feed. +- `landscape`: GitHub repositories linked from `docs/landscape.md`, checked + for new releases using the same release reader as `upstream_release`. + +Example: + +```yaml +runs: + spektacular: + recheck: + discovery: + enabled: true + timeout: 30s + max_total_items: 100 + sources: + - {kind: upstream_release, name: kubernetes, url_or_repo: kubernetes/kubernetes, max_items: 10} + - {kind: repo_activity, name: competitor-runtime, url_or_repo: example/runtime, allow_prerelease: true} + - {kind: standards_feed, name: ietf-http, url_or_repo: https://www.ietf.org/archive/id/atom.xml} + - {kind: landscape, name: project-landscape, max_items: 20} +variables: + security: + http_allowlist: [api.github.com, www.ietf.org] +``` + +Findings are recorded on the revision as +`drift.external[] = {source, kind, title, url, published_at, summary}` with +summaries capped at 280 characters, plus `external_count` and +`sources_failed[] = {name, reason}`. A failed source is non-fatal: Hive records +the typed reason and continues with the rest. The same evidence is included in +the `spec` stage assignment prompt as context so the spec generator can decide +what, if anything, belongs in the revised spec; the spec and plan human +checkpoints remain the only approval gates. Campaign list/detail JSON includes -`recheck: {enabled, interval, last_at, next_at, in_flight, last_delta_count}`; -the dashboard Campaigns panel renders the cadence and provides an owner-only +`recheck: {enabled, interval, last_at, next_at, in_flight, last_delta_count, +external_count, sources_failed}`; recheck revisions also include the `drift` +block above. The dashboard Campaigns panel renders the cadence, shows +`N external signals` with an expandable list, and provides an owner-only Recheck action. ## Defensive handling of the open questions diff --git a/src/hive.yaml.example b/src/hive.yaml.example index 7b55cdbf23..c059925fb1 100644 --- a/src/hive.yaml.example +++ b/src/hive.yaml.example @@ -569,6 +569,15 @@ governor: # enabled: false # default_interval: 168h # max_delta_tasks: 50 +# discovery: +# enabled: false +# timeout: 30s +# max_total_items: 100 +# sources: +# - {kind: upstream_release, name: kubernetes, url_or_repo: kubernetes/kubernetes, max_items: 10} +# - {kind: repo_activity, name: competitor-runtime, url_or_repo: example/runtime, allow_prerelease: true} +# - {kind: standards_feed, name: ietf-http, url_or_repo: https://www.ietf.org/archive/id/atom.xml} +# - {kind: landscape, name: project-landscape, max_items: 20} # max_stage_retries: 2 # max_worktrees: 8 diff --git a/src/pkg/config/runs_config.go b/src/pkg/config/runs_config.go index 6e97782253..9ea9dfbf7e 100644 --- a/src/pkg/config/runs_config.go +++ b/src/pkg/config/runs_config.go @@ -29,6 +29,12 @@ const ( // DefaultSpektacularMaxDeltaTasks caps how many changed/new plan tasks a // recheck imports into the planner in one generation. DefaultSpektacularMaxDeltaTasks = 50 + // DefaultSpektacularDiscoveryTimeoutS bounds each outward discovery pass. + DefaultSpektacularDiscoveryTimeoutS = 30 + // DefaultSpektacularDiscoveryMaxItems caps one configured discovery source. + DefaultSpektacularDiscoveryMaxItems = 20 + // DefaultSpektacularDiscoveryMaxTotalItems caps all discovery evidence for a revision. + DefaultSpektacularDiscoveryMaxTotalItems = 100 // DefaultRunsWaitTimeoutSeconds is how long a run checkpoint may wait for // owner approval before escalation. It also floors how far a held plan @@ -165,6 +171,26 @@ type SpektacularRecheckConfig struct { // MaxDeltaTasks caps imported changed/new tasks. Zero or negative means // DefaultSpektacularMaxDeltaTasks. MaxDeltaTasks int `yaml:"max_delta_tasks,omitempty" json:"max_delta_tasks,omitempty"` + // Discovery configures opt-in outward evidence gathered before recheck spec. + Discovery SpektacularRecheckDiscoveryConfig `yaml:"discovery,omitempty" json:"discovery,omitempty"` +} + +// SpektacularRecheckDiscoveryConfig is the bounded, declared-source-only +// outward discovery step for recheck spec revisions. +type SpektacularRecheckDiscoveryConfig struct { + Enabled bool `yaml:"enabled,omitempty" json:"enabled,omitempty"` + Sources []SpektacularRecheckDiscoverySource `yaml:"sources,omitempty" json:"sources,omitempty"` + Timeout time.Duration `yaml:"timeout,omitempty" json:"timeout,omitempty"` + MaxTotalItems int `yaml:"max_total_items,omitempty" json:"max_total_items,omitempty"` +} + +// SpektacularRecheckDiscoverySource declares one operator-approved source. +type SpektacularRecheckDiscoverySource struct { + Kind string `yaml:"kind,omitempty" json:"kind,omitempty"` + Name string `yaml:"name,omitempty" json:"name,omitempty"` + URLOrRepo string `yaml:"url_or_repo,omitempty" json:"url_or_repo,omitempty"` + AllowPrerelease bool `yaml:"allow_prerelease,omitempty" json:"allow_prerelease,omitempty"` + MaxItems int `yaml:"max_items,omitempty" json:"max_items,omitempty"` } // MaxStageRetriesOrDefault returns the configured retry budget or the default. @@ -238,3 +264,24 @@ func (s SpektacularConfig) MaxDeltaTasks() int { } return s.Recheck.MaxDeltaTasks } + +func (s SpektacularConfig) DiscoveryTimeout() time.Duration { + if s.Recheck.Discovery.Timeout <= 0 { + return time.Duration(DefaultSpektacularDiscoveryTimeoutS) * time.Second + } + return s.Recheck.Discovery.Timeout +} + +func (s SpektacularConfig) DiscoveryMaxTotalItems() int { + if s.Recheck.Discovery.MaxTotalItems <= 0 { + return DefaultSpektacularDiscoveryMaxTotalItems + } + return s.Recheck.Discovery.MaxTotalItems +} + +func (s SpektacularRecheckDiscoverySource) EffectiveMaxItems() int { + if s.MaxItems <= 0 { + return DefaultSpektacularDiscoveryMaxItems + } + return s.MaxItems +} diff --git a/src/pkg/config/spektacular_recheck_discovery_test.go b/src/pkg/config/spektacular_recheck_discovery_test.go new file mode 100644 index 0000000000..88f863d49f --- /dev/null +++ b/src/pkg/config/spektacular_recheck_discovery_test.go @@ -0,0 +1,74 @@ +package config + +import ( + "strings" + "testing" + "time" +) + +func TestSpektacularRecheckDiscoveryDefaults(t *testing.T) { + var cfg SpektacularConfig + if got := cfg.DiscoveryTimeout(); got != 30*time.Second { + t.Fatalf("DiscoveryTimeout = %s, want 30s", got) + } + if got := cfg.DiscoveryMaxTotalItems(); got != 100 { + t.Fatalf("DiscoveryMaxTotalItems = %d, want 100", got) + } + if got := (SpektacularRecheckDiscoverySource{}).EffectiveMaxItems(); got != 20 { + t.Fatalf("EffectiveMaxItems = %d, want 20", got) + } +} + +func TestValidateSpektacularRecheckDiscoveryRejectsBadKind(t *testing.T) { + cfg := Config{ + Project: ProjectConfig{Org: "org"}, + GitHub: GitHubConfig{Token: "x"}, + Agents: map[string]AgentConfig{"scanner": {Backend: "claude"}}, + } + cfg.Runs.Spektacular.Recheck.Discovery = SpektacularRecheckDiscoveryConfig{ + Enabled: true, + Sources: []SpektacularRecheckDiscoverySource{{Kind: "search_engine", Name: "bad", URLOrRepo: "https://example.com/feed.xml"}}, + } + err := cfg.validate() + if err == nil || !strings.Contains(err.Error(), "invalid kind") { + t.Fatalf("validate() = %v, want invalid kind", err) + } +} + +func TestValidateSpektacularRecheckDiscoveryRejectsDisallowedHost(t *testing.T) { + cfg := Config{ + Project: ProjectConfig{Org: "org"}, + GitHub: GitHubConfig{Token: "x"}, + Agents: map[string]AgentConfig{"scanner": {Backend: "claude"}}, + } + cfg.Variables.Security.HTTPAllowlist = []string{"allowed.example.com"} + cfg.Runs.Spektacular.Recheck.Discovery = SpektacularRecheckDiscoveryConfig{ + Enabled: true, + Sources: []SpektacularRecheckDiscoverySource{{Kind: "standards_feed", Name: "ietf", URLOrRepo: "https://blocked.example.com/feed.xml"}}, + } + err := cfg.validate() + if err == nil || !strings.Contains(err.Error(), "blocked.example.com") || !strings.Contains(err.Error(), "http_allowlist") { + t.Fatalf("validate() = %v, want host allowlist error", err) + } +} + +func TestValidateSpektacularRecheckDiscoveryAcceptsAllowlistedSources(t *testing.T) { + cfg := Config{ + Project: ProjectConfig{Org: "org"}, + GitHub: GitHubConfig{Token: "x"}, + Agents: map[string]AgentConfig{"scanner": {Backend: "claude"}}, + } + cfg.Variables.Security.HTTPAllowlist = []string{"api.github.com", "standards.example.com"} + cfg.Runs.Spektacular.Recheck.Discovery = SpektacularRecheckDiscoveryConfig{ + Enabled: true, + Sources: []SpektacularRecheckDiscoverySource{ + {Kind: "upstream_release", Name: "up", URLOrRepo: "owner/repo"}, + {Kind: "repo_activity", Name: "activity", URLOrRepo: "owner/other"}, + {Kind: "standards_feed", Name: "feed", URLOrRepo: "https://standards.example.com/feed.xml"}, + {Kind: "landscape", Name: "landscape"}, + }, + } + if err := cfg.validate(); err != nil { + t.Fatalf("validate() = %v, want nil", err) + } +} diff --git a/src/pkg/config/validate.go b/src/pkg/config/validate.go index 0bb1b0919f..776c99224b 100644 --- a/src/pkg/config/validate.go +++ b/src/pkg/config/validate.go @@ -16,6 +16,69 @@ func (c *Config) Validate() error { return c.ValidateWithOptions(ValidateOptions{RequireAgents: true}) } +func (c *Config) validateSpektacularRecheckDiscovery() error { + d := c.Runs.Spektacular.Recheck.Discovery + if !d.Enabled && len(d.Sources) == 0 { + return nil + } + allowedKinds := map[string]bool{ + "upstream_release": true, + "repo_activity": true, + "standards_feed": true, + "landscape": true, + } + for i, src := range d.Sources { + kind := strings.TrimSpace(src.Kind) + if !allowedKinds[kind] { + return fmt.Errorf("runs.spektacular.recheck.discovery.sources[%d]: invalid kind %q (must be upstream_release, repo_activity, standards_feed, or landscape)", i, src.Kind) + } + if strings.TrimSpace(src.Name) == "" { + return fmt.Errorf("runs.spektacular.recheck.discovery.sources[%d]: name is required", i) + } + if kind != "landscape" && strings.TrimSpace(src.URLOrRepo) == "" { + return fmt.Errorf("runs.spektacular.recheck.discovery.sources[%d]: url_or_repo is required", i) + } + for _, host := range discoverySourceHosts(kind, src.URLOrRepo) { + if !hostAllowedByHTTPList(host, c.Variables.Security.HTTPAllowlist) { + return fmt.Errorf("runs.spektacular.recheck.discovery.sources[%d]: host %q is not covered by variables.security.http_allowlist", i, host) + } + } + } + return nil +} + +func discoverySourceHosts(kind, value string) []string { + value = strings.TrimSpace(value) + switch kind { + case "upstream_release", "repo_activity": + return []string{"api.github.com"} + case "standards_feed": + u, err := url.Parse(value) + if err != nil || u.Hostname() == "" { + return []string{value} + } + return []string{strings.ToLower(u.Hostname())} + case "landscape": + return []string{"api.github.com"} + default: + return nil + } +} + +func hostAllowedByHTTPList(host string, allowlist []string) bool { + host = strings.ToLower(strings.TrimSpace(host)) + if host == "" { + return false + } + for _, allowed := range allowlist { + allowed = strings.ToLower(strings.TrimSpace(allowed)) + if allowed == host || allowed == "*" { + return true + } + } + return false +} + func (c *Config) ValidateWithOptions(opts ValidateOptions) error { if c.Project.Org == "" { return fmt.Errorf("project.org is required") @@ -63,6 +126,9 @@ func (c *Config) ValidateWithOptions(opts ValidateOptions) error { if err := c.validateUpstreamWatch(); err != nil { return err } + if err := c.validateSpektacularRecheckDiscovery(); err != nil { + return err + } if normalized, err := ValidateSnapshotFrameAncestors(c.Dashboard.SnapshotFrameAncestors); err != nil { return err } else { diff --git a/src/pkg/dashboard/api_campaigns.go b/src/pkg/dashboard/api_campaigns.go index 03a4f3899f..a7cea2cf7d 100644 --- a/src/pkg/dashboard/api_campaigns.go +++ b/src/pkg/dashboard/api_campaigns.go @@ -37,21 +37,40 @@ type Campaign struct { } type CampaignRecheck struct { - Enabled bool `json:"enabled"` - Interval string `json:"interval,omitempty"` - LastAt string `json:"last_at,omitempty"` - NextAt string `json:"next_at,omitempty"` - InFlight bool `json:"in_flight"` - LastDeltaCount int `json:"last_delta_count,omitempty"` + Enabled bool `json:"enabled"` + Interval string `json:"interval,omitempty"` + LastAt string `json:"last_at,omitempty"` + NextAt string `json:"next_at,omitempty"` + InFlight bool `json:"in_flight"` + LastDeltaCount int `json:"last_delta_count,omitempty"` + ExternalCount int `json:"external_count,omitempty"` + SourcesFailed []CampaignSourceFailure `json:"sources_failed,omitempty"` } type CampaignDrift struct { - CodebaseChanged bool `json:"codebase_changed"` - PriorHeadSHA string `json:"prior_head_sha,omitempty"` - CurrentHeadSHA string `json:"current_head_sha,omitempty"` - PriorRevision string `json:"prior_revision,omitempty"` - DeltaCount int `json:"delta_count"` - RecheckReason string `json:"recheck_reason,omitempty"` + CodebaseChanged bool `json:"codebase_changed"` + PriorHeadSHA string `json:"prior_head_sha,omitempty"` + CurrentHeadSHA string `json:"current_head_sha,omitempty"` + PriorRevision string `json:"prior_revision,omitempty"` + DeltaCount int `json:"delta_count"` + RecheckReason string `json:"recheck_reason,omitempty"` + External []CampaignExternalEvidence `json:"external,omitempty"` + ExternalCount int `json:"external_count,omitempty"` + SourcesFailed []CampaignSourceFailure `json:"sources_failed,omitempty"` +} + +type CampaignExternalEvidence struct { + Source string `json:"source"` + Kind string `json:"kind"` + Title string `json:"title"` + URL string `json:"url,omitempty"` + PublishedAt string `json:"published_at,omitempty"` + Summary string `json:"summary,omitempty"` +} + +type CampaignSourceFailure struct { + Name string `json:"name"` + Reason string `json:"reason"` } type CampaignArtifact struct { @@ -390,13 +409,18 @@ func campaignFromInceptionArchive(archive knowledge.InceptionCampaignArchive) Ca for _, file := range archive.WikiFiles { artifacts = append(artifacts, CampaignArtifact{Kind: "fact", Label: file}) } - return Campaign{ + campaign := Campaign{ ID: archive.ID, Title: title, Source: firstRunNonEmpty(archive.Source, "inception"), Repos: repos, CurrentStage: stage, CurrentStep: step, Artifacts: artifacts, LinkedIssues: linkedIssues, Contributors: contributors, LastActivity: formatRunTime(last), Status: status, Engine: firstRunNonEmpty(archive.Engine, "Spec Kit"), Type: firstRunNonEmpty(archive.Type, "inception"), RunKey: runKey, RunURL: runURL, LeaseOwner: leaseOwner, RevisionOf: archive.RevisionOf, Revision: archive.Revision, Recheck: campaignRecheckFromArchive(archive, false), Drift: campaignDriftFromArchive(archive), } + if campaign.Recheck != nil && campaign.Drift != nil { + campaign.Recheck.ExternalCount = campaign.Drift.ExternalCount + campaign.Recheck.SourcesFailed = campaign.Drift.SourcesFailed + } + return campaign } func campaignFromRun(run Run) Campaign { diff --git a/src/pkg/dashboard/campaign_recheck.go b/src/pkg/dashboard/campaign_recheck.go index 977fc3dd89..49ec103b20 100644 --- a/src/pkg/dashboard/campaign_recheck.go +++ b/src/pkg/dashboard/campaign_recheck.go @@ -149,6 +149,7 @@ func (s *Server) triggerCampaignRecheck(ctx context.Context, id, actor, reason s evidence, _ := (codebaseHeadEvidenceSource{}).Evidence(ctx, base, priorCampaignHead(base)) evidence.PriorRevision = firstRunNonEmpty(base.RevisionOf, base.ID) evidence.RecheckReason = reason + external, failures := s.collectRecheckDiscovery(ctx, base, parseCampaignTime(base.LastActivity)) revision.Drift = &knowledge.CampaignDrift{ CodebaseChanged: evidence.CodebaseChanged, PriorHeadSHA: evidence.PriorHeadSHA, @@ -156,6 +157,9 @@ func (s *Server) triggerCampaignRecheck(ctx context.Context, id, actor, reason s PriorRevision: evidence.PriorRevision, DeltaCount: evidence.DeltaCount, RecheckReason: evidence.RecheckReason, + External: external, + ExternalCount: len(external), + SourcesFailed: failures, } if _, err := s.deps.Inception.SetCampaignRecheck(revision.ID, revision.Recheck); err != nil { return Campaign{}, err @@ -282,9 +286,41 @@ func campaignDriftFromArchive(archive knowledge.InceptionCampaignArchive) *Campa PriorRevision: archive.Drift.PriorRevision, DeltaCount: archive.Drift.DeltaCount, RecheckReason: archive.Drift.RecheckReason, + External: campaignExternalEvidenceFromKnowledge(archive.Drift.External), + ExternalCount: archive.Drift.ExternalCount, + SourcesFailed: campaignSourceFailuresFromKnowledge(archive.Drift.SourcesFailed), } } +func campaignExternalEvidenceFromKnowledge(in []knowledge.CampaignExternalEvidence) []CampaignExternalEvidence { + if len(in) == 0 { + return nil + } + out := make([]CampaignExternalEvidence, 0, len(in)) + for _, ev := range in { + out = append(out, CampaignExternalEvidence{ + Source: ev.Source, + Kind: ev.Kind, + Title: ev.Title, + URL: ev.URL, + PublishedAt: formatRunTime(ev.PublishedAt), + Summary: ev.Summary, + }) + } + return out +} + +func campaignSourceFailuresFromKnowledge(in []knowledge.CampaignSourceFailure) []CampaignSourceFailure { + if len(in) == 0 { + return nil + } + out := make([]CampaignSourceFailure, 0, len(in)) + for _, fail := range in { + out = append(out, CampaignSourceFailure{Name: fail.Name, Reason: fail.Reason}) + } + return out +} + func (s *Server) importRecheckDelta(runKey, repo, taskList, source string) (bool, error) { if s == nil || s.deps == nil || s.deps.Inception == nil { return false, nil diff --git a/src/pkg/dashboard/recheck_discovery.go b/src/pkg/dashboard/recheck_discovery.go new file mode 100644 index 0000000000..dff754cb15 --- /dev/null +++ b/src/pkg/dashboard/recheck_discovery.go @@ -0,0 +1,354 @@ +package dashboard + +import ( + "context" + "encoding/xml" + "fmt" + "io" + "net/http" + "net/url" + "os" + "regexp" + "sort" + "strings" + "time" + "unicode/utf8" + + gh "github.com/google/go-github/v72/github" + "github.com/hivecommons/hive/pkg/config" + "github.com/hivecommons/hive/pkg/knowledge" +) + +const ( + externalSummaryMaxRunes = 280 + discoveryFeedReadLimit = 1 << 20 +) + +type recheckDiscoveryCollector struct { + cfg config.SpektacularConfig + gh *gh.Client + client *http.Client +} + +func (s *Server) collectRecheckDiscovery(ctx context.Context, base Campaign, since time.Time) ([]knowledge.CampaignExternalEvidence, []knowledge.CampaignSourceFailure) { + if s == nil || s.deps == nil || s.deps.Config == nil { + return nil, nil + } + cfg := s.deps.Config.Runs.Spektacular + if !cfg.Recheck.Discovery.Enabled || len(cfg.Recheck.Discovery.Sources) == 0 { + return nil, nil + } + timeout := cfg.DiscoveryTimeout() + if timeout > 0 { + var cancel context.CancelFunc + ctx, cancel = context.WithTimeout(ctx, timeout) + defer cancel() + } + var ghc *gh.Client + if s.deps.GHClient != nil { + ghc = s.deps.GHClient.GoGitHub() + } + collector := recheckDiscoveryCollector{cfg: cfg, gh: ghc, client: http.DefaultClient} + return collector.collect(ctx, base, since) +} + +func (c recheckDiscoveryCollector) collect(ctx context.Context, base Campaign, since time.Time) ([]knowledge.CampaignExternalEvidence, []knowledge.CampaignSourceFailure) { + maxTotal := c.cfg.DiscoveryMaxTotalItems() + out := []knowledge.CampaignExternalEvidence{} + failures := []knowledge.CampaignSourceFailure{} + for _, src := range c.cfg.Recheck.Discovery.Sources { + if len(out) >= maxTotal { + break + } + items, err := c.collectSource(ctx, src, base, since) + if err != nil { + failures = append(failures, knowledge.CampaignSourceFailure{Name: src.Name, Reason: discoveryFailureReason(err)}) + continue + } + remaining := maxTotal - len(out) + if len(items) > remaining { + items = items[:remaining] + } + out = append(out, items...) + } + return out, failures +} + +func (c recheckDiscoveryCollector) collectSource(ctx context.Context, src config.SpektacularRecheckDiscoverySource, base Campaign, since time.Time) ([]knowledge.CampaignExternalEvidence, error) { + switch strings.TrimSpace(src.Kind) { + case "upstream_release": + return c.githubReleases(ctx, src, since) + case "repo_activity": + return c.githubActivity(ctx, src, since) + case "standards_feed": + return c.feedItems(ctx, src, since) + case "landscape": + return c.landscape(ctx, src, base, since) + default: + return nil, fmt.Errorf("bad_kind") + } +} + +func (c recheckDiscoveryCollector) githubReleases(ctx context.Context, src config.SpektacularRecheckDiscoverySource, since time.Time) ([]knowledge.CampaignExternalEvidence, error) { + if c.gh == nil { + return nil, fmt.Errorf("github_unavailable") + } + owner, repo, ok := splitRepoSlug(src.URLOrRepo) + if !ok { + return nil, fmt.Errorf("bad_repo") + } + opt := &gh.ListOptions{PerPage: src.EffectiveMaxItems()} + releases, _, err := c.gh.Repositories.ListReleases(ctx, owner, repo, opt) + if err != nil { + return nil, err + } + out := []knowledge.CampaignExternalEvidence{} + for _, rel := range releases { + if rel.GetDraft() || (rel.GetPrerelease() && !src.AllowPrerelease) { + continue + } + published := rel.GetPublishedAt().Time + if !since.IsZero() && !published.After(since) { + continue + } + out = append(out, knowledge.CampaignExternalEvidence{ + Source: src.Name, + Kind: "upstream_release", + Title: rel.GetName(), + URL: rel.GetHTMLURL(), + PublishedAt: published, + Summary: truncateDiscoveryRunes(rel.GetBody(), externalSummaryMaxRunes), + }) + if len(out) >= src.EffectiveMaxItems() { + break + } + } + return out, nil +} + +func (c recheckDiscoveryCollector) githubActivity(ctx context.Context, src config.SpektacularRecheckDiscoverySource, since time.Time) ([]knowledge.CampaignExternalEvidence, error) { + releases, err := c.githubReleases(ctx, src, since) + if err != nil { + return nil, err + } + if c.gh == nil { + return nil, fmt.Errorf("github_unavailable") + } + owner, repo, ok := splitRepoSlug(src.URLOrRepo) + if !ok { + return nil, fmt.Errorf("bad_repo") + } + prs, _, err := c.gh.PullRequests.List(ctx, owner, repo, &gh.PullRequestListOptions{ + State: "closed", Sort: "updated", Direction: "desc", ListOptions: gh.ListOptions{PerPage: src.EffectiveMaxItems()}, + }) + if err != nil { + return nil, err + } + out := releases + for _, pr := range prs { + if pr.MergedAt == nil { + continue + } + merged := pr.GetMergedAt().Time + if !since.IsZero() && !merged.After(since) { + continue + } + out = append(out, knowledge.CampaignExternalEvidence{ + Source: src.Name, + Kind: "repo_activity", + Title: pr.GetTitle(), + URL: pr.GetHTMLURL(), + PublishedAt: merged, + Summary: truncateDiscoveryRunes(pr.GetBody(), externalSummaryMaxRunes), + }) + if len(out) >= src.EffectiveMaxItems() { + break + } + } + sortEvidence(out) + if len(out) > src.EffectiveMaxItems() { + out = out[:src.EffectiveMaxItems()] + } + return out, nil +} + +func (c recheckDiscoveryCollector) feedItems(ctx context.Context, src config.SpektacularRecheckDiscoverySource, since time.Time) ([]knowledge.CampaignExternalEvidence, error) { + u, err := url.Parse(strings.TrimSpace(src.URLOrRepo)) + if err != nil || u.Scheme == "" || u.Host == "" { + return nil, fmt.Errorf("bad_feed_url") + } + req, err := http.NewRequestWithContext(ctx, http.MethodGet, u.String(), nil) + if err != nil { + return nil, err + } + client := c.client + if client == nil { + client = http.DefaultClient + } + resp, err := client.Do(req) + if err != nil { + return nil, err + } + defer resp.Body.Close() + if resp.StatusCode < http.StatusOK || resp.StatusCode >= http.StatusMultipleChoices { + return nil, fmt.Errorf("http_%d", resp.StatusCode) + } + body, err := io.ReadAll(io.LimitReader(resp.Body, discoveryFeedReadLimit)) + if err != nil { + return nil, err + } + items := parseFeedEvidence(src.Name, body) + out := []knowledge.CampaignExternalEvidence{} + for _, item := range items { + if !since.IsZero() && !item.PublishedAt.IsZero() && !item.PublishedAt.After(since) { + continue + } + out = append(out, item) + if len(out) >= src.EffectiveMaxItems() { + break + } + } + return out, nil +} + +func (c recheckDiscoveryCollector) landscape(ctx context.Context, src config.SpektacularRecheckDiscoverySource, _ Campaign, since time.Time) ([]knowledge.CampaignExternalEvidence, error) { + raw, err := readLandscape() + if err != nil { + return nil, err + } + repos := githubReposFromMarkdown(string(raw)) + out := []knowledge.CampaignExternalEvidence{} + for _, repo := range repos { + child := src + child.URLOrRepo = repo + child.Name = firstRunNonEmpty(src.Name, "landscape") + ":" + repo + items, err := c.githubReleases(ctx, child, since) + if err != nil { + continue + } + for _, item := range items { + item.Source = src.Name + item.Kind = "landscape" + out = append(out, item) + if len(out) >= src.EffectiveMaxItems() { + sortEvidence(out) + return out, nil + } + } + } + sortEvidence(out) + return out, nil +} + +func parseFeedEvidence(source string, body []byte) []knowledge.CampaignExternalEvidence { + type atomEntry struct { + Title string `xml:"title"` + Summary string `xml:"summary"` + Content string `xml:"content"` + Updated string `xml:"updated"` + Link []struct { + Href string `xml:"href,attr"` + } `xml:"link"` + } + type atomFeed struct { + Entries []atomEntry `xml:"entry"` + } + var atom atomFeed + if err := xml.Unmarshal(body, &atom); err == nil && len(atom.Entries) > 0 { + out := make([]knowledge.CampaignExternalEvidence, 0, len(atom.Entries)) + for _, e := range atom.Entries { + link := "" + if len(e.Link) > 0 { + link = e.Link[0].Href + } + out = append(out, knowledge.CampaignExternalEvidence{Source: source, Kind: "standards_feed", Title: strings.TrimSpace(e.Title), URL: link, PublishedAt: parseFeedTime(e.Updated), Summary: truncateDiscoveryRunes(firstRunNonEmpty(e.Summary, e.Content), externalSummaryMaxRunes)}) + } + return out + } + type rssItem struct { + Title string `xml:"title"` + Link string `xml:"link"` + Description string `xml:"description"` + PubDate string `xml:"pubDate"` + } + type rss struct { + Items []rssItem `xml:"channel>item"` + } + var r rss + _ = xml.Unmarshal(body, &r) + out := make([]knowledge.CampaignExternalEvidence, 0, len(r.Items)) + for _, item := range r.Items { + out = append(out, knowledge.CampaignExternalEvidence{Source: source, Kind: "standards_feed", Title: strings.TrimSpace(item.Title), URL: strings.TrimSpace(item.Link), PublishedAt: parseFeedTime(item.PubDate), Summary: truncateDiscoveryRunes(item.Description, externalSummaryMaxRunes)}) + } + return out +} + +func parseFeedTime(value string) time.Time { + value = strings.TrimSpace(value) + for _, layout := range []string{time.RFC3339, time.RFC3339Nano, time.RFC1123Z, time.RFC1123} { + if t, err := time.Parse(layout, value); err == nil { + return t + } + } + return time.Time{} +} + +func splitRepoSlug(value string) (string, string, bool) { + value = strings.TrimSpace(value) + if u, err := url.Parse(value); err == nil && u.Hostname() != "" { + parts := strings.Split(strings.Trim(u.Path, "/"), "/") + if len(parts) >= 2 { + return parts[0], strings.TrimSuffix(parts[1], ".git"), true + } + } + owner, repo, ok := strings.Cut(value, "/") + return strings.TrimSpace(owner), strings.TrimSpace(repo), ok && strings.TrimSpace(owner) != "" && strings.TrimSpace(repo) != "" +} + +func truncateDiscoveryRunes(value string, maxRunes int) string { + value = strings.TrimSpace(value) + if maxRunes <= 0 || utf8.RuneCountInString(value) <= maxRunes { + return value + } + runes := []rune(value) + return string(runes[:maxRunes]) +} + +func discoveryFailureReason(err error) string { + if err == nil { + return "" + } + if err == context.DeadlineExceeded { + return "timeout" + } + return strings.ReplaceAll(strings.TrimSpace(err.Error()), "\n", " ") +} + +func sortEvidence(items []knowledge.CampaignExternalEvidence) { + sort.SliceStable(items, func(i, j int) bool { + return items[i].PublishedAt.After(items[j].PublishedAt) + }) +} + +func readLandscape() ([]byte, error) { + for _, path := range []string{"docs/landscape.md", "src/docs/landscape.md"} { + if raw, err := os.ReadFile(path); err == nil { + return raw, nil + } + } + return nil, fmt.Errorf("landscape_unavailable") +} + +func githubReposFromMarkdown(markdown string) []string { + re := regexp.MustCompile(`https://github\.com/([A-Za-z0-9_.-]+/[A-Za-z0-9_.-]+)`) + seen := map[string]bool{} + out := []string{} + for _, match := range re.FindAllStringSubmatch(markdown, -1) { + repo := strings.TrimSuffix(match[1], ".git") + if !seen[repo] { + seen[repo] = true + out = append(out, repo) + } + } + return out +} diff --git a/src/pkg/dashboard/recheck_discovery_test.go b/src/pkg/dashboard/recheck_discovery_test.go new file mode 100644 index 0000000000..fe27526682 --- /dev/null +++ b/src/pkg/dashboard/recheck_discovery_test.go @@ -0,0 +1,161 @@ +package dashboard + +import ( + "context" + "errors" + "io" + "net/http" + "net/http/httptest" + "net/url" + "strings" + "testing" + "time" + + gh "github.com/google/go-github/v72/github" + "github.com/hivecommons/hive/pkg/config" + "github.com/hivecommons/hive/pkg/knowledge" +) + +func TestRecheckDiscoveryFeedFiltersSinceAndCaps(t *testing.T) { + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) { + w.Header().Set("Content-Type", "application/atom+xml") + _, _ = w.Write([]byte(` +old2026-01-01T00:00:00Zold summary +new one2026-10-05T12:00:00Znew summary +new two2026-10-05T13:00:00Znew summary +`)) + })) + defer srv.Close() + + collector := recheckDiscoveryCollector{cfg: config.SpektacularConfig{}, client: srv.Client()} + items, err := collector.feedItems(context.Background(), config.SpektacularRecheckDiscoverySource{ + Kind: "standards_feed", Name: "standards", URLOrRepo: srv.URL, MaxItems: 1, + }, time.Date(2026, 10, 5, 0, 0, 0, 0, time.UTC)) + if err != nil { + t.Fatalf("feedItems: %v", err) + } + if len(items) != 1 || items[0].Title != "new one" || items[0].Kind != "standards_feed" { + t.Fatalf("items = %#v, want first new item only", items) + } +} + +func TestRecheckDiscoveryCollectRecordsFailureNotFatal(t *testing.T) { + cfg := config.SpektacularConfig{} + cfg.Recheck.Discovery.MaxTotalItems = 1 + cfg.Recheck.Discovery.Sources = []config.SpektacularRecheckDiscoverySource{ + {Kind: "standards_feed", Name: "bad", URLOrRepo: "http://127.0.0.1:1/nope"}, + {Kind: "standards_feed", Name: "good", URLOrRepo: "https://example.com/feed.xml"}, + } + + collector := recheckDiscoveryCollector{ + cfg: cfg, + client: recheckRoundTripFunc(func(req *http.Request) (*http.Response, error) { + if req.URL.Host == "127.0.0.1:1" { + return nil, errors.New("dial blocked") + } + return &http.Response{ + StatusCode: http.StatusOK, + Body: io.NopCloser(strings.NewReader(`okhttps://example.com/okMon, 05 Oct 2026 12:00:00 +0000ok`)), + Header: make(http.Header), + }, nil + }).Client(), + } + items, failures := collector.collect(context.Background(), Campaign{}, time.Time{}) + if len(items) != 1 || items[0].Title != "ok" { + t.Fatalf("items = %#v, want successful source evidence", items) + } + if len(failures) != 1 || failures[0].Name != "bad" { + t.Fatalf("failures = %#v, want bad source recorded", failures) + } +} + +func TestRecheckDiscoveryGitHubReleasesFiltersSincePrereleaseAndCaps(t *testing.T) { + const releasesPath = "/repos/owner/repo/releases" + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + if r.URL.Path != releasesPath { + t.Fatalf("path = %s, want %s", r.URL.Path, releasesPath) + } + _, _ = w.Write([]byte(`[ + {"name":"old","html_url":"https://github.com/owner/repo/releases/tag/v1","published_at":"2026-01-01T00:00:00Z","body":"old"}, + {"name":"pre","html_url":"https://github.com/owner/repo/releases/tag/v2-rc1","published_at":"2026-10-05T11:00:00Z","prerelease":true,"body":"pre"}, + {"name":"new","html_url":"https://github.com/owner/repo/releases/tag/v2","published_at":"2026-10-05T12:00:00Z","body":"new"} + ]`)) + })) + defer srv.Close() + client := gh.NewClient(srv.Client()) + client.BaseURL = mustURL(t, srv.URL+"/") + collector := recheckDiscoveryCollector{cfg: config.SpektacularConfig{}, gh: client} + items, err := collector.githubReleases(context.Background(), config.SpektacularRecheckDiscoverySource{ + Kind: "upstream_release", Name: "up", URLOrRepo: "owner/repo", MaxItems: 1, + }, time.Date(2026, 10, 5, 0, 0, 0, 0, time.UTC)) + if err != nil { + t.Fatalf("githubReleases: %v", err) + } + if len(items) != 1 || items[0].Title != "new" || items[0].Kind != "upstream_release" { + t.Fatalf("items = %#v, want only non-prerelease new release", items) + } +} + +func TestRecheckDiscoveryRepoActivityIncludesMergedPR(t *testing.T) { + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + switch r.URL.Path { + case "/repos/owner/repo/releases": + _, _ = w.Write([]byte(`[]`)) + case "/repos/owner/repo/pulls": + _, _ = w.Write([]byte(`[{"title":"merged feature","html_url":"https://github.com/owner/repo/pull/7","merged_at":"2026-10-05T12:00:00Z","body":"body"}]`)) + default: + t.Fatalf("unexpected path %s", r.URL.Path) + } + })) + defer srv.Close() + client := gh.NewClient(srv.Client()) + client.BaseURL = mustURL(t, srv.URL+"/") + collector := recheckDiscoveryCollector{cfg: config.SpektacularConfig{}, gh: client} + items, err := collector.githubActivity(context.Background(), config.SpektacularRecheckDiscoverySource{ + Kind: "repo_activity", Name: "activity", URLOrRepo: "owner/repo", + }, time.Date(2026, 10, 5, 0, 0, 0, 0, time.UTC)) + if err != nil { + t.Fatalf("githubActivity: %v", err) + } + if len(items) != 1 || items[0].Title != "merged feature" || items[0].Kind != "repo_activity" { + t.Fatalf("items = %#v, want merged PR activity", items) + } +} + +func TestRecheckDiscoveryLandscapeExtractsGitHubRepos(t *testing.T) { + repos := githubReposFromMarkdown(`[one](https://github.com/owner/repo) and [dup](https://github.com/owner/repo) plus https://github.com/other/project/releases`) + if len(repos) != 2 || repos[0] != "owner/repo" || repos[1] != "other/project" { + t.Fatalf("repos = %#v, want stable unique GitHub repos", repos) + } +} + +func TestRecheckSpecPromptTitleIncludesExternalEvidence(t *testing.T) { + drift := campaignDriftFromArchive(knowledge.InceptionCampaignArchive{Drift: &knowledge.CampaignDrift{ + ExternalCount: 1, + External: []knowledge.CampaignExternalEvidence{{ + Source: "source", + Kind: "standards_feed", + Title: "signal", + URL: "https://example.com/signal", + Summary: "summary", + }}, + }}) + if drift == nil || drift.ExternalCount != 1 || !strings.Contains(drift.External[0].Summary, "summary") { + t.Fatalf("drift = %#v, want external evidence shape", drift) + } +} + +type recheckRoundTripFunc func(*http.Request) (*http.Response, error) + +func (f recheckRoundTripFunc) RoundTrip(req *http.Request) (*http.Response, error) { return f(req) } + +func (f recheckRoundTripFunc) Client() *http.Client { return &http.Client{Transport: f} } + +func mustURL(t *testing.T, value string) *url.URL { + t.Helper() + u, err := url.Parse(value) + if err != nil { + t.Fatalf("url.Parse(%q): %v", value, err) + } + return u +} diff --git a/src/pkg/dashboard/stage_leases.go b/src/pkg/dashboard/stage_leases.go index e43b40ce52..5649ef04eb 100644 --- a/src/pkg/dashboard/stage_leases.go +++ b/src/pkg/dashboard/stage_leases.go @@ -884,11 +884,41 @@ func (a *runStageAccessor) PendingRunStages(_ context.Context) ([]worksource.Run if l.stage == StageImplement && l.identity != runFanoutIdentity && s.runPlanHasWaves(l.runKey) { continue } - out = append(out, worksource.RunStage{RunKey: l.runKey, Stage: l.stage, Repo: l.repo, Title: l.runKey}) + title := l.runKey + if l.stage == StageSpec { + title = s.recheckSpecStageTitle(l.runKey, title) + } + out = append(out, worksource.RunStage{RunKey: l.runKey, Stage: l.stage, Repo: l.repo, Title: title}) } return out, nil } +func (s *Server) recheckSpecStageTitle(runKey, fallback string) string { + if s == nil || s.deps == nil || s.deps.Inception == nil { + return fallback + } + archive, err := s.deps.Inception.LoadCampaignArchive(runKey) + if err != nil || archive == nil || archive.Drift == nil || len(archive.Drift.External) == 0 { + return fallback + } + var b strings.Builder + b.WriteString(fallback) + b.WriteString(" — recheck external evidence for spec input: ") + for i, ev := range archive.Drift.External { + if i > 0 { + b.WriteString("; ") + } + b.WriteString(ev.Source) + b.WriteString(": ") + b.WriteString(ev.Title) + if ev.URL != "" { + b.WriteString(" ") + b.WriteString(ev.URL) + } + } + return b.String() +} + // StageHasLiveLease reports whether a connection is currently working that // exact run stage, in which case it must not be offered again. func (a *runStageAccessor) StageHasLiveLease(_ context.Context, runKey, stage string) (bool, error) { diff --git a/src/pkg/dashboard/static/index.html b/src/pkg/dashboard/static/index.html index b1a3fc8a6b..29cf7ab155 100644 --- a/src/pkg/dashboard/static/index.html +++ b/src/pkg/dashboard/static/index.html @@ -21874,6 +21874,10 @@

How Agents Use Knowledge (Priming)

const engine = c.engine || 'unknown engine'; const rc = c.recheck || {}; const recheckText = rc.enabled ? `Rechecks every ${escapeHtml(rc.interval || 'default')} · next ${escapeHtml(rc.next_at ? relativeAge(rc.next_at) : 'not scheduled')} · last delta ${Number(rc.last_delta_count || 0)}` : 'Rechecks off'; + const drift = c.drift || {}; + const external = Array.isArray(drift.external) ? drift.external : []; + const failed = Array.isArray(rc.sources_failed) ? rc.sources_failed : (Array.isArray(drift.sources_failed) ? drift.sources_failed : []); + const recheckExternal = external.length ? `
${Number(drift.external_count || external.length)} external signals
${external.map(ev => `
${escapeHtml(ev.source || '')} · ${escapeHtml(ev.kind || '')}: ${ev.url ? `${escapeHtml(ev.title || ev.url)}` : escapeHtml(ev.title || '')}${ev.summary ? `
${escapeHtml(ev.summary)}` : ''}
`).join('')}${failed.length ? `
Failed sources: ${failed.map(f => `${escapeHtml(f.name || '')} (${escapeHtml(f.reason || '')})`).join(', ')}
` : ''}
` : (Number(rc.external_count || 0) ? `
${Number(rc.external_count || 0)} external signals
` : ''); const recheckDisabled = rc.in_flight ? ' disabled' : ''; return `
@@ -21886,6 +21890,7 @@

How Agents Use Knowledge (Priming)

${escapeHtml(activity)}
${recheckText}
+ ${recheckExternal}
diff --git a/src/pkg/knowledge/inception.go b/src/pkg/knowledge/inception.go index d92090ff03..777a4813f7 100644 --- a/src/pkg/knowledge/inception.go +++ b/src/pkg/knowledge/inception.go @@ -784,12 +784,29 @@ type CampaignRecheck struct { // CampaignDrift records the evidence attached to a Spek recheck revision. type CampaignDrift struct { - CodebaseChanged bool `json:"codebase_changed"` - PriorHeadSHA string `json:"prior_head_sha,omitempty"` - CurrentHeadSHA string `json:"current_head_sha,omitempty"` - PriorRevision string `json:"prior_revision,omitempty"` - DeltaCount int `json:"delta_count"` - RecheckReason string `json:"recheck_reason,omitempty"` + CodebaseChanged bool `json:"codebase_changed"` + PriorHeadSHA string `json:"prior_head_sha,omitempty"` + CurrentHeadSHA string `json:"current_head_sha,omitempty"` + PriorRevision string `json:"prior_revision,omitempty"` + DeltaCount int `json:"delta_count"` + RecheckReason string `json:"recheck_reason,omitempty"` + External []CampaignExternalEvidence `json:"external,omitempty"` + ExternalCount int `json:"external_count,omitempty"` + SourcesFailed []CampaignSourceFailure `json:"sources_failed,omitempty"` +} + +type CampaignExternalEvidence struct { + Source string `json:"source"` + Kind string `json:"kind"` + Title string `json:"title"` + URL string `json:"url,omitempty"` + PublishedAt time.Time `json:"published_at,omitempty"` + Summary string `json:"summary,omitempty"` +} + +type CampaignSourceFailure struct { + Name string `json:"name"` + Reason string `json:"reason"` } // CampaignLease is the dashboard-level lock that prevents two operators from