Skip to content

✨ knowledge: seed spec research from Hive knowledge, ADRs and Context7 - #65

Open
castrojo wants to merge 2 commits into
hivecommons:mainfrom
castrojo:feat/59-research-knowledge-sources
Open

castrojo wants to merge 2 commits into
hivecommons:mainfrom
castrojo:feat/59-research-knowledge-sources

Conversation

@castrojo

Copy link
Copy Markdown

Fixes #59

What

  • Pluggable research sources. internal/research defines a Source interface and a Seed function. Four adapters ship with it:
    • knowledge: the configured knowledge stores, same ranking as knowledge search.
    • adr: Markdown ADRs in each registered repo's code root, under adr/, adrs/, doc/adr/, docs/adr/, docs/adrs/, docs/decisions/, docs/architecture/decisions/ or architecture/decisions/. The root is resolved through repo.Set.LocalSource.
    • hive: the Hive knowledge export that bin/contributor-agent.sh writes to ~/agent.md (# Agent Knowledge, ## <type>, ### <title>, Tags:). Active under Hive (HIVE_HUB set).
    • context7: GET /api/v2/libs/search, then a bounded /<id>/llms.txt?topic=<query>&tokens=2000 per library; the excerpts come from the fetched docs. Active under Hive or when CONTEXT7_API_KEY is set. The key is optional, as in Hive's own context7Get.
  • New command knowledge research <query> [--limit N].
    • It reports every source as used, skipped or unavailable with a reason.
    • Each finding has a cite (knowledge:<tier>/<name>/<path>, adr:<repo>/<path>, hive:<section>/<title>, context7:<id>) and a read field saying how to fetch the full body.
    • A source that is not configured or fails never fails the command.
  • Spec workflow.
    • The interview step runs the research before its first question and writes research_sources.md.
    • The spec scaffold gains a ## Research Sources section, and step 08 assembles it from that file. Citations therefore survive the rm -rf .spektacular/work/<spec> cleanup.
  • One ranking scale. knowledge.ScoreHit/CutoffFloor and store.DescribeBytes are exported so ADR and Hive-export findings use the same formula and evidence rules as store hits. Set.Search now calls ScoreHit, with no behaviour change.
  • Docs and changelog. docs/knowledge-base.md gains a "Seeding spec research" section and a command-table row; changelog record 000061_seed-spec-research is added and CHANGELOG.md updated; the spek-new skill (template plus installed .claude/.bob copies) mentions the research step.

Security choices

  • Spektacular only reads the launcher-written Hive export. It never fetches /api/knowledge/export itself and never touches HIVE_REGISTRATION_TOKEN. /api/knowledge/search needs dashboard auth, so contributors cannot use it anyway.
  • Context7 requests go only to the fixed https://context7.com base, and redirects are refused.
  • A library ID must be a single-/-prefixed [A-Za-z0-9/._-] path with no .., or it is dropped.
  • A standalone run with no key makes no network call.

Acceptance criteria

  • Under Hive, spec research pulls relevant knowledge entries and cites them. Covered by TestHiveSource_CitesRelevantExportEntries and TestHiveSource_TitleCountsAsEvidence, plus a smoke run with HIVE_HUB set and a fixture ~/agent.md:
    • hive used [('hive:Decisions/CLI flags follow cobra conventions', …)]
    • context7 used [('context7:/spf13/cobra', ['By default, Cobra ignores local flags on parent commands…'])] from the live Context7 API.
  • Standalone, it degrades to local knowledge with no error. Covered by TestSeed_DegradesPerSourceAndKeepsGoing, TestHiveSource_Degrades and TestContext7Source_Degrades. A smoke run without HIVE_HUB/CONTEXT7_API_KEY returned error: false, exit 0, with knowledge used and hive/context7 skipped with reasons.

Verification

  • go vet ./... passes.
  • go test -shuffle=on ./... passes except internal/autocommit TestIntegration_CommitPushesNothing. That test fails on my host because git-upload-pack is not installed; it is unrelated to this change.
  • End-to-end smoke in a fresh spektacular init claude project:
    • spec goto interview renders the research instruction and the research_sources.md path.
    • A docs/adr/0001-cobra.md in the repo code root comes back as adr:<repo>/docs/adr/0001-cobra.md.
    • A local knowledge entry comes back as knowledge:repo/<repo>/decisions/cobra-flags.md.
    • A blank query is refused with knowledge_query_required.

Not in this PR

The registered docs repo (spektacular-website) documents configuration and workflows. It may want a page for knowledge research and the new spec section; that belongs in a separate PR against that repo.

— hive: backend=omp

🐝 Hive Agent: contributor | SHA: 84be2b5

Add a pluggable research source interface (internal/research) and a
`knowledge research <query>` command that searches, in one pass, the
project's knowledge stores, ADR folders in registered repos' code roots,
the Hive knowledge export the Hive launcher writes to ~/agent.md, and
Context7 library documentation. Each source reports used, skipped or
unavailable with a reason, so a standalone run degrades to local
knowledge with no error. Spektacular only reads the Hive export and
never handles launcher credentials; the optional Context7 key is sent
only to context7.com and redirects are refused.

The spec interview step now seeds itself from this research, and the
findings it draws on are cited in a new `## Research Sources` spec
section assembled from research_sources.md, so citations outlive the
workflow's working directory.

Knowledge ranking exports ScoreHit/CutoffFloor and the store exports
DescribeBytes, so ADRs and Hive export entries rank on the same formula
and evidence rules as knowledge store hits.

Fixes hivecommons#59

Hive-Run: hivecommons#59
Hive-Plan: issue-59 seed spec research with Hive knowledge and Context7 domain docs
Hive-Spec: 000061_seed-spec-research#acceptance-criteria
Signed-off-by: castrojo <castrojo@users.noreply.github.com>
@kubestellar-prow

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign nicholasjackson for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

A failed docs fetch did not count toward the per-source limit, so a long
Context7 result list whose docs all failed triggered one request per
result, each up to the 15s timeout. Count attempts instead of findings.

Hive-Run: hivecommons#59
Hive-Plan: issue-59 seed spec research with Hive knowledge and Context7 domain docs
Hive-Spec: 000061_seed-spec-research#acceptance-criteria
Signed-off-by: castrojo <castrojo@users.noreply.github.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Seed spec research with Hive knowledge and Context7 domain docs

1 participant