Skip to content

feat(research): gap validation protocol — automated sweep + decision matrix #3734

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
ryanklee wants to merge 2 commits into
base: main
Choose a base branch
from
Open

feat(research): gap validation protocol — automated sweep + decision matrix #3734

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
b3203fa
feat(research): gap validation protocol — automated sweep + decision …
ryanklee May 21, 2026
a0c13c2
fix(lint): remove unused imports in gap validation tests
ryanklee May 21, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 41 additions & 0 deletions docs/research/gap-portfolio-SCHEMA.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# Gap Portfolio Registry Schema

**Co-located with:** `gap-portfolio-registry.yaml`

## Record Fields

| Field | Type | Required | Description |
|-------|------|----------|-------------|
| gap_id | string | yes | Unique identifier (GAP-NNN) |
| title | string | yes | Human-readable gap name |
| request_ref | string | yes | Cross-reference to REQ-* in hapax-requests |
| disposition | enum | yes | `execute` / `publish` / `hold` |
| validation_status | string | yes | Current validation state |
| uniqueness_score | float 0-1 | yes | How unique to this apparatus |
| composability_score | float 0-1 | yes | How well it composes with other gaps |
| decay_rate_halflife_days | int | yes | Time until gap value halves |
| unique_apparatus_required | bool | yes | Whether moat components are necessary |
| apparatus_justification | string | yes | Which moat components and why |
| last_reviewed | date | yes | Last operator review date |

## Disposition Values

- **execute** — actively being implemented/validated. WIP limit: exactly 1.
- **publish** — gap is documented and ready for publication as a gap map or paper.
- **hold** — gap is real but not being worked. Value decays per half-life.

## WIP Invariant

Exactly one gap may have `disposition: execute` at any time. The registry enforces this as a constraint, not a guideline.

## Moat Components

The 7 moat components referenced in `apparatus_justification`:

1. stigmergic_coordination
2. temporal_grounding
3. perceptual_embodiment
4. axiom_governance
5. single_operator_architecture
6. bayesian_claim_tracking
7. publication_bus_provenance
222 changes: 222 additions & 0 deletions docs/research/gap-portfolio-registry.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,222 @@
schema_version: 1
registry_id: research-gap-portfolio-v1
authority_case: CASE-20260509-RESEARCH-PO
wip_limit: 1
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot May 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

WIP invariant is currently violated by two active execute gaps.

wip_limit: 1 conflicts with two records marked disposition: execute (GAP-001 and GAP-002). This breaks the registry contract and can invalidate downstream prioritization logic.

Also applies to: 18-18, 30-30

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/research/gap-portfolio-registry.yaml` at line 4, The WIP invariant fails
because wip_limit: 1 conflicts with two records marked disposition: execute
(GAP-001 and GAP-002 — also applies to the entries at 18-18 and 30-30); fix by
either increasing the wip_limit value to at least 2 (update the wip_limit key)
or changing one or more of those records' disposition from execute to a
non-active state (e.g., backlog/defer) so the number of execute dispositions
does not exceed wip_limit; update the wip_limit or the disposition fields for
GAP-001/GAP-002 (and the entries at 18-18 and 30-30) accordingly.

created_at: '2026-05-20'
moat_components:
- stigmergic_coordination
- temporal_grounding
- perceptual_embodiment
- axiom_governance
- single_operator_architecture
- bayesian_claim_tracking
- publication_bus_provenance
gaps:
- gap_id: GAP-001
title: Multimodal grounding research pipeline
request_ref: REQ-20260509-multimodal-grounding-research-pipeline
disposition: execute
validation_status: validated_novel
uniqueness_score: 0.85
composability_score: 0.9
decay_rate_halflife_days: 180
unique_apparatus_required: true
apparatus_justification: Requires perceptual_embodiment (camera/IR fleet) + temporal_grounding
(bands architecture) + stigmergic_coordination (filesystem-as-bus context)
last_reviewed: '2026-05-20'
- gap_id: GAP-002
title: Epistemic quality infrastructure
request_ref: REQ-20260512-epistemic-quality-infrastructure
disposition: execute
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot May 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Enforce single active execute gap in registry

The registry declares wip_limit: 1 and the schema states exactly one gap may be disposition: execute, but this entry introduces a second execute gap alongside GAP-001. That breaks the documented invariant and makes downstream tooling/reporting ambiguous about which gap is the single active execution target.

Useful? React with 👍 / 👎.

validation_status: blocked_on_labels
uniqueness_score: 0.75
composability_score: 0.8
decay_rate_halflife_days: 365
unique_apparatus_required: true
apparatus_justification: Requires bayesian_claim_tracking + publication_bus_provenance
for calibrated epistemic scoring
last_reviewed: '2026-05-20'
- gap_id: GAP-003
title: Unb-AIRy discursive plane
request_ref: REQ-20260509-unb-airy-discursive-plane
disposition: publish
validation_status: validated_novel
uniqueness_score: 0.92
composability_score: 0.7
decay_rate_halflife_days: 270
unique_apparatus_required: true
apparatus_justification: Requires axiom_governance + single_operator_architecture
for assertion-plane enforcement unique to single-operator AI systems
last_reviewed: '2026-05-20'
- gap_id: GAP-004
title: Research positioning and publication pipeline
request_ref: REQ-20260509-research-positioning-and-publication
disposition: publish
validation_status: phase1_complete
uniqueness_score: 0.6
composability_score: 0.95
decay_rate_halflife_days: 120
unique_apparatus_required: false
apparatus_justification: Publication pipeline is standard tooling; positioning requires
unique content but not unique apparatus
last_reviewed: '2026-05-20'
- gap_id: GAP-005
title: Visibility engine and egress safety
request_ref: REQ-20260510-visibility-engine
disposition: hold
validation_status: phase1_complete
uniqueness_score: 0.5
composability_score: 0.85
decay_rate_halflife_days: 90
unique_apparatus_required: false
apparatus_justification: Standard publication automation; uniqueness comes from
content not tooling
last_reviewed: '2026-05-20'
- gap_id: GAP-006
title: Quality-preserving subscription routing
request_ref: REQ-20260509195830-quality-preserving-subscription-routing
disposition: hold
validation_status: spec_complete
uniqueness_score: 0.45
composability_score: 0.75
decay_rate_halflife_days: 60
unique_apparatus_required: false
apparatus_justification: Routing optimization is operational, not research-unique
last_reviewed: '2026-05-20'
- gap_id: GAP-007
title: Deliberative council engine
request_ref: REQ-20260515-deliberative-council-engine
disposition: publish
validation_status: validated_novel
uniqueness_score: 0.88
composability_score: 0.65
decay_rate_halflife_days: 365
unique_apparatus_required: true
apparatus_justification: Requires bayesian_claim_tracking + axiom_governance for
multi-agent claim disconfirmation unique to this architecture
last_reviewed: '2026-05-20'
- gap_id: GAP-008
title: 3D compositor direct v4l2 output
request_ref: REQ-20260514-3d-compositor-v4l2-direct-output
disposition: hold
validation_status: in_progress
uniqueness_score: 0.55
composability_score: 0.6
decay_rate_halflife_days: 90
unique_apparatus_required: false
apparatus_justification: Compositor plumbing, not a research contribution
last_reviewed: '2026-05-20'
- gap_id: GAP-009
title: CCTV benchmark and perplexity evaluation
request_ref: REQ-20260516-cctv-perplexity-benchmark
disposition: publish
validation_status: data_collected
uniqueness_score: 0.78
composability_score: 0.7
decay_rate_halflife_days: 180
unique_apparatus_required: true
apparatus_justification: Requires bayesian_claim_tracking + deliberative council
for claim-level perplexity benchmarking
last_reviewed: '2026-05-20'
- gap_id: GAP-010
title: Axiom gate coverage and property hardening
request_ref: REQ-20260515-axiom-gate-coverage-and-property-hardening
disposition: hold
validation_status: ongoing
uniqueness_score: 0.8
composability_score: 0.55
decay_rate_halflife_days: 365
unique_apparatus_required: true
apparatus_justification: Requires axiom_governance — constitutional constraint enforcement
is unique to this system
last_reviewed: '2026-05-20'
- gap_id: GAP-011
title: Monetization decision agent
request_ref: REQ-20260510-monetization-decision-agent
disposition: hold
validation_status: spec_complete
uniqueness_score: 0.4
composability_score: 0.7
decay_rate_halflife_days: 120
unique_apparatus_required: false
apparatus_justification: Standard revenue automation
last_reviewed: '2026-05-20'
- gap_id: GAP-012
title: Wyoming LLC formation
request_ref: REQ-20260510-wyoming-llc-formation
disposition: hold
validation_status: operator_action_required
uniqueness_score: 0.1
composability_score: 0.3
decay_rate_halflife_days: 365
unique_apparatus_required: false
apparatus_justification: Legal/business, not research
last_reviewed: '2026-05-20'
- gap_id: GAP-013
title: Livestream full surface success gate
request_ref: REQ-20260511152000-livestream-full-surface-success-gate
disposition: hold
validation_status: in_progress
uniqueness_score: 0.65
composability_score: 0.8
decay_rate_halflife_days: 60
unique_apparatus_required: false
apparatus_justification: Operational gate, not research
last_reviewed: '2026-05-20'
- gap_id: GAP-014
title: Disaster recovery plan
request_ref: REQ-20260512-disaster-recovery-plan
disposition: hold
validation_status: phase2_complete
uniqueness_score: 0.15
composability_score: 0.4
decay_rate_halflife_days: 365
unique_apparatus_required: false
apparatus_justification: Standard DR, not research
last_reviewed: '2026-05-20'
- gap_id: GAP-015
title: Governance repair strengthening program
request_ref: REQ-20260515-governance-repair-strengthening-program
disposition: publish
validation_status: ongoing
uniqueness_score: 0.82
composability_score: 0.6
decay_rate_halflife_days: 270
unique_apparatus_required: true
apparatus_justification: Requires axiom_governance + single_operator_architecture
for constitutional repair documentation
last_reviewed: '2026-05-20'
- gap_id: GAP-016
title: AoA triangular pane information surfaces
request_ref: REQ-20260516122534-aoa-triangular-pane-information-surfaces
disposition: hold
validation_status: avsdlc_complete
uniqueness_score: 0.7
composability_score: 0.55
decay_rate_halflife_days: 180
unique_apparatus_required: true
apparatus_justification: Requires perceptual_embodiment for self-visualization of
system state on compositor
last_reviewed: '2026-05-20'
- gap_id: GAP-017
title: Audio topology hardening program
request_ref: REQ-20260520-audio-topology-hardening-program
disposition: hold
validation_status: in_progress
uniqueness_score: 0.35
composability_score: 0.5
decay_rate_halflife_days: 90
unique_apparatus_required: false
apparatus_justification: Audio plumbing hardening, not research
last_reviewed: '2026-05-20'
- gap_id: GAP-018
title: HN launch preparation
request_ref: REQ-20260510-hackernews-launch-prep
disposition: hold
validation_status: in_progress
uniqueness_score: 0.2
composability_score: 0.4
decay_rate_halflife_days: 30
unique_apparatus_required: false
apparatus_justification: Launch ops, not research
last_reviewed: '2026-05-20'
90 changes: 90 additions & 0 deletions docs/research/gap-validation-observation-guide.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
# Phase 3: Practitioner Observation Guide

## Purpose

Remote contextual inquiry protocol for validating research gaps that remain
ambiguous after Phase 1 (automated sweep) and Phase 2 (community probe).
Use when the decision matrix yields fewer than 4 agreeing signals.

## When to Use

- Phase 1 sweep returned "low_confidence_needs_phase2" or "medium_confidence_novel"
- Phase 2 community probes returned mixed or no responses
- The gap involves tacit knowledge that published sources may not capture

## Participant Selection

Target 3-5 domain practitioners who:
- Work in the intersection area described by the gap
- Have 5+ years domain experience
- Are NOT the gap author (avoid confirmation bias)
- Represent different organizational contexts (academic, industry, open source)

## Interview Protocol (Remote Contextual Inquiry)

Duration: 30-45 minutes, recorded with consent.

### Opening (5 min)

Explain the purpose: "I'm validating whether a specific research gap is
genuinely novel. I'll describe a capability intersection and ask whether
you've encountered anything similar."

### Core Questions (20-30 min)

1. **Current practice:** "In your work on [domain], how do you currently
handle [core capability described by the gap]?"

2. **Combination awareness:** "Have you seen a system that combines
[component A] with [component B] in a single architecture? If so,
what were the results?"

3. **Barrier identification:** "What prevents practitioners in your field
from building something like [gap description]? Is it technical
complexity, lack of need, or something else?"

4. **Prior attempts:** "Are you aware of any projects — published or
unpublished — that attempted this combination? What happened to them?"

5. **Tacit knowledge:** "Is there domain knowledge about why this
combination is difficult or unnecessary that wouldn't appear in
published literature?"

6. **Community awareness:** "If someone had solved this, where would you
expect to find evidence? Which conferences, forums, or communities?"

7. **Validation of novelty:** "On a scale of 1-5, how surprised would
you be to learn that no existing system combines these capabilities?
(1 = not surprised at all, 5 = very surprised)"

### Closing (5 min)

- Ask for referrals to other practitioners
- Offer to share findings
- Confirm consent for anonymized use in publications

## Scoring

Map each interview to a signal vote:
- Surprise score 4-5 + no prior examples cited → **novel**
- Surprise score 2-3 + some partial examples → **inconclusive**
- Surprise score 1 + specific prior art cited → **prior_art_exists**

Aggregate across 3-5 participants. Majority vote becomes the
`practitioner_observation` signal in the decision matrix.

## Output

Write results to `{gap_id}-phase3-observation.json` with:
- participant_count
- anonymized responses per question
- individual votes
- aggregate vote
- referenced prior art (if any)

## Ethics

- Obtain informed consent before recording
- Anonymize all participant data in outputs
- Do not name participants in publications without explicit permission
- Offer co-authorship if contributions are substantial
Loading
Loading