Research Orchestrator Productization — runtime-backed multi-source research with citations and progress

[luci-efe]

Summary

Turn research into a runtime-backed feature instead of a partially broken or spec-only capability: decompose tasks, gather sources, synthesize output, and expose progress/citations through a real product flow.

Current State

Research exists in partial code/spec form, but there is still a mismatch between intended behavior and what the runtime/dashboard product actually delivers. Prior fixes focused on API compatibility, not product completeness.

Why This Matters

Research is one of the clearest higher-order parity features versus OpenClaw/OpenFang, but it only matters if it actually runs with source tracking, progress visibility, and stable runtime ownership.

Repo Evidence

• SPEC-025-research-orchestrator-fix.md addresses a compatibility break, not end-to-end productization.
• convex/research.ts and related research codepaths exist, but the product flow is still not treated as a fully shipped surface.
• Older specs and sprint documents around research are fragmented and stale.

External Grounding

• OpenFang documents a dedicated researcher capability with cited reports and verification-oriented flows: https://www.openfang.sh/ and https://www.openfang.sh/docs/workflows
• OpenClaw emphasizes tool-enabled multi-source workflows rather than a fake “research” label without execution: https://docs.openclaw.ai/
• Mastra workflows and agents should run in the persistent runtime, not in Convex actions: https://mastra.ai/docs and https://mastra.ai/llms.txt

Goals

• Define the supported research execution path in runtime.
• Support multi-source gathering with citations and progress updates.
• Provide a coherent CLI/dashboard entry point.
• Remove or archive stale research specs that no longer match the architecture.

Non-Goals

• Autonomous swarm features beyond the current runtime model.
• Slide deck/video output generators.
• Managed cloud research delivery.

Acceptance Criteria

• Research requests execute through supported runtime codepaths.
• Source URLs/citations are preserved in output.
• Progress or run status is visible to the user.
• Tests cover success, partial-failure, and empty-source scenarios.
• Docs and UI reflect actual supported depth/options.

Implementation Notes

• Keep orchestration in runtime and persistence in Convex.
• Reuse existing browser/search/tooling primitives where they are already stable.
• Do not treat a CLI stub or partial action as feature parity.

Validation Plan

• Run a full research task end to end.
• Verify citations, progress reporting, and saved outputs.
• Add tests around planner/executor/synthesis boundaries.

Dependencies

• Depends on: MCP Lifecycle Completion — real connection management, dynamic tools, and end-to-end validation #15, Sandbox Execution Productization — stabilize Docker/local sandbox providers and close parity gap #38.