-
Notifications
You must be signed in to change notification settings - Fork 0
feat: stub agent driver for QA acceleration #33
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
Fullstop000
wants to merge
19
commits into
main
Choose a base branch
from
claude/stub-agent-driver
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
19 commits
Select commit
Hold shift + click to select a range
39f2041
docs: add stub agent driver design spec
Fullstop000 7344e11
docs: address review feedback on stub driver spec
Fullstop000 b8a1b09
docs: add stub agent driver implementation plan
Fullstop000 845f4d4
build: convert to Cargo workspace, add stub-agent crate scaffold
Fullstop000 5825e15
feat(agent): add AgentRuntime::Stub enum variant
Fullstop000 41fd953
feat(agent): add StubDriver implementation
Fullstop000 bfdb3ab
feat(agent): hide stub runtime from /runtimes API
Fullstop000 bc9131b
feat(stub-agent): implement MCP client binary with echo logic
Fullstop000 3e6b80f
feat(stub-agent): add STUB_DELAY_MS configurable response delay
Fullstop000 751c3d5
feat(qa): add stub-trio preset and ensureStubTrio helper
Fullstop000 ab2c060
feat(qa): wire MSG-002 to support CHORUS_E2E_LLM=stub mode
Fullstop000 55afb16
feat(qa): wire all specs for CHORUS_E2E_LLM=stub mode
Fullstop000 e4a53b2
fix(stub-agent): address review — chat MCP key, skip footer lines, no…
Fullstop000 8f21e22
fix(qa): stub Playwright pass — workspace build, invites, ERR-001, teams
Fullstop000 a0afcd2
fix(qa): stub E2E stability — timeouts, members panel, AGT-004 stub path
Fullstop000 75f8034
fix(qa): stub MSG-001/REC-002 assert agent-only content; longer app r…
Fullstop000 a3ca813
fix(qa): stub MSG-001 fan-out poll + gotoApp retry; REC-002 reload be…
Fullstop000 c0143cc
style: cargo fmt (stub-agent, StubDriver)
Fullstop000 50a00ba
docs(qa): sync stub E2E notes with Playwright behavior
Fullstop000 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,17 @@ | ||
| [package] | ||
| name = "chorus-stub-agent" | ||
| version = "0.1.0" | ||
| edition = "2021" | ||
|
|
||
| [[bin]] | ||
| name = "chorus-stub-agent" | ||
| path = "src/main.rs" | ||
|
|
||
| [dependencies] | ||
| rmcp = { version = "0.16", features = ["client", "transport-async-rw"] } | ||
| serde = { version = "1", features = ["derive"] } | ||
| serde_json = "1" | ||
| tokio = { version = "1", features = ["full"] } | ||
| anyhow = "1" | ||
| regex = "1" | ||
| uuid = { version = "1", features = ["v4"] } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,314 @@ | ||
| use std::sync::atomic::{AtomicU64, Ordering}; | ||
|
|
||
| use anyhow::{Context, Result}; | ||
| use regex::Regex; | ||
| use rmcp::model::CallToolRequestParams; | ||
| use rmcp::{ClientHandler, ServiceExt}; | ||
| use serde::Deserialize; | ||
| use tokio::process::Command; | ||
|
|
||
| static SEQ: AtomicU64 = AtomicU64::new(0); | ||
|
|
||
| // --------------------------------------------------------------------------- | ||
| // CLI args (minimal hand-parse to avoid adding clap) | ||
| // --------------------------------------------------------------------------- | ||
|
|
||
| struct Args { | ||
| mcp_config: String, | ||
| #[allow(dead_code)] | ||
| prompt: String, | ||
| } | ||
|
|
||
| fn parse_args() -> Result<Args> { | ||
| let args: Vec<String> = std::env::args().collect(); | ||
| let mut mcp_config = None; | ||
| let mut prompt = None; | ||
| let mut i = 1; | ||
| while i < args.len() { | ||
| match args[i].as_str() { | ||
| "--mcp-config" => { | ||
| i += 1; | ||
| mcp_config = Some(args.get(i).context("missing --mcp-config value")?.clone()); | ||
| } | ||
| "--prompt" => { | ||
| i += 1; | ||
| prompt = Some(args.get(i).context("missing --prompt value")?.clone()); | ||
| } | ||
| _ => {} | ||
| } | ||
| i += 1; | ||
| } | ||
| Ok(Args { | ||
| mcp_config: mcp_config.context("--mcp-config is required")?, | ||
| prompt: prompt.context("--prompt is required")?, | ||
| }) | ||
| } | ||
|
|
||
| // --------------------------------------------------------------------------- | ||
| // MCP config parsing | ||
| // --------------------------------------------------------------------------- | ||
|
|
||
| #[derive(Deserialize)] | ||
| struct McpConfig { | ||
| #[serde(rename = "mcpServers")] | ||
| mcp_servers: std::collections::HashMap<String, McpServerEntry>, | ||
| } | ||
|
|
||
| #[derive(Deserialize)] | ||
| struct McpServerEntry { | ||
| command: String, | ||
| #[serde(default)] | ||
| args: Vec<String>, | ||
| } | ||
|
|
||
| fn load_mcp_config(path: &str) -> Result<(String, Vec<String>)> { | ||
| let data = std::fs::read_to_string(path) | ||
| .with_context(|| format!("Failed to read MCP config at {path}"))?; | ||
| let config: McpConfig = | ||
| serde_json::from_str(&data).context("Failed to parse MCP config JSON")?; | ||
| let entry = config | ||
| .mcp_servers | ||
| .get("chat") | ||
| .context("No MCP server entry named 'chat' in config")?; | ||
| Ok((entry.command.clone(), entry.args.clone())) | ||
| } | ||
|
|
||
| // --------------------------------------------------------------------------- | ||
| // JSON stdout protocol | ||
| // --------------------------------------------------------------------------- | ||
|
|
||
| fn emit(value: serde_json::Value) { | ||
| // Print to our own stdout — the manager reads these lines. | ||
| println!("{}", serde_json::to_string(&value).unwrap()); | ||
| } | ||
|
|
||
| fn emit_session_init(session_id: &str) { | ||
| emit(serde_json::json!({"type": "session_init", "session_id": session_id})); | ||
| } | ||
|
|
||
| fn emit_text(text: &str) { | ||
| emit(serde_json::json!({"type": "text", "text": text})); | ||
| } | ||
|
|
||
| fn emit_tool_call(name: &str, input: &serde_json::Value) { | ||
| emit(serde_json::json!({"type": "tool_call", "name": name, "input": input})); | ||
| } | ||
|
|
||
| fn emit_turn_end() { | ||
| emit(serde_json::json!({"type": "turn_end"})); | ||
| } | ||
|
|
||
| fn emit_error(message: &str) { | ||
| emit(serde_json::json!({"type": "error", "message": message})); | ||
| } | ||
|
|
||
| // --------------------------------------------------------------------------- | ||
| // MCP client handler (no-op — we only call tools, never receive requests) | ||
| // --------------------------------------------------------------------------- | ||
|
|
||
| struct StubClientHandler; | ||
| impl ClientHandler for StubClientHandler {} | ||
|
|
||
| // --------------------------------------------------------------------------- | ||
| // Tool helpers | ||
| // --------------------------------------------------------------------------- | ||
|
|
||
| async fn call_tool( | ||
| peer: &rmcp::service::Peer<rmcp::RoleClient>, | ||
| name: &str, | ||
| args: serde_json::Value, | ||
| ) -> Result<String> { | ||
| let params = CallToolRequestParams { | ||
| name: std::borrow::Cow::Owned(name.to_string()), | ||
| arguments: Some(args.as_object().cloned().unwrap_or_default()), | ||
| meta: None, | ||
| task: None, | ||
| }; | ||
| let result = peer.call_tool(params).await?; | ||
| let text: String = result | ||
| .content | ||
| .iter() | ||
| .filter_map(|c| c.raw.as_text().map(|t| t.text.as_str())) | ||
| .collect::<Vec<_>>() | ||
| .join("\n"); | ||
| Ok(text) | ||
| } | ||
|
|
||
| async fn wait_for_message(peer: &rmcp::service::Peer<rmcp::RoleClient>) -> Result<String> { | ||
| let args = serde_json::json!({}); | ||
| emit_tool_call("wait_for_message", &args); | ||
| call_tool(peer, "wait_for_message", args).await | ||
| } | ||
|
|
||
| async fn send_message( | ||
| peer: &rmcp::service::Peer<rmcp::RoleClient>, | ||
| target: &str, | ||
| content: &str, | ||
| ) -> Result<String> { | ||
| let args = serde_json::json!({"target": target, "content": content}); | ||
| emit_tool_call("send_message", &args); | ||
| call_tool(peer, "send_message", args).await | ||
| } | ||
|
|
||
| // --------------------------------------------------------------------------- | ||
| // Token extraction from message content | ||
| // --------------------------------------------------------------------------- | ||
|
|
||
| fn extract_token(content: &str) -> Option<String> { | ||
| // Patterns: reply with "TOKEN", reply with TOKEN, token: TOKEN, echo "TOKEN", say "TOKEN" | ||
| let patterns = [ | ||
| r#"(?i)reply\s+with\s+"([^"]+)""#, | ||
| r#"(?i)reply\s+with\s+(\S+)"#, | ||
| r#"(?i)token:\s*(\S+)"#, | ||
| r#"(?i)echo\s+"([^"]+)""#, | ||
| r#"(?i)say\s+"([^"]+)""#, | ||
| ]; | ||
| for pat in &patterns { | ||
| if let Ok(re) = Regex::new(pat) { | ||
| if let Some(caps) = re.captures(content) { | ||
| if let Some(m) = caps.get(1) { | ||
| return Some(m.as_str().to_string()); | ||
| } | ||
| } | ||
| } | ||
| } | ||
| None | ||
| } | ||
|
|
||
| fn next_fallback_token() -> String { | ||
| let seq = SEQ.fetch_add(1, Ordering::Relaxed); | ||
| format!("stub-reply-{seq}") | ||
| } | ||
|
|
||
| // --------------------------------------------------------------------------- | ||
| // Parse target from bridge message format | ||
| // --------------------------------------------------------------------------- | ||
|
|
||
| fn parse_target(line: &str) -> Option<String> { | ||
| // Format: [target=#channel msg=... time=... type=...] @sender: content | ||
| let re = Regex::new(r"\[target=(\S+)\s").ok()?; | ||
| re.captures(line) | ||
| .and_then(|c| c.get(1).map(|m| m.as_str().to_string())) | ||
| } | ||
|
|
||
| fn parse_content(line: &str) -> Option<String> { | ||
| // After "] @sender: " comes the content. Sender may contain spaces (OS usernames); | ||
| // do not use `\S+` here — that breaks token extraction and yields empty content. | ||
| let re = Regex::new(r"\]\s+@([^:]+):\s*(.+)$").ok()?; | ||
| re.captures(line) | ||
| .and_then(|c| c.get(2).map(|m| m.as_str().to_string())) | ||
| } | ||
|
|
||
| // --------------------------------------------------------------------------- | ||
| // Main | ||
| // --------------------------------------------------------------------------- | ||
|
|
||
| #[tokio::main] | ||
| async fn main() { | ||
| if let Err(e) = run().await { | ||
| emit_error(&format!("{e:#}")); | ||
| std::process::exit(1); | ||
| } | ||
| } | ||
|
|
||
| async fn run() -> Result<()> { | ||
| let Args { | ||
| mcp_config, | ||
| prompt: _, | ||
| } = parse_args()?; | ||
| let (command, cmd_args) = load_mcp_config(&mcp_config)?; | ||
|
|
||
| // Drain stdin in background to prevent buffer fill-up. | ||
| // The manager writes stdin notifications but the bridge handles delivery via wait_for_message. | ||
| tokio::spawn(async move { | ||
| use tokio::io::AsyncBufReadExt; | ||
| let stdin = tokio::io::stdin(); | ||
| let reader = tokio::io::BufReader::new(stdin); | ||
| let mut lines = reader.lines(); | ||
| while let Ok(Some(_line)) = lines.next_line().await { | ||
| // consumed — bridge handles delivery | ||
| } | ||
| }); | ||
|
|
||
| // Spawn bridge as child process | ||
| let mut child = Command::new(&command) | ||
| .args(&cmd_args) | ||
| .stdin(std::process::Stdio::piped()) | ||
| .stdout(std::process::Stdio::piped()) | ||
| .stderr(std::process::Stdio::null()) | ||
| .spawn() | ||
| .with_context(|| format!("Failed to spawn bridge: {command}"))?; | ||
|
|
||
| let child_stdout = child.stdout.take().context("No stdout from bridge child")?; | ||
| let child_stdin = child.stdin.take().context("No stdin from bridge child")?; | ||
|
|
||
| // Connect as MCP client | ||
| let service = StubClientHandler | ||
| .serve((child_stdout, child_stdin)) | ||
| .await | ||
| .map_err(|e| anyhow::anyhow!("MCP handshake failed: {e}"))?; | ||
| let peer = service.peer().clone(); | ||
|
|
||
| // Emit session init | ||
| let session_id = uuid::Uuid::new_v4().to_string(); | ||
| emit_session_init(&session_id); | ||
|
|
||
| let delay_ms: u64 = std::env::var("STUB_DELAY_MS") | ||
| .unwrap_or_else(|_| "200".to_string()) | ||
| .parse() | ||
| .unwrap_or(200); | ||
|
|
||
| // Short status only — full `--prompt` can be large and may contain sensitive context. | ||
| emit_text("Processing prompt"); | ||
|
|
||
| // Main loop: wait for messages, extract token or use fallback, send reply | ||
| loop { | ||
| let response = match wait_for_message(&peer).await { | ||
| Ok(r) => r, | ||
| Err(e) => { | ||
| emit_error(&format!("wait_for_message failed: {e:#}")); | ||
| break; | ||
| } | ||
| }; | ||
|
|
||
| if response.contains("No new messages.") { | ||
| // No messages — loop back and wait again | ||
| continue; | ||
| } | ||
|
|
||
| // Process each line (multiple messages may arrive). Bridge output can include | ||
| // footers such as "Reply instructions:" — only handle real message header lines. | ||
| for line in response.lines() { | ||
| let line = line.trim(); | ||
| if line.is_empty() || line.contains("No new messages.") { | ||
| continue; | ||
| } | ||
| if !line.starts_with("[target=") { | ||
| continue; | ||
| } | ||
|
|
||
| let Some(target) = parse_target(line) else { | ||
| emit_error(&format!("Could not parse target from line: {line}")); | ||
| continue; | ||
| }; | ||
| let content = parse_content(line).unwrap_or_default(); | ||
| let token = extract_token(&content).unwrap_or_else(next_fallback_token); | ||
|
|
||
| tokio::time::sleep(std::time::Duration::from_millis(delay_ms)).await; | ||
| emit_text(&format!("Replying with: {token}")); | ||
|
|
||
| if let Err(e) = send_message(&peer, &target, &token).await { | ||
| emit_error(&format!("send_message failed: {e:#}")); | ||
| break; | ||
| } | ||
| } | ||
|
|
||
| emit_turn_end(); | ||
| } | ||
|
|
||
| // Clean up | ||
| drop(peer); | ||
| drop(service); | ||
| let _ = child.kill().await; | ||
| Ok(()) | ||
| } | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
wait_for_messageresponses include a trailing "Reply instructions" section (see bridge formatting), and this loop currently processes every line. Lines like "Reply instructions:" won’t match the[target=...]header, so the stub will still attempt to generate a token and send an extra message (often to the fallback target). Filter the loop to only handle actual message header lines (e.g., lines starting with[target=), and ignore the instruction/footer lines entirely.