Repository navigation
✨ Betty times every answer stage and answers live-state questions without waiting for search - #18629
clubanderson wants to merge 1 commit into
Conversation
…hout waiting for search Each answer carries heard, retrieval, model and answer-ready timings with its source and budget (4 s for the hive's live state, 8 s otherwise); the live session adds delivery and say times and keeps the last 20 with a latency summary in the live state served by the meeting detail API. Gather stops waiting once the hive status retriever answers. Part of #18590. Signed-off-by: Andy Anderson <andy@clubanderson.com>
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
Open PR overlap checkBase branch: Conflicts are checked with every PR merged onto the current tip of
|
Mutation testinggremlins found 4 surviving mutant(s) in changed files: the code ran under tests, but changing it did not fail any test.
5 mutant(s) in changed files are not reached by any test (NOT COVERED). A maintainer may apply the
ℹ️ 1 mutant(s) on added lines were SKIPPED because gremlins v0.6.0 only honours the first
Verdict: |
|
Combined into #18635 to get through CI and review together. |
Part 4 of the concurrent-requests work on #18590: speed visibility and the live-state fast path.
Stage timings. Every answer to a request carries
Reply.Timing: heard (final line), ack (engine side, andearly_ackwhen it was said from the partial transcript), retrieval, model call, answer ready, source (hive-status,retrieval,model,fallback) and budget:HiveStatusBudget4 s for questions about the hive's live state,AnswerBudget8 s otherwise. Over-budget answers are logged with their stages.The live session adds delivery (answer ready to posted or handed to the voice) and the
saycall time (Quorum accepts after TTS on Discord), and keeps the last 20 inState.PersonaTimingsplusState.PersonaLatency(answers, over budget, p50, max, last ack say time). The meeting detail API already returns the live state (GET /api/meetings/{id}/live→live), so both appear there with no API change.Live-state fast path.
Gatherwaited for every retriever up to 1.5 s, so a hive-status question waited for SearXNG. It now stops as soon as the hive status retriever returns its snippet: the in-memory status plus one model call, well inside the 4 s budget on the API backends.Expected latencies (estimates, not measured live: running hives and the meetings namespace are off limits for this work): hive-status question on Haiku: retrieval about 10 to 50 ms (was up to 1.5 s behind web search), model about 1 to 2 s, answer ready about 1 to 2 s after the final line. Knowledge/web question: retrieval up to 1.5 s, model 1 to 2.5 s, about 3 to 4 s. On the
claudeCLI backend add a few seconds per warm turn. The new timings give the real numbers per meeting.Touches the same lines as #18625 (attribution) in
onAnswer/onAddressedAnswer; whichever merges second rebases and keeps both.Tests:
betty/timing_test.go(stage timings with a stepped clock, hive-status budget and over-budget flag, fallback source,Gatherstops at the live state but waits otherwise),live/timing_test.go(timings and summary in the live state after an answer, last 20 kept, untimed replies not counted).🤖 Generated with Claude Code