Conversation
npm audit fix, no --force needed: vite 6.4.1 -> 6.4.3, plus transitive bumps to postcss, nanoid, and picomatch. All within the ranges package.json already declares, so only the lockfile moves. The one that mattered here is the Vite dev server's arbitrary file read via WebSocket (GHSA-p9ff-h696-f583) — this project's documented workflow runs that dev server. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Nine call sites sent JARVIS's reply only inside `if audio:`, so whenever
Fish Audio was unconfigured, rate-limited, or failing, the reply was
dropped with no trace in the UI. The startup greeting and every proactive
notification — research complete, build finished, project connected — went
straight to nothing. Each site now falls back to a "text" message, which
the client can surface.
Three of those sites also passed the raw `bytes` from synthesize_speech
straight to send_json as `"data": audio`, without base64. Bytes are not
JSON-serializable, so the send raised — and in two cases the exception was
swallowed by a bare `except Exception: pass`. Those paths ("Fix Yourself"
and the calendar/mail lookups) delivered neither audio nor text even with
a working Fish key, and did so silently.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The client's stated "text fallback when TTS fails" only ran console.log, so a reply that arrived without audio reached the devtools console and nowhere else. With no Fish Audio key the interface was not merely silent but blank — the orb reacted to nothing and said nothing. Add a caption below the orb, in the existing palette. It is shown whether or not audio arrived, so it doubles as subtitles for a working voice. It holds for 4s plus 60ms per character, capped at 20s. Reading time rather than a fixed delay, because without a voice the caption is the entire response and vanishing mid-sentence loses it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
recognition.lang was hardcoded to en-US, so anyone speaking another
language was transcribed phonetically into English nonsense — "come stai"
arrived as "comic Style". Add a Spoken Language selector to the settings
panel, persisted as SPEECH_LANG and read live, so the choice applies
without a restart.
voice.ts rebuilds the recognition object on a language change rather than
reassigning .lang. Chrome reads that property at construction and ignores
it on a session that has already run, so the reassignment silently kept
the old language. The retired session's onend checks whether it has been
superseded, or it would restart and race the new one for the microphone.
main.ts resolves the language before opening the microphone, so the first
session is built correctly and the fragile switch path is reserved for
live changes from the panel.
The system prompt is English and would answer an Italian question in
English, so a non-English selection appends a language directive. Kept to
a bare directive deliberately: wordings that also discussed action
selection measurably weakened the language adherence they were added to
enforce.
Known limitation: action routing is less reliable outside English. On one
repeated question, English chose the right action 4/4 while Italian
misrouted to a screen capture 4/4. The 46 hardcoded English strings in
server.py ("Right away, sir.") also stay English — translating them is an
i18n project, not a setting.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Chrome closes a recognition segment at every pause, and the client sent each "final" result the instant it arrived. A sentence spoken with any hesitation reached JARVIS as several questions: he answered half a thought, then received the rest as a new one. The logs show single words like "hey" arriving alone. Collect fragments and send once the speaker has actually stopped, after a 1s gap. Two edges that would otherwise bite: - A 6s ceiling. Steady dictation never produces the gap, so without a cap the timer re-arms forever and the reply never comes. - Muting discards what is pending, so half a sentence cannot arrive a second after the user silenced him. Barge-in stays immediate: the audio is cut on the first fragment, not at flush, since interrupting is what the user wanted the moment they spoke. Costs 1s before JARVIS starts thinking. That is the price of knowing the sentence ended; UTTERANCE_GAP_MS trades it back for more split sentences. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Three complaints from one Italian session, three separate causes.
**He was called the wrong thing.** HONORIFIC was written to .env, served by
the preferences API and offered in the settings panel — and never read.
The prompt hardcoded `Address {user_name} as "sir"`, so the dropdown did
nothing. Worse, the model translated that "sir" freely and settled on
"signora": it addressed him with a feminine honorific, five times out of
five, while his configuration said otherwise.
The honorific is now a prompt variable, and the language directive states
it must be reproduced letter for letter rather than translated or
inflected. Placing that in the language block matters — the same
instruction in the personality section was ignored 5/5, and obeyed 5/5 at
the end. Measured, both ways round: "sir" and "signore" now each hold 5/5.
The panel field becomes free text with suggestions. It offered sir, ma'am
and none, so "signore" was not expressible in the first place.
**The greeting arrived in English.** It was hardcoded, so every Italian
session opened with "Good evening, sir." and switched language on the next
sentence. It is the one line a user hears every single time. Now built
from a small table for the languages the panel offers, falling back to
English.
**It could not hear its own name.** An Italian recogniser mangles an
English name: the logs show "arbiss", "e gli arbis" and "hey Yaris" where
he plainly said JARVIS. Added as corrections, taken from the transcripts
rather than invented.
"yaris" is handled only after a term of address. It is also a very common
Toyota, and a bare rule rewrote "la mia auto Yaris" into a summons.
Verified in both directions, including that "arbitro", "arbitri" and
"arbitrario" survive untouched.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This was referenced Aug 11, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Three complaints from one Italian session — "why does it call me by another name?", "I keep hearing English and then immediately Italian", "it often doesn't understand when I call it" — and three separate causes.
He was called the wrong thing
HONORIFICwas written to.env, served by the preferences API, and offered as a dropdown in the settings panel. Nothing ever read it. The prompt hardcoded:So the dropdown was decorative. And in Italian the model translated that "sir" freely, settling on "signora" — addressing him with a feminine honorific, 5 times out of 5, while his configuration said
sir.The honorific is now a prompt variable, and the language directive states it must be reproduced letter for letter rather than translated or inflected for gender.
Where that instruction lives turned out to matter. In the personality section it was ignored 5/5. Moved into the language block appended at the end, it holds 5/5. Measured both ways round, so the fix is not merely "it stopped saying signora":
sirsignoreThe panel field also becomes free text with suggestions. It offered
sir,ma'amandnone— sosignorewas not expressible in the first place, and no amount of prompt work would have helped.The greeting arrived in English
Hardcoded, sent on connect. Every Italian session therefore opened in English and switched language on the very next sentence — exactly the "first English, then immediately Italian" the user described. It is the one line heard every single time.
Now built from a small table covering the languages the panel offers, falling back to English for anything else. Thirty short strings, bounded — not an attempt to solve the wider hardcoded-string problem.
It could not hear its own name
An Italian recogniser mangles an English name. From the logs, where he was plainly saying JARVIS:
Added to the existing correction table — taken from real transcripts rather than invented.
yarisis corrected only when preceded by a term of address. It is also a very common Toyota, and the obvious bare rule rewrote "la mia auto Yaris" into a summons. Verified in both directions, including thatarbitro,arbitriandarbitrariopass through untouched — the word-boundary anchors hold.Testing
pytest tests/gives 35 passed, 8 failed — identical to this branch's base, so no regressions. The 8 pre-existing failures are unrelated: 7 needplaywright install, andtest_browse_action_keywordsimportsACTION_KEYWORDS, a symbol that no longer exists inserver.py.tsc --noEmitclean.🤖 Generated with Claude Code