From 2ef5b44b6ca21924070b12e3d8ac0aac85482ebc Mon Sep 17 00:00:00 2001 From: enumura1 Date: Sat, 26 Sep 2026 20:41:45 +0900 Subject: [PATCH] docs: add publication article drafts --- articles/EDITORIAL_NOTES.md | 88 ++++++++++++++ articles/devto-speech-captions-en.md | 138 ++++++++++++++++++++++ articles/zenn-web-speech-caption-ui-ja.md | 134 +++++++++++++++++++++ 3 files changed, 360 insertions(+) create mode 100644 articles/EDITORIAL_NOTES.md create mode 100644 articles/devto-speech-captions-en.md create mode 100644 articles/zenn-web-speech-caption-ui-ja.md diff --git a/articles/EDITORIAL_NOTES.md b/articles/EDITORIAL_NOTES.md new file mode 100644 index 0000000..7209f7c --- /dev/null +++ b/articles/EDITORIAL_NOTES.md @@ -0,0 +1,88 @@ +# GlyphFlow article notes + +These notes record the editorial research behind the Zenn and DEV Community +drafts in this directory. They are deliberately kept with the drafts so that a +future revision does not turn them into release announcements. + +## What the research changed + +The articles are not translations of each other. + +| Draft | Question it answers | Reader's outcome | +| ---------------------------------- | ------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------- | +| `zenn-web-speech-caption-ui-ja.md` | How can a browser-only caption move inside a word when the browser only reports word boundaries? | Understand the event model, the CSS rendering choice, and the limits. | +| `devto-speech-captions-en.md` | Why use one gradient-clipped word instead of changing a whole word or splitting it into letters? | Understand the UI trade-off and the bounded between-event animation. | + +Both drafts lead with a concrete mismatch between an expected UI and a first, +simpler implementation. They retain the project's real constraints rather than +claiming precise character or phoneme timing. + +## Research sources and publishing constraints + +- [Zenn community guidelines](https://zenn.dev/guideline): Zenn values concrete + experience and trial-and-error, asks authors to avoid advertising as the main + purpose, and recommends a clear overview, audience, context, and reproducible + conditions. +- [Zenn: writing a better technical article](https://zenn.dev/northward/articles/how-to-write-better-tech-articles): check that the title matches the article, + name the environment, include enough detail to reproduce the result, and read + the finished piece aloud for awkward rhythm. +- [Zenn: using AI support while writing](https://zenn.dev/rin/articles/bdbdab57997e1e) and + [a personal AI-writing workflow](https://zenn.dev/yoshiko/articles/my-ai-writing): use AI for structure or review, but rewrite the final voice from actual + experience rather than accepting generic prose. +- [DEV: high-quality posts](https://dev.to/devteam/how-to-write-a-high-quality-post-on-dev-3me0): use accurate titles, correct tags, alt text for images, and + deliver what the title promises. +- [DEV: AI-assisted article guidelines](https://dev.to/guidelines-for-ai-assisted-articles-on-dev): AI assistance must be disclosed. The author must stand + behind factual accuracy, and an article must not be created mainly to drive + external traffic or backlinks. +- [MDN: `SpeechSynthesisUtterance` `boundary`](https://developer.mozilla.org/en-US/docs/Web/API/SpeechSynthesisUtterance/boundary_event): a boundary event + marks a word or sentence boundary. It is not a per-character timing API and is + not Baseline across widely used browsers. + +## Voice rules applied to these drafts + +1. Start from an observed mismatch, not a product definition. In this project, + changing an entire word's color did not convey the motion the UI needed. +2. Preserve decisions that another author could not make generically: one word + span, gradient clipping, `charIndex` mapping, bounded progress, and no audio + alignment service. +3. Include the trade-offs next to the technique. A boundary event starts a word; + it does not reveal when every letter is spoken. +4. Do not manufacture a founder story, performance result, compatibility claim, + or user outcome. Before publication, replace any first-person wording with + the author's own, factually correct memory of the work. +5. Delete generic sales language. In particular, avoid opening with phrases such + as "a library that enables", ending with a broad request for feedback, or + presenting a symmetric feature list as the substance of the article. +6. Vary paragraph length only where it follows the thought. Artificially clipped + fragments and deliberate typos are not a human voice. +7. Use a small, verified code sample. Link to the repository for the complete + implementation instead of pasting a large, context-free excerpt. + +## Pre-publication checklist + +- [ ] The author has replaced or approved every first-person statement. +- [ ] Browser, OS, voice, rate, and language claims match a manual check. +- [ ] Code samples have been copied from, or tested against, the release source. +- [ ] Screenshots/GIFs show the actual inside-glyph wipe; no mock UI is presented + as a live boundary-timed result. +- [ ] The final call to action is a short project-links section, not the article's + purpose. +- [ ] The DEV post includes an AI-assistance disclosure if this draft materially + shaped the final prose, as DEV requires. +- [ ] The author reads the Japanese and English drafts aloud and removes wording + they would not naturally use. + +## Project facts verified from this repository + +- Package: `@enumura/glyphflow`, version `1.0.0`. +- Runtime package contents: `dist`, `README.md`, and `LICENSE` only. +- Rendering keeps each word as one `.kn-word` element and changes + `--kn-progress`; it does not split a word into character elements. +- `boundary` event handling uses `charIndex`, ignores stale/duplicate/out-of-order + events, and records a measured duration only when a following boundary is + received. +- The visual predictor uses lightweight spelling costs and bounded progress. It + targets 94% by its predicted end and never moves beyond 98.5% until the next + event or completion resolves the word. +- The documented initial support target is Chrome Desktop, English text, and a + voice that emits usable word boundaries. diff --git a/articles/devto-speech-captions-en.md b/articles/devto-speech-captions-en.md new file mode 100644 index 0000000..b12226a --- /dev/null +++ b/articles/devto-speech-captions-en.md @@ -0,0 +1,138 @@ + + +# I wanted browser speech captions to move, not blink + +The first version of this UI only changed the color of the current word. + +That did answer one question: which word is the browser speaking? It did not +look like the caption was moving with the voice. A word would sit unchanged and +then switch all at once. For a read-along interface, I wanted the color boundary +to travel through the letters instead. + +I built [GlyphFlow](https://github.com/speech-caption/glyphflow) around that +constraint. It is a small TypeScript library for Web Speech captions. This post +is about the rendering and timing decisions behind it, not a claim that the +browser can provide character-level speech alignment. + +## The useful event is also an incomplete one + +`SpeechSynthesisUtterance` can emit a `boundary` event while an utterance is +spoken. The event includes a character index and elapsed time. + +```ts +utterance.onboundary = (event) => { + console.log(event.name, event.charIndex, event.elapsedTime); +}; +``` + +That is enough to identify a word start in the original input. It is not enough +to say when the voice reached the second `t` in `letter`. + +This distinction shaped the whole implementation. The browser is the source of +truth for word starts. The motion between word starts is a visual estimate. + +It also puts a real limit on the project: Web Speech behavior depends on the +browser, OS, and installed voice. The initial support target is Chrome Desktop, +English text, and voices that produce usable word-boundary events. MDN marks +the event as limited availability rather than a Baseline feature. [Its boundary +event documentation](https://developer.mozilla.org/en-US/docs/Web/API/SpeechSynthesisUtterance/boundary_event) +is worth reading before treating it as a universal captioning API. + +## One word, one element + +I did not split each word into character elements. That would make it easy to +light up letters one at a time, but that is a different visual language: a +sequence of discrete changes rather than one boundary moving through a glyph. + +The rendered word stays as one span. + +```html +caption +``` + +The CSS background is the moving part. + +```css +.kn-word { + color: transparent; + background: linear-gradient( + 90deg, + var(--kn-highlight-color, #f3f7ff) 0%, + var(--kn-highlight-color, #8da7ff) var(--kn-progress), + var(--kn-unhighlight-color, #475061) var(--kn-progress), + var(--kn-unhighlight-color, #475061) 100% + ); + background-clip: text; + -webkit-background-clip: text; + -webkit-text-fill-color: transparent; +} +``` + +Updating `--kn-progress` from `0%` to `100%` moves the transition through the +actual letter shapes. The DOM still represents a word as a word, and punctuation +and whitespace can remain faithful to the source text. + +## A boundary starts the motion; the next one measures it + +There is a quiet gap in this model. A boundary tells me a word has started, but +the next boundary has not arrived yet. Leaving the word static for that whole +gap looks broken, so GlyphFlow starts a `requestAnimationFrame` loop. + +The initial duration estimate is intentionally lightweight: word length, a few +letter patterns, speech rate, and nearby punctuation. There is no dictionary, +phoneme model, audio file, or network call involved. + +The estimate does not complete the word at exactly 100%. It reaches 94% at the +predicted end, then eases no farther than 98.5%. When the next word boundary +arrives, the previous word becomes 100% and the elapsed-time difference becomes +an observed duration. + +```ts +const actualDuration = nextBoundary.elapsedTime - currentBoundary.elapsedTime; +``` + +That observed duration can gently influence later estimates in the same +utterance. It is useful when a particular voice and rate are consistently faster +or slower than the initial guess. It is not phoneme alignment, and a final word +has no following boundary from which to derive an observed duration. + +## Browser events need defensive state handling + +The more interesting bugs were not in the gradient. They were in event timing. + +A word boundary can be duplicated, arrive out of order, point near rather than +at a token start, or arrive after a narration was cancelled. A paused caption +must not keep progressing just because an animation frame is still scheduled. + +GlyphFlow assigns each `speak()` call a session identity. Events from an older +utterance cannot update a newer session's DOM or state. Duplicate and stale +boundaries are ignored. Pause time is excluded from visual progress. If speech +finishes without usable word boundaries, the caller receives an `unsupported` +state instead of a caption that quietly pretends it was synchronized. + +Those cases are less impressive in a demo than a flowing gradient, but they are +the difference between a visual experiment and a component that can be embedded +in another page. + +## What this is, and what it is not + +The result is useful when a browser voice should give a reader a place to look: +read-along captions, language exercises, or a narrated explanation. It is not a +tool for synchronizing supplied audio, producing MP3 files, or deriving +per-letter pronunciation timestamps. + +The project is MIT-licensed. If you want to inspect the implementation or try +the demo: + +- Source: https://github.com/speech-caption/glyphflow +- Package: https://www.npmjs.com/package/@enumura/glyphflow +- Demo: https://speech-caption.github.io/glyphflow/ diff --git a/articles/zenn-web-speech-caption-ui-ja.md b/articles/zenn-web-speech-caption-ui-ja.md new file mode 100644 index 0000000..060bc42 --- /dev/null +++ b/articles/zenn-web-speech-caption-ui-ja.md @@ -0,0 +1,134 @@ +--- +title: "Web Speech APIの単語境界だけで、文字の内側を塗り進める字幕UIを作る" +emoji: "🔊" +type: "tech" +topics: ["typescript", "webapi", "css", "webspeechapi"] +published: false +--- + +## はじめに + +ブラウザの読み上げに合わせて字幕を動かすなら、今読んでいる単語だけ色を変えればよいと思っていました。 + +実際に表示してみると、それでは足りませんでした。単語の切り替わりは分かりますが、音声と文字が一緒に進んでいる感じが出ません。欲しかったのは、単語が一瞬で明るくなる表示ではなく、文字の形の中を左から右へ色が進む表示です。 + +この記事では、Web Speech APIが返す単語境界イベントだけを使い、音声ファイルや強制アラインメントなしでその表示を作るまでの設計をまとめます。完成コードは [GlyphFlow](https://github.com/speech-caption/glyphflow) にあります。 + +## この記事で扱う範囲 + +対象は、Chrome Desktopで英語テキストを読み上げ、単語境界を返すvoiceです。Web Speech APIの挙動はブラウザ、OS、voiceで変わります。特に`boundary`イベントは広く一律に使えるAPIではありません。 [MDN](https://developer.mozilla.org/ja/docs/Web/API/SpeechSynthesisUtterance/boundary_event) + +この記事は、文字や音素ごとの正確な発音時刻を求めるものではありません。そこまで必要なら、音声ファイルとアラインメント情報を使う別の問題になります。 + +## まず分かるのは「単語の開始」だけ + +`SpeechSynthesisUtterance` の `boundary` イベントには、発話テキスト内の位置である `charIndex` と、発話開始からの `elapsedTime` が入ります。 + +```ts +utterance.onboundary = (event) => { + console.log(event.name, event.charIndex, event.elapsedTime); +}; +``` + +ここで重要なのは、イベントがアニメーションのフレームではないことです。`charIndex` が教えてくれるのは「このあたりの単語が始まった」であって、「この文字を発音した」ではありません。 + +そのため、元のテキストを先に単語化し、各単語の開始・終了位置を保存します。句読点や連続する空白を壊さずに表示したいので、単語そのものと、その後ろに続く文字列を分けて保持します。 + +```ts +type WordToken = { + text: string; + start: number; + end: number; + trailing: string; + units: number; +}; +``` + +同じ単語が続く文でも、単語の文字列ではなく`charIndex`で探せば区別できます。実際のvoiceが単語の先頭ぴったりではない位置を返す場合もあるため、GlyphFlowでは最も近い該当トークンを安全に探します。 + +## 1文字ずつDOMにしなかった + +最初に考えたのは、単語を1文字ずつ`span`へ分割して順番に色を変える方法でした。ただ、この表示で欲しいのは文字ごとの段階的な点灯ではありません。境界が連続的に通過する見た目です。 + +そこで、単語は1要素のままにし、文字だけを背景グラデーションで切り抜く形にしました。 + +```html +caption +``` + +```css +.kn-word { + color: transparent; + background: linear-gradient( + 90deg, + var(--kn-highlight-color, #f3f7ff) 0%, + var(--kn-highlight-color, #8da7ff) var(--kn-progress), + var(--kn-unhighlight-color, #475061) var(--kn-progress), + var(--kn-unhighlight-color, #475061) 100% + ); + background-clip: text; + -webkit-background-clip: text; + -webkit-text-fill-color: transparent; +} +``` + +更新するインラインスタイルは`--kn-progress`だけです。DOMを文字単位に増やさず、単語の途中にも連続した境界を作れます。 + +## イベントが来ない間をどう見せるか + +単語の開始イベントを受け取った直後は、次の単語がいつ始まるか分かりません。それでも、その間ずっと表示を止めると字幕らしく見えません。 + +GlyphFlowでは、単語の長さ、簡単な文字パターン、rate、直後の句読点から、その単語が終わるまでの時間を軽く見積もります。辞書や音声モデルは使いません。これは実測値ではなく、次のイベントまでの見た目を動かすための仮の時間です。 + +予測どおりの時刻に100%まで到達させると、次のイベントが少し遅れたときに、単語が完了したまま待ってしまいます。そこで、予測時間では94%まで進め、その後は最大98.5%までゆっくり近づけます。 + +次の単語境界が届いた瞬間に、直前の単語を100%にします。同時に、2つの開始イベントの`elapsedTime`の差から、直前の単語に実際にかかった時間を得られます。 + +```ts +const actualDuration = nextBoundary.elapsedTime - currentBoundary.elapsedTime; +``` + +この値は同じ読み上げセッション内だけで使います。計測値の一部を後続の予測に反映しますが、極端に短い・長い値は学習に使いません。長い文章ほど材料が増える一方、これはvoiceの発音を完全に理解する仕組みではありません。 + +## イベントをそのまま信じない + +ブラウザイベントの扱いでは、見た目より状態管理の方が難しい部分でした。 + +- 同じ`boundary`が重複して届く +- 順番が逆のイベントが届く +- pause中も`requestAnimationFrame`だけは進んでしまう +- cancel後に古いutteranceのイベントが届く +- 終了まで単語境界を返さないvoiceがある + +このため、再生ごとにセッションIDを持ち、現在のセッションと一致しないイベントは無視します。pause中はワイプを進めず、resume後はpause時間を補正します。終了まで有効な単語境界が来なかった場合は、同期できなかったことを`unsupported`として利用側に伝えます。 + +ここは「すべてのvoiceを支援する」と言い切らないための設計でもあります。音声品質やイベントの粒度は、ライブラリ側で改善できるものではありません。 + +## 使う側のコードは小さくした + +ライブラリの利用側は、テキスト、描画先、voice、rateを渡して`speak()`するだけです。 + +```ts +import { createKaraokeNarrator } from "@enumura/glyphflow"; +import "@enumura/glyphflow/styles.css"; + +const narrator = createKaraokeNarrator({ + text: "Every letter follows the voice.", + target: document.querySelector("#caption")!, + rate: 1, +}); + +narrator.speak(); +``` + +読み上げに伴う単語ごとの予測値・実測値は`onWordTiming`と`getDiagnostics()`から取得できます。ただし、最後の単語には次の境界がないため、実測時間サンプルはありません。この点も、表示用の推定と、観測できた時間データを混同しないために明示しています。 + +## まとめ + +Web Speech APIの単語境界だけでは、文字ごとの発音時刻は分かりません。それでも、単語を1要素のまま残し、CSSグラデーションの境界を動かし、次のイベントまでの表示を控えめに予測すれば、単語の内側を進む字幕UIは作れます。 + +実装とデモは以下です。 + +- GitHub: https://github.com/speech-caption/glyphflow +- npm: https://www.npmjs.com/package/@enumura/glyphflow +- Demo: https://speech-caption.github.io/glyphflow/