Skip to content

Repository files navigation

Sightline logo

Sightline

Turn the pages of your sheet music with your eyes — so both hands stay on your instrument.

✔ Runs entirely in your browser  ·  ✔ Nothing uploaded  ·  ✔ No account  ·  ✔ Free to host


Sightline watches where you're looking through your webcam and gently scrolls your score so the music you're reading stays in a comfortable band on the screen. When you reach the end of a line — or glance down at the next one — it turns the page for you. No pedals to tap, no hands off the instrument.

Three ways to turn the page, pick whichever fits how you play:

  • Wink tracking (default) — no calibration needed; wink your left eye to scroll up, your right to scroll down.
  • Iris tracking — watches exactly where your eyes point for the most precise, natural page-following.
  • Auto-scroll (tempo-based) — no camera at all; set your tempo and time signature and the page scrolls itself, like a metronome for sheet music.

Contents

Try it

Online (nothing to download): Sightline is hosted for free on GitHub Pages at https://wizoi.github.io/Sightline/ — just open that link in Chrome or Edge on a computer with a webcam. That's the recommended way to use it; no install, no build step, nothing to keep up to date yourself.

Want to run your own copy or work on the code? See Development and Host your own copy.

Quick start

Sightline turns pages two independent ways — hands-free eye/wink tracking (below) or time-based auto-scroll, which follows a tempo you set instead of your eyes. Switch between them with the tabs at the top of the panel. This quick start covers eye/wink tracking; see Auto-scroll (time-based) for the other.

  1. Load your music — click Load PDF and choose any PDF of your score or part.
  2. Start the camera — click Start camera and allow access when your browser asks.
  3. Calibrate — a dot appears in the middle, then moves slowly around the screen for about twenty seconds, pausing a few times. Follow it with your eyes and don't look away. Do this sitting the way you'll actually play. (Wink tracking needs no calibration — skip to the next step.)
  4. Follow eyes — click it (or press Space) and the page starts following you.
  5. Play! Read normally; when you reach the end of a line or look at the next system, the page advances.

Calibration is saved, so next time you can skip straight to loading your music.

Your privacy

Everything happens on your own computer, inside your browser. Your camera feed is used only to work out where you're looking, moment to moment — it is never recorded, saved, or sent anywhere. The same goes for your microphone if you turn on Live tempo correction: audio is analyzed locally, in real time, and never recorded or uploaded. There is no account, no server, and no upload. Close the tab and nothing is kept except your saved settings (which live only in your browser).

Using Sightline

Tracking type (in the Eye/Wink tab) picks how Sightline reads your intent to turn the page:

  • Wink tracking (default) — no calibration needed to get started. Wink your left eye to scroll up, your right eye to scroll down; a blink (both eyes together) is ignored, only a one-eyed wink counts. Wink scroll strength controls how firm a push each wink gives. If winks are missed or blinks trigger by mistake, Calibrate wink sensitivity measures your own resting and winking eyes for a few seconds and sets personal thresholds instead of the one-size-fits-all default.
  • Iris tracking — watches exactly where your eyes point, for the most precise page-following. Calibrating takes about twenty seconds: a dot sits still in the middle for a moment, then moves slowly around the screen, pausing a few times along the way. Just follow it with your eyes and don't look away. If that doesn't work well for you, Calibrate (9-dot fallback) offers the older method instead — clicking nine fixed dots, one at a time. Either way the result is saved, so you only do it once.

Switching tracking types is instant — pick whichever is more reliable for your face/lighting/glasses.

The reading band is the horizontal stripe where your current line sits (shown by default). You choose where it sits on screen and how tall it is. You read within it; the page moves to keep the music there.

Turning the page happens two ways, whichever feels natural: read to the right edge of the band (you've finished the line), or simply look down at the next system. A brief hold prevents accidental turns from a quick glance.

Pause instantly with the spacebar — or with a foot pedal. A Bluetooth page-turner pedal usually sends a mouse click, and a click anywhere on the music toggles pause. Handy for the moments you look away and don't want the page to move.

Recenter (the R key or the button) pops a target in the middle of the band; look at it for a second and tracking snaps back into alignment.

Presets let you save a whole setup (speed, band size, everything) per piece — a fast étude and a slow ballad can each have their own feel.

Auto-scroll (time-based)

An alternative to eye/wink tracking — instead of watching your eyes, the page scrolls in time with a tempo you set, like a metronome for the page. Switch to it with the 🎵 Tempo tab in the panel (the two are alternatives — starting one automatically pauses the other).

  1. Load your music, same as above.
  2. Scroll to your starting point — wherever in the piece you want playback to begin.
  3. Set the time signature and tempo — beats per measure and BPM.
  4. Analyze score — Sightline scans the page for systems (a system is one line of music — see Using Sightline for scores with multiple staves per line) and estimates how many measures are in each. Detection isn't perfect, especially on dense or unusually-spaced scores — check the Measures per system list it produces and fix any count that looks wrong; the schedule uses exactly what's there. If it spots a printed time signature it doesn't recognize with confidence, it may offer a "detected — use this?" suggestion instead of guessing — accept it or leave your own setting as-is. If your music prints its own tempo, Analyze also reads it and sets your starting Tempo to match automatically (and flags any later tempo changes it finds) — this happens right away, with no confirmation step, so if a change it shows doesn't match your actual music, it misread the page; just set the Tempo slider back to what you want.
  5. Pick a section, if there is one — a combined "Score and Parts" PDF (a full conductor score followed by individual instrument parts, all in one file) gets split into named sections automatically, shown in a dropdown. Pick your own part and everything below — measures, tempo, playback — scopes to just that section, so you're not scheduled against the whole document. A plain single-part PDF has just one section and skips this entirely.
  6. Start auto-scroll — the page scrolls and highlights the current system in time, starting from wherever you're scrolled to. Pause stops it in place; Start again resumes from your current scroll position, so you can nudge things while paused.
  7. Playback speed nudges the overall pace up or down (50–150%) without re-entering a new BPM.

Live tempo correction is a toggle inside this tab. While auto-scroll is playing, it listens through your microphone for note attacks and gently nudges the scroll speed to track your actual playing tempo — a small, bounded correction, not full tempo-following. It shows a live status: listening (mic connected, nothing heard yet), tracking tempo (actively adjusting), or no signal (quiet for a while, e.g. during a rest). It's opt-in, off by default, and does nothing while paused.

Tuning

Every player, webcam, and room is a little different, so a minute spent adjusting pays off. A few controls only make sense for Iris tracking (they watch a screen position, which Wink tracking doesn't have) and disappear when Wink tracking is active — that's expected, not a bug. Eye-tracking smoothing and Ignore glances past the sides are tucked under the Advanced disclosure at the bottom of the Eye/Wink tab rather than sitting loose in the main list.

The reading band you shape by dragging it, not from the panel — it's right there on the page, so you can see the effect as you make it:

On the band What it changes Do it when…
Drag the middle up or down Where on screen you read You want more look-ahead of what's coming — move it toward the top
Drag the top or bottom edge (or pinch it on a touch screen) How tall the reading zone is It turns while you're still on a line — make it taller; you want it to advance sooner — make it shorter
Drag the line-end marker sideways (Iris only) How far along a line you look before the page turns Turns come too late — drag it left; it turns before you've finished the line — drag it right

Everything else is a slider in the side panel, under how it scrolls:

Slider What it does Turn it…
Page scroll speed How fast it moves Up for quick page turns, down for slow passages
Eye-tracking smoothing (Iris only, Advanced) Steady vs. responsive Up if it jitters; down if it lags your eyes
Wait before turning Delay before a turn commits Up to ignore more stray glances; down for snappier turns
Ignore glances past the sides (Iris only, Advanced) How close to the left/right edge still counts as reading, vs. looking away Up if a glance toward the edge of the screen accidentally scrolls; down if it ignores real reading near the edges
Music size Zoom of the score (100% = fit width) Down to see more of the page at once; up to enlarge

Getting the best accuracy

Webcam eye-tracking isn't laser-precise, but a good setup makes it reliable:

  • Light your face evenly (a lamp in front beats a bright window behind you).
  • Put the camera near eye level and sit roughly centered in its view.
  • Leave Auto-frame on — it zooms in on your face automatically so your eyes are well-resolved even if you sit back from the laptop.
  • With Iris tracking, use Verify (in the Eye/Wink tab): it shows you 7 targets and reports how close your gaze lands, whether up/down or sideways is weaker, and your room brightness — with specific fixes. Aim for "you'd land on the right line" being high.
  • If it drifts mid-piece, tap R to recenter; if your setup changes (new camera, resized window), it'll suggest a quick recalibration. (Both are Iris-tracking concepts — Wink tracking has no drift to correct, since it never looks at a screen position.)

Troubleshooting

The camera won't turn on. Allow camera access when prompted. Camera access requires a secure context (HTTPS or localhost) — the hosted GitHub Pages link and npm run dev / npm run preview both satisfy that automatically.

It keeps scrolling when I look away (Iris tracking). That's tracking drift. Add light, tap R to recenter, or recalibrate — and use the pedal/spacebar pause when you glance away. Running Verify will tell you what's off.

It feels inaccurate (Iris tracking). Recalibrate, keeping your eyes on the moving dot the whole way and resisting the urge to glance ahead of it; improve lighting, and keep Head-pose comp on so moving your head doesn't throw it off.

Wink tracking misses winks, or a blink triggers a page turn by mistake. Run Calibrate wink sensitivity (next to the Wink scroll strength slider) — it measures your own resting eyes and each individual wink, and sets thresholds for your face instead of the shared default. Good, even lighting on your face helps here too.

Auto-scroll's measure counts look wrong. Detection isn't perfect on dense, unusually-spaced, or handwritten scores. Open Measures per system after analyzing and correct any counts by hand — the schedule uses exactly what's there.

Live tempo correction won't turn on, or keeps asking for the microphone. Allow microphone access when prompted — same secure-context requirement as the camera (HTTPS or localhost). It also only does anything while auto-scroll is actually playing; while paused it just waits.

Under the hood

Gaze tracking

Sightline uses Google's MediaPipe face-landmark model to locate your eyes and irises in the webcam image. Rather than using raw iris position (which changes when you move your head), it reconstructs where your eyes point relative to your head from the 3D face geometry, so calibration survives you swaying and turning while you play. Blinks are detected and ignored.

Calibration

The nine calibration points fit a small quadratic model (plus the model's own eye-look signals) mapping your eye direction to a point on screen, with feature standardization and ridge regression so it stays stable and doesn't over-fit. The result is saved locally and restored automatically; it's re-validated if your camera or window size changes.

Detecting musical systems

Sightline renders each page and finds the staff lines (long horizontal strokes), clusters them into staves, then groups staves into systems — accepting the grouping only when it's consistent. That's how a four-staff clarinet-quartet score is understood as whole systems while a single-staff part is read line by line. Auto-scroll's measure counting builds directly on this.

Auto-scroll: measures and scheduling

Analyzing a score for auto-scroll reuses the same staff-line/system detection described above, then scans each system's columns for barlines (tall vertical strokes) to estimate its measure count. Those counts, together with your time signature and BPM, build a simple schedule — how long each system should take — that the scroll position and highlight interpolate through smoothly. If the PDF has no real text layer (a scanned or photographed page), the printed measure numbers themselves are read back off the image via on-device OCR instead, as a more accurate fallback than the barline estimate alone.

Sections: splitting a combined score

A PDF containing a full conductor score followed by individual instrument parts (a common "Score and Parts" export) is automatically split into named sections, using the PDF's own real embedded text — instrument names, tempo markings, and printed measure-number resets — rather than guessing from pixels. Each section keeps its own measure counts, time signature, and tempo, so picking your own part schedules auto-scroll against just that part, not the whole document. A section detected only from a measure-number reset (no matching instrument name nearby) is labeled as an auto-detected split rather than a real name, so an approximate boundary is never presented as more certain than it is.

Live tempo correction

An AudioWorklet analyzes microphone input off the main thread for note onsets, using a simple rising-energy detector rather than full pitch or beat tracking. Each detected onset is compared to when the schedule expected the next beat, and the timing error nudges a small, clamped correction multiplier (0.85×–1.15×) rather than re-estimating tempo from scratch — much simpler and more robust, and it decays back to neutral whenever playing stops, so a rest or a missed note can never leave a stale correction stuck in place.

Camera zoom

Auto-frame crops and upscales the view around your face before detection so your eyes get more pixels when you sit back — it follows your face and periodically widens to the full frame to re-lock. If your webcam exposes a hardware zoom, the manual zoom uses that instead for real optical detail.

Rendering

Mozilla's PDF.js renders your score into one tall scrollable column that Sightline scrolls smoothly (or snaps to a system) based on whatever is currently driving it — your eyes, a wink, or the auto-scroll schedule.

Requirements

  • A desktop or laptop with a webcam.
  • Chrome or Edge (they support the camera and the face model well).

MediaPipe's face-tracking model and WASM runtime, and PDF.js, are all bundled with the app itself and served from the same place as everything else — no third-party CDN requests, so school networks that filter external domains won't block it.

No sample scores are included — load your own PDF. (PDFs are git-ignored so your music never ends up in the repo.)

Development

Sightline is a normal Vite-based static web app: plain JS modules, no framework, no server component. The build output is still just static HTML/CSS/JS — Node/npm are only needed to build it, not to run it.

npm install       # install dependencies
npm run dev        # start a local dev server with hot reload
npm test            # run the unit test suite (Vitest)
npm run lint          # lint with ESLint
npm run build           # production build → dist/
npm run preview          # serve the production build locally

Accuracy over time: Analyze-score detection is checked against a 39-file real-world corpus (band parts, duets, full scores) on every meaningful change, comparing each PDF's actual systems/ sections/measures/tempo against hand-verified ground truth. Earliest snapshot vs. the latest one on record:

Detected correctly 2026-07-20 (earliest) 2026-07-23 (latest)
Systems (lines of music) 80.6% 92.9%
Section names (in multi-part PDFs) 63.8% 72.5%
Measures per system¹ 3.1% 83.8%
Printed tempo marks 43.6% 96.8%

¹ Only counted on files where system detection itself was already correct (a prerequisite for a fair per-system comparison) — 20 of 39 files at the earliest snapshot, 28 of 39 at the latest; not yet measurable on the rest.

Run npm run benchmark:report for the full trend across every snapshotted commit, straight from the committed history in benchmarks/snapshots/ — no setup or personal music corpus needed, it works right after cloning. See scripts/benchmark/ for the runner that produces those snapshots (it drives the app against a real, git-ignored corpus of PDFs — not something a clone needs in order to just read the trend).

Layout:

  • src/lib/ — pure, dependency-free logic (calibration math, gaze math, staff/system detection, the follow-controller's decision logic). This is the part covered by unit tests, colocated as *.test.js next to each module.
  • src/ (top level) — the DOM-facing modules that wire that logic up to the page: camera capture, calibration UI, PDF rendering, the accuracy test, settings/presets, and the follow controller's per-frame loop.
  • src/appState.js — the shared runtime state (calibration, toggles, camera state) that those modules read and write.

Pushing to main runs the test suite and, if it passes, builds and publishes the app — see Host your own copy.

Host your own copy

GitHub Pages hosts Sightline for free with a shareable HTTPS link (HTTPS means the camera works anywhere, no local-server step). A GitHub Actions workflow (.github/workflows/deploy.yml) builds the app and publishes it to a gh-pages branch on every push to main:

  1. Push this repo to GitHub and push to main at least once (or trigger the Deploy to GitHub Pages workflow manually) so the gh-pages branch gets created.
  2. On GitHub: Settings → Pages.
  3. Under Build and deployment, set Source: Deploy from a branch, Branch: gh-pages / (root), and Save.
  4. Wait a minute, then visit https://wizoi.github.io/Sightline/.

From then on, every push to main that passes tests automatically redeploys.

Share that link with anyone — they just open it and play.

Credits & license

Built with PDF.js (Mozilla) and MediaPipe Tasks (Google).

Released under the MIT License.

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages