Repository navigation
Loudness readings belong to the bytes: keep them in the history, keyed by content hash - #157
Closed
AliceLafox wants to merge 10 commits into
Closed
AliceLafox wants to merge 10 commits into
AliceLafox wants to merge 10 commits into
Conversation
…h the file's modification time A loudness reading lived in the slot table's memory and died with the window: the next connect showed a dash for every slot (#140). A reading is a pure function of a take's bytes, and the history already names takes by exactly that. Store version 9 adds loudness_readings, keyed by the 32-byte content hash and by nothing else - no slot, no operation, no foreign key to blobs_meta - since a reading holds for bytes slot_audio only names, kept or released, and for bytes read off the card that no operation archived. The raw numbers go in: integrated LUFS (NULL when unmeasurable), sample peak, true peak in dBTP, the count of impossible samples, and when it was measured. Never the verdict: the target moves in Settings, and describeReading judges an old number against today's target instead of keeping an old opinion. The same hash again replaces. A hash of the wrong length, a negative count and a value that is not a number are refused - SQLite would file a NaN as NULL, which here reads "unmeasurable". slot_audio gains the take's modification time off the card's directory entry, on every row that says what a slot holds: landed by a write, found in place after one, photographed by the first sighting. With the name and the size it is what #141 needs to tell whether the file in the slot is still the one that was measured, without reading it. The stamp is read through JUCE: file_time_type has no portable conversion to the system clock across the three standard libraries the app builds with, and #141 compares two sightings for equality. NULL only on rows older than the column, and on archived takes, which are leaving their slot. The hash function moves to ContentHash.h so the player's read pass can compute the same key without linking the store; HistoryStore::contentHash keeps its name for every caller that knows the store.
… it measured normalize meters the take before deciding, and the reading went out with the result. A new optional journal hook beside audioWritten, loudnessMeasured, hands it out with the very bytes measured, once the command knows they are audio: before the archive, before any rewrite, and whether a write follows or not. A slot found at target and a slot refused as silent are both measurements of their bytes; bytes refused as damaged report no reading. The bytes a gain lands get no derived number - the next read measures them. The recorder files it through HistoryRecorder::reading, under the hash the archive names the original by when a write follows. A reading is a fact about bytes, not about an operation, so none needs to have begun - but it is written only while a card's session is open: with no card in front of it the history is not told about bytes it cannot place, and the store is never opened for a reading alone. Inside an operation the session is open, so the hook refuses loudly if it is not.
…nd check and the player's pass The Check-slot job and the background check read the take's bytes anyway; readSlotLoudness measures them and files the reading under their hash before the words go to the screen. The filing is the read's tail, not its condition: with no card in front of the history nothing is filed, and a store that refuses is a line in operations.log while the reading still lands in the column. The player asked how loud the loop is, and that answer does not depend on the history being able to remember it. The player's pass metered the loaded loop while drawing its waveform. It now reads the file into memory in slices and decodes from there, so the hash is of every byte the meter saw and the card is still read once; stop() is heard between slices, since one read of a long take over USB would outlast its patience. The hash rides up with the reading and crosses to the worker - the store's thread - as a quiet job that needs no card. A lane that is only drawn, in a multi-track memory, streams from the file as before. The import path is left as it is: it meters only with normalize-on- upload on, and then usually applies a gain. Proving the gain-free stream byte-identical to what push lands means auditing the resampler at ratio 1.0, and the next read of the slot measures the landed bytes themselves.
…e bound A reader that wants to know whether the core would accept a file - the player's read pass, before it files a loudness reading under the file's hash (#140) - should not have to read the whole file to ask. readWavInfo gains an overload over the file's first bytes and its whole size: the same chunk walk, the same refusals, and the core's truncation check against the real size, which is the one lie a probe alone cannot see. A chunk header or a fmt body that lies past the probe is a refusal with its own words, never a guess. readWavInfo(data) is this with the whole file as the probe, and reads exactly as it did.
… stream the take again The read pass filed what JUCE's decoder read, and JUCE reads more than the core does: it pads a file shorter than its header claims with silence and reads a number (0.3 s present and 1 s claimed filed -23.0 where the core refuses; 2 s of 3 filed -20.34 where the audio measures -20.00), and it decodes pcm16 and mono files the pedal gate refuses (review of #140, P0). The two agree on exactly one shape, bit for bit: the pedal's own stereo float32, whole. The pass streams again as before - progressive waveform, the JUCE meter for the number on screen, no whole-file read. The hash goes up with the reading only when the core would measure those very bytes, decided off the file's first 4 KiB and its size through the core's own header walk and upload gate (ReaderLoudness.h), and only when the file is still the one the decode began on: size and stamp are read before the decode and after the hash, so a take replaced under the pass is not filed under another file's bytes. The hash is streamed in a second pass through a 1 MiB buffer (FileHash.h): the digest ends on the first short read, so the bytes it saw are counted against the file's length, and a read the pass cut short is no hash at all. A take of any length stays out of memory, and stop() is heard between decoded blocks and between hash buffers. ReaderMeter is the one place a decoded block is fed to the meter, shared by the pass and by reader_loudness_tests, which pins the decoder-side reading to wav::measureLoudness bit for bit - LUFS, sample peak, true peak, wild count - on float32 takes of three levels, silence, a damaged take and one under a gating block, and shows the gate refusing the truncated, pcm16, mono and 48 kHz files JUCE still decodes. The player harness loads a pcm16 take and must get its number with no key, and a float32 take with the hash of its every byte.
… saw hashHeldBefore matched an earlier after-row on name and size and skipped any newer row without a hash. Take A renamed, then re-recorded on the pedal as B under the same name and size, then renamed again, gave B's row A's hash with B's stamp; with a rename in between, the hash came from two operations back (review of #140, P1). #141 would read A's loudness for B. It now looks only at the slot's newest after-row for the file name - the slot's last word about that file - and carries its hash only when the row has one and its size and modification time are the file's now. A newer row without a hash blocks every older one; a row from before the store kept stamps cannot vouch for a file; a stamp or size that differ are another file under the same name. The recorder reads the stamp once and asks it of the row too. Both of the reviewer's scenarios run end to end in history_recorder_tests, and the store's rules on their own in history_store_tests.
… a take once Three small ones from the review of #140. A sample peak is the largest |sample|, a magnitude: recordReading refuses one below zero, and the new table carries CHECK (sample_peak >= 0) - it is this branch's table, so no migration pays for it. The two background filings caught only loopercat::Error; anything else the store threw would have failed the read, or crossed Thread::run: they catch std::exception. modifiedMs stat'ed a file twice - existsAsFile, then the time - and a file gone in between would have stored the epoch as a real stamp; one stat now, and the epoch, JUCE's answer for a file it cannot read, is the error it is. It no longer includes Sqlite.h for a UTF-8 helper.
…would measure
The gate proved the core would measure the file; it did not prove JUCE
decoded the same frames. JUCE stops its chunk walk at the RIFF size field
and sizes the data chunk with the frame size it knows when it meets it, so
a RIFF size that ends after fmt, or inside a LIST ahead of data, or a data
chunk ahead of fmt in a file under the probe, gives a reader of 0 frames
over a file the core reads whole. The pass then filed {absent, 0, -inf}
under the hash the check job files the core's number under, and the
stored reading depended on which path ran last (re-review of #140, P0).
The gate now returns the core's Info, and the pass files only when the
reader's frame count, sample rate and channel count are the core's
(decodesAsCore). reader_loudness_tests builds the reviewer's three shapes,
shows the core reading each whole and JUCE opening each as a reader of no
frames, and the second half of the gate failing on every one; the good
tones and the LIST-ahead-of-data shape pass both halves.
…s written A newest after-row with no modification time vouched for nothing, which is every row of a history migrated from version 5 to 8 - every preview tester's. Until audio landed in a slot, its rename, tempo and settings states carried no hash, and the History window refused to restore them: "that row recorded no state that can go back as a whole" (re-review of #140, P2). hashHeldBefore now holds a row without a stamp to the rule it was written under, name and size, for that row only; a stamped row is still held to its stamp. The staleness the allowance permits stays confined to rows written before the upgrade: the first operation on the slot writes a stamped row, and from there the slot is held to the new rule. The store suite covers the allowance and its edge, and the recorder suite runs a migrated history through a rename that carries the hash, a re-record that the old row still vouches for, and a second re-record that the new row does not.
…sked for The stamp test set a modification time with milliseconds and expected the same integer back. JUCE on Linux sets and reads whole seconds, so the CI lanes read 1600000000000 where the test asked for 1600000000123 and the rows, which faithfully carry what the directory entry says, failed against the asked-for value. The invariant is that a row says what the file system says: compare against the stamp read back, and check only that the setter took (rounded down to a second at most).
Member
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #140. Stacked on #156 (
feat/operation-subject, schema v8); the base moves tomainonce #156 is merged. #141 and #142 build on this.A reading lived only in the table's memory and died with the window. The history already identifies takes by the hash of their bytes; a reading is a pure function of those bytes.
loudness_readingskeyed by the 32-byte content hash (the raw reading, not the verdict: integrated LUFS nullable, sample peak, true peak dBTP, impossible-sample count, when measured; standalone key, replaced on re-record, survives release and forget), andslot_audio.modified, the file's modification time on every new after-row (NULL only on migrated rows), which is what Fill the LUFS column on connect from what the history already knows #141 needs to match a take without reading it.app/history/SlotLoudness.h),commands::normalizethrough a new journal callback (the before-bytes' reading, once, also when nothing is written; never for damaged bytes), and the player's own pass for the loaded slot, filed only when the core's gate accepts the file off a 4 KiB probe and JUCE decoded exactly the frames the core would measure (app/ReaderLoudness.h, pinned bit-for-bit againstwav::measureLoudness), with a streamed hash (app/history/FileHash.h) and the file's size and mtime unchanged around the pass. The import path is left out: the metered stream is not provably byte-identical to what lands.hashHeldBeforecarries a hash to a new take row only for the very file an earlier row saw (newest row for the slot and name, same size, same mtime; a row from before v9 vouches by name and size as it was written).describeReadingstays the one place that turns a reading into words, so an old number is re-judged against today's target.Tests:
history_store_tests128+,history_schema_tests(v8→v9 migration, idempotent),history_recorder_tests(normalize through the journal; the stale-hash scenarios; migrated rows),commands_tests(the hook: once with the before bytes, once on already-at-target, once on silence, never on damaged),reader_loudness_tests(new: the JUCE meter equals the core bit-for-bit; the gate refuses truncated, pcm16, mono, 48 kHz, damaged RIFF sizes; the streamed hash equals the one-shot hash and never returns a prefix),wav_tests(the header probe equals the whole-file parse at every probe size). Non-GUI ctest 59/59; the six GUI harnesses 6/6 locally (the player harness checks the hash handed up equals the file's).Known, pre-existing: a swap renames takes, so a hash does not carry across it; the gate under-files conservatively for DAW-dropped takes with large metadata chunks (the check job still files the core's number).