Repository navigation
fix(vrowzer): write binary files given to ready() as bytes in both Workers - #66
Merged
Merged
Conversation
…eb Worker
The Web Worker wrote every file given in V_WW_SETUP with { encoding: 'utf8' }, so an ArrayBuffer was written as the string "[object ArrayBuffer]".
updateFile() now writes an ArrayBuffer as bytes, as the @vrowzer/fs subscriber does for binary content, and a string as UTF-8 text as before. SetupWorkerMessage.files and setupWorker() accept ArrayBuffer values.
…kers initialize() treated config.files as text. The Web Worker received the ArrayBuffers in V_WW_SETUP, and the Service Worker in the text field of V_FS_INIT, so both wrote them as the string "[object ArrayBuffer]". The Service Worker now receives binary files in the binaryFiles field of V_FS_INIT, which @vrowzer/fs writes as bytes. The Web Worker writes the ArrayBuffers in V_WW_SETUP as bytes since the previous commit. With text files only, V_FS_INIT is the same as before. ready() copies the files when it is called, ArrayBuffers included, and sends the copies without a transfer list. Each Worker receives its own copy through structured cloning, and the caller's buffers stay usable. Changes that the caller makes while ready() runs no longer reach the Workers; before, the files were read later, so such changes could reach one Worker and not the other. The JSDoc of VrowzerConfig.files and the README describe the copy, and the API references are regenerated.
The playground now passes two binary files to ready(): a public image and a file outside the public directory. The tests check that the Service Worker serves both with the same bytes, through servePublicMiddleware and serveStaticMiddleware, that the Web Worker inlines the same bytes with ?import&inline, and that the caller's ArrayBuffers keep their size and content after ready(). Without either fix, the matching test fails with the bytes of "[object ArrayBuffer]": sending the binary files in the text field of V_FS_INIT fails the Service Worker test, and writing them as UTF-8 in the Web Worker fails the Web Worker test.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #65
Binary files (
ArrayBuffers) passed toready({ files })are now written with their bytes in both Workers, instead of as the string[object ArrayBuffer].Cause
initialize()treatedconfig.filesasRecord<string, string>and sent the files as text:filesofV_WW_SETUP, andupdateFile()wrote every value with{ encoding: 'utf8' }.filesofV_FS_INIT, which the@vrowzer/fssubscriber writes as UTF-8. The binary field,binaryFiles, was not used.Binary files added later with
addFile()orupdateFile()were not affected.Fix
initialize()splits the files into text and binary, and sends the binary files inbinaryFilesofV_FS_INIT. The subscriber already writes them as bytes and emitsaddevents, so public images also enter the public file list (fix(vite-dev-server): serve public files given to the runtime from the Service Worker #64).@vrowzer/fsis unchanged. With text files only, the message is the same as before.updateFile()writes anArrayBufferas bytes, as the subscriber does, and a string as UTF-8 text as before.SetupWorkerMessage.files,setupWorker()andupdateFile()acceptArrayBuffervalues. The types are only widened, so existing callers keep working.ready()copies the files when it is called,ArrayBuffers included, and sends the copies without a transfer list. Each Worker receives its own copy through structured cloning, and the caller's buffers stay usable, as withaddFile()since feat(vrowzer): resolve file operations after the Workers apply them #61.VrowzerConfig.filesand the README section ofready()describe the copy. The API references are regenerated.FileSystemPublisher.initFiles()is not used. It would also send the files to the Web Worker, which already loaded them during setup, and it has its own bug (#63).Behavior change
ready()now sends the files as they were when it was called. Before, it readconfig.fileslater, after loading the client files, and each Worker received theArrayBuffers as they were when its message was sent. So changes that the caller made whileready()ran could reach one Worker and not the other, and anArrayBufferthat the caller detached in the meantime could makeready()fail.This keeps one extra copy of each binary file in memory until
ready()finishes.Not included
FileSystemPublisher.initFiles()(Copy binary files for each target in FileSystemPublisher.initFiles() #63): it still transfers the caller's buffers to every target. The JSDoc ofFSInitMessage, which says thatbinaryFilesare transferred, will be updated there.Blobs or typed arrays: they are not validated, as withaddFile().ready(), text or binary, are lost when the Service Worker restarts.Tests
Unit tests:
transformer.test.ts:updateFile()and the initial files ofsetupWorker()write anArrayBufferas bytes and a string as UTF-8 text. Both tests failed before the change.index-file-sync.test.ts:filesofV_WW_SETUPand in thebinaryFilesofV_FS_INIT, without a transfer list, and the caller's buffer keeps its bytes;ready(), to anArrayBufferor to the file record, do not reach the Workers.Both tests failed before the change, and fail again without the copy in
ready(). The existing test of text-only initialization passes unchanged.Integration tests in the shared playground, which now passes a public image and a binary file outside the public directory to
ready():servePublicMiddlewareandserveStaticMiddleware;?import&inline;ArrayBuffers keep their size and content afterready().Reverting either fix makes the matching test fail with the bytes of
[object ArrayBuffer]. This also confirms in the browser that both Workers were affected.Validated locally with:
vp run test:unit: 119 files, 1,911 testsvp run test:integration:vrowzer: 67 passed and 1 conditional skip, in three runsvp run test:e2e: 46 passed, 6 existingtest.skip