Improve canvas text#831
Conversation
There was a problem hiding this comment.
Pull request overview
This PR updates the canvas text rendering path to avoid getImageData pixel readback by using GPU-friendly sources (ImageBitmap via transferToImageBitmap on OffscreenCanvas, or passing an HTMLCanvasElement directly) as texture inputs.
Changes:
- Extend texture source typing to allow
HTMLCanvasElement(and broaden internal image sources inImageTexture). - Update the canvas text renderer to capture output as
ImageBitmap/HTMLCanvasElementinstead ofImageData. - Update WebGL and Canvas texture upload paths to accept
HTMLCanvasElementas a valid source type.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| src/core/textures/Texture.ts | Adds HTMLCanvasElement to the core TextureData union. |
| src/core/textures/ImageTexture.ts | Broadens src typing to include ImageBitmap/HTMLCanvasElement and unifies non-string load handling. |
| src/core/text-rendering/TextRenderer.ts | Updates CanvasRenderInfo.imageData to `ImageBitmap |
| src/core/text-rendering/CanvasTextRenderer.ts | Switches capture from getImageData to transferToImageBitmap/direct-canvas source. |
| src/core/renderers/webgl/WebGlCtxTexture.ts | Allows uploading textures from HTMLCanvasElement in the WebGL path. |
| src/core/renderers/webgl/WebGlCtxSubTexture.ts | Extends subtexture data union to include HTMLCanvasElement. |
| src/core/renderers/canvas/CanvasTexture.ts | Allows HTMLCanvasElement as a direct image source in the Canvas renderer path. |
| src/core/CoreTextNode.ts | Passes through the new canvas text image source type to ImageTexture. |
Comments suppressed due to low confidence (1)
src/core/renderers/webgl/WebGlCtxTexture.ts:209
- HTMLCanvasElement support was added to the WebGL upload path, but there’s no unit test to ensure this new branch is exercised (and that upload + UNPACK_PREMULTIPLY_ALPHA_WEBGL behavior matches the ImageData/HTMLImageElement path). Consider extending WebGlCtxTexture.test.ts with a stubbed
HTMLCanvasElementand asserting it goes through the same upload branch.
const memoryPadding = 1.1; // Add padding to account for GPU Padding
const isImageBitmap =
typeof ImageBitmap !== 'undefined' && tdata instanceof ImageBitmap;
const isHTMLCanvas =
typeof HTMLCanvasElement !== 'undefined' &&
tdata instanceof HTMLCanvasElement;
// If textureData is null, the texture is empty (0, 0) and we don't need to
// upload any data to the GPU.
if (
isImageBitmap ||
isHTMLCanvas ||
tdata instanceof ImageData ||
// not using typeof HTMLI mageElement due to web worker
isHTMLImageElement(tdata) === true
) {
| // For OffscreenCanvas we reuse the shared canvas and snapshot via | ||
| // transferToImageBitmap at the end. For HTMLCanvasElement we allocate a | ||
| // fresh canvas per text node so it can be passed directly as the texture | ||
| // source — no pixel readback or intermediate copy needed. | ||
| let drawCanvas: HTMLCanvasElement | OffscreenCanvas; |
|
Nice thanks!! we should also add support for this in the Platform layers right? else we would never utilize offscreen canvas properly unless you have your own custom platform. |
a473c48 to
def25ed
Compare
|
|
Use bitmap/canvas instead of ImageData
def25ed to
a559cfd
Compare
Updated WebPlatform, not sure that is what you meant. |
4a71316
into
lightning-js:fix/improve-canvas-text
getImageDataforces the browser to copy the pixels drawn by the canvas to JS world. By using ImageBitmap instead the pixels can be drawn on the GPU and stay there.