Regression for custom get_context() integrations (v2.4.0)

[rafaelbeckel]

Hey team, I had a custom integration working a few months ago, originally from this thread.

My lib uses get_context("fragmentcolor") and it worked perfectly for months.

I think a recent update ¹ may have broken this contract, as running my lib with RenderCanvas now raises this error:

python main.py
Traceback (most recent call last):
  File "/Users/rafaelbeckel/LocalProjects/Python/fragmentcolor_example/main.py", line 30, in <module>
    target = renderer.create_target(canvas)
  File "/Users/rafaelbeckel/LocalProjects/Python/fragmentcolor_example/.venv/lib/python3.13/site-packages/rendercanvas/base.py", line 260, in get_context
    raise TypeError(
        "The given context type is invalid: {context_type!r} is not 'bitmap' or 'wgpu'."
    )
TypeError: The given context type is invalid: {context_type!r} is not 'bitmap' or 'wgpu'.

Should I do something differently for a custom Rust+Python integration now?

Best,
Rafael

Footnotes

1. EDIT: v2.4.0 was the release that broke it; it still works with v2.3.0. ↩

[almarklein]

Hi Rafael, thanks for reaching out! We indeed made some changes to how contexts work, and we removed the possibility for 3d party contexts.

My apologies for breaking your code! Let's discuss about a solution for your use-case.

Maybe good to know why we made the change. Previously, rendercanvas only implemented the bitmap-context, and relied on wgpu-py to provide the wgpu-canvas. We had generalized the hook mechanism to allow 3d party libs to provide other contexts. But we decided that rendercanvas should implement its own context classes for a few reasons:

• We can focus on making these classes user-friendly, and document them in our docs.
• More direct control over the contract between context and canvas.
• We can work on more advanced contexts for sending data in a compressed way over the network (rendering to a remote client).
• The context in wgpu-py can become simpler (more low-level).

Some contexts in rendercanvas now wrap a wgpu.GPURenderCanvas where needed. I suspect that similarly, you could implement a context class that wraps a rendercanvas.BitmapContext or .WgpuContext. Does that make sense?

Maybe if helps if you tell a bit more about your use case. If the code is public, maybe you can link to it?

[rafaelbeckel]

Yes, it's public on https://fragmentcolor.org

My implementation wraps RenderCanvas and manages the hook internally, so it should be easy to adapt the internals without changing my API.

[almarklein]

Cool project!

Looks like the core is about here: https://github.com/vista-art/fragmentcolor/blob/ebb83ffda151ffbe39a97cbc16fc53bb1901eef3/src/renderer/platform/python/mod.rs#L36

Looks like further down you call _rc_get_present_methods(), and use that directly to query the window id, to eventually render to screen. From what I can see, you don't even need the context?

Do you use the event system of rendercanvas? And what about wgpu-py?

[rafaelbeckel]

That's the place, indeed.

Anything that happens inside that closure happens in Python (the closure acquires the GIL).

Calling get_context here makes RenderCanvas call the constructor of my Target object (implemented in another file).

It's a bridge that implements some interfaces making both RenderCanvas and my Renderer happy. This Target object is later used as an input to the Renderer's render() method.

This design of using a Target as input in render() was inspired by pygfx when I first met it a few years ago (around 2023).

About querying for the context: this piece of code is one of my oldest in the lib. I implemented Python support first before Js, and back in the day I didn't support bitmap rendering, but I planned to. I ended up building a specialized ImageTarget instead of reusing the same Target for both, but the query stayed there. I'll revisit it eventually when I refine the user experience for Jupyter (then I'll probably need to use this context).

Finally, about wgpu-py, I used it back in the day for a Python-specific project and it worked well. It's not suited for this one, though, because it uses Rust's wgpu directly and the goal is to implement exactly the same API surface across all languages. It's a bit higher-level API compared to wgpu and lower-level compared to pygfx.

[almarklein]

@rafaelbeckel feel free to reach out if you need help in updating to the new context mechanics!

[rafaelbeckel]

@almarklein Thank you! Just describe how you'd do it in Python (or point to the updated docs) and I'll figure the rest out.

[almarklein]

Since you're getting the current texture and presenting it via Rust, you're basically bypassing the normal canvas.

One thing you could try is to not create a rendercanvas context at all. Just call _rc_get_present_methods, and do your thing. I have never tried this, but I think it should work for your purpose.

Another option is something like:

context = rendercanvas.contexts.BaseContext(present_methods['screen'])  # or a subclass
context._context_type = 'fragmentcolor'
canvas._canvas_context = context

A bit more hacky with these private vars, but then you have an actual context which has .physical_size, pixel_ratio, etc.
If this works well, I think it makes sense in rendercanvas to allow setting custom canvases, so that this path becomes a proper way to go.

Code: https://github.com/pygfx/rendercanvas/blob/main/rendercanvas/contexts/basecontext.py
Docs: https://rendercanvas.readthedocs.io/stable/contexts.html

[rafaelbeckel]

Thanks for your support!

In the specific RenderCanvas case, my Target object acts as the context provider, and I let RenderCanvas control the presentation: https://github.com/vista-art/fragmentcolor/blob/ebb83ffda151ffbe39a97cbc16fc53bb1901eef3/src/target/platform/python.rs#L91

For all other platforms, my Renderer calls the Target's present() method: https://github.com/vista-art/fragmentcolor/blob/ebb83ffda151ffbe39a97cbc16fc53bb1901eef3/src/renderer/mod.rs#L449 (auto-present returns true by default, but is disabled in the RenderCanvas implementation of the Target interface).

If I don't call get_context(), I think RenderCanvas won't call present() anymore, and I should call it myself, like in the other platforms. Then I'd still be using RenderCanvas to get the surface and integrate at a lower level. Is that what you mean?

[almarklein]

Ah right. So it would indeed make more sense to subclass BaseContext and implement its _rc_present() method.

[almarklein]

So now you can do:

class MyContext(rendercanvas.contexts.BaseContext):
    
    present_methods = ["screen"]

    def _rc_present(self):
        # ... do your thing
        return {"method", "screen"}  # or {"method": "fail", "message": "..."}

    def _rc_close(self):
        ...


context = canvas.get_context(MyContext)