Clipboard events and interactions

[hmaarrfk]

It might be cool to interact with an other kind of "input", the clipboard.

https://developer.mozilla.org/en-US/docs/Web/API/Clipboard_API

In one level, I want to be able to place things on the clipboard:

• Images
• Text
• URLs
• HTML

In other ways we want to be able to "read" them.

This would really help with strange backends that don't necessary live on the same system like jupyter-rfb

[Korijn]

You mean you want to interact with the OS clipboard?

It's not immediately obvious (to me) what role rendercanvas should play here?

[hmaarrfk]

Yeah, it might be a little hard to articulate.

For me, the rendercanvas is somewhat a "gateway" to the OS through a backend.

At least, when writing our abstraction code, it is useful to have a "single entry point" for an abstraction.

say we have a button that is "copy to clipboard" with IMGUI, we need a way to dispatch this appropriately based on the backend that is being used.

[hmaarrfk]

I guess lets consider this usecase:

1. An event is received either by PyGFX or imgui through PyGFX.
2. What mechanism do we have to say "put this on the user's clipboard"?

If this is out of scope for rendercanvas (which it might be) its ok.

I just feel like keyboard events, mouse events, are pretty similar to clipboard events.

Mostly just trying to think how I pass that "action" from the application (written with PyGFX) up and back to the user in a "backend agnostic way".

[Korijn]

Well, to me, the ability to abstract over various backends for window creation is sensible in the context of varied python environments.

If I were to build a proper complete production application, then I have control over the python environment, and so I would pick a backend and use its facilities directly. That gives access to that backend's full suite of functionality (such as the clipboard, sound, etc).

So, just keep that in mind! Don't let the abstraction get in your way unnecessarily.

[hmaarrfk]

I have control over the python environment

this is often true initially when projects start.

However, as projects evolve and requirements evolve, it may be that the original environment is incompatible with a new deployment target.

Suddenly something as simple as "copy+paste" seems impossible when you try to support 2 backends (even if just for experimentation).

[almarklein]

I think this is out of scope. Even if we'd add it, it easily gets out of hand with e.g. supporting images, etc.

Why not just use clipboard or pyperclip

>>> import clipboard

>>> clipboard.copy("asd")

[hmaarrfk]

I think images and etc. seem to be "mime types".
https://dreamswork.github.io/qt4/classQClipboard.html#a128baa0791a37f2d41fc850f998822b5

clipboard is pyperclip, and it seems ok. its concerning that on linux they say "xsel doesn't always seem to work, use Qt".

I think the usecase I'm considering is:

1. Jupyter-rfb "like" backend.
2. An error is generated.
3. You want to provide a way to the user to copy the traceback, without seeing the whole text.

So you kinda want a magic button, that just moves things to the clipboard.

[almarklein]

If that "magic button" is a <button> you can also add the JS to copy to the clipboard.

But I think you are thinking about the button being an imgui button. The browser can only copy something to the clipboard in a an event that originated from a click. So that complicates this.

Would it make sense to go with <button>? Maybe if an error occured, send a message to the client so it creates a <div> that shows (a summary) of the error plus a button to copy it?

[hmaarrfk]

Would it make sense to go with ? Maybe if an error occured, send a message to the client so it creates a

that shows (a summary) of the error plus a button to copy it?

I think AI found a similar solution to this.

1. imgui/pygfx sends data to the frontend
2. frontend creates a javascript button to "ask the user to copy (again....)".
3. animation created to acknowledge success

The question at this stage is, is it worth it to do something like:

canvas.send_to_clipboard

which can pipe the data to Qt/GLFW/Javascript as needed?

Each backend would have to choose what to do about the frontend display and interaction.

The javascript issues you bring up are real.... and annoying....

The permission model is slightly annoying, since it means unpredictable behavior from the applications side.

For example, I found that getting images on the clipboard on iOS was somewhat "difficult" compared to text based data.

Similar to the wgpu device capabilities, it might warrant this 'canvas capabilities'....

I'm remembering Korjin's point:

So, just keep that in mind! Don't let the abstraction get in your way unnecessarily.

[Korijn]

Can you give more details about your full stack? Is this Electron?

[hmaarrfk]

Generally speaking it is pretty close to the work that you all have pioneered with jupyter-rfb.

So you have one "lightweight client" and a "server" elsewhere, the standard "desktop metaphore" doesn't exist anymore.