Experiment with trio support

[miracle2k]

I tried to make this run on top of the trio event loop. In a way, gino is uniquely positioned to support other event loops because it's dependency on asyncio is so light. It's basically just timeouts and one `asyncio.Lock. The rest is cleanly encapsulated within the dialect implementation.

So this does two things:

• It introduced a simple "backend" class, to customize which Lock is used, and how timeouts work.
• It uses sniffio to automatically pick the right backend based on the loop used.
• It ads a dialect based on riopg, which is psycopg2 on trio.

At it stands, it runs the Showcase example successfully up until the last part where gino.iterate() is used. I stumbled there because gino core uses prepared statements here, and psycopg2 does not support them. Or at least that was my impression.

This is absolutely not merge-ready, but I did want to ask for some feedback, in particular if this is something that you see gino supporting.

[wwwjfy]

This is really cool! Thanks a lot!
I've been a bit busy lately, will try to do review this weekend.
Or maybe Fantix has time.

[fantix]

Yeah I’ll try to do it this week, thanks for the PR!

[wwwjfy]

I took a look at riopg implementation. It's using psycopg2, though it's wrapped by async, but it's actually synchronous, i.e. I/O with PostgreSQL blocks the current thread.

But trio doesn't have to be bound to riopg, right? triopg could be a better candidate I think, which is a wrapper of asyncpg.

I'll try this route to see if everything works.

[wwwjfy]

In the spirit of explicitness, I prefer letting the user set current event loop (default to asyncio), but not using sniffio, which is a bit intrusive (adding ContextVar current_async_library_cvar)

[miracle2k]

@wwwjfy Is riopg synchronous? (cc @Fuyukai). I thought that it uses psycopg2 in async mode:

http://initd.org/psycopg/docs/advanced.html#asynchronous-support

As implemented here:

https://github.com/Fuyukai/riopg/blob/master/riopg/connection.py#L78

The reason I chose it, vs triopg, is that the latter uses trio_asyncio as a compatibility layer, so it isn't "trio native".

As for sniffio vs explicit setting the loop, I do not have a strong opinion.

[wwwjfy]

I didn't dig deep, but only tested connect. async=1 is not used in riopg, and since async is not a keyword in Python, it can't be used as a parameter name, getting syntax error.

I don't have too much problem with trio_asyncio as a compatibility layer, as long as it works well. Not realistic to ask each async library to implement its own db driver :)
I'll give a try.

BTW, the simple script I used to test riopg. Ideally, connect shouldn't block printing 1, but it does, when I listen on 5433 and don't return.

import trio
import riopg
import multio
import psycopg2

multio.init('trio')


async def connect():
    print(await riopg.Connection.open('postgresql://127.0.0.1:5433/postgres'))


async def a():
    while True:
        print('1')
        await trio.sleep(1)


async def main():
    async with trio.open_nursery() as nursery:
        nursery.start_soon(connect)
        nursery.start_soon(a)


trio.run(main)

[wwwjfy]

Plus, asyncpg has better support to PostgreSQL and better performance, at least it claims so.
https://magic.io/blog/asyncpg-1m-rows-from-postgres-to-python/

[Fuyukai]

Hm, I always wondered why that function was never covered in riopg. Whoops!

[miracle2k]

I can verify that riopg now works in async mode. As for the suitable postgres backend - since the backends are self-contained anyway, I guess ultimately both could be supported.

But yes, I think to support anything based on psycopg2 , gino core would need to be changed to not use prepared statements.

[wwwjfy]

Sorry for being dormant.

Lately I've been thinking which one is a better solution, riopg or triopg. But yes you're right, it's really not my position to choose for users, as there is no obvious winner.
I've tried to integrate triopg, which I think can be done, but will be a bit tricky, as trio doesn't expose event loop.
Anyway, I'll see what I can do to support both of them.

[miracle2k]

@wwwjfy I am happy to help out where I can, or continue working on this pull request if you think it's the right direction. I also have a decent amount of experience with trio at this point, so if you have any questions regarding how to deal with the differences to asyncio, I might be able to help.

[wwwjfy]

Thanks!
I'm trying to integrate trio in triopg way. I'm not sure how much triopg need to be involved at all. I may need to re-implement triopg which is about 100 lines to fit in.

As for riopg, I want to spend time reading the code, to see if there is any issue. Not that I don't trust it, but since it's a relatively new project, with few users, there can be potential bugs. I want to have a better understanding on it (and its dependencies) before putting it as a dependency.

As for prepared statements, looks like psycopg2 still hasn't supported that. So yes if we want to integrate psycopg2, we have to make it optional.

[wwwjfy]

Reading this again, I thought I was clear. Sorry about that.

I think we can support as many backends as possible, and please go on with this PR.

A few things I have in mind now:

• I prefer explicitly specifying the event loop or backend, but not by some way to smartly detect. The reason is 1) this adds another dependency, which can be avoided 2) event loop upgrades may break the compatibility.
• If possible, I'll try not to bind riopg to trio, as it also supports curio, which means we'll need combinations: trio+riopg, curio+riopg, trio+triopg, asyncio+asyncpg. How to make it more friendly is another thing to think about.
  Anyway, no need to put those in this PR I think. That would be too big. I'm considering to create a new branch to have those WIP commits. And put them in next version when ready.

[fantix]

Thanks to everyone for working on the Trio support! This is now fixed in cb9eecc as part of GINO 2.