Skip to content

[WIP] Async ORM#9

Open
nicolaslara wants to merge 48 commits into
andrewgodwin:async_viewsfrom
nicolaslara:async_orm
Open

[WIP] Async ORM#9
nicolaslara wants to merge 48 commits into
andrewgodwin:async_viewsfrom
nicolaslara:async_orm

Conversation

@nicolaslara

@nicolaslara nicolaslara commented Oct 31, 2019

Copy link
Copy Markdown

I've started experimenting with allowing the ORM to make async calls to the DB. I am very slowly, but surely making progress.

This is far from ready, but I wanted to open this PR to provide visibility into what I'm doing.

My next steps are more or less:

  • General cleanup and making sure things work both for the sync and async cases
  • Allow this to work via DefaultConnectionProxy
  • Experiment with how to use this from a higher level perspective. Does the current approach (connection.a.cursor().execute()) even make sense?
  • Consider the possibility of the ORM using async internally in a way that is invisible to the user.

Once that is done, I'll be in a position to start scoping out the work needed to move this forward and I'll post to the forum to get some discussions going (and hopefully recruit someone to help me with it)

(Some of my personal notes about the work can be found here: https://github.com/nicolaslara/django/blob/async_orm/orm_notes.md)


At the moment, this is implemented:

>>> from django.db import connections
>>> conn = connections.all()[0]
>>> await conn.a.connect()
>>> async with conn.a.cursor() as cur:
...    await cur.execute('select 1')
...    print(await cur.fetchall())
[(1,)]

>>> cur = await conn.a.cursor()
>>> await cur.execute('select 1')
>>> await cur.fetchall()
[(1,)]

andrewgodwin and others added 30 commits August 10, 2019 11:45
They actually do as of this commit, and the middleware appears to work
fine, but I'm sure there's some threading issues to be caught if nothing
else.
…go#4)

* Add decorator to mark middleware as async; fix middleware tests

* Simplify async middleware decorator

* Don't log about adapting middleware if DEBUG is false
They actually do as of this commit, and the middleware appears to work
fine, but I'm sure there's some threading issues to be caught if nothing
else.
…go#4)

* Add decorator to mark middleware as async; fix middleware tests

* Simplify async middleware decorator

* Don't log about adapting middleware if DEBUG is false
@andrewgodwin andrewgodwin force-pushed the async_views branch 3 times, most recently from 1df9d06 to a22d053 Compare January 28, 2020 17:41
@andrewgodwin andrewgodwin force-pushed the async_views branch 4 times, most recently from 9ecc9b4 to 2c1d742 Compare February 18, 2020 16:30
@felixxm felixxm force-pushed the async_views branch 6 times, most recently from c637aef to f8d0904 Compare March 12, 2020 06:40
@felixxm felixxm force-pushed the async_views branch 6 times, most recently from c3a94e3 to c86f88b Compare March 17, 2020 06:57
@felixxm felixxm force-pushed the async_views branch 2 times, most recently from 40ca60c to fc0fa72 Compare March 18, 2020 18:59

@auvipy auvipy left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you try to fix the conflicts?

@nicolaslara

Copy link
Copy Markdown
Author

@auvipy I was planning to work on this before covid, but I've been pretty low on time this year unfortunately, so I'm not sure any of the work on this branch is something we can reuse; it was more of an initial experiment at integrating an async db backend.

There are probably easier ways to do this: implement an async backend with minimal changes to the internal backends, or start by buidling the async interface while wrapping the actual calls in sync_to_async. Hopefully I'll have time to look at this again in the near future, but if I do, it will probably differ from this branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants