[performance] Make the migration middleware faster, second attempt#13018
Conversation
|
|
||
| @memoize(ttl=20) | ||
| def is_migrating(): | ||
| last_applied = MigrationRecorder(connection).migration_qs.order_by('-applied').only('name').first().name |
There was a problem hiding this comment.
This is a stripped-down query compared to what we had before, which did multiple queries to return the full list of migrations... which grows over time.
But I think it should be stripped down even more. Probably want to reverse the order, and find the largest migration number from the module, then do a .exists() query.
Also think I had it locally wrong, where we need to .filter(applied__isnull=False), so that we only count migrations that have been applied. We might also arguably add app="main" to the filter.
|
Ran the tests and they look good, I'd say this is ready for review! |
|
@AlanCoding I'm not a big fan of what we are doing in the first place. If I remember right, we consider the app to be in a "migrating" state if there are any migrations unapplied, so we can't temporarily go backwards in the migrations stack, for example. It might be simpler to just set a lock or a single value table in the database with the pre_migrate signal, and unset it with the post_migrate signal. |
|
Perhaps a part of the intention is to raise the migration screen after the source is updated but before it gets to the actual migration task? Also, the approach here only does a local redis call for most requests, which is much faster than a database query. Nonetheless, a single simple query would be much better than the many complex queries we run now. On every. Single. Request. I don't have a big issue with your suggested approach, I just want something to get done for this, because if I ever look at profiling data from customers this winds up constituting a significant amount of time. I also don't like that we have this in this first place, which is why I want to get something to mitigate it. As with my prior attempt, best is the enemy of good. |
7f502e6 to
65add01
Compare
65add01 to
6c46aeb
Compare
6c46aeb to
e09f682
Compare
SUMMARY
I was looking at cprofile data from requests, and found myself balking at these numbers again.
ISSUE TYPE
COMPONENT NAME
ADDITIONAL INFORMATION
I tried this before in #5239, but this revised approach is more standard.