Repository navigation
Make email.message.Message.__contains__ faster #100792
Description
Right now the implementation of Message.__contains__ looks like this:
Lines 450 to 451 in 2f2fa03
There are several problems here:
- We build intermediate structure (
listin this case) - We use
listforinoperation, which is slow
The fastest way to do check if actually have this item is simply by:
def __contains__(self, name):
name_lower = name.lower()
for k, v in self._headers:
if name_lower == k.lower():
return True
return FalseWe do not create any intermediate lists / sets. And we even don't iterate longer than needed.
This change makes in check twice as fast.
Microbenchmark
Before
» pyperf timeit --setup 'import email; m = email.message_from_file(open("Lib/test/test_email/data/msg_01.txt"))' '"from" in m'
.....................
Mean +- std dev: 1.40 us +- 0.14 us
pyperf timeit --setup 'import email; m = email.message_from_file(open("Lib/test/test_email/data/msg_01.txt"))' '"missing" in m'
.....................
Mean +- std dev: 1.42 us +- 0.06 us
After
» pyperf timeit --setup 'import email; m = email.message_from_file(open("Lib/test/test_email/data/msg_01.txt"))' '"missing" in m'
.....................
Mean +- std dev: 904 ns +- 55 ns
» pyperf timeit --setup 'import email; m = email.message_from_file(open("Lib/test/test_email/data/msg_01.txt"))' '"from" in m'
.....................
Mean +- std dev: 715 ns +- 24 ns
The second case is now twice as fast.
It probably also consumes less memory now, but I don't think it is very significant.
Importance
Since EmailMessage (a subclass of Message) is quite widely used by users and 3rd party libs, I think it is important to be included.
And since the patch is quite simple and pure-python, I think the risks are very low.
Activity
pythongh-100792: Make `email.message.Message.__contains__` twice as fast
pochmann commentedon Jan 7, 2023
I'd say it's one of the fastest linear searches you can do.
What times do you get if you simply change the listcomp to a genexp, i.e., just change
[/]to(/)?return name.lower() in map(str.lower, self)might also be good.Btw in "After," you switched the order of the two benchmarks, better use the same as in "Before".
sobolevn commentedon Jan 7, 2023
and
Nope:
pochmann commentedon Jan 7, 2023
Hmm, I hoped the "from" search would be good. Odd that it's slower than the "missing" search in the map solution.
gh-100792: Make `email.message.Message.__contains__` twice as fast (#…