Practical Decorators

Reuven M. Lerner • PyCon 2019

reuven@lerner.co.il • @reuvenmlerner

1
Free “Better developers”

weekly newsletter

Corporate Python
training

Weekly Python
Exercise

Online video courses

2
3
4
Let’s decorate a function!
See this: But think this:

@mydeco

def add(a, b): def add(a, b):

return a + b return a + b

add = mydeco(add)

5
 Three callables!
(2) The decorator

(1) The decorated

function
@mydeco

def add(a, b):

return a + b

(3) The return value

from mydeco(add),

assigned back to “add”

6
 Defining a decorator
(2) The decorator

(1) The decorated

def mydeco(func): function

def wrapper(*args, **kwargs):

return f'{func(*args, **kwargs)}!!!'

return wrapper

(3) The return value

from mydeco(add),

assigned back to “add”

7
 Another perspective

Executes once,
def mydeco(func): when we decorate
the function

def wrapper(*args, **kwargs):

return f'{func(*args, **kwargs)}!!!'

return wrapper
Executes each time
the decorated
function runs

8
Wow, decorators are cool!

9
 Better yet:
Decorators are useful

10
 Example 1: Timing
How long does it take for a function to run?

11
 My plan

• The inner function (“wrapper”) will run the original function

• But it’ll keep track of the time before and after doing so

• Before returning the result to the user, we’ll write the

timing information to a logfile

12
def logtime(func):

def wrapper(*args, **kwargs):

start_time = time.time()

result = func(*args, **kwargs)

total_time = time.time() - start_time

with open('timelog.txt', 'a') as outfile:

outfile.write(f'{time.time()}\t{func.__name__}\t{total_time}\n')

return result

return wrapper

13
@logtime

def slow_add(a, b):

time.sleep(2)

return a + b

@logtime

def slow_mul(a, b):

time.sleep(3)

return a * b

14
1556147289.666728 slow_add 2.00215220451355

1556147292.670324 slow_mul 3.0029208660125732

1556147294.6720388 slow_add 2.0013420581817627

1556147297.675552 slow_mul 3.0031981468200684

1556147299.679569 slow_add 2.003632068634033

1556147302.680939 slow_mul 3.0009829998016357

1556147304.682554 slow_add 2.001215934753418

15
 def logtime(func): (1) The decorated

function
def wrapper(*args, **kwargs):

start_time = time.time()

(2) The decorator

result = func(*args, **kwargs)

total_time = time.time() - start_time

with open('timelog.txt', 'a') as outfile:

outfile.write(f'{time.time()}\t{func.__name__}\t{total_time}\n')

return result

(3) The return value

return wrapper from logtime(func),

assigned back to func’s name

16
Example 2: Once per min
Raise an exception if we try to run
a function more than once in 60 seconds

17
 Limit
def once_per_minute(func): (1) The decorated

function
(2) The decorator

def wrapper(*args, **kwargs):

# What goes here?

return func(*args, **kwargs)

(3) The return value

return wrapper from once_per_minute(func),

assigned back to func’s name

18
 We need “nonlocal”!
def once_per_minute(func):

last_invoked = 0

def wrapper(*args, **kwargs):

nonlocal last_invoked

elapsed_time = time.time() - last_invoked

if elapsed_time < 60:

raise CalledTooOftenError(f"Only {elapsed_time} has passed")

last_invoked = time.time()

return func(*args, **kwargs)

return wrapper

19
 We need “nonlocal”!
def once_per_minute(func): Executes once,
when we decorate
last_invoked = 0 the function

def wrapper(*args, **kwargs):

nonlocal last_invoked

elapsed_time = time.time() - last_invoked

if elapsed_time < 60:

raise CalledTooOftenError(f"Only {elapsed_time} has passed")

last_invoked = time.time()

return func(*args, **kwargs)

return wrapper Executes each

time the decorated
function is executed
20
print(add(2, 2))

print(add(3, 3))

__main__.CalledTooOftenError: Only 4.410743713378906e-05 has passed

21
Example 3: Once per n
Raise an exception if we try to run
a function more than once in n seconds

22
 Remember
When we see this: We should think this:
@once_per_minute

def add(a, b): def add(a, b):

return a + b return a + b

add = once_per_minute(add)

23
So what do we do now?
This code: Becomes this:
@once_per_n(5)

def add(a, b): def add(a, b):

return a + b return a + b

add = once_per_n(5)(add)

24
That’s right: 4 callables!
(1) The decorated
def add(a, b):
function

return a + b
(2) The decorator
(3) The return value
from once_per_n(5),
itself a callable, invoked on “add”
add = once_per_n(5)(add)

(4) The return value

from once_per_n(5)(add),
assigned back to “add”

25
 How does this
look in code?
For four callables,
we need three levels of function!

26
def once_per_n(n): (2) The decorator
def middle(func): (1) The decorated

function
last_invoked = 0

def wrapper(*args, **kwargs):

nonlocal last_invoked

if time.time() - last_invoked < n:

raise CalledTooOftenError(f"Only {elapsed_time} has passed")

last_invoked = time.time() (4) The return value

from middle(func)
return func(*args, **kwargs)

return wrapper

return middle (3) The return value

from the one_per_n(n)

27
 Executes once,
def once_per_n(n):
when we get an argument
def middle(func):
Executes once,
last_invoked = 0 when we decorate
the function

def wrapper(*args, **kwargs):

nonlocal last_invoked

if time.time() - last_invoked < n:

raise CalledTooOftenError(f"Only {elapsed_time} has passed")

last_invoked = time.time()

return func(*args, **kwargs)

return wrapper Executes each time

return middle
the function is run

28
 Does it work?
print(slow_add(2, 2))

print(slow_add(3, 3))

__main__.CalledTooOftenError: Only 3.0025641918182373 has passed

29
Example 4: Memoization
Cache the results of function calls,
so we don’t need to call them again

30
 def memoize(func): (1) The decorated function

cache = {}

def wrapper(*args, **kwargs):

if args not in cache:

(2) The decorator
print(f"Caching NEW value for {func.__name__}{args}")

cache[args] = func(*args, **kwargs)

else:

print(f"Using OLD value for {func.__name__}{args}")

return cache[args]
(3) The return value

from memoize(func),

return wrapper
assigned back to the function

31
 Executes once, when we
decorate the function
def memoize(func):

cache = {} Executes each

time the decorated
function is executed

def wrapper(*args, **kwargs):

if args not in cache:

print(f"Caching NEW value for {func.__name__}{args}")

cache[args] = func(*args, **kwargs)

else:

print(f"Using OLD value for {func.__name__}{args}")

return cache[args]

return wrapper

32
 Does it work?
@memoize

def add(a, b):

print("Running add!")

return a + b

@memoize

def mul(a, b):

print("Running mul!")

return a * b

33
 Caching NEW value for add(3, 7)

Running add!

10
print(add(3, 7))
Caching NEW value for mul(3, 7)

print(mul(3, 7)) Running mul!

print(add(3, 7)) 21

Using OLD value for add(3, 7)

print(mul(3, 7))
10

Using OLD value for mul(3, 7)

21
34
 Wait a second…

• What if *args contains a non-hashable value?

• What about **kwargs?

35
 Pickle to the rescue!

• Strings (and bytestrings) are hashable

• And just about anything can be pickled

• So use a tuple of bytestrings as your dict keys, and you’ll

be fine for most purposes.

• If all this doesn’t work, you can always call the function!

36
def memoize(func):

cache = {}

def wrapper(*args, **kwargs):

t = (pickle.dumps(args), pickle.dumps(kwargs))

if t not in cache:

print(f"Caching NEW value for {func.__name__}{args}")

cache[t] = func(*args, **kwargs)

else:

print(f"Using OLD value for {func.__name__}{args}")

return cache[t]

return wrapper

37
Example 5: Attributes
Give many objects the same attributes,
but without using inheritance

38
Setting class attributes

• I want to have a bunch of attributes consistently set

across several classes

• These classes aren’t related, so I no inheritance

• (And no, I don’t want multiple inheritance.)

39
 Let’s improve __repr__

def fancy_repr(self):

return f"I'm a {type(self)}, with vars {vars(self)}"

40
 Our implementation
(2) The decorator

def better_repr(c): (1) The decorated class

c.__repr__ = fancy_repr

def wrapper(*args, **kwargs):

o = c(*args, **kwargs)

return o

return wrapper (3) Return a callable

41
Our 2nd implementation
(2) The decorator

def better_repr(c): (1) The decorated class

c.__repr__ = fancy_repr

return c (3) Return a callable —

here, it’s just the class!

42
 Does it work?
@better_repr

class Foo():

def __init__(self, x, y):

self.x = x

self.y = y

f = Foo(10, [10, 20, 30])

print(f)

I'm a Foo, with vars {'x': 10, 'y': [10, 20, 30]}

43
 Wait a moment!
We set a class attribute.
Can we also change object attributes?

44
Of course.

45
 Let’s give every object
its own birthday

• The @object_birthday decorator, when applied to a class,

will add a new _created_at attribute to new objects

• This will contain the timestamp at which each instance

was created

46
 Our implementation
(2) The decorator

def object_birthday(c): (1) The decorated class

def wrapper(*args, **kwargs):

o = c(*args, **kwargs)

o._created_at = time.time()
(3) The returned object —

return o what we get when we

invoke a class, after all

return wrapper

47
 Does it work?
@object_birthday

class Foo():

def __init__(self, x, y):

self.x = x
<__main__.Foo object at 0x106c82f98>
self.y = y
1556536616.5308428

f = Foo(10, [10, 20, 30])

print(f)

print(f._created_at)

48
 Let’s do both!
def object_birthday(c):

c.__repr__ = fancy_repr Add a method

to the class

def wrapper(*args, **kwargs):

o = c(*args, **kwargs)

o._created_at = time.time()

return o
Add an attribute

return wrapper to the instance

49
 Conclusions
• Decorators let you DRY up your callables

• Understanding how many callables are involved makes it

easier to see what problems can be solved, and how

• Decorators make it dramatically easier to do many things

• Of course, much of this depends on the fact that in

Python, callables (functions and classes) are objects like
any other — and can be passed and returned easily.

50
 Questions?
• Get the code + slides from this talk:

• http://PracticalDecorators.com/

• Or: Chat with me at the WPE booth!

• Or contact me:

• reuven@lerner.co.il

• Twitter: @reuvenmlerner

51