Using Decorators in Python

Posted on 04-02-19 in Computing

Introduction

I've ended up relearning about decorators a fair few times over the years. In the end, like most things, it comes down to seeing useful examples. In this article I will outline a few cases where decorators can be used.

On top of that I'll make sure to use doctests and a couple of very useful functions from the standard library which make decorators much nicer to work with.

Case Zero - Err what is a decorator?

A decorator is a function that allows us to put stuff around other functions. For instance we may want to keep a log of the variables being passed to a function or prevent the function from even being called if a condition is not met.

Syntax wise we write them like so :

In [1]:
def name_of_decorator(func):
    
    def wrapper(*args, **kwargs):
        # Do whatever you want in here before the function is called
        print("This is before we execute!")
        returned = func(*args, **kwargs) # Calls the function
        print("This is after we execute!")
        # Do whatever you want afterwards
        return returned
    
    return wrapper

This can be then be applied to any function! Like :

In [2]:
@name_of_decorator
def some_function():
    pass
In [3]:
some_function()
This is before we execute!
This is after we execute!

This can be immensely powerful as we will be able to alter inputs and outputs to a function. Let's look at a few examples

Case One - Only allow positive integers into a function

Validation of inputs to a function seems to be my most common use-case for decorators so I'll cover two. In this first case we want to reject all negative numbers and all numbers that are not integers. In reality we should have a better test for integers then the one used, but it'll do for this example.

In [4]:
from functools import wraps  # neccessary to ensure docstring is preserved
from itertools import chain # so we can iterate over args and kwargs
import doctest

def positive_numbers_required(func):
    @wraps(func)  # Needed to preserve the docstring oddly enough
    def wrapper(*args, **kwargs):
        
        all_arguments = chain(args, list(kwargs.values()))
        
        for arg in all_arguments:
            fail_on_argument = False
            try:
                if arg < 0:
                    fail_on_argument = True
            except TypeError:
                fail_on_argument = True

            if fail_on_argument:
                raise ValueError("Only positive numbers allowed!")
                
        return func(*args, **kwargs)
        
    return wrapper

@positive_numbers_required
def test_decorator(*args, **kwargs):
    """
    Test Behaviour of Decorator
    
    >>> test_decorator(2)
    'pass'
    >>> test_decorator(-2)
    Traceback (most recent call last):
    ...
    ValueError: Only positive numbers allowed!
    >>> test_decorator(value=2.0)
    'pass'
    >>> test_decorator(a="2")
    Traceback (most recent call last):
    ...
    ValueError: Only positive numbers allowed!
    >>> test_decorator(arg=-1)
    Traceback (most recent call last):
    ...
    ValueError: Only positive numbers allowed!
    """
    return 'pass'

doctest.testmod()
Out[4]:
TestResults(failed=0, attempted=5)

Here I've put a load of tests onto a dummy function just to see if it works as we expect. We can use this function in a useful way, for instance!

In [5]:
@positive_numbers_required
def loan_interest(value, interest):
    """
    Give the value in pence and the interest as a percentage
    
    returns value in pence
    """
    return value * (1 + interest / 100)
In [6]:
principal = 100  # pounds
pounds = 100
interest = 2.2  # percent
result = loan_interest(1000*pounds, interest)
print(f"Amount is now £{result // pounds:.2f}")
Amount is now £1022.00
In [7]:
principal = 100  # pounds
pence = 100
interest = -5.1  # Oops 'accidental' mistake!
result = loan_interest(1000*pence, interest)
print(f"Amount is now £{result // 100:.2f}")
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-7-39fe23f3213a> in <module>()
      2 pence = 100
      3 interest = -5.1  # Oops 'accidental' mistake!
----> 4 result = loan_interest(1000*pence, interest)
      5 print(f"Amount is now £{result // 100:.2f}")

<ipython-input-4-67b4749f92d1> in wrapper(*args, **kwargs)
     18 
     19             if fail_on_argument:
---> 20                 raise ValueError("Only positive numbers allowed!")
     21 
     22         return func(*args, **kwargs)

ValueError: Only positive numbers allowed!

As you can see this is rather useful for catching cases when you :

  1. don't want to clutter your functions with constant validation checks
  2. when you don't have express control over the inputs into your functions.

Aside - Why is their a decorator in the decorator?

The reason this is required is simple - without it, we'll lose the docstring on the original function.

In [8]:
def dec_one(func):  # WITHOUT the wraps decorator
    def wrap(*args, **kwargs):
        return func(*args, **kwargs)
    return wrap
    
@dec_one
def some_function(*args, **kwargs):
    """
    Here's the vital data on how this function works
    """
    return

help(some_function)
Help on function wrap in module __main__:

wrap(*args, **kwargs)

In [9]:
def dec_two(func):  # WITH the wraps decorator
    @wraps(func)
    def wrap(*args, **kwargs):
        return func(*args, **kwargs)

    return wrap

@dec_two
def some_other_function(*args, **kwargs):
    """
    Here's the vital data on how this function works
    """
    return
help(some_other_function)
Help on function some_other_function in module __main__:

some_other_function(*args, **kwargs)
    Here's the vital data on how this function works

Case Two - Requiring a Valid Email Address

This is a slightly more complex example with a lazy regex check to see if something approximating a valid email address has been entered.

In [10]:
# case 2 - require valid email address

class error(Exception):
    pass

class EmailError(Exception):
    def __init__(self, *args, **kwargs):
        print("Email Error has been Raised!")
    
def regex_decorator_maker(func, pat, param, raise_error):
    """
    This isn't a decorator yet!
    """
    @wraps(func)
    def wrapper(*args, **kwargs):
        
        parameter = kwargs.get(param, '')
        print("Received", f"'{parameter}'")
        if not parameter:
            raise raise_error(f"No {param} was found")
            
        if re.findall(pat, parameter):
            func(*args, **kwargs)
        else:
            raise raise_error(f"Invalid {param} given!")
            
    return wrapper

import re
from functools import partial


pat = re.compile('[A-z]+@[A-z]+\.[A-z]+')
require_valid_email = partial(regex_decorator_maker, pat=pat, param='email', raise_error=EmailError)


def test_decorator(*args, **kwargs):
    """
    Dummy function
    
    >>> send_message('cheese@toast.iop')
    Sending email to cheese@toast.iop
    >>> send_message(email='cheese@toast.iop')
    Sending email to cheese@toast.iop
    >>> send_message(email='cheese')
    Traceback (most recent call last):
    ...
    EmailError: Invalid email address given!
    >>> send_message('cheese')
    Traceback (most recent call last):
    ...
    EmailError: Invalid email address given!
    >>> send_message()
    Traceback (most recent call last):
    ...
    EmailError: No email address was found
    """
    pass
In [11]:
@require_valid_email
def send_message(email):
    print("Sending email to", email)
                    
valid_email = "cheese@toast.iop"

send_message(email=valid_email)
Received 'cheese@toast.iop'
Sending email to cheese@toast.iop
In [12]:
try:
    send_message(email='@google.com')
except EmailError as e:
    print(e)
Received '@google.com'
Email Error has been Raised!
Invalid email given!

Okay so this is a bit more complex. In this case I've created a function which creates decorators. Why would you unleash such a horror? Well there are plenty of cases where we will want to alter the behaviour of the decorator. It would be rather annoying if we had to type it out again! For instance :

In [13]:
pat = re.compile('[A-Z][a-z]+')
requirecapitalname = partial(regex_decorator_maker, pat=pat, param='name', raise_error=ValueError)

pat = re.compile("[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}")
requireip4 = partial(regex_decorator_maker, pat=pat, param='ip', raise_error=ValueError)

@requirecapitalname
@requireip4
def convoluted_example(name, ip):
    print(f"Contacting {name} at {ip}")
In [14]:
convoluted_example(name='Mark', ip='192.12.68.21')
Received 'Mark'
Received '192.12.68.21'
Contacting Mark at 192.12.68.21
In [15]:
convoluted_example(name='Jeff', ip='192.0.0')
Received 'Jeff'
Received '192.0.0'
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-15-e9d840eb83d6> in <module>()
----> 1 convoluted_example(name='Jeff', ip='192.0.0')

<ipython-input-10-4bac534b173c> in wrapper(*args, **kwargs)
     21 
     22         if re.findall(pat, parameter):
---> 23             func(*args, **kwargs)
     24         else:
     25             raise raise_error(f"Invalid {param} given!")

<ipython-input-10-4bac534b173c> in wrapper(*args, **kwargs)
     23             func(*args, **kwargs)
     24         else:
---> 25             raise raise_error(f"Invalid {param} given!")
     26 
     27     return wrapper

ValueError: Invalid ip given!
In [16]:
convoluted_example(name='mary', ip='8.8.8.8')
Received 'mary'
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-16-3993c5af8094> in <module>()
----> 1 convoluted_example(name='mary', ip='8.8.8.8')

<ipython-input-10-4bac534b173c> in wrapper(*args, **kwargs)
     23             func(*args, **kwargs)
     24         else:
---> 25             raise raise_error(f"Invalid {param} given!")
     26 
     27     return wrapper

ValueError: Invalid name given!

Case Three - logging parameters

Another useful case is simply one where we output the arguments passed to a function to a textfile.

In [17]:
import os


def log_decorator_maker(func, location, filename):
    @wraps(func)
    def log_wrap(*args, **kwargs):

        if not os.path.exists(location):
            os.makedirs(location)

        full_path = os.path.join(location, filename)

        mode = 'a' if os.path.isfile(full_path) else 'w'
        with open(full_path, mode) as output:
            print("".center(80, '='), file=output)
            print("Function {}".format(func.__name__), file=output)
            for j, arg in enumerate(args):
                print("Argument {} is {}".format(j, arg), file=output)
            for kw in kwargs:
                print("Keyword argument {} is {}".format(kw, kwargs[kw]), file=output)

            return func(*args, **kwargs)
    return log_wrap



filename = 'specific_functions.txt'
location = 'logs/'
log = partial(log_decorator_maker, location=location, filename=filename)

@log
def dummy_function(*args, **kwargs):
    """
    Count how many arguments we have been given
    """
    print("There are {} arguments".format(len(args)))
    print("There are {} keyword arguments".format(len(kwargs)))
    
    
In [18]:
from random import randint, choice
from string import ascii_letters

for _ in range(3):
    random_args = [randint(0, 9) for _ in range(randint(0, 9))]
    random_kwargs = {choice(ascii_letters) : randint(0, 9) for _ in range(randint(0, 9))}
    dummy_function(*random_args, **random_kwargs)
There are 4 arguments
There are 1 keyword arguments
There are 7 arguments
There are 1 keyword arguments
There are 9 arguments
There are 0 keyword arguments
In [19]:
full_path = os.path.join(location, filename)
!cat $full_path
================================================================================
Function dummy_function
Argument 0 is 3
Argument 1 is 0
Argument 2 is 9
Argument 3 is 9
Keyword argument Y is 7
Keyword argument Q is 7
Keyword argument U is 3
Keyword argument x is 8
Keyword argument F is 0
================================================================================
Function dummy_function
Argument 0 is 7
Argument 1 is 3
Argument 2 is 5
Argument 3 is 9
Argument 4 is 8
Argument 5 is 6
Argument 6 is 8
Keyword argument g is 9
Keyword argument J is 7
Keyword argument G is 4
Keyword argument l is 0
Keyword argument V is 7
================================================================================
Function dummy_function
Keyword argument E is 3
Keyword argument w is 8
Keyword argument l is 4
================================================================================
Function dummy_function
Argument 0 is 1
Argument 1 is 1
Argument 2 is 5
Argument 3 is 4
Argument 4 is 4
Argument 5 is 8
Argument 6 is 7
Argument 7 is 3
Argument 8 is 1
Keyword argument z is 2
Keyword argument Q is 6
Keyword argument a is 2
Keyword argument K is 4
Keyword argument C is 8
Keyword argument T is 7
================================================================================
Function dummy_function
Argument 0 is 1
Argument 1 is 5
Argument 2 is 7
Argument 3 is 5
Argument 4 is 7
Argument 5 is 9
Argument 6 is 5
Argument 7 is 7
Argument 8 is 8
Keyword argument U is 1
Keyword argument B is 2
Keyword argument c is 8
Keyword argument m is 2
Keyword argument l is 7
Keyword argument L is 0
Keyword argument Z is 2
================================================================================
Function dummy_function
Argument 0 is 2
Argument 1 is 2
Argument 2 is 8
Keyword argument S is 4
Keyword argument u is 0
Keyword argument q is 0
Keyword argument I is 7
Keyword argument e is 9
================================================================================
Function dummy_function
Argument 0 is 9
Argument 1 is 4
Argument 2 is 7
Argument 3 is 3
Argument 4 is 1
Argument 5 is 8
Argument 6 is 9
Keyword argument M is 2
Keyword argument X is 0
================================================================================
Function dummy_function
Argument 0 is 3
Argument 1 is 8
Argument 2 is 2
Argument 3 is 6
Argument 4 is 4
Argument 5 is 0
Keyword argument Y is 5
Keyword argument v is 0
Keyword argument J is 4
Keyword argument n is 0
Keyword argument Z is 5
================================================================================
Function dummy_function
Argument 0 is 7
Argument 1 is 9
Argument 2 is 6
Argument 3 is 4
Argument 4 is 5
Argument 5 is 7
Argument 6 is 7
Keyword argument O is 2
Keyword argument q is 7
Keyword argument F is 2
Keyword argument T is 7
Keyword argument V is 5
================================================================================
Function dummy_function
Argument 0 is 5
Argument 1 is 9
Keyword argument m is 2
Keyword argument E is 4
Keyword argument k is 7
Keyword argument H is 0
Keyword argument O is 0
Keyword argument h is 5
Keyword argument z is 6
Keyword argument a is 2
================================================================================
Function dummy_function
Argument 0 is 8
Argument 1 is 3
Keyword argument o is 3
Keyword argument f is 4
Keyword argument Y is 1
================================================================================
Function dummy_function
Argument 0 is 6
Argument 1 is 6
Argument 2 is 3
Argument 3 is 2
Argument 4 is 7
Keyword argument U is 7
Keyword argument J is 5
Keyword argument s is 3
Keyword argument h is 3
================================================================================
Function dummy_function
Argument 0 is 9
Argument 1 is 0
Keyword argument p is 0
Keyword argument I is 4
Keyword argument F is 2
Keyword argument d is 2
Keyword argument s is 0
Keyword argument c is 6
Keyword argument D is 2
================================================================================
Function dummy_function
Argument 0 is 9
Argument 1 is 3
Keyword argument e is 4
================================================================================
Function dummy_function
Argument 0 is 0
Argument 1 is 3
Argument 2 is 6
Keyword argument j is 8
Keyword argument O is 0
Keyword argument c is 1
Keyword argument C is 3
================================================================================
Function dummy_function
Keyword argument O is 3
Keyword argument a is 7
Keyword argument R is 2
Keyword argument z is 7
Keyword argument f is 9
Keyword argument H is 0
Keyword argument E is 1
================================================================================
Function dummy_function
Argument 0 is 5
Keyword argument B is 2
Keyword argument v is 8
Keyword argument V is 9
Keyword argument e is 3
================================================================================
Function dummy_function
Argument 0 is 8
Argument 1 is 8
Argument 2 is 9
Argument 3 is 5
Keyword argument D is 1
Keyword argument f is 1
Keyword argument O is 2
Keyword argument x is 3
Keyword argument s is 8
Keyword argument y is 0
================================================================================
Function dummy_function
Argument 0 is 0
Argument 1 is 9
Argument 2 is 4
Argument 3 is 9
Keyword argument F is 6
================================================================================
Function dummy_function
Argument 0 is 4
Argument 1 is 1
Argument 2 is 5
Argument 3 is 9
Argument 4 is 1
Argument 5 is 6
Argument 6 is 8
Keyword argument T is 3
================================================================================
Function dummy_function
Argument 0 is 3
Argument 1 is 8
Argument 2 is 2
Argument 3 is 6
Argument 4 is 1
Argument 5 is 1
Argument 6 is 3
Argument 7 is 8
Argument 8 is 9

Case Four - Crude Login Validation

Finally we will look at a usage of decorators to ensure functions cannot be executed if permission is not given. I will create a couple of users and we will not store plaintext versions of the password in our code. Here we have used https://www.vitoshacademy.com/hashing-passwords-in-python/ to describe how to 'hash' our passwords.

In [20]:
import hashlib, binascii, os

def hash_password(password):
    """Hash a password for storing."""
    salt = hashlib.sha256(os.urandom(60)).hexdigest().encode('ascii')
    pwdhash = hashlib.pbkdf2_hmac('sha512', password.encode('utf-8'), 
                                salt, 100000)
    pwdhash = binascii.hexlify(pwdhash)
    return (salt + pwdhash).decode('ascii')

def verify_password(stored_password, provided_password):
    """Verify a stored password against one provided by user"""
    salt = stored_password[:64]
    stored_password = stored_password[64:]
    pwdhash = hashlib.pbkdf2_hmac('sha512', 
                                  provided_password.encode('utf-8'), 
                                  salt.encode('ascii'), 
                                  100000)
    pwdhash = binascii.hexlify(pwdhash).decode('ascii')
    return pwdhash == stored_password
In [21]:
permission_granted = False

users = {'mark' : '1e1771130586b42e918e44df5680a95e78a2cfd124447b439de85cc9c2616b55b24ffc324021d9baa9f86d1c77912b1cba9955f883e1ed6d69331b68636ac884f3b0fbeac3acb14d2ce99599c1793fbf611c8f37dcc5636aeb1e47145f10cee9',
         'mary' : '04058ebdc157564f9e4dd334eefaa0aecfcb1c2a76355d07625c2bc1ccd00ab741d48c0dab4cd481757af8a3217957cff8840694c82a15cda0821bd24fe699a53494c49c48dcc7c2be0fa401c897eea5afa22b06f5cd77982eb32070974390fc'} 

class error(Exception):
    pass

class PermissionError(Exception):
    def __init__(self, *args, **kwargs):
        print("You do not have permission to run this function!")
    

def login_screen(attempts=3):
    for attempt in range(attempts, 0, -1):
        print(" Login Screen ".center(80, '='))
        print(f"You have {attempt} remaining!")
        usr = input("Please enter your username >>")
        pw = input("Please enter your password >>")
        if usr in users:
            if verify_password(users[usr], pw):
                print("Login Successful!")
                return True
            
    return False

def login_required(func, *args, **kwargs):
    @wraps(func)
    def wrap(*args, **kwargs):
        global permission_granted
        if permission_granted:
            return func(*args, **kwargs)
        else:
            raise PermissionError
  
    return wrap
    
@login_required
def main_menu():
    """
    Dummy Function Listing User Options
    """
    options = ('new data', 'edit data', 'import data', 'export data', 'settings')
    for j, option in enumerate(options, start=1):
        print(f"{j:<5} : {option}")
In [22]:
permission_granted = login_screen(attempts=1)
main_menu()
================================= Login Screen =================================
You have 1 remaining!
Please enter your username >>mark
Please enter your password >>potato
Login Successful!
1     : new data
2     : edit data
3     : import data
4     : export data
5     : settings
In [23]:
permission_granted = login_screen(attempts=1)
main_menu()
================================= Login Screen =================================
You have 1 remaining!
Please enter your username >>mary
Please enter your password >>apples
You do not have permission to run this function!
---------------------------------------------------------------------------
PermissionError                           Traceback (most recent call last)
<ipython-input-23-f816de310210> in <module>()
      1 permission_granted = login_screen(attempts=1)
----> 2 main_menu()

<ipython-input-21-79bd0b852d31> in wrap(*args, **kwargs)
     32             return func(*args, **kwargs)
     33         else:
---> 34             raise PermissionError
     35 
     36     return wrap

PermissionError: 

Here the function, main_menu, will not be executed if the variable permission_granted is False. Of note in this example is that the decorator can alter which function is even executed. In this case, unless permission is granted I raise a new error. This could be altered to call the login function. In this case it might be wise to introduce a fixed number of attempts as well as a time delay. Again, further stuff to investigate for the interested student!

Summary

Overall decorators are a useful way of augmenting existing functions with code you're likely to reuse. They can add useful functionality and keep your codebase cleaner. For most my students this will be simply removing the repetitive validation and putting it away from the code!