6. Functions and Modules Part 1

Introduction

In this module we will formally introduce functions, generator functions and lambda functions in Python, starting with formal definitions before exploring their structures and core components, including:

• Functions - bespoke functions, parameters, arguments, default parameters, name scope, and important keywords such as return, None and global
• Generator Functions - bespoke generators, lazy evaluation and lazy iterators, generator termination and generator expressions
• Lambda Functions - functional programming, anonymous functions, and lambda functions within filter, map, reduce, sorted and reversed functions

The source code for this module may be found in the public GitHub repository for this course. Where code snippets are provided in this module, you are strongly encouraged to type and execute these Python statements in your own Jupyter notebook instance.

1. Functions

In this course thus far, we have come across numerous functions available via the Python standard library, including but not limited to print(), min(), max(), len() and type(). As a consequence, you have most likely started to form an intuitive definition in your minds as to what functions are. The goal of this module is to formalise this definition, and to explore their structure and core components.

In Python, a function is a block of related Python statements that together is designed to undertake a specific task. As such, they are a common means by which to break down complex applications into smaller, specific and more manageable tasks. They also help sofware engineers avoid repetition, and promote reusability i.e. the ability for a Python function to be used by more than one application or software service wihtout the need to refactor the function code.

Functions need to be called in order to run. If a Python function is never called, then it will never run. A function may expect zero or more arguments to be passed to it by the code calling the function. And a function may return zero or more values. A simple example of a user-defined function called product() is provided below. This function expects to be given two numbers number_1 and number_2. The product() function will then return the product of these two numbers. We then call the product() function, outside of the function definition, and display its output via the print statement.

# A simple example of a user-defined function
def product(number_1, number_2):
    return number_1 * number_2

# Call the user-defined function
a = 12
b = 20
c = product(a, b)
print(c)

1.1. Defining Functions

A Python function is defined using the def keyword followed by the name of the function. This has the effect of binding the function name to the function object, where the function object is a wrapper around the executable code for a given function.

Following the function name is a set of parentheses (). These may be left blank, in which case no information is expected to be passed to the function by the calling code. Otherwise information can be passed to the function by populating the parentheses with parameters. Parameters are Python objects on which the function may be dependent in order to execute its task. Multiple parameters can be passed to the function by simply separating individual parameters with a comma.

The terms parameters and arguments are often used interchangeably to denote information passed to the function. Technically speaking, parameters are the Python objects listed inside the parentheses in the function definition, and arguments (commonly referred to as args in Python) are the values sent to the function when it is called.

Following the parentheses is the colon : character which marks the end of the function header or signature. Thereafter follows an indented but optional documentation string (docstring) which is recommended for all non-trivial user-defined functions, followed by the indented function body itself. The function body is the block of related Python statements that together is designed to undertake a specific task and which may use the parameters defined in the function header. Finally a function can optionally return values using the indented return keyword followed by the value to be returned.

The following example is a simple user-defined Python function that will test whether a given number is a prime number. This function may be broken down as follows:

• Function Name - the function name is is_prime.
• Function Parameters - a single function parameter called num is defined and expected to be passed to the function when it is called.
• Function Body - the function body will first test to see if the given argument value of num is less than or equal to 1 (as 0, 1 and negative numbers are not prime numbers), or whether there is a remainder left when dividing the given argument value by 1 (i.e. testing whether it is a whole number as there should be no remainder when dividing a whole number by 1). If either condition is met, then the boolean value of False is returned (denoting that the given argument value is not a prime number) and the remaining function body is not executed. If neither condition is met, then the function body iterates through the evaluation of range(2, num//2) which evaluates to the immutable sequence of numbers from 2 to the floor division of num by 2. If the given argument value num can be divided by a number in this range, this means that it is not a prime number, in which case False is returned. If by the end of the for loop the inner return statement is never executed, then the for loop terminates and the Python function returns True denoting that the given argument value is indeed a prime number.

# Define a function to test whether a given number is a prime number or not
def is_prime(num):
    """ Test whether a given number is a prime number or not.

    Tests whether a given number is a prime number or not,
    by first testing whether it is 0, 1, negative or not a
    whole number. If neither of these conditions are met, 
    then the function proceeds to test whether the given
    number can be divided by the numbers from 2 to the
    floor division of the given number by 2 without a
    remainder. If not, then the given number is indeed a 
    prime number.

    Args:
        num (int): The number to test

    Returns:
        True if the number is a prime number,
        False otherwise.
    """

    if num <= 1 or num % 1 > 0:
        return False

    for i in range(2, num//2):
        if num % i == 0:
            return False

    return True

A recognised standard is to document all non-trivial Python functions with a documentation string (docstring) defined by enclosing the docstring within three quotation marks """. The standard is to begin the docstring with a simple one-line summary of the function's purpose. Thereafter follows a blank line, followed by a more detailed description of the function. Following that is a description of the function arguments and return values. For further information regarding recognised styling conventions in Python, including identifier naming conventions, please refer to the PEP 8 Style Guide for Python Code, and the Google Python Style Guide respectively.

1.2. Calling Functions

To call our is_prime() user-defined function, we simply call the function name and pass any expected arguments within the parentheses, as follows:

# Call the is_prime() function on a given integer
a = 8
print(f'Is the number {a} a prime number? {is_prime(a)}')
b = 13
print(f'Is the number {b} a prime number? {is_prime(b)}')
c = 277
print(f'Is the number {c} a prime number? {is_prime(c)}')
d = -23
print(f'Is the number {d} a prime number? {is_prime(d)}')
e = 7.181
print(f'Is the number {e} a prime number? {is_prime(e)}')
f = 0
print(f'Is the number {f} a prime number? {is_prime(f)}')

Unless we are using Keyword Arguments to send arguments to functions, then the order of the arguments (referred to as positional arguments) is important and must match the order defined in the function header. Furthermore, when calling a function, it must be called with the correct number of arguments as defined in the function header. If not, a TypeError exception is raised, as follows:

# Call a function with an incorrect number of positional arguments
print(is_prime())

1.3. Arbitrary Arguments

In some cases, you may not know beforehand the number of arguments that will be passed to a function. In this case, we can define a function to expect a tuple of arguments using the unpacking * operator immediately prefixing the parameter name in the function header, as follows:

# Define a function to expect an arbitrary number of arguments
def product(*nums):
    x = 1
    for num in nums:
        x *= num
    return x

# Call this function with a tuple of arguments
print(product(2, 10, 5))

my_numbers = (9, 11, 2)
print(f'\nThe product of the numbers {my_numbers} is {product(*my_numbers)}')

In Python documentation, arbitrary arguments are often referred to by the shorthand *args.

1.4. Unpacking Argument Lists

The * unpacking operator works by unpacking the given iterable (such as a tuple or list). If a function is defined expecting separate positional arguments, then the * unpacks the given iterable and passes them as positional arguments to the function. In the following example, we define a function that expects to be passed two numerical arguments start_num and end_num. The function then finds all the prime numbers between these two numbers. When we call our function, we pass it a list which is then unpacked and passed as positional arguments to our function, as follows:

# Define a function expecting separate positional arguments
def findPrimesBetween(start_num, end_num):
    return [num for num in range(start_num, end_num) if is_prime(num)]

# Call this function using argument unpacking
args = [1, 100]
print(findPrimesBetween(*args))

1.5. Keyword Arguments

When calling a function and sending arguments to it, we may also use key=value syntax. By using this syntax, the positional order of the arguments is no longer important, as long as we send the required number of arguments, as follows:

# Call a function using keyword arguments
print(findPrimesBetween(start_num = 100, end_num = 200))

In Python documentation, keyword arguments are often referred to by the shorthand kwargs.

1.6. Arbitrary Keyword Arguments

Similar to arbitrary arguments, if we do not know beforehand the number of keyword arguments that will be passed to a function, we can define a function to expect a dictionary of arguments using the unpacking ** operator immediately prefixing the parameter name in the function header, as follows:

# Define a function to expect an arbitrary number of keyword arguments
def concatenate_words(**words):
    return ' '.join(words.values())

# Call this function with a dictionary of arguments
print(concatenate_words(word_1="Wonderful", word_2="World", word_3="of", word_4="Python"))

In Python documentation, arbitrary keyword arguments are often referred to by the shorthand **kwargs.

1.7. Default Parameter Values

By default, when calling a function we must send the correct number of arguments as defined by the number of parameters in the function header. However we can assign default values to a subset (or all) of the parameters by using the = assignment operator in the parentheses in the function header. If we assign a default parameter value and then call the function without sending that argument, the function will use the default value, as follows:

# Define a function with default parameter values
def findPrimesBetween(start_num = 2, end_num = 100):
    return [num for num in range(start_num, end_num) if is_prime(num)]

# Call this function without sending selected arguments
print(findPrimesBetween())
print(findPrimesBetween(end_num = 50))
print(findPrimesBetween(start_num = 25))
print(findPrimesBetween(start_num = 1000, end_num = 1250))
print(findPrimesBetween(100, 200))

1.8. Passing Mixed Arguments

We have now seen the various different types of arguments that can be passed to a function:

• Position Arguments - passing arguments to a function using the order of the parameters as defined in the function header
• Keyword Arguments - passing arguments to a function using key=value syntax
• Default Parameter Values - not passing specific arguments to a function, and instead relying on the default values for these parameters as defined in the function header

A problem now arises whereby a function header defines different types of parameters for a single function, for example it expects a mix of position arguments and keyword arguments to be passed to it. Fortunately Python permits us to include two special parameter operators, namely / and * that have the effect of indicating how those arguments should be passed to the function, as follows:

• Position or Keyword Arguments - if neither of the special parameter operators are present in the function header, then arguments may be passed to a function by position or keyword.
• Position-only Parameters - the / operator is used to indicate that the preceding one or more parameters are position-only parameters, and expect arguments to be passed as such. If there is no / operator, then there are no position-only parameters. Furthermore, parameters following the / operator may be either position-or-keyword or keyword-only parameters.
• Keyword-only Parameters - the * operator is used to indicate that the following one or more parameters are keyword-only parameters, and expect arguments to be passed as such.

These special parameter operators are demonstrated in the following example:

# Define a function with parameters of mixed types
def calculate_bmi(fname, lname, /, dob, *, weight_kg, height_m):
    bmi = round((weight_kg / (height_m * height_m)), 2)
    print(f'{fname} {lname} (DOB: {dob}) has a BMI of: {bmi}')

# Call this function with mixed arguments
calculate_bmi("Barack", "Obama", height_m = 1.85, weight_kg = 81.6, dob = '04/08/1961')

In the example above, fname and lname are position-only parameters, dob is a position-or-keyword parameter, and weight_kg and height_m are keyword-only parameters.

1.9. Pass Statement

There may be times when you wish to define a function without a function body to act as a placeholder for a member of the development team to subsequently work on at a later date. Python will not allow you to define a function without a body - attempting to do so will raise a SyntaxError exception. In these circumstances, we can simply use the pass keyword in place of the function body to avoid an error being raised, as follows:

# Define an empty function using the pass keyword
def my_empty_function():
    pass

1.10. None Keyword and Returning None

As introduced in the Control and Evaluations Part 1 module of this course, the special literal None is used to specify nothing (i.e. no value at all) or a null value, and is a data type in its own right (of type NoneType).

When applied to Python functions, if a Python function either does not contain a return statement, or the return statements are never executed given conditional logic within the function body, then the Python function will actually return None, as follows:

# Define a function that does not return a value
def my_logging_function(message):
    print(message)

# Call this function and examine the type that it returns
log = my_logging_function("01/09/2020 00:01 - Audit Log 1")
print(type(log))

1.11. Function Recursion

Function recursion enables functions to call themselves. Function recursion is commonly used to evaluate mathematical structures such as mathematical and statistical sequences in an elegant manner. However, and especially for beginner software engineers, function recursion is quite difficult to get right initially and developers may end up creating never-ending functions that incrementally require more CPU and/or memory resources until the limits of your hardware is reached and your system becomes unresponsive. Therefore it is important to understand beforehand, when designing the function and before writing any code, when the recursion should stop.

In the following example, we define a function that uses function recursion to calculate the Nth element in the Fibonacci sequence.

# Calculate the Nth element in Fibonacci sequence using function recursion
def fibonacci_sequence(n):
    if n <= 1:
        return n
    else:
        return(fibonacci_sequence(n-1) + fibonacci_sequence(n-2))

# Calculate the Nth element in the Fibonacci sequence
print(fibonacci_sequence(10))

# Print the first N elements in the Fibonacci sequence
for num in range(0, 10):
    print(fibonacci_sequence(num), end = ' ')

1.12. Name Scope and Global Keyword

Variables that are created within the body of a function are not visible outside of that function. These variables are referred to as having local scope, and are deleted from memory once the function terminates and control is handed back to the calling application. This is demonstrated in the following example, where a NameError exception is raised when we attempt to access a variable with local scope outside of the function:

# Define a function that contains variables with local scope
def calculate_bmi(weight_kg, height_m):
    weight_lb = weight_kg * 2.205
    height_inches = height_m * 39.37
    bmi = round((weight_lb / (height_inches * height_inches)) * 703, 2)
    return bmi

# Try to access a variable with local scope
print(weight_lb)

On the other hand, variables that are created outside of a function have global scope and are consequently visible and may be used inside the function body, as follows:

# Define the kg to lb ratio
kg_lb = 2.205

# Define the metres to inches ratio
m_inches = 39.37

# Define a function that uses variables created outside of the function
def calculate_bmi(weight_kg, height_m):
    weight_lb = weight_kg * kg_lb
    height_inches = height_m * m_inches
    bmi = round((weight_lb / (height_inches * height_inches)) * 703, 2)
    return bmi

# Call this function
print(calculate_bmi(weight_kg = 81.6, height_m = 1.85))

In the previous example, we were able to use the variables kg_lb and m_inches within the function body as they had global scope. However we are not able to modify the values assigned to these variables within the function body, as demonstrated in the following example:

# Define variables with global scope
x = 1
y = 2

# Define a simple function
def sum():
    x = 10
    y = 20
    return x + y

# Print the value of x
print(sum())
print(x)

In the example above, the sum() function assigns the value 10 to the variable x. However this variable, though it has the same name, is different to the variable x with global scope. The sum() function actually creates a new variable x with local scope. This is demonstrated when we print the value of x outside of the function - this statement returns the value assigned to the variable x with global scope.

When a variable declared within a certain scope, such as local scope within a function body, has the same name as a variable declared in an outer scope, such as global scope, that is called name hiding or shadowing. Though it is functionally acceptable to use shadowing, and your Python code will still execute correctly, it is generally not encouraged as it reduces the readability of your code, and as a consequence may introduce additional and unnecessary complexity during code reviews and refactoring by another developer.

In order to modify the value assigned to a variable with global scope within the function body itself, we must declare those variables as global variables using the global keyword within the function body, as follows:

# Define variables with global scope
x = 1
y = 2

# Define a simple function
def sum():
    global x
    x = 10
    y = 20
    return x + y

# Print the value of x
print(sum())
print(x)

Finally, in order to create a new variable within the function body but which has global scope, again we must declare that variable as a global variable using the global keyword, as follows:

# Define a function that creates new variables with global scope
def product():
    global alpha, beta
    alpha = 1000
    beta = 1_000_000
    return alpha * beta

# Print the value of alpha and beta
print(product())
print(alpha)
print(beta)

2. Generator Functions

Generator functions in Python are a special type of function that return a lazy iterator. To understand what a lazy iterator is, we have to first understand what lazy evaluation. In Python, as well as other programming languages, lazy evaluation refers to the evaluation strategy where the evaluation of an expression is delayed until such time it is needed by the application. The benefits of lazy evaluation include optimisation of memory-intensive workloads, the ability to define infinite data structures, and performance optimisation. The opposite of lazy evaluation is eager evaluation. In eager evaluation, the expression is evaluated and the entire object is created as soon as it is bound to a variable, which is the traditional and default evaluation strategy employed by most programming languages including Python.

Lazy iterators return iterable objects. But instead of the entire iterable object being created and stored in memory at the time of creation (as is normally the case when instantiating iterable objects such as lists and dictionaries), instead lazy evaluation is employed where the next element in the iterable object is only evaluated when requested.

2.1. Defining Generator Functions

To further help explain this concept, which for beginner developers may seem quite novel, let us examine an example. In the following example, we define a generator function that generates an infinite sequence. We cannot use a normal Python function to generate an infinite sequence as otherwise we would need to return an iterable object, such as a list or tuple, that contains an infinite number of elements which would then require an infinite amount of memory. If we attempted to create a normal Python function that did return an infinite sequence, we would soon be bounded by the memory limits of our hardware when we called that function. Instead, we can define a generator function, as follows:

# Define a generator function to generate an infinite sequence
def infinite_sequence_generator():
    counter = 0
    while True:
        yield counter
        counter += 1

In this example, we initialise an integer called counter and assign it the value 0. We then define a while loop that will continue to iterate indefinitely, but will only move onto the next iteration when requested to do so (i.e. lazy evaluation). When an application does request the next iteration of the while loop, the current value of counter will be returned to the calling application using the yield keyword, and then the counter will be incremented by 1.

In order to call this generator function, and hence return an iterable object whose next element is lazily evaluated only when requested, please examine the following code:

# Call this generator function and lazily evaluate the next element in the iterable object
infinite_generator = infinite_sequence_generator()
print(next(infinite_generator))
print(next(infinite_generator))
print(next(infinite_generator))
print(next(infinite_generator))
print(next(infinite_generator))

In the code above, we assign the variable infinite_generator to the lazily evaluated iterable object returned by the infinite_sequence_generator generator function. Thereafter, in order to evaluate the next element in this iterable object, we call the next() function applied to the iterable object. When we call the next() function, the next iteration of the while loop in the generator function body is executed, including returning the next element in the sequence via the yield keyword. We can now call the next() function indefinitely in order to generate an indefinite sequence of numbers.

The yield keyword is similar to the return keyword in that it is used to return a value from a function, in this case a generator function. However whilst return causes a normal Python function to terminate, yield causes the generator function to pause and save its state, ready for the next call, and control is handed back to the calling application. As such, generator functions must contain one or more yield statements, whilst normal Python functions may contain zero or more return statements

2.2. Generator Termination

Iterable objects returned by a generator function can only be iterated over in its entirety once. This means that when the last element in a lazily evaluated iterable object is evaluated, further calls are not possible (Python will actually raise a StopIteration when the generator function terminates, meaning that it can not be restarted). This is demonstrated in the following example:

# Define a generator function to lazily return an ordered sequence of letters given a starting letter and an ending letter
def letter_sequence_generator(start, stop, step=1):
    for ord_unicode_int in range(ord(start.lower()), ord(stop.lower()), step):
        yield chr(ord_unicode_int)

# Call this generator function until there are no further elements in the sequence to be evaluated
alphabet = letter_sequence_generator("a", "e")
print(next(alphabet))
print(next(alphabet))
print(next(alphabet))
print(next(alphabet))

# Attempt to call the next() function again on the terminated generator function
print(next(alphabet))

The only option available at this stage is to create an new iterable object, i.e. alphabet = letter_sequence_generator("a", "z") and restart the calls to lazily evaluate the next element in the iterable object.

2.3. Generators and For Loops

We can use the iterable objects returned by generator functions with for loops directly. This works because a for loop requires an iterable object to iterate through. The for loop will automatically know when to terminate i.e. when the StopIteration is raised, as follows:

# Use a for loop to iterate over the iterable object returned by our letter sequence generator function
alphabet = letter_sequence_generator("a", "z")
for letter in alphabet:
    print(letter)

2.4. Generator Expressions

Similar to list and dictionary comprehension where we can create and initialise lists and dictionaries with elegant one-line Python statements, generator expressions allow us to quickly and elegantly create generator objects without the need to define a full generator object. Generator expressions use parentheses () containing any valid Python expression that will return an iterable object. As with generator functions, the iterable objects created using generator expressions are lazily evaluated (unlike list and dictionary comprehension where the entire list or dictionary is created and stored in memory). Therefore generator expressions are particularly useful to optimise workloads in memory-intensive applications.

In the following example, we create a lazily evaluated iterable object using a one-line generator expression with which we can generate a sequence of the first one million integers:

# Use a generator expression to create a lazily evaluated iterable object
first_million_numbers = (num for num in range(1, 1_000_000))
print(next(first_million_numbers))
print(next(first_million_numbers))
print(next(first_million_numbers))

2.5. Generator Objects as Lists

We can use the list() constructor function to convert generator objects into lists (naturally this will not work for infinite sequences), as follows:

# Create a lazily evaluated generator object
alphabet = letter_sequence_generator("a", "z")

# Convert the generator object into a list
alphabet_list = list(alphabet)
print(alphabet_list)

3. Lambda Functions

Lambda functions are supported by many programming languages, including Python. They are small and anonymous functions, meaning that they are created without defining a function name, and may take any number of arguments but only one expression.

3.1. Defining Lambda Functions

Whilst a normal Python function is defined using the def keyword, lambda functions are defined using the lambda keyword. Lambda functions are formed of the following three primary components:

• The lambda keyword.
• Any number of comma-separated arguments, declared before the : colon operator.
• A single expression (only one expression is allowed) defined after the : colon operator that operates on the given arguments, the evaluation of which is returned by the lambda function.

An example lambda function is provided below. In this example, we have a single argument named n declared before the : colon operator. The expression is defined after the : colon operator, and in this case is n * n which has the effect of squaring the given argument. The lambda function object is then assigned to the variable square. Finally, we call the lambda function given an argument value, in this case the value 8, as follows:

# Create a simple lambda function to square a given number
square = lambda n: n * n

# Call this lambda function
print(square(8))

Another example lambda function is provided below. In this example, we have three arguments x, y and z. The expression x * y * z simply evaluates the product of these three arguments. The lambda function object is then assigned to the variable product which is then called with the values 3, 10, 5, as follows:

# Create a simple lambda function with three arguments
product = lambda x, y, z: x * y * z

# Call this lambda function
print(product(3, 10, 5))

3.2. Using Lambda Functions

To understand why and when we use lambda functions, it is necessary to briefly explore a couple of different programming language paradigms.

• Functional Programming - functional programming is based on the concept of mathematical functions, which given an input will return an output. Programs written using the functional programming paradigm are made up of pure functions (a function, given the same inputs, will always return the same outputs) that avoid shared states (objects that are shared between different scopes such as global and local scopes are avoided), avoid mutating states (objects are immutable and cannot be modified after creation) and avoid side effects (any application state changes that are observable outside the called function are avoided, such as logging, displaying to the console, persisting to the file-system, and modifying a variable in an external scope). As such, functional programming is declarative i.e. it defines a set of instructions on what needs to be done rather than how it should be done. Structured Query Language (SQL) is an example of a declarative language in which you define a series of operations to query and transform data without providing specific instructions on how to perform those operations.
  
  
• Imperative Programming - imperative programming explicitly defines how tasks should be achieved, as opposed to declarative programming that defines only what needs to be done. Imperative programs consist of control flow and step-by-step instructions descrbing every event that needs to be undertaken to achieve the task in question. As such, imperative programs must manage state and promotes mutability of objects.

In order to make the distinction clearer, let us write a computer program to filter a given list of numbers, keeping only the even numbers. Using the imperative paradigm, we define the step-by-step instructions required to filter our list, including creating an empty list to hold the even numbers, using a for loop to iterate over the given number list to be filtered, checking whether the current number is even and if so adding it to the list of even numbers, and finally returning the list of even numbers after termination of the for loop, as follows:

# Create a list of numbers
my_numbers = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

# Use the imperative paradigm to filter the list of integers, keeping only the even numbers
def filter_evens_imperative(number_list):
    even_number_list = []
    for number in number_list:
        if number % 2 == 0:
            even_number_list.append(number)
    return even_number_list

# Filter the list of numbers, keeping only the even numbers
even_numbers = filter_evens_imperative(my_numbers)
print(even_numbers)

Writing the same computer program using functional programming is much shorter and concise. We can simply use a lambda expression within the filter() function to filter the given list, as follows:

# Use a functional approach to filter the same list of integers
even_numbers = list(filter(lambda num: num % 2 == 0, my_numbers))
print(even_numbers)

Functional programming tends to deliver code that is shorter, more concise, more predictable and hence easier to test (as objects are immutable, and shared states and side effects are avoided) and easier to maintain as it can be independent of a wider application. Functional programming combined with lazy evaluation can also provide significant performance improvements. However imperative programming tends to be easier to learn and more intituive, at least for junior software engineers, as the conceptual model can be followed more easily within an application.

Pure functional programming languages are those which only support the functional programming paradigm, such as Haskell. However many common programming languages including Python, Javascript and Java (since version 8), which are not pure functional languages, do indeed support functional concepts and hence may be used for functional programming. In Python, lambda functions are widely used to write functional programs, as we have seen in the previous example.

To learn more about functional programming, including a detailed exploration of the fundamental principles of functional programming and the core differences between the declarative and imperative approaches, please refer to this great article on Medium by Eric Elliott.

We use lambda functions when we require an anonymous (nameless) function for a brief period of time. The power of lambda functions becomes apparent when applied to functional programming where they are used as anonymous functions within normal Python functions, including user-defined Python functions and common Python standard library functions such as map(), filter(), reduce(), reversed() and sorted(). In the following subsections, we will explore all of these use-cases in further detail.

3.2.1. User Defined Functions

The following Python function is defined with a parameter multiplier. The purpose of this function is to multiply an unknown number by that multipler. To that end, we use an anonymous lambda function with the argument x representing the as-of-yet unknown number. The expression multiplies x by the given multipler argument.

We then bind the variable doubler to an instantiation of the function given a multiplier argument of 2. This creates a function that will always double a given number. We then call this function with the value 10. Similarly, we bind the variable quadrupler to an instantiation of the function given a multipler argument of 4. This creates a function that will always quadruple a given number. We then call this function with the value of 100, as follows:

# Create a Python function containing a lambda anonymous function
def multiply(multiplier):
    return lambda x: x * multiplier

# Instantiate a function that always doubles a given number
doubler = multiply(2)

# Call this function with the value 10
print(doubler(10))

# Instantiate a function that always quadruples a given number
quadrupler = multiply(4)

# Call this function with the value 100
print(quadrupler(100))

3.2.2. Filter Function

The Python filter(<function>, <iterable>) function filters a given iterable object through a given function to test, for each item in the iterable, whether it should be filtered or not. The filter() function then returns an iterator object itself of type filter which we can convert into a list, for example, using the list() constructor function. In the following example, we filter a given list of numbers given a user-defined function that will test whether an item in the list is even or not.

# Create a list of numbers
my_numbers = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

# Create a function that will test whether a given number is even or not
def test_even(num):
    if num % 2 == 0:
        return True
    return False

# Apply this function to the list of numbers using the filter() function
my_even_numbers = list(filter(test_even, my_numbers))
print(my_even_numbers)

However a much shorter and more concise way to achieve the same task is to use a lambda function within the filter() function, where all the items in the iterable as passed to the lambda function as an argument value, as follows:

# Use a lambda function within filter() to perform the same task
my_even_numbers = list(filter(lambda num: num % 2 == 0, my_numbers))
print(my_even_numbers)

In general, we can use lambda functions as an argument in a higher-order function, where that higher-order function requires another function as an argument, such as filter(,). In the remaining subsections, we will explore other higher-order functions that require function arguments for which lambda functions may be used.

3.2.3. Map Function

The Python map(<function>, <iterable>) function executes a given function on each item in a given iterable. The map() function then returns an iterator object itself of type map which we can convert into a list, for example, using the list() constructor function. In the following example we apply a function that squares a given number, but only if that number is odd, to a given list of numbers using the map() function:

# Create a function that will square a given number but only if that number is odd
def square_odd_number(num):
    if num % 2 == 0:
        return num
    return num * num

# Apply this function to the list of numbers using the map() function
my_squared_odd_numbers = list(map(square_odd_number, my_numbers))
print(my_squared_odd_numbers)

Again however, a much shorter and more concise way to achieve the same task is to use a lambda function within the map() function, where all the items in the iterable as passed to the lambda function as an argument value, as follows:

# Use a lambda function within map() to perform the same task
my_squared_odd_numbers = list(map(lambda num: num if num % 2 == 0 else num * num, my_numbers))
print(my_squared_odd_numbers)

Note the use of an if statement within the expression of the lambda function in the example above. We can use conditional logic within lambda function expressions generally in order to conditionally change the values that are returned. In this case, we return the given number unchanged if it is even, else we square it.

3.2.4. Reduce Function

The Python reduce(<function>, <iterable>[, initializer]) function, found within the functools module as of Python 3, executes a given function initially on the first two items in the iterable, and the result is returned. The given function is then called again but this time on the result that was just returned along with the next item in the iterable. This process is repeated until all the items in the iterable have been processed by the given function, and finally a single value is returned. In effect, the reduce() function acts to reduce a given iterable into a single cumulative value using a given function. In the following example, we reduce a given list of numbers to the product of all those numbers using a user-defined function that calculates the product of two given numbers:

from functools import reduce 

# Create a function that will calculate the product of two given numbers
def product(a, b):
    return a * b

# Apply this function to the list of numbers using the reduce() function
my_product_of_numbers = reduce(product, my_numbers)
print(my_product_of_numbers)

# Apply this function to the list of numbers using the reduce() function and an initializer value of 2
my_product_of_numbers = reduce(product, my_numbers, 2)
print(my_product_of_numbers)

# Apply this function to the list of numbers using the reduce() function and an initializer value of 10
my_product_of_numbers = reduce(product, my_numbers, 10)
print(my_product_of_numbers)

The reduce function accepts an optional initializer argument that, if provided, will be placed before the items in the iterable in the reduction calculation, and serves as the default value when the iterable is empty. If the initializer is not provided and the iterable only contains one item, then the first item is returned.

Again however, a much shorter and more concise way to achieve the same task is to use a lambda function within the reduce() function, as follows:

# Use a lambda function within reduce() to perform the same task
my_product_of_numbers = reduce(lambda a, b: a * b, my_numbers)
print(my_product_of_numbers)

3.2.5. Sorted Function

The Python sorted(<iterable>, <key>, <reverse>) function sorts the items in a given iterable in a specific order, and then returns the sorted iterable as a list. By default, the items in the given iterable are sorted in ascending order. However an optional reverse boolean argument may be provided that, if True, will sort the items in the given iterable in descending order, as follows:

# Create a list of unordered numbers
my_unordered_numbers = [2, 100, 99, 3, 7, 8, 13, 48, 88, 38]

# Sort the list of numbers
sorted_numbers = sorted(my_unordered_numbers)
print(sorted_numbers)

# Sort the list of numbers in descending order
sorted_numbers_desc = sorted(my_unordered_numbers, reverse=True)
print(sorted_numbers_desc)

An optional key function may also be provided to the sorted() function that serves to generate the key for the sort comparison operation. For example, suppose we have a list of tuples where each tuple represents a football (soccer) team competing in a league that has just finished. The elements within that tuple represent the team name, the total number of points achieved in the season, the total number of goals scored, and the total number of goals conceded respectively. The task is to sort this list of tuples in descending order by total number of points. Where teams have finished on level points, the team with the better goal difference should come first i.e.sort by multiple keys. We can achieve this sort by multiple keys by creating a user-defined sorting function that returns a tuple of (total points, goal difference) given an item (in this case a tuple) from the iterable. This tuple then serves as the key which can be used to compare items and subsequently sort them, as follows:

# Create a list of tuples modelling a football league (team name, total points, total goals scored, total goals conceded)
completed_football_league = [
    ('Manchester United', 75, 64, 35), 
    ('Aston Villa', 56, 48, 44), 
    ('Arsenal', 90, 73, 26), 
    ('Newcastle United', 56, 52, 40), 
    ('Liverpool', 60, 55, 37), 
    ('Chelsea', 79, 67, 30)
]

# Create a function to create a compound sorting key
def sorting_key(item):
    total_points = item[1]
    goal_difference = item[2] - item[3]
    return (total_points, goal_difference)

# Use this sorting function with the sorted() function to sort the football league by multiple keys i.e. total points and goal difference
sorted_football_league = sorted(completed_football_league, key=sorting_key, reverse=True)
print(sorted_football_league)

Again however, a much shorter and more concise way to achieve the same task is to use a lambda function as the key function within the sorted() function. This lambda function, given an item (in this case a tuple from the list of tuples) will return another tuple i.e. (total points, goal difference), as follows:

# Use a lambda function within sorted() as the key function to perform the same task
sorted_football_league = sorted(completed_football_league, key=lambda item: (item[1], item[2] - item[3]), reverse=True)
print(sorted_football_league)

3.2.6. Reversed Function

The Python reversed(<iterable>) function will return a reversed iterator object, of type list_reverseiterator which we can convert into a list, for example, using the list() constructor function, as follows:

# Use the reversed() function to return a reversed iterable object
my_reversed_numbers = list(reversed(my_numbers))
print(my_reversed_numbers)

# Reverse a range of numbers
my_range = range(11, 21)
my_reversed_range = list(reversed(my_range))
print(my_reversed_range)

# Reverse the characters in a string
my_string = 'abracadabra'
my_reversed_string = list(reversed(my_string))
print(my_reversed_string)

3.2.7. Sort Method

The list sort(<reverse>, <key>) method, as we explored in the Data Aggregates Part 1 module of this course, sorts the items in the list and in place (i.e. the original list is modified). Similar to the sorted() function introduced above in section 3.2.5. Sorted Function, the list sort() method accepts an optional reverse boolean argument that, if True, will sort the list in descending order (where the default is reverse=False). Furthermore, the list sort() method also accepts an optional key function, the return value of which is used as the key during sort comparison operations, as follows:

import copy

# Create a deepcopy of the previous list of tuples representing a football league
completed_football_league_deep_copy_1 = copy.deepcopy(completed_football_league)

# Sort the previous list of tuples representing a football league using the list sort() method
completed_football_league_deep_copy_1.sort(reverse=True, key=sorting_key)
print(completed_football_league_deep_copy_1)

And again, similar to the sorted() function, we can use a lambda function as the key function within the list sort() method, as follows:

# Use a lambda function within the list sort() method as the key function to perform the same task
completed_football_league_deep_copy_2 = copy.deepcopy(completed_football_league)
completed_football_league_deep_copy_2.sort(reverse=True, key=lambda item: (item[1], item[2] - item[3]))
print(completed_football_league_deep_copy_2)

Summary

In this module we have covered functions in Python. We now have an in-depth understanding of the differences between normal Python functions, generator functions and lambda functions. We know how to define Python functions and pass arguments to functions when they are being called. We can also define generator functions that return lazy iterators which may be used to improve the performance of memory-intensive workloads. And finally we can use lambda functions as anonymous functions to undertake functional programming in Python.

Homework

1. Generator Function - Prime Numbers
   Write a Python generator function that will lazily generate an indefinite sequence of prime numbers when called, starting from and including 2.
   
   

2. Map Function - Temperature Conversion
   Using a lambda function within the Python map() function, write a Python program that will return an equivalent list of temperatures in Celsius given a list of temperatures in Fahrenheit. To convert Fahrenheit to Celsius, please refer to this link. For example your Python program should return the following list in Celsius given a list in Fahrenheit of [0, 10, 20, 30]: [-17.78, -12.22, -6.67, -1.11]
   
   

3. Sorted Function - Air Travel
   Using a lambda function within the Python sorted() function, write a Python program that will order a given list of tuples where each tuple represents the information associated with a one-way non-stop flight from a fixed origin to a fixed destination (e.g. London to Tokyo). The elements within that tuple represent the airline name, the total cost of the flight, take-off time represented as a string 'HH:MM' (at the origin), and arrival time represented as a string 'HH:MM' (also at the origin). The list of tuples should be sorted in ascending order by total cost. Where the total cost is the same between different journeys, the journey with the shortest travel time should come first. For example, given the original list of tuples below, your Python program should return the following ordered list of tuples: [('Finnair', 650.0, '10:45', '21:30'), ('Cathay Pacific', 650.0, '11:30', '23:55'), ('British Airways', 850.0, '08:00', '19:50'), ('ANA', 1025.0, '12:00', '23:25'), ('Japan Airlines', 1025.0, '06:15', '17:45')]

london_to_tokyo_flights = [
    ('British Airways', 850.00, '08:00', '19:50'), 
    ('Cathay Pacific', 650.00, '11:30', '23:55'), 
    ('Japan Airlines', 1025.00, '06:15', '17:45'), 
    ('ANA', 1025.00, '12:00', '23:25'), 
    ('Finnair', 650.00, '10:45', '21:30')
]

Hint: To calculate the time interval between two time strings of the format 'HH:MM', you can use the following:

from datetime import datetime
start_time = '06:15'
end_time = '17:45'
time_format = '%H:%M'
interval = datetime.strptime(end_time, time_format) - datetime.strptime(start_time, time_format)

What's Next

In the next module, we will formally introduce Python modules and packages, including how to write and use Python modules, how to construct and distribute Python packages, how to hide Python module entities, how to document Python modules, and Python hashbangs.