Imagine you want to write an e-commerce application, where people can search for interesting stuff, place orders, which keep informed about new articles and much more. You might write a file that contains all necessary functions, for example (pseudo code):

add_item_to_basket(customer_id, item_id)
remove_item_from_basket(customer_id, item_id
place_order(customer_id)
get_items_from_basket(customer_id)
subscribe_to_newsletter(customer_id, newsletter_id)
unsubscribe_from_newsletter(customer_id, newsletter_id)
search_for_item(name_pattern)
change_user_name(customer_id, name)
change_password(customer_id, password)

Of course these are just a few high level functions an application like this will need. You will end up with a few thousand functions more with certainly tens to hundreds of thousands of lines of code in a single file.

What’s wrong with this?

First of all, no one wants to edit a single file of that size. It is hard to keep an overview where to find a certain function.

The main drawback, though, is that all the different functionalities like basket, user management and so on are mixed up in one file! Wouldn’t it be great if every component were independent and individually maintainable? Or maybe you want to share components between different parts of the application? Or use the work of others, save time and avoid error prone development? Or even make your code available for others?

This is where modularity steps in!

Learning goals

In this tutorial you will learn how to organize your code by modular programming. The basic concept of modular programming in Python is to place your classes and functions in modules and packages.

5795690693_e56323af56_o
Let’s fight spaghetti code!

What’s a module?

All programming tutorials start with the well known “Hello, World” program, so let’s create a file hello_world.py with the following content:

print("Hello World!")

and execute it with your Python interpreter

python3 hello_world.py

hello_world.py is a called a Python source file.

When your program gets larger and more complex it might be helpful to print out some meaningful messages to track which part of the code is executed, but only if during development or testing. You could write something like this:

def print_debug_message(message):
     if show_debug_messages == True:
         print("Debug: " + message)

show_debug_messages = True
print_debug_message("Begin of Hello World")
print("Hello World!")
print_debug_message("End of Hello World")

This works as expected. But the function print_debug_message and the variable show_debug_messages have nothing to do with our “Hello, World” program. They bloat up the code and do not fit in context. And maybe you want them to use elsewhere. Let’s create a new file for our debug tools and name it debug_tools.py. Move print_debug_message and show_debug_messages to this file and add an init() function:

def init():
     global show_debug_messages
     show_debug_messages = False
 def print_debug_message( message ):
    if show_debug_messages == True:
       print( "Debug: " + message )

Bravo, you just created your first module!
And you can instantly use it in your hello_world.py code:

import debug_tools

debug_tools.init()
 debug_tools.show_debug_messages = True
 debug_tools.print_debug_message("Begin of Hello World")
 print("Hello World!")
 debug_tools.print_debug_message("End of Hello World")

Note: In this example we use the global statement that hasn’t been introduced yet. If used in a module, it defines a variable that is visible throughout the whole module but only in the module.

A Python module is in fact a Python source file and the name of the module is the filename without the .py suffix. You can use a module by loading it with the import statement.

As you can see, you have to prepend debug_tools followed by a . to the functions of the module you want to use. We will talk about this in a later chapter where we will take a deeper look at the import statement.

Note: You might have heard that “Python has batteries included”. That means Python comes up with a lot of built-in functionalities. And in most cases there is no need to write a debug module from scratch again. This is just for learning purposes.

What’s a package?

Larger software projects use many modules. Saving all these module files in the project root directory is not a best practice, as it is – again – hard to maintain and to track. You can group your module files by moving them into packages. A package is simply a directory containing a special file __init__.py that is called package initialization file. Whenever Python discovers this file in a folder, this folder will be used as a package. The __init__.py file may be empty, but it also may contain initialization code for you module (that´s why it is called package initialization file).

How to import a package

To use a module inside a package you invoke the import statement:

import debug_tools.debug_to_console
debug_tools.debug_to_console.print_debug_message(“My debug”)

Here you use the print_debug_message function from the debug_to_console module that is part of the debug_tools package. The file system would look like this:

myMachine:~/pythonDemo> ls -R
debug_tools  my_program.py
./debug_tools:
__init__.py     debug_to_console.py

In fact, we imported a module from a package. Let’s take a deep dive into the import statement.

The import statement

Simple import

There are several ways to import modules and packages. Until now we used the import statement as follows:

import my_fancy_module

to import a module and

import my_cool_package.some_module

to import a module inside a package. You can also import multiple modules in one statement:

import my_module1, some_other_module, an_even_better_module

You may also import only certain functions from a module

from my_math import pi

And use it then directly as follows:

print(pi)

Importing and renaming

Sometimes it may be useful to change the name of a module. To change the name you can import like this:

import my_module_that_does_math as my_math

You can now use functions of that module as:

value = my_math.get_square_root(2)

Be careful! If you change the name of a module it may be hard to track where the module is used.

Another use case for renaming a module is to avoid naming conflicts. For example you have two packages with a module named debug. You may import them as:

import frontend_package.debug as frontend_debug
import backend_package.debug as backend_debug

This applies also to the from ... import ... as ... statement.

Global vs. local imports

If you import a module or package on top of your program (like we did), the module or package is loaded in the global namespace. That means you can use the imported code everywhere in your code file. This is generally best practice.

Next to the global namespace there is the local namespace. Take a look at this example:

def my_local_function():
    import some_module
    some_module.just_do_it()

All the functions and variable of the imported module are only visible and usable from within the function my_local_function. We say the module is imported to the local namespace.

Generally, you should avoid importing in a local namespace. However there are some circumstances when importing locally is recommendable. For example, assume you have a huge package that is used rarely. Because importing it executes a lot of initialization code, this might take a while, so you only want to import if it is really necessary.

To see what is exposed to your current namespace you can use the dir() function. Start your Python interpreter, execute dir() and you should see something like this:

>>> dir()
['__builtins__', '__doc__', '__name__', '__package__']

Now import the math package:

>>> import math
>>> dir()
['__builtins__', '__doc__', '__name__', '__package__', 'math']

math is added to your namespace.

dir() accepts also an argument. If the argument is a module object, the list contains the names of the module’s attributes. Try it with the imported math module:

>>> dir(math)
['__doc__', '__file__', '__name__', '__package__', 'acos', 'acosh', 'asin', 'asinh', 'atan', 'atan2', 'atanh', 'ceil', 'copysign', 'cos', 'cosh', 'degrees', 'e', 'erf', 'erfc', 'exp', 'expm1', 'fabs', 'factorial', 'floor', 'fmod', 'frexp', 'fsum', 'gamma', 'hypot', 'isinf', 'isnan', 'ldexp', 'lgamma', 'log', 'log10', 'log1p', 'modf', 'pi', 'pow', 'radians', 'sin', 'sinh', 'sqrt', 'tan', 'tanh', 'trunc']

Exercise

A fundamental data structure is a stack. It is a collection of data with two basic functions, push and pop. For a much more detailed description see Wikipedia.

Another data structure is a FIFO. Wikipedia again gives you a detailed explanation. The basic FIFO functions are enqueue and dequeue.

Here is the challenge:

Implement a package data_structures with two modules named stack and fifo. All functions have an array as an argument that push, pop, enqueue and dequeue elements to and from the array.

Example:

data_structures.stack.push(my_array, 5)
my_value = data_structures.stack.pop(my_array)
data_structures.fifo.enqueue(my_array, ”FSE”)
my_value = data_structures.fifo.dequeue(my_array)

Write reusable code

Always design and implement your module or package in a way that makes it useful not only for a single or very special purpose. For example, is your stack module limited only to numbers? Or just strings? It should work with all kinds of objects and data types.

Use a style convention. Do not mix up different styles in your modules. The most commonly used and recommended style guide for Python is PEP 8.

Document your code both for yourself and your users. It helps others to use your module easily. There are many ways to document Python code. For further information about documentation we recommend  docstrings and Sphinx.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s