Best practices

Virtual environments

When you install packages via pip, you might want to create a virtual environment to isolate package installation from the global installation directory. You can create a virtual environment with the following command:

python3 -m venv <name of venv>

After you create a virtual environment, you can activate it by sourcing the script that is located in <name of venv>/bin/activate and then deactivate it with the command deactivate. You can verify your current venv by checking your shell prompt that should now be prefixed with <name of venv>. Once using a venv, all pip installed packages are placed in

<name of
venv>/lib/python3.11/site-packages

. You can reference the following commands:

$ python3 -m venv my_venv
$ . my_venv/bin/activate
(my_venv) $ pip3 install <package>         # this will install into the venv 
(my_venv) $ deactivate                     # this will deactivate the venv
$                                          # note the shell prompt is no longer prefixed

Note: If you want to use any of the bundled packages such as Numpy in your venv, you must add the option --system-site-packages. To verify that these packages are in your venv by simply running the following commands:

(my_venv) $ pip3 list
Package    Version
----------    -------
numpy       1.23.4

For more information on virtual environments, refer to the CPython official documentation at venv.

Security and pip

pip is a tool that connects to the internet and executes setup files. It is advised that you never run pip as a privileged user or with any elevated permissions, since pip runs arbitrary code to install packages. Additionally, if you run pip as an elevated user, you might inadvertently globally install packages, which can alter the intended behavior of Python for all users.

Multiprocessing

Multiprocessing enables side-stepping of the Global Interpreter Lock in CPython, which is useful not only for task parallelization, but also for preventing long-lived tasks, for example, handling https requests from blocking the main flow of the program. Multiprocessing is available in Python as the multiprocessing package, where processes can be created by using the Pool class, which offers a convenient way for setting up the parallel processing and the Process class. It is useful for controlling individual processes.

Here is a multiprocessing example with the Pool class:

from multiprocessing import Pool

def times_two(x):
    return 2 * x

if __name__ == "__main__":
    pool = Pool(100)
    print(pool.map(times_two, [x for x in range(1000)]))    #[0, 2, 4, … , 1998]

Here is a multiprocessing example with the Process class:

from multiprocessing import Process
import time

def long_process(seconds):
    print("long_process started.")
    time.sleep(int(seconds))
    print("long_process finished.")


if __name__ == "__main__":
    proc = Process(target=long_process, args=('10',))    
    proc.start()
    print("Hi from the main process.")
    proc.join()

For further information on multiprocessing, see multiprocessing topic in the official Python reference.

Unit testing and code coverage

Python includes two separate but similar tools to aid you to test your code. The first tool is unittest, which is used to create and run distinct unit tests:

import unittest
def test_func(a, b):
    return a + b
class ExampleTest(unittest.TestCase):
    def test_add(self):
        self.assertEqual(test_func(1,2), 3)
if __name__ == '__main__':
    unittest.main()

You can run the above example with the following commands:

python3 <filename.py>
python3 -m unittest <filename.py>

For more information about the built-in unit testing framework, see unittest in the official Python documentation.

The second module that Python includes for testing is doctest. Instead of having separate unittest files, doctest is used for inline testing directly into the function itself. It is useful for testing pure functions, and not meant to replace tradition unit testing:

def test_func(a, b):
    '''
    Return the sum of a + b
    >>> test_func(5, 5)
    10
    >>> test_func(5.0, 5.0)
    10.0
    '''
    return a + b
if __name__ == "__main__":
    import doctest
    doctest.testmod()

You can run the above example with the following commands:

python3 -m doctest -v <filename.py>
python3 <filename.py>

For more information about doctest, see doctest in the official Python documentation.

In addition to unit testing, code coverage tooling can assist with determining what hasn’t been tested by showing which lines haven’t been run by the interpreter. While there are no built-in code coverage tools in a Python distribution, PyPI has multiple code coverage frameworks that integrate with both the built-in unit testing framework and other PyPI testing frameworks.