Information on using distutils module

Distutils is the primary way of building and distributing Python packages. For more information about distils, see distutils — Building and installing Python modules in the official Python documentation.

Writing a module

You can use the typical layout for a Python package or module as follows:

README
LICENSE
setup.py
requirements.txt
src/
    module.py
    module.c
include/
    module.h
docs/
    conf.py
    index.rst
tests/
    test_module.py

The setup.py is the makefile equivalent for Python modules and it is often invoked through the following commands:

python3 setup.py build: builds the package, but does not install it.
python3 setup.py sdist: builds a source distributable tape archived file of the package and contains all the source of your modules.
python3 setup.py bdist: builds a binary distributable tape archived file of the package and contains only object files of your compiled code.
python3 setup.py install: installs the package to <python install location>/lib/site-packages/<your package here>.
python3 setup.py check: checks the package for correctness.

Distutils by default uses the compiler located at /bin/xlc to compile C source files. If you have set the environment variable CC, the compiler defined by the CC variable is used instead. If the Python package requires a C++ compiler, /bin/xlc++ is used as by default unless the CXX environment variable is set, in which case the compiler defined by the CXX variable is used. Similarly, /bin/xlc is used as the default linker for both shared and static libraries. If LD or LDSHARED are set, LD and LDSHARED, are used for each library type respectively.

When building packages with distutils or pip you may encounter build errors related to compiler argument processing. When building packages with distutils or pip you may encounter build errors related to compiler argument processing. Distutils may not always emit compile commands where it is true. For more tips about using IBM® XLC for z/OS®, see setting CCMODE step in Customization and environment configuration.

The usage of xlclang and xlclang++ is also supported. You can export

CC=<path to
xlclang>

and CXX=<path to xlclang++> to enable xlclang or xlclang++.

Note: There might be compatibility issues when mixing xlc and xlclang for compiled code and thus only one should be used consistently for building and linking modules.

On z/OS, DLL (.dll) and shared object (.so) files require a special file called a definition side-deck. The side-deck describes the functions and the variables that can be imported from a DLL by the binder. These files are generated automatically by the compiler when creating a DLL or shared object. For more information about side-decks, see Binding z/OS XL C/C++ programs in z/OS XL C/C++ User's Guide.

Side-deck considerations: Python.x is included by default and for other libraries, distutils automatically attempts to find the relevant side-decks. However, side-decks can be explicitly added to the build by using the extra_compile_args parameter to the Extension Class in setup.py.

Note: By default, distutils automatically supplies compilation and linking parameters for Python header files and libpython side-decks.

Note: If you use a dynamic library for Python packages, you should ensure that all .so or .dll files are found in your LIBPATH.

Troubleshooting

For more information about errors using distutils, see Errors when using distutils.

Examples

A simple setup.py for a pure Python module is as follows:

from distutils.core import setup
setup(name='example',
   version='1.0',
   description='An example package for distutils',
   author='John Doe',
   author_email='john.doe@ibm.com',
   url='https://www.ibm.com',
   packages=["ibm_example"],
   )

The corresponding file layout would be as follows:

example/
   setup.py
   ibm_example/
      __init__.py

If you want to add a C source file to the module, you can do it with the following lines:

from distutils.core import setup
setup(name='example',
   version='1.0',
   description='An example package for distutils',
   author='John Doe',
   author_email='john.doe@ibm.com',
   url='https://www.ibm.com',
   packages=["ibm_example"],
   ext_modules=[Extension('foo', ['src/foo1.c', 'src/foo2.c'], include_dirs=['include'])]
   )

The file layout would be as follows:

example/
    setup.py
ibm_example/
    __init__.py
include/
    foo.h
src/
    foo1.c
    foo2.c

Note: You can also add C++ files in an analogous manner. Make sure that you use the appropriate file extensions, since this is how Python determines which compiler to invoke for the source files. If you include several modules that are specified with different extensions, a separate shared library is produced per extension.

A setup.py example with an explicit side-deck is as follows:

from distutils.core import setup
setup(name='example',
   version='1.0',
   description='An example package for distutils',
   author='John Doe',
   author_email='john.doe@ibm.com',
   url='https://www.ibm.com',
   packages=["ibm_example"],
   ext_modules=[Extension('foo', ['src/foo1.c', 'src/foo2.c'], include_dirs=['include'], extra_compile_args=[/usr/lib/example.x])]
   )

If your module requires the use of dll or .so files, distutils automatically attempts to find them. When the side-deck is in a non-standard location, you should modify your setup.py to include the side-deck with extra_compile_args as shown above.

Best practice

When writing modules for Python, you should consider external dependencies, which can be located in non-typical locations, or in locations that are platform-dependent. To alleviate the non-typical locations issue, you can create a setup.cfg file that allows you to specify values at installation time. For more information on setup.cfg files, see Writing the Setup Configuration File. For more information on how to extend Python with C or C++, see Python/C API Reference Manual.