2.3 Describing extension modules

Just as writing Python extension modules is a bit more complicated than writing pure Python modules, describing them to the Distutils is a bit more complicated. Unlike pure modules, it's not enough just to list modules or packages and expect the Distutils to go out and find the right files; you have to specify the extension name, source file(s), and any compile/link requirements (include directories, libraries to link with, etc.).

All of this is done through another keyword argument to setup(), the ext_modules option. ext_modules is just a list of Extension instances, each of which describes a single extension module. Suppose your distribution includes a single extension, called foo and implemented by foo.c. If no additional instructions to the compiler/linker are needed, describing this extension is quite simple:

Extension('foo', ['foo.c'])

The Extension class can be imported from distutils.core along with setup(). Thus, the setup script for a module distribution that contains only this one extension and nothing else might be:

from distutils.core import setup, Extension
setup(name='foo',
      version='1.0',
      ext_modules=[Extension('foo', ['foo.c'])],
      )

The Extension class (actually, the underlying extension-building machinery implemented by the build_ext command) supports a great deal of flexibility in describing Python extensions, which is explained in the following sections.

2.3.1 Extension names and packages

The first argument to the Extension constructor is always the name of the extension, including any package names. For example,

Extension('foo', ['src/foo1.c', 'src/foo2.c'])

Extension('pkg.foo', ['src/foo1.c', 'src/foo2.c'])

describes the same extension in the pkg package. The source files and resulting object code are identical in both cases; the only difference is where in the filesystem (and therefore where in Python's namespace hierarchy) the resulting extension lives.

If you have a number of extensions all in the same package (or all under the same base package), use the ext_package keyword argument to setup(). For example,

setup(...
      ext_package='pkg',
      ext_modules=[Extension('foo', ['foo.c']),
                   Extension('subpkg.bar', ['bar.c'])],
     )

2.3.2 Extension source files

The second argument to the Extension constructor is a list of source files. Since the Distutils currently only support C, C++, and Objective-C extensions, these are normally C/C++/Objective-C source files. (Be sure to use appropriate extensions to distinguish C++ source files: .cc and .cpp seem to be recognized by both Unix and Windows compilers.)

However, you can also include SWIG interface (.i) files in the list; the build_ext command knows how to deal with SWIG extensions: it will run SWIG on the interface file and compile the resulting C/C++ file into your extension.

This warning notwithstanding, options to SWIG can be currently passed like this:

setup(...
      ext_modules=[Extension('_foo', ['foo.i'], 
                             swig_opts=['-modern', '-I../include'])],
      py_modules=['foo'],
     )

> python setup.py build_ext --swig-opts="-modern -I../include"

On some platforms, you can include non-source files that are processed by the compiler and included in your extension. Currently, this just means Windows message text (.mc) files and resource definition (.rc) files for Visual C++. These will be compiled to binary resource (.res) files and linked into the executable.

2.3.3 Preprocessor options

Three optional arguments to Extension will help if you need to specify include directories to search or preprocessor macros to define/undefine: include_dirs, define_macros, and undef_macros.

For example, if your extension requires header files in the include directory under your distribution root, use the include_dirs option:

Extension('foo', ['foo.c'], include_dirs=['include'])

You can specify absolute directories there; if you know that your extension will only be built on Unix systems with X11R6 installed to /usr, you can get away with

Extension('foo', ['foo.c'], include_dirs=['/usr/include/X11'])

You should avoid this sort of non-portable usage if you plan to distribute your code: it's probably better to write C code like

#include <X11/Xlib.h>

If you need to include header files from some other Python extension, you can take advantage of the fact that header files are installed in a consistent way by the Distutils install_header command. For example, the Numerical Python header files are installed (on a standard Unix installation) to /usr/local/include/python1.5/Numerical. (The exact location will differ according to your platform and Python installation.) Since the Python include directory--/usr/local/include/python1.5 in this case--is always included in the search path when building Python extensions, the best approach is to write C code like

#include <Numerical/arrayobject.h>

from distutils.sysconfig import get_python_inc
incdir = os.path.join(get_python_inc(plat_specific=1), 'Numerical')
setup(...,
      Extension(..., include_dirs=[incdir]),
      )

Even though this is quite portable--it will work on any Python installation, regardless of platform--it's probably easier to just write your C code in the sensible way.

You can define and undefine pre-processor macros with the define_macros and undef_macros options. define_macros takes a list of (name, value) tuples, where name is the name of the macro to define (a string) and value is its value: either a string or None. (Defining a macro FOO to None is the equivalent of a bare #define FOO in your C source: with most compilers, this sets FOO to the string 1.) undef_macros is just a list of macros to undefine.

Extension(...,
          define_macros=[('NDEBUG', '1'),
                         ('HAVE_STRFTIME', None)],
          undef_macros=['HAVE_FOO', 'HAVE_BAR'])

#define NDEBUG 1
#define HAVE_STRFTIME
#undef HAVE_FOO
#undef HAVE_BAR

2.3.4 Library options

You can also specify the libraries to link against when building your extension, and the directories to search for those libraries. The libraries option is a list of libraries to link against, library_dirs is a list of directories to search for libraries at link-time, and runtime_library_dirs is a list of directories to search for shared (dynamically loaded) libraries at run-time.

For example, if you need to link against libraries known to be in the standard library search path on target systems

Extension(...,
          libraries=['gdbm', 'readline'])

If you need to link with libraries in a non-standard location, you'll have to include the location in library_dirs:

Extension(...,
          library_dirs=['/usr/X11R6/lib'],
          libraries=['X11', 'Xt'])

(Again, this sort of non-portable construct should be avoided if you intend to distribute your code.)

2.3.5 Other options

The extra_objects option is a list of object files to be passed to the linker. These files must not have extensions, as the default extension for the compiler is used.

extra_compile_args and extra_link_args can be used to specify additional command line options for the respective compiler and linker command lines.

export_symbols is only useful on Windows. It can contain a list of symbols (functions or variables) to be exported. This option is not needed when building compiled extensions: Distutils will automatically add initmodule to the list of exported symbols.